Experience quitting a contract position?

Hi. So I need some advice. I'm usually good with stuff like this but I'm really stuck here and borderline panicking.

I am currently working a 2 month contract position, 1 month in. The pay is average for what I do, and they've made no indications that they want to keep me permanently after the 2 months. But I am comfortable here and like what I do (IT).

Recently I got a job offer, should I choose to except, for a 4 month contract position (with possible extension or possibility of permanent hire), paying wayyy more and with benefits. The offer expires in 4 days.

In your opinion, do you think its worth quitting the current contract job that I'm on? If so should I contact the recruiting agency that got me this job or should I go to my manager on the job site? I would ask if there is any chance that they'd keep me permanently, and if not then I'd give my two weeks...(i guess lol)

I'm leaning towards staying at my current job to avoid the stress of the on-boarding process and having to tell people that I'm leaving. Pay is not a huge concern for me, and I also enjoy the people I work with.. I'm really stuck on this one. Any input is appreciated. Thanks!