I like a boy... he says just friends?

I liked him even before we met. We hung out 3 times and talked almost every day on the phone before we did have sex the 4th time we hung out. He kept saying he doesn't want a relationship and we're just friends. We did it again then after that we talked on the phone the next day and I said sex kills friendship and I like him too much to do that, we agreed not to have sex again and he still talks to me and tells me to call him everyday. I wanna do it more, but not if it's really just so casual. I just wanna do it with him if he's not doing other gurls as well. I don't want to get married or have him meet my family or anything serious! Does he really want to be just friends? Is this a guy's way of saying they want to have sex with other people too? I think he does care at least a little bit, cuz why else would he still wanna talk to me after deciding together that we wont do it anymore? Maybe he's just protecting himself from getting hurt? (I know I sound like a kid but were both in our 30's)