Does my boyfriend like my roommate and her friend?

I have been dating him for about 6 weeks. He is very nice to me and very romantic and we meet almost everyday. He usually comes to my place and most of the times we encounter my roommate and one of her female friends as well.

These two girls are more than friends, though I don't know exactly the extent of their intimacy. In the beginning they were extremely flirty with my boyfriend and he was reciprocating as well. One day, we were hanging out a lot of friends and he was sticking to those two only and ignored me for a long time. I didn't like this behavior and told him to stop that. He said he was just having fun and wasn't intentionally ignoring me. And then one day one of the girls got drunk and said some very offensive things to my boyfriend. After that, he stopped being "too friendly" to them and never flirted with them again. Although my roommate still does inappropriate things sometimes.

The problem is he says certain things to me occasionally which are very upsetting. He has mentioned many times that he fantasizes about threesome and I made it perfectly clear that he isn't getting it from me. Last week, my roommate, her friend and another guy cuddled together (this is what they told me. I don't know if it went any further.) When I told my boyfriend about it, he said "I am much happier dating you." When I asked why he said so, he mentioned "That could have been me instead of that guy."

The next day, he said something about my roommate being too flirty and in the beginning and said that he would have asked her to cut that out if it had continued.

Yesterday, he said that one of the two girls may have made him cheat on me if he had not stopped "being nice" to them. I asked how can anyone make you cheat, if you don't want it yourself. He started saying that "No, I will never cheat on you. But they could have tried."

I trust him that he will not cheat on me. I really like him a lot and he likes me too. I know I am much nicer than any of those girls and I really am a catch (I really don't mean to be impolite or arrogant by saying so). And he is smart enough to see that. But I can't help thinking that somewhere in his mind, he is fantasizing about my roommate or her friend because it keeps coming out at regular intervals in some form or the other. What should I do? I have told him a number of times and don't want to sound like a jealous and suspicious girlfriend.