"Breaking up" with someone who doesn't even think you are in a relationship...how to?

You have to ask yourself a couple questions first:

Why did you and your ex break up? That whole, its broke for a reason thing" always comes into my head when my friends get back together with an ex.

It could be love with him, but you don't always love whats right.

Will your ex leave you again, or will similar problems that lead to your break up become an issue in a future relationship?

As for the casual guy, it sounds to me like you two have an understanding. Is he seeing anyone, as well as having sex with anyone while he is with you?

If he is noncommittal, then you do not have to worry about breaking it off with him. He will know that you got back together with an ex and that was the end of the bed buddy situation. Trust me, he will take the hint when you don't take him up on his offer when hes 'up'

However, if he has feelings for you like you assume, let him know. Say, hey ___ my ex wants me back. The only thing stopping me is you. I care for you. If you see a future for us, then I already made up my mind. But if all you see for us is what we have now, then my mind is made up too. I need more than this.

That way it gives him the option to stop you and come clean!

Nope, not really a question! Sounds like you're totally together, and you know what you want and why. The casual guy will deal with it and stay friends if he really likes you as a person. Be gentle with him though - as you say, it's quite possible he feels more than he lets on, or even more than he realises himself. But that's life!