Ex-husband staying with me...tough situation?

This will be somewhat a long story, so I hope you all can bear with me...

About a year ago my ex-husband's father past away, he ended up quitting his job because they wouldn't let him come home to be with his dying father, he asked if he could stay with me for a while.  I said sure since he had no really else to go at the time.  Over the last few months he's asked me if I want him out and I kept saying yes.  He's tried to get back with me but even though I still love him I don't think it's strong enough to take him back, if that makes sense. Also during this time I've gotten really close with a guy friend I work with.  My ex seems to get really upset that we talk on the phone a lot and I never really said anything about us hanging out, which I never brought my friend back to my place! Now yesterday evening my ex saw us hugging outside of work, even exchanged a kiss (after our shift ended), I came home and basically got an *** chewing.  So he decided he'll be out of my hair within a week or 2, he won't be a imposition (sp?) on me and I will never see or talk to him ever again.  I could also tell he was really hurt by all of this too.

Why would he be so upset over this?  Is it because how he still feels about me and he also can't get over the past either.  Now why do I feel so bad about this whole situation?  Also, why should I feel guilty, about all this?  Or am I just being silly? Sure I wanted him to move out, but I honestly didn't want this to end on bad terms.