Boyfriend constantly freaking out because he thinks im leaving?

I need a little advice here guys. Warning though. its gunna be long.

Over the past year my boyfriend and i have been in an LDR due to him being in college. The entire time he was there i was off my Birth control. I went back on it roughly about a month before he came home. And, by the time he got home i was a total side show due to the stupid hormones from the bc. I was on it for about two months after that and decided to stop it because i honest to god hated being a total hormone troll.. But, ever since i stopped it my boyfriend has been on some kick that im going to leave him, and its driving me nuts. I was no where near leaving him until he started this nonsense. Ive spent everyday with him since hes gotten home. The last two weeks are the only two weeks ive been home due to my horse being sick. But even then, i still find time for my boyfriend. The only thing thats been a little off with us is the fact that i havent been into sex much lately (due to a health factor). and its driving him nuts.. all he wants is sex lately. And i know its not all hes in this relationship for, but still i cant stand it. I dont know whats happend but all of a sudden he thinks im leaving him all the time.. if i dont answer a text, or come over he freaks out and gets upset. We've talked and talkeed and talked about this but nothing changes.. I wasnt leaving him but all this worrying and freaking out on me is really starting to push me away.

Any advice would be awesome. thanks :)