My boyfriend's depression is depressing me?

So I'm fairly certain that my boyfriend has manic depression/bipolar disorder. He'll have a few weeks where he's in a normal mood and acting like a normal person, followed by a few weeks of manic/depressive episodes. Mostly depressive. And when he's depressive, he doesn't want to talk much and is just kinda...not exactly hostile towards me but acts like he doesn't like me very much. I try not to take it personally because I know he loves me and I know what it's like to hate the world and have depression (I suffer from it myself) but it's really hard to deal with.

He refuses to go on meds and won't do much else to control it either. We're long distance and he doesn't get like this when we're together in person. But we only get to be together for a few weeks at a time every 3-5 months. So his being not talkative really puts a damper on the relationship. My ex left me because I had depression and I don't want to do that to my current boyfriend, he's really a great guy, but his depressive episodes are so hard to deal with. My mood generally reflects his mood and when he's in a bad mood, it puts me in a bad mood. How can I better deal with this?