What do you think and what would you do?

I've been dating my boyfriend for about 5 months now. I love him, I really do. Hes sweet and hes good to my son and I. Plus hes totally honest, maybe even brutally honest, which is alright. I'd rather have that then someone who lies lol. But theres another side of him that I'm starting to see. I asked him if he would get up to go to church with me this sunday and he said no. I got somewhat upset because when I ask for favors, he always says no. Though he lives 20 miles away. And he works the night shift so hes used to staying up until 4 or 5 in the morning. I told him just to go to bed early and he just kept saying no. Then he said he wasn't going to be miserable just to make me happy. I can understand that but I put in more effort then him when it comes to that. I drive to see him almost everyday. It wouldn't kill him to put a little more effort in on his part. I tried telling him that and he just said that's how he is and hes nothing changing. Love him for him or leave him. Do I have a reason to be somewhat upset? Should he put in a little more effort or am I making it a big deal? I don't want him miserable, but I would like a little more effort on his part. I don't know what to do...thanks for your opinions!