When should one say "I love you?" Too soon?

My bf of 4 months and I were joking around and he said "I love you, but Im going to throw you over the balcony." I know its too early to say I love you, that was the first he said it. i got all excited and he said he didnt mean it that way. He likes me A LOT but not love, that takes time. He will tell me when he loves me.

I thought he did bc, hes been much more into me and our rs lately. He introduced me to all his friends and siblings, he met my parents, he helped me move homes, find a new job, buys me surprise gifts all the time, called me pretty after 4 months of dating before it was always cute! He even told me im unique and there is no one else like me. He constantly wants to hang out, and ive spent all weeknights at his place last week. When he went out of town he said it may sound lame and girlish but he was going to miss me and would call. He even left me a shirt of his while he left bc I love his scent! Are these things a man would do if he is NOT in love? All my friends get guys that fall in love with them after 3 months, but.it seems hr may take awhile. What gives?