do i have a right to be upset or am i being immature?

my boyfriend and i have been dating for 2 years. lately, he's been talking a lot about being with me in the future and spending his life with me. we've also been joking around about putting all our future money into gold (people say gold worth 1600 now will be worth thousands in the future).

he's selling some of his furniture for thousands and he says he'll put that money into gold. i joke around and ask him if he'll let me hold onto it and put it in my sock drawer and i get really hurt and offended when he says no. when i ask him why, he says "because it'll be our money and i'll protect it. i have a right to be cautious with my money. what if i put all my money into gold and give it to you and then we break up and it's worth thousands?"

do i have a right to be upset or am i being unreasonable