Is it okay to eat a few bites of expired ice cream?

You will be fine. But I wouldn't eat no more of it just cause the sell by date was 4 months ago. It might be no good by now. Sometimes it very hard to tell if ice cream is good or bad. I had a bowel of ice cream last week, SELL BY date was not until October 1, 2014. And within the hour after I consumed it, I had an upset stomach and loose bowel movements, lucky it was just in a 12 hour course period. Didn't know if it was already expired and they lied or mix up and put expired milk in the ice cream, just say wasn't fun when my stomach was turning on me and running to the toilet to have the poops. Just make sure to get the freshest one you can find and always check the dates even if there only sell by dates. Never miss with dairy products that have expired, they can really make you very sick

Chances are it didn't expire in January, if it says "sell by" or "best by" that's exactly what it means and it didn't expire, if it looks, smells, and tastes fine it's likely safe to eat but, you should take it back just out of the principle that they sold you something they shouldn't have.

If it has been frozen for all that time it would be ok. After all Mawson left foods behind in the Antartic and it was frozen for 20 years or so and when they found it, it was still ok.

Did it look and taste ok?

If it didn't smell or taste bad it's probably ok. Just don't eat more of it just incase.