Could it be that many "That's racist" claims are a product of the the accuser's own offended FEELINGS, and NOT based on fact or reality?

absolutely. it's the sign that one party is losing the argument or can't defend themselves intelligently. the 'racist' cry is an effort to silence opposition.

Sure, you could make that argument...but that doesnt make racism right.  

Lets take workplace sexual harassment as an example here.  It doesnt matter what the intention of the comment, its all about the perception.  A simple compliment can be taken as the boss hitting on their subordinate...when they just thought the person was wearing a nice sweater...

Its the same thing with racism.  Sure, some people could be considered oversensitive...but I'd argue that the source of the over-sensitivity cannot be dismissed...