Dating a guy when you are not his type?

I have been seeing a guy and he told me that I am generally not his type of woman. He dates the skinny barbie doll type and I am the complete opposite. But with that being said, the ones he does date use him and treat him like crap. I am the opposite of that also. So here are my questions: Is it a good thing to date someone you are not used to being with? Am I just setting myself up to be hurt? Can it really work out? We have a LOT in common...it's almost scary. I care about him immensely and he told me that he cares about me and has feelings for me. But can it work? I am afraid that when he looks at me he sees the outside and is like "what the hell". I need advice.