Why am I so mean to him?

There's a guy I recently met here at school and we play around in a joking way. He tends to pick on me to see how far he can push his buttons while I react back by punching him back. (A little brother would to his older sister). When I react back to him, I'll usually say "I don't like you". Yet, he and I have the best, intellectual conversations, especially since he and I come from two separate countries. It is amazing how both countries have a lot in common and I enjoy learning a lot from him. He's one of my favorite people.

But, I have noticed his characteristics (free-spirit, adventurous, attractive, extroverted, very optimistic, Caucasian, great bod, youngest child in a very loving family) are my opposite of mine (structured, cautious, introverted, not very attractive, pessimistic, African-American, struggles with weight, eldest child from a separated family) that I am exceptionally attracted to him. And, when I am around him, my sarcasm is at its highest or I'll say mean things to him.

Deep down, I feel if I go along with his happy-go-lucky personality, I'll fall for him, like the other men I've liked in the past similar to his personality, and he'll reject me like the other men have with me.

Why am I so mean to him and other men like him? Am I pushing him away?