Can I charge him with assault?

I've had history in the past with this guy, I honestly have never done anything to him and I asked what I did to make him have so much spite towards me and he just told me to shut up. I'm assuming he hates me because I used to be really into drugs and got expelled but now I'm clean. He always picks on me at parties saying he could beat me up. So I was at another party last night and I was really drunk and there were a pair of shoes that were really similar to mine and mistakenly thinking they were mine I put them on. It turns out those were his shoes. Out of no where he came out and pushed me to the ground and ripped off his shoes and said he would beat me up if there wernt people around. This is really bugging me and I feel like this is never gonna stop. I don't even want money out of this I just want to get a restraining order or something. I'm not a small guy either he's just bigger, but I just don't want to degrade myself and start fighting again because I have changed my morals. If anyone can help I'd really appreciate it.