Broken restraining order and PFA?

My ex broke a restraining order against him and a PFA(protection from abuse) and his mother has felony forgery charges pending against her. He is disabled and I know he is going to go to jail for breaking the restraining order because the man who got it against him was attacked by my ex with a 5 foot steel pipe. Would I be able to take his visitation rights away from him because he violated the order? My kids are scared to death of going to his house. He lives 3 1/2 hours away from me in NY and I'm in PA. They only see him once a month but they always fight me to go and they come home in tears each time. He hits them, calls them names, doesn't feed them, lays in bed, they have no clothes or beds and he refuses to heat the house so they can be warm. I have to send them with thermal underwear, sweats and normal clothes just so they can make it through one day with him. He makes them sleep on ceramic tile in sleeping bags. He called my son on Saturday screaming at him over a broken game system(PSP) because I traded it in at GameStop and got him a non broken PSP. Then he yelled at my son 2 days ago for asking for a bed. He told him, "You can have bed or Christmas presents. One or the other." I know he has to have adequate sleeping arrangements for them, but he doesn't. So, can I take his visitation?