How to control an "out-of-control" temper?

I have an issue involving "fights." When ever someone disrespects me and we get into a heated argument, I usually end up walking away and hitting something with my fist. Its usually only one punch, but it calms me down (I also don't punch break-able things such as walls, shelves, cars, etc.) Lately, however, my hand becomes extremely sore and cut up from "releasing my anger in a bad way." I have tried calming my self with other means, deep breathing, taking a walk, etc. and they were followed by failure. I'm not the kind of guy to hold grudges, so I get over things pretty quickly. However, I have a "photographic" memory. I remember things and tend to slowly build up, even if I forgive the person who had been the reason behind the fights. Sooner or later, I have a feeling I might end up punching something a bit too hard and busting my fist open or break bones.

My question is, What can I do to calm myself or rather, how can I stop myself from getting from 1 to 100 in a half-second during a simple argument?