Multi-cat household, one cat is a bully!?

Hi, I would appreciate any and all suggestions. I have 4 indoor cats, all are very well cared for (up to date on vaccines, spayed/neutered), all receive plenty of attention! Here is the problem...my 3 year old cat is mean to the others, they are all afraid of him and go out of their way to avoid him. The litter boxes are in the laundry room (I have tried having litter boxes upstairs for the other cats, but it was too much), that is where they also eat. One of my other cats is afraid to come downstairs, we have to "escort" her to and from kitty box and her food so she won't get bullied. Any ideas for calming this kitty down and helping him be nice to others? He is a neutered male, was previously returned to humane society because he had this problem in other households. He is a wonderful cat in every other way...sweet, loving, laid back, just has this problem with other cats! Is there medication for this? Help!