Why do photographers have more rights than their subject?

The courts have ruled over and over again that people in a public setting have no expectation of privacy. The moral of this story is: don't do anything you are ashamed of in public. As for monetary reward, you are only entitled to a payment if a contract exists.

If the subject knows he is being photographed, he can make a claim too. There are lawyers who can make it happen but timing is essential.

The laws in the United States state that if anyone is out in public on public property when privacy is not expected, then anyone can photograph them without permission.  However, the law restricts what can be done with the image.  If the image is to be used for commercial use (e.g. in an ad or promoting a business) then the photographer must get a signed model release from anyone who's recognizable in the shot.  Recognizable means being able to see their face as opposed to shooting them from behind.

So you need a model release for commercial work, but you would not need any permission for that same image if you were to be selling it as fine art or publishing it as an editorial or newspaper.

Then there's "embarrassing but true."  Publishing an image that causes someone embarrassment (usually not a public figure) even out in public is grounds for a law suit.

What you can't do is photograph someone who's in a privately-owned place (e.g. their home), which is an example of an area where privacy is to be expected.  So photographing a person with a telephoto lens in their bedroom while standing out in the street is not legal.

Is this fair?  Yes.  You can always ask the photographer to not take your photo or delete the ones that have been taken, or take the photo down.  Most photographers will comply.  But if the image has social importance (i.e. newsworthy) then you're likely to be out of luck.

Photographers have no more right than you do, sir, so stop trying to make something out of nothing.