Is it ok to pick the smegma out of a horse's sheath without water? Will it hurt him?

A tiny bit of mineral oil can help loosen up some of the flakes, but I agree with the others that outright washing can cause more harm than good. Getting the bean out is the most important point of sheath cleaning, so as long as you keep that clear, he should be fine without an all-out cleaning.

Honestly, not much point in sheath cleaning. Observe him for lesions and swelling as a routine going over, make sure he is feeling fine and urinating normally, dropping, the usual. But, it's rather a self-cleaning mechanism.

Within five days after a thorough clean, enough dirt and smegma has built back up to rival what you had so recently removed. The oil production of the average horse is not enough to become an issue where smegma build up is concerned. (If your horse actually requires his sheath cleaned per the vet's advice, that's another matter. If he is an average Joe, chances are good he doesn't need it and you're being an annoying bother.)

Frequent cleaning, especially with soaps and products, can also disrupt the natural pH balance and bacteria that help keep the area healthy. If it's mandatory, doing it with nothing is preferable, but do be gentle. If it's unyielding, it'll come off in its own time. You don't want to pick and pick and pick until you do peel off skin, because that provides an open sore for bad bacteria to enter. Once you've got that, the chance for infection sky-rockets and you'll wind up with much worse problems than a touch of dirt and oil.

I'm chiming in to agree with Bard. You can wear disposable gloves and gently grasp the end of the penis with one hand while you run the other hand up the shaft to knock off the loose smegma, but don't rub or scrub it with towels, etc. Sheath cleaning is mostly about human ideals, and is not beneficial to horses except when done as part of medical treatment for cancer or infection. Cleaning a healthy sheath is more likely to cause lesions and infections than it is to prevent them.

I agree with the two previous answers, but what I do check for fairly regularly (whenever my gelding "hangs low" is that there is no "peanut" stuck in the end of his penis. I pull back the head of his penis a bit and there is often a little yellowish-grey "peanut" that I pick out. Years ago we had an older gelding who was having trouble peeing because of a peanut as it was big enough that it plugged him up... I just check our horses when they give me the chance to avoid future problems.