Can I use any disc brake with any rotor?

You can keep your rotors... provided they are the same size.

I have two different hayes rotors on my bike, but in our shop, I often sell different rotors than brand of brake.

Now, hayes soles are single piston calipers... that means only one brake pad moves. If adjusted properly the braking power can be great. But they really have to be VERY finely adjusted.

If you switch to any double piston caliper, you will notice a difference for sure, as both pads will sqeeze the rotor. Hayes HFX 9's or El Caminos, Avid Juicy 5 and 7 are good and better respectively for each company.

If the braking is "mushy" you might just need to bleed the line.

As for mechanical, you can get INSANE braking power if set up correctly with proper calipers and levers- this has a lot to do with the lever. A crazy good set up I have seen is the Avid BB7's with XTR levers, or XT with the pivot point moved inwards. The braking power is crazy, but there is absolutely no modulation. This is really only ideal for dirt jumping and downhill - not XC. Usually the stock levers are only okay...

EDIT***

A double piston caliper will provide better modulation, but to me it sounds like you might be a candidate for a really good set of V-brakes - check out the XC pro circuit, they are used frequently. They are lighter, and XTR brakes with XTR levers will provide just as good stopping power in dry conditions as a double piston brake. Even the XT or LX levers with the pivot point adjusted are crazy good.

The modulation with v-brakes is better for sure!

Some rotors can be used on some other calipers. I had Shimano XT on Hayes rotors without any problems, but I would hestitate to go viceversa.

Hayes Sole are the low end of the spectrum for Hayes, so don't judge all hydros for this.

Hope, yeah! Id love some M4, but the Mini sounds pretty nice. Nothing wrong with the Avid, they're great, but I'd love the Hope. Other options are Magura Marthas SL or Louises.

Word up to 'ct' He has it pretty much right on. I use disc brakes on my bike with two diff size rotors. I am used to them and don't miss the v-brake "feel" However I do use v-brakes on my SS MTB just because of the "feel" I do switch pads for different conditions. A basic pad is good in dry to damp (no water or mud over rim) and will switch to a wet weather pad when is raining or muddy.

Hayes Sole is the entry level brake for Hayes. The 9s really a much better brake and has good "feel" for a disc. I like mine. My son has Avid BB7 with Avid SD levers, which is a mechanical setup, and those are nice as well ... although they are a bit heavier than the Hayes they have plenty of power and are easy to feel. The reviews on the Juicys are positive in the riders I know with them. I can't comment on them from experience tho as haven't tried em. The other thing to consider, I didn't see in your question, are you Downhilling or XC? DH will really benefit from a bigger disc and feel as well. XC a 6 is plenty big when setup with the right pads. I go 225 and 6'1" and I have a 6 on front and 8 on back. Waaaaay more than enuf stopping power and smooooth when I am just feathering into the brakes as well.

Hayes Sole Brakes

the discs are the rotors which you dont could desire to interchange frequently. You replace the pads that touch the rotor greater frequently. and you have the rotors "became"( milled soft) in the event that they get grooves. while the rotors get became sufficient cases or in the event that they get warped, you alter them

get Hayes nine they are excellent, Hayes sole aren't so good my friend has them and he is planing on the nine and he knows more about bikes and bike parts more then any one I know, I would have gone on avid juicy,