Why do potatoes grow plants out of them when they rot?

Potatoes don't "grow plants". They are plants.

The potato itself is a tuber, a storage root. It's how the plant gets itself through the winter. Naturally, the potato would sit in the ground all winter, then in the spring when the soil warms up, it would start sprouting and grow into a whole new potato plant. That plant then creates a dozen or more new potato tubers, which would then sit in the soil until the next year, and grow again, and so on.

Potatoes stored indoors often start sprouting early because its warm, so they are tricked into thinking it is spring already. 

THEY 'GO TO SEED', SO IF YOU PLANT THEM YOU COULD GROW YOUR OWN POTATOES FROM THEM.