Transformer based Regulated power supplies, IC7812, IC 7805, and loading issues?

Voltage drop and ripple are issues when the regulator has insufficient voltage across it. This applies to a 12V regulator here. Power dissipation is the issue for a 5V regulator.

Transformer

Transformers are usually rated to provide their voltage at rated load, and will be higher voltage with less current.

Diodes

These diodes will drop around 2V or more as the peak current is high. Use 6A diodes for a better result as they will have lower voltage drop, and get less hot. With the right schottky diodes you could save maybe 1V here.

Expected peak voltage

This is the highest peak voltage of the capacitor ripple. You need higher current diodes, but something around 14V peak is likely.

Capacitor and ripple

The ripple percentage is due to capacitor size, load current and also capacitor ESR and the ESR (equivalent series resistance) at the peak current of other components like diodes. Even with 10,000uF it is more than 2%. Several capacitors in parallel works better. Lets say the lowest voltage is then 13.3V. This is the main issue with a 12V regulator.

With a 5V regulator, load 0.55A and 1000uF the ripple is around 20% of the peak voltage, so 14V - (14V * 0.2) = 11.2V approx, so a bit high, but ok for the regulator. The average, as read by a meter, probably about half way, so 12.6V. Your 13.1V does not seem unreasonable for 550mA, only half the transformer current rating.

Regulator

The 12V regulator dropout voltage of a 7812 is something like 2.6V for line regulation, especially when approaching 1A current load. This means the input must be at least 2.6V higher than 12V always. This is not so, even with the upgrades suggested (13.3V). The only choice is to use a different "low dropout" regulator chip rated for 12V at 1A output. You have around 1V available, so a suitable device might be the LM2940 in the third link, which requires only 0.8V, but needs 2V for good line regulation. With large capacitor and schottky diodes it just makes this. It could need a small heat-sink depending on the load.

The 5V regulator is dissipating something like (13.1V-5V) * 0.55A = 4.5W. This needs a heat sink to keep it down in temperature. A T0220 package needs a good heat sink rated at least 1 degree C/W, which with the junction to case 4C/W is a total of 5.5C/W (that is for 25C rise). With 10C/W as in the link, the rise is 66C, very hot, but not in thermal shutdown. The diodes must be increased to higher current anyway, then you need no fan.

Fan

Yes the current of the fan plus the current of the supply is too much.

A smaller fan would allow the 10W heatsink to run cool. Or get one that is mains operated. Or think of a switching regulator. The losses should be small enough that it needs no heat-sink.

You could also use a pre-regulator with 8V output to supply the 5V regulator. Needs more capacitance at the diodes, maybe 4000uF instead of 1000uF. This splits the dissipation between 2 regulators so both need the 10C/W heatsinks or smaller even, much easier than a 1C/W heat-sink. Cool, no fan. This is what I recommend.