What would cause a Woofer to buzz?

Some level of deterioration other than the surround itself. Pull the speaker and look at the spider as it may be coming loose at the basket or the voice coil. Also the point where the cone and surround are attached may be coming loose. I've seen many, many cones come loose from the surrounds. You may also have a slag of glue rattling under the dust cap. Don't rule out a dust cap coming loose. Lastly look at the electrical leads from the terminal plate to the cone near the voice coil as they can be the culprit.

Keep in mind that the woofer cone and it's attached parts move at an extreme velocity, stopping and reversing constantly. So while a problem may not be detectable when pressing on the cone carefully, a deteriorating adhesive at the surround to cone or spider/voice coil to cone can easily be the problem.

I would take the bare woofer, connect wire direct to it in a free air state and carefully play music with full bass / minimum treble. Pay attention to if the buzzing is louder from the front or back. With the volume just high enough to buzz, use your fingers and feel around the cone / surround / spider. When the buzz goes away - you found your trouble spot.

If you find a loose surround, use Eileens or another adhesive that remains flexible. If the problem is at the voice coil/spider, you do not want flex... but you want more grip and strength than elmers.

The voice coil is probably coming apart.

It's just breaking down over time. Sounds like you've checked everything I would.