I'd like to know how to clean this without issue?

yes there is, here is a website i used to remove mold from my grandmothers wedding dress:

http://urbanext.illinois.edu/thriftyliving/tl-remo...

Don't take it to the dry cleaners and don't throw it in the washing machine. I've done both and the colors will run all over the place. You can try washing the hat in the dishwasher. Doing that is the only luck I've had with not completely ruining the hat.

Try with Borax and white vinegar mixture. Scrub off the mildew gently using an old toothbrush then wipe off with a damp cloth. Repeat this until all the mildews are gone.