Home-Grown Peppermint has strange Musky/Leafy taste?

So, I harvested some of the Peppermint I've been growing, and brew some fresh leaf tea with it, but I've realized that while it DOES taste like peppermint, its got another, woody/musky flavor to it as well.
While it is not undrinkable, it is not great anyone know what might be the Cause of it?
The only things I can think of are;
1). I harvested to late in the growth, only started to pick it because I noticed it was started to Flower
2). harvested it to late in the day, around 6-7 PM.

Cara2016-08-19T02:06:16Z

Are you sure it's actually peppermint, and not some other kind of mint? There are many kinds, and many flavors, of mint.