Weird skin growth behind ear (Picture)?

http://www.flickr.com/photos/64546272@N02/71683406...

This strange growth has been there for as long as I can remember. I am female aged 18 if that helps. It doesn't hurt or bleed, but has been growing steadily over the years. It only hurts if I pick at it or squeeze it. It is soft and flimsy to the touch. I had it checked by a gp and was told it was a benign fungal growth. Later, I went to a dermatologist. She told me it could be epidermal nevus, but was not completely sure. Both times I was told it was non-cancerous, which is a relief.

The dermatologist referred me to a plastic surgeon to have it removed because it could potentially deform the ear due to it growing larger. Her other option was to just leave it alone. It wasn't very encouraging to hear she didn't exactly know what it was. I just want to know if it should be removed and what exactly is it. Thank you!