What is wrong with my eyelid?

I've noticed i get a tiny white little dot on my lower eyelid everytime i use eyeliner on my lower lids! This is the 3rd time it has happened and everytime it is on the same lower lid almost on the same spot,above my eyelashes! I went to the doctor the second time it happened and he said it was an infection, he pulled the white dot and gave my eye drops but now it's happened again! I don't think it's a stye because its not exactly painful or swollen, its more like a white head that you can remove by rubbing or picking on it. Is it because of my eyeliner? I use loreal gel liner. Nothing happens on my upper lid but why my lower lid?