People start talking to me when I dress nicely, and otherwise ignore me. It's embarrassing.?

I find that I'd rather people liked me for who I am, so I rarely dress nicely when I go to church or school because when I do, people that otherwise ignore me try to be my friends and it bothers me. I dress nicely at home and in the general public where I don't know most people, and people who see me there see a different person. But I *want* to look nice, I just don't want to have to be rude to the people who I'm frankly not interested in being friends with because they weren't nice to me until I put on something "pretty".