Why are my teeth in such bad conditions?

I'm 16, and I get cavities every year despite brushing twice a day! Let me first present a history of my oral hygiene:

Firstly, both my parents have bad teeth/gum problems. I'm not sure how bad but I can definitely say that both of them have gum diseases right now. Ok anyway, I've always had yellow teeth since I lost my baby teeth. I can remember that when I was little, I never had good oral hygiene AT ALL. I didn't like brushing, and I guess there were days that went by without any brushing (which is disgusting to think of now!). I started brushing at least once everyday when I was in my pre-adolescence and early adolescence. I got braces in 8th grade and that motivated me to keep a good oral hygiene, so I mostly stopped neglecting brushing teeth. I started firmly brushing twice a day everyday in 9th grade. I regret to say that there was a short period when I brushed vigorously, and I just learned recently that that wears down your enamel... Anyway, after I got my braces off (10th grade/last year), I started taking care of my teeth really well. I really wanted to get rid of my yellow teeth too, so aside from brushing routinely twice a day, I started flossing from time to time. I've recently (since half a year ago) started flossing every single week, and I've bumped that up to every single day and nowadays I use Listerine mouthwash from time to time (though I don't like it much because it burns).

I do eat sugary things often, but don't we all? I'm talking about fruits, ok and sometimes pastries fine... But I'm not a big fan of candies. I rarely drink soda, if not at all. I've been trying to eat healthily.

The problem is that every time I visit the dentist, I still get multiple cavities. A month ago, I was shocked to hear that I have two new cavities. I still have yellow teeth. I have gum pockets and receding lower gum (mostly due to the damage caused when I was little). My lower gum also looks red, not at all light pink. My teeth are just a TINY BIT loose but I expect that from braces + wearing retainers every night then taking them off in the morning + throughout the day. I have fillings on just about every teeth at the back of my mouth. Many teeth have black stains on them (since forever and the dentist didn't say they were cavities so...). Don't get me wrong, my teeth don't look rotten, fortunately, but upon close inspection they are certainly not completely healthy.

So what's wrong with my teeth? I know friends who don't take care of their teeth as well as I do now and they never get cavities, plus they have perfectly white teeth!

Why are my teeth yellow despite attempts to clear them (I've tried whitening toothpastes but I haven't tried actual whitening for fear of weak enamel)? Why do I get cavities so often despite brushing routinely? Are these results of the damage I've caused when I was little? Will I ever have healthy gum/teeth again?