should i be concerned about fertility?

I'm 22 years old, and in good physical health (other than I'm currently dealing with extreme fatigue and tension headaches but that's recent.) my husband and I want to have kids in the next few years.

I'm 5 feet tall, 113lbs. I get my period on a regular basis, the first day is quite bad for cramps and it only lasts for 2-3 days. I have noticed my pms symptoms have changed recently but idk if that matters.

I am not on birth control.

Sorry for TMI, but my husband and I have only used the "pull out method". I have been told it is very possible to end up pregnant from this because of pre *** and whatever else. He also has not pulled out a handful of times in the past.

He has had one other partner and I have had multiple, all protected sex.

Although we aren't trying for kids or anything (we would be ready and financially prepared if it were to happen though), I haven't ever been pregnant or showed signs or anything, despite possible pre *** or not pulling out.

Should I be worried about fertility or is it just "luck" that I haven't gotten pregnant?