Why is it that gravity doesn't separate out the various elements of the atmosphere so we would have a nitrogen layer and an oxygen layer?

The kinetic molecular theory is where we start. One of its postulates is that molecules are in constant random motion. Because of that gas molecules will diffuse throughout their container. In this case the "container" is the entire atmosphere up to an altitude of about 80 km (50 miles). Therefore, N2 and O2 (as well as the trace amounts of other gases) are distributed in the same proportion throughout this portion of the atmosphere.

Above the 80-90 km range (which is approaching the commonly used limit of the atmosphere) the gases are stratified by molecular weight. But keep in mind that many scientists place the "top" of the atmosphere (and the beginning of space) at an arbitrary altitude of 100 km (62 miles). At this point the molecules are widely distributed and the pressure is extremely low. The pressure has gone from 1 atm at sea level to about 10^-6 atm at 100 km.

The bottom line is that for the portion of the atmosphere which "supports life" the atmosphere thins and the pressure drops but the gases do not stratify by molecular weight. Instead the proportions of the gases making up the dry atmosphere remain in the same proportion.