Son just turned 2 and he hasn't grown much. Should I be concerned?

To me, it seems like he’ll grow later. Give it time. If he’s not growing, after a certain age, see a doctor. But to me, this seems normal and nothing to be worried about. 

If he stays small, then that's how he's supposed to be.

I would also have been the smallest in all my classes until I reached high school except I had a girl who was a midget/dwarf/little person (or whatever is today's accept term) in all my classes.  (By random change when my youngest sister went to an out of state college, she ended up with this girl as her room mate for four years.)  Today I am a little over 6' tall.

All my kids were also the smallest in their classes.  Both my girls are about 5' 10" now, and my youngest son is 6' 2" tall.  My oldest grandson is 6' 5".  (My other son, however is only 5' 4" ????)

My oldest daughter was so tiny that when she when she was in middle school she looked like a 1st grader.  Her aunt was the private assistant for Gloria Gaither (of the gospel group the Gaither Trio).  She got my daughter a summer job traveling with the Gaither for about four months as the lead singer for one of the children's albums because everyone thought she was 6 rather than almost 16. 

Over the next year I swear she grow an inch a week,  But the next summer she was 5' 10" (and traveled with Sandi Patty and Carmen the next summer.)