My dog doesn't trust everyone?

I have a 3 year old pit bull mix. He's very friendly with 99% of people but every so often he decides that he doesn't like someone. He has never bit or bared he teeth but he will bark until I tell him to stop. It doesn't happen often enough to truly be concerned but I'm curious as to where it's coming from. From what I can tell it seems to be people that are leary of him that he's uncomfortable with. Example- 2 men came to my house to drop off a couch. One was happy to see him and bent down to pat him and they made fast friends. The other guy clearly was uncomfortable (kept his hands in his pockets and stood back) Loki stared at him and then started barking, I guess it's a chicken/egg question. Do you think he gets uncomfortable because the person is or do you think he senses something about them?

There's been a few people that I thought were really nice that he was uncomfortable with and all of them have turned out to not be as great as I thought they were. (One was a sex offended that lived next door that was extremely friendly but niether of my dogs would go anywhere near him, it wasn't until the school released the sex offender registry at the beginning of the school year that I realized who he was)