Hi, my kids were (and are) small too. Not in terms of height, but in terms of weight. Right at the bottom of the charts - like bottom 5%.
It's probably genetic, because both my wife and I were skinny kids, and she's still skinny today. (Me, I was skinny up to age 30, then started growing a little fat
)
As long as the kids are healthy and happy, it's nothing to worry about.
Like Spirit4711 said, if a parent wants to obsess, there'll always be something to obsess about. The kid is too short / too tall / too fat / too thin / too shy / too boisterous / too stupid / too smart / eats too much / eats too little etc etc.
Really, the worst thing you could do is inadvertently affect your own child, with your own concerns. Like, he grows very self-conscious about his height.