Depends what you mean by "smart," as that term can have a number of different definitions. But I'm assuming that when you say "smart," you're referring to something like an IQ (as opposed to being "street smart" or "money smart" or similar).
Doku makes a very, very good point about cause-effect relationships. The smarter you appear to be, the more that's expected of you. And if you measure up to those expectations, the bar is raised even higher. Meet those, and the bar goes higher again. There's no upper limit, and even the smartest, most positive and driven personalities will eventually come to a point where they see a law of diminishing returns; that is, all that extra work for only a fraction of payback (however that payback is defined).
Ever see the movie, Good Will Hunting with Robin Williams, Matt Damon and Ben Affleck? That's a good look at someone who's brilliant but has come up against the wall of diminishing returns.
Some smart folks, then, simply step off the "smart folk" treadmill. And those that do are often perceived as failures, because they're not playing by the rules that social convention expects.