scottyp,
This could very well be a self confidence thing... but that is a very culturally derived diagnosis. That is how we perceive it in the west. See the
wikipedia entry on eye contact and cultures.
If you don't feel that you lack self confidence, but still find yourself not maintaining eye contact... try to give this a try. Interact with people from the cultures where direct eye contact is considered offensive. Feel it out. Maybe you will feel more at home with that group.
Bottom line I want to convey is that these notions that "eyes are windows into the soul", etc are culturally derived ideas and are directly correlated to time and location. It isn't really bad or good to maintain eye contact!
(Yeah, I know its a crazy idea)