Sounds like he's not the one for you, girl.
You need to be happy with yourself, babe. It's cliche, but true. What can he give you that you can't give yourself?
Make a list of a reason to stay and a reason to leave. I just sort of fell into a relationship with a very special man, and can provide an example list:
Why I Stay with My Sweetie:
1) He provides great, unbiased advice about what he thinks would be best for my life.
2) He looks out for me when he has no obligation to.
3) He is incredibly brave about his personal pain.
4) He is great with animals.
5) He has a phenomenol bod.

I mean, wow. Wow.
6) He is so emotionally honest and intelligent.
7) He is outstanding in bed.
Reasons to Leave:
1) He does have an illness that will continue to challenge my personal weaknesses.
2) He can be indecisive and scared sometimes.
3) He can be lazy to his detrement.
4) He's frightened of being in love again.
But when I look it over, you see why I stay.
Also, as Erin suggested in her blog, make a list of what's important in a lover. My top four:
1) Urbane and smart with original sense of humor and unique intelligence. Good test takers are a dime a dozen. Brilliant musicians? Someone who gets Eco? *Faints.*
2) Kind, especially to animals and kids. Baby talk to kitty cats from grown man? I will swoon.
3) Handsome as all get out. I like my partners in good shape. Cyclists and runners and swimmers are delicious. Must have nice face and be clean shaven and have great hair. By great, I mean touchable, sexy stuff I can play with. The Boy's losing his a little, but OMG I love having my hands in there! Such a dirty blonde!
4) Open-minded in all matters. Willing to look at new perspectives and wants to learn more. The Boy impressed me sooo much there. He's still growing as a person as he's approaching 40. That is hot.