Faye, I hear a lot of "yeah, but" in your responses to the responses -- some devil's advocate stuff going on (and guess who is the devil in this situation!

) What I mean is: it sounds like you have a pretty good sense of what it takes to live a life you love, but you are second-guessing it, doubting your own inner truth. That's fine, and it can lead to a new perspective and personal growth. You can make your choices in your own good time, and you can't really make a mistake.
And... from my perspective, it looks like one thing you could generate that would make a difference is Trust -- trust for yourself primarily. Can you see anything that you could generate, completely independently of what he says, does, or thinks, that would make a difference? Also, something you could let go of?
I know it's a tough spot. I've been there -- a different flavor of it, not polyamory. But a similar feeling.