This might be way off, but what does you intuition say about being together in the long long term. Deep down, what does the quiet voice inside say about it. Is it really the best you can do, and are you really the best people for each other.
From the doubt in your first post, I would say not. Even if you love someone, and even if you connect on an emotional and spiritual level, you still might not be compatible. Perhaps he just came into your live to teach you about yourself, and more importantly, about forgiveness. If you can forgive him 100%, then you can forgive everyone in your life that deserves it, your family, your friends, everyone. And if you choose to divorce and find a new partner, you can forgive all his future flaws before they become a problem.
I don't think you are stupid for seeing him at all, but I do think it's the wrong question.