Two things jumped out at me from your post. (Welcome, by the way!)
First, he wants you to 'stick with it and see what happens' and he knows that you are hurting in this situation. Your well-being is way, way, way down his list of priorities.
Second, he says you make him happy and he needs you for his happiness. This is not a man who takes 100% responsibility for his life.
He is who he is. If you stay in this relationship the way it is, you are in a relationship with a man to whom your well-being is not a priority, and who doesn't take 100% responsibility for his life. Regardless of his polyamory or the state of his relationship with the Gamble Girl, is this the kind of partner you want to be with? The answer may be yes or it may be no, but the important thing is: the choice is all yours.