This is going to be really left wing, but I think the reason why you like him so much is because deep down you know he will treat you like crap and your current self hate of yourself wants to bring more of that into your life through another person.
How you have described him, as a guy who only wants one thing, really sounds like there's nothing more to him than his looks and his bravado. How could anyone like that with no substance like the true and real substance in another. I can see that he would cause misery in another person that isn't at least as forceful metal-emotionally as he is. I could be wrong though, he could be very nice and sweet, but I'm just not hearing that.
Right now though you should take an opportunity to look into yourself and find out why you like him so much. What in you resonates with him and makes you like him so much? It's good that you are doing the work to raise your self esteem but I think it's like building a house on quicksand. The foundations are far from solid. There's a deep core of belief in you that sees yourself as not good enough, and that you deserve to be with an arsehole. Take some time out this easter to have a deep look in yourself and try and discover why you have such a negative vew of yourself. Use this self discover to free yourself from those negative beliefs and recreate yourself as the whole and perfect person that you are, flaws and all.
The real truth is that all of us humans really are perfect and it's just our own views that keep us away from seeing that. Discover that for yourself by deep self discovery and your life will change for the better.