What Agota says is spot on. I used to be like you - thought I was ugly and no one could possibly want me. It was my own perception. At this point in my life I am still 35 pounds over weight. I can tell you when I was 50 pounds overweight I'd see guys kind of look my way but pretend they were looking at something else as soon as they'd see my body it was like "oops she's fat - lets ignore her " And I would be hurt and upset.
I think just working on myself ane acknowledging my feelings and accepting myself as I am helped me to care less what men think. Now that I am down 15 pounds I get more attention from guys but there are still guys that I know find me too fat. I just pass them by.
I think you are putting too much of your value on how other people see you. If a guy seems to respond positively - you have a good day. If they respond negatively - you have a bad day. Don't let other people have so much power over you.
Next time you overhear someone say "she's ugly" I'd go right up to them and say "what did you say? I'm ugly? How rude!" They'd probably be totally embarassed.