It's possible to dress comfortably, without going for the latest trends in hair style and clothing, and still look great. And frankly doing that would surely be more empowering than adapting to fashion trends because one friend said so, and a few girls turned you down.
Fashion doesn't have to mean the latest look, nor excessive effort. I used to think that I was completely unfashionable. I didn't care about how I looked until a few years ago when I started to compare myself to others and found myself lacking. But a few friends told me that they liked my style when I said I wanted to improve it. Casual but neat, and not too reserved or common. And I don't put in any more effort than ironing a shirt! (now that I actually wear shirts. Didn't for ages; why bother with ironing when I don't have to?
) These are girls who always dress well, and one who works in fashion, so I take their word for it and don't worry about how I look.
I like Sheryl's advice; and I think it means that it may not be necessary to change your look at all. And from what I've heard and experienced, as long as you're well groomed and 'neat', women won't care if your clothes are designer labels or not. (though they can speak for themselves if their opinion differs). On the other hand, beauty is in the eye of the beholder; it could turn out that the kind of woman you want to attract likes the way you look now, but the kind of women you've gone after haven't been the right ones.
However, if you really do want to improve your image, sure, go for it. As Elaine suggested, take a lady friend shopping. Or go with a few different friends on different occasions and get some varied opinions. Try a few different styles and find out what works for you. And be prepared for some discomfort initially, and a lot of uncertainty, but I'm sure you'll grow to appreciate your new look, whatever it is, as long as it's what you truly want.
btw I've been complimented a few times on the way my glasses make me look. So, as with everything else, there are exceptions to the glasses/contacts rule. But if you have got glasses get an opinion on how you look with or without them.