Sanity Panda, to me you come across as a very kind person. So in my opinion the problem is not a flaw or a lack of heart on your side. I agree with Cloud: I too think it's probably about being poorly connected to yourself.
How well are you connected with your body? Do you live completely in your head, or can you feel your inner body? That's the energy field you feel in your body from
inside. How much of your awareness resides in your body usually?
And something else, how well do you feel what your body needs, in terms of food, sleep, exercise...? Some people are very well aware of their body's needs whereas others have a hard time figuring out what they want to eat or when they are tired or why they feel bad physically.
How well are you connected with your emotions? Do you feel them strongly or not? When you feel an emotion, can you describe in concrete words how it feels, or is it more like an idea? Can you tell where in or around your body it manifests? Can you deal with them without suppressing them?
Are you aware of your feelings? Can you describe them? Do you notice them immediately when they appear, or only later? Do you tend to push them away or deny them, or do you acknowledge them?
Do you know what you want? Are you aware of your desires?
In case you lack connection to your body, emotions etc, it could be worthwhile to practice increasing your awareness in those areas. I know maybe this feels completely off-topic now, but I bet it's not. Get in touch with yourself, you'll get in touch with others
There's also the energy level: do you cleanse your chakras? Maybe some of your chakras are clogged, like heart or throat. Erin wrote
a post about what chakras are, how to cleanse them and what the benefits are.
And for the mindset/beliefs level, Steve wrote an interesting post about
Soulful Relationships, I think it could help you a lot.
Also, reading
The Power of Now by Eckhart Tolle might help if you haven't read it yet. And
The Work of Byron Katie could be applied in this case too.
Hope some of this helps in some way!
Much Love to you

Keep us posted, let us know how you're doing!