I like the repeating "No, Thank You" over and over again. If they keep goading or talking, look them right in the eye and say it again. Eventually they give up. Or you can walk away if you have the chance.
You could also examine the view of why this bothers you so much that he said you did not have a kind heart. Is it because you secretly agree with him? If you truly 100% believed you did have a kind heart, it would not bother you so much.
I'll tell you a secret, giving money to others may not be an indication of a kind heart. Some people give money out of guilt, fear, or to pay the other person to go away. So the act in an of itself is not an indication of kindness.