oh yes, PNOTGs are everywhere!!
since I 'officially' started looking deeper and deeper into myself and my concept of reality, or started on this path or however you would like to call it, I have met such people everywhere.
even people who before I thought as quite enlightened, I began to realize even they were stuck in some old unrecognized patterns and limiting beliefs.
and I too had your reaction. I felt that they drain me, and I unconsciously resented them for it. I felt that they held me back. I pulled back in my interactions and focused on introspection.
and you know what happened? first I realized I was being resentful. that got me off my newlyfound high horse. and it got me to realize how I was doing the same unconscious-living trick again, I was pointing the finger at them for holding me back, when really it was all me not developing my inner power to think and act how I chose to think and act.
there was also an element of control there. I realized that the only person able to drag me down is me. the only person holding me back is me. and I chose to allow them to influence me because I had my own inner doubts about the path I was taking and because my gremlins were desperately acting out to preserve themselves, masking giving away my power and not being personally responsible for my life in the newfound shroud of personal development.
and that as much as I want them to let me be the way I am, I should also grant them the freedom of being who they are and making their own choices in life. I have the right to my own choice no matter what anyone thinks, but then so do they no matter what I think. and no matter how wiser and enlightened and spiritual and personally developed I am. seriously, I'm laughing hard at myself remembering that now, so thank you so much for a good laugh!
anyways, what I meant to say is that what you're feeling is perfectly normal. and that those people, no matter how much they annoy you, are PRICELESS for your personal growth. they were for me. I still meet them, and they still annoy me, and when that happens I take my pretty high horsie for a walk and then when I'm done I have new things to learn about myself and my perception of reality. sometimes the horsie doesn't even get his walk before this happens!

now that's progress
these days, I don't feel the need to eliminate them from my life, especially family, and most times I don't even chose to give away my own power to them. and somehow, along the way, some of it got rubbed off on the people I thought would be the last people on earth to ever change (my family). some connections even got closer! funny that, huh?
and there are those that it didn't rub off on, that are perfectly fine with me now. and my connection to them is different, sometimes more distant, but always more loving, because even if we drift apart I find myself loving them for who they are much more and I find myself loving myself for keeping the relationships at a level that suits me and doing the loving thing for myself.
I babbled a lot here, but basically, if I had to say one thing, it would be that those people that you think are most awful for you are actually the best opportunity for you to learn about yourself and develop. personally.