It's all in the intention. It doesn't really matter (from a spiritual/metaphysical standpoint) what you do, but what your intention is in doing it (serving God and the others, bringing Love into the world). I'm reminded of Nisargadatta Maharaj, who was a bidi (cigarette/smoke) shop owner in India and who became enlightened. While Mother Theresa was out running around saving India's poor and hungry, he just sat in his bidi shop for the rest of his life, lecturing spiritual seekers
Also remember, you personally are also part of the "others" you're trying to help

Following your passions is probably in the highest good of all.
Don't change anything because what others do or say, do it because it is a natural consequence of who you've become (or want to become). It should feel more like you've outgrown your previous career/goals, and that something else is more important to focus on now.
I guess what I want to say is, keep the main goal of raising consciousness in mind, but there are a million paths to do so. The outward form or shape of the path doesn't matter as much as the intention and dedication to your main goal, raising consciousness. You'll be helping people spiritually, though it may not seem like it in the physical realm, just as a consequence of your higher consciousness.