Yup, I can definately identify with this. My close family know what I'm truly like, but nearly everybody else is only of the opinion that I have slightly unconventional views. They are unaware of the true extent of my habits (journal writing, meditation, IM etc).
My advice to anyone out there with this problem there would be trying to mix a bit more with people who have similar interests to you, then you wont be so fearful of people you know rejecting and leaving you alone when you truly express yourself.
Not sure of the best places to meet people like us though - apart from this forum of course!