He probly still wants to sleep in your bed for the feeling of security. Nothing wrong with that.
My two boys are grown now, and I raised them alone since they were young. If they wanted to sleep in my bed, generally they were welcome. I wasn't the "family bed" sort, but I understood that there was value in allowing that closeness. Especially when they were sick. But if I needed my space, I'd just walk them back to their bed and tuck them in. Because it wasn't a forbidden thing in general, they rarely fought it. They're wonderfully well-adjusted now, and I sense it did much good rather than harm.
You might want to try something along the lines of "Here's your big-boy bed, your own special place. But if you need to, you can still come into daddy's bed." That way it's not forbidden. And if he comes to your bed in the middle of the night, you can walk him back to his bed if you need to. "Daddy's gonna sleep alone tonight, but I'm right here if you need me."
And my suggestion...don't stress about it at all. He'll grow up and want his own space soon enuf.