Don't worry about it, you're not being rude. Perhaps I misunderstood what you were asking.
How can you stop thinking about this guy? Hmmm...
Maybe there's nothing wrong with thinking about him after all. But then I can understand your hesitation because of his personal situation.