I suspect a lot of the difficulty with specific comes from internal conflict - in my past, for example, I would often try to envision being in love with a certain girl and desperately wanting her approval. Rather unsurprisingly, I was extremely successful in manifesting unrequited love with that particular girl as my ever-loving universe gave me just what I wanted: desperately seeking approval while being in love!
Another qualm is that there's a fundamental sense of 'this isn't right!' which, of course, is easier for us darkworkers. Anything that makes us happy is right, so that particular block is avoided.
Its so much easier with new people since we work on a cleaner emotional slate, and thus have fewer opportunities to sabotage ourselves.
Manifesting a specific person just takes a little bit more strength and discipline. As with all things, in the end, you're most bond by the limitations that you set upon your own mind.