From my point of view it seems that you are worried about what MAY happen if you and the guy you love have sex. One thing I have learnt is that a lot of the times most of what we worry about never comes to pass, and if it does come to pass, then it is really isnt as bad as we thought it would be.
You have stated repeatedly that you are afraid of what will happen if you have sex. Well, if you can for a moment, try to think of what will happen and how you will feel if you do NOT have sex with the guy. Think of not just the physical pleasure you will feel, but also of the intimate connection that the two of you will share, even for just a moment.
I will give you a little story: A year and a half ago I was in a relationship with a girl who, at the time, was my perfect match. At the same time she was terrified of me leaving her, actually going so far as to break up with me first to prevent being hurt. At that time I was planning on moving to a different part of the country to begin a new life so in fact I was leaving her. In the end I had a long talk with her and one of the things I said was, "You and I both know I am moving far away. Instead of worrying about what will happen in the future, why not enjoy the time we can spend in the present. That way you will have fond memories after I do leave." She did eventually see my point and we ended up having fun with each other for a couple of more months before we broke up for reasons other than what I had mentioned.
As for feeling past-your-prime? Don't even think about things like that. In fact you have the enviable position of dating guys almost half your age but are in their sexual prime. In fact ask any guy in their twenties what their number two sexual fantasy is, and most of them will say it is sex with someone in their 30s or 40s (I can tell you it sure was one of MY fantasies).