Your husband hasn't forgiven you. He is holding onto his pain and he is punishing you.
So what's your next right action? You could ask him what it would take for him to really forgive you; that is, to grant you the freedom to be exactly who you are and exactly who you are not, to take your word that you are committed to your relationship and to make a choice to trust you (that's all trust is - a choice), and to make a similar commitment to you and your relationship. I would ask him if he's willing to do that. Then you can make your choice: are you willing to do what he asks, and are you willing to hold him to his word?
If he's not willing or you're not willing, it's time to let each other go with love. If you're both willing, then invite him to some of your counseling sessions!
p.s... the fact that this woman wrote to you saying she wouldn't wreck your home pretty much indicates what she and your husband are up to, don't you think?