He sounds like a good man. And in the end, this may simply be a case where you found someone who really attracts and interests you. For what it is worth to you, this happened to me a long time ago, a woman who was my superior. But she was not married but did have a relationship at the time. We did spend some time together. She wound up going back to her old BF.

As for my relationship, well, I could fill pages. And that would not be fair to you. But I will say this. I went into my relationship believing that I had good intentions, and I did. But I was coming into it from an emotionally needy place. And I was not even aware of this until after the relationship was over and with a lot of help from this wonderful forum.
When we have unmet emotional needs, and some think it is actually emotional hunger, we are prone to choose bad partners, or make poor relationship decisions. And therefore we are bound to react to our partner’s behavior rather than respond. A healthy relationship, in my opinion, is where each party stays as an individual rather than as a "couple". It is a relationship of two, rather than one. If we look at it as "we" we can find ourselves submerging into an emotionally dependent place. We then ask, or rather demand (this can be sub conscious) that our partner meet our unmet emotional hunger, or to feel whole or complete. Sometimes we can be hoping our partners "fix" us in some way.
Now, it is true, we need others to meet some of our needs. But, we need to come at them open, vulnerable and emotionally honest. We need to ask them, rather than expect them, to meet our needs. See, many of us do not do this. Not really. We assume our partners know what we need. See, this is based on fear. That if we tell them what we need, they may so no. So, we stay silent and perhaps unfulfilled or some get angry or feel unloved, because their partners are not responding to their emotional cues. Because the partner simply does not know.
We can also be prone to "idealize" the love of someone, and create what is called a fantasy bond with another. We think we are in love or are happy with our partners. But we cling to the idea of love, rather than judging correctly if our partners are right for us. According to experts in this field, many marriages and long term relationships are fantasy bonds, or the illusion of a relationship/marriage and not an authentic loving and intimate relationship.
In my story, I became involved with someone who had a Personality Disorder, with tremendous intimacy fears. She initially idealized me, and then I idealized her. But her fears and heavily defended personality (we all have defenses) led us in the classic, but ultimately unhealthy romantic dance of pursue and distance. I pushed for emotional closeness (my hunger) and she pushed it away (her intimacy fears). But because I was in a fantasy bond, I stayed.
In the end, I needed to assess why I was so filled with emotional hunger and what I needed to fix to be in a loving relationship rather than a fantasy bond.