Uncontrolable Wandering Eye?

I’m 28, and I’ve been in a relationship with my boyfriend for just over a year and a half, and we have lived together for most of it. We run a new business together, and have a great emotional, intellectual, and friendship connection. I am really devoted to this person, love him deeply, and if things were a bit less rocky, would love to marry him. We often say that if the issue I'm writing in for is the only thing that keeps us from this sort of commitment.

He says that I have a wandering eye. He says that I wait until his back is turned, and then visually follow men around with my eyes. He doesn't believe that this behaiviour is unconscious because it seems calculated. He says that it is quite pronounced, and we fight about this almost every week. I don’t think that he is lying to me about my behaviour, but I am completely unaware that I do this. As in completely unaware. I do know that I often feel attracted or “pulled in” by many different sorts of people, which is why I can even believe him at all. But as much as he insists that I am doing this, I am completely lost as to when this is going on. If I try, I can sort of reconstruct things I might have done, or must have done based upon what he says, but I would never be at all aware of doing anything of the sort if he didn’t become upset.

I have tried to stop looking at all people, at all men, at all bodies in space, of course this has failed. We don’t go out nearly as much any more, and our social life is suffering. He says that I manage to make myself the target for sexual attention everywhere we go. But I don’t dress provocatively (on a normal day I am actually painfully plain), and don’t engage in any behavior that I am actually aware of that would cause this to happen. Yet he says that whatever energy I am putting out is undeniably sexual. I don’t know what to do.

I have never had any intention of cheating on him (and haven't been a cheater in past relationships), and don't want to be with other people. I don’t approach or talk to any men, don’t email or text men I used to know, and don’t fantasize about other people. I don’t feel unfulfilled in my relationship, except that the fights this causes have been going on for a long time and they are really taking their toll. Like I said, I feel deeply committed to this person, but whatever I am doing is really hurting him. And I just simply don’t know when I’m doing it.

Any Advice?
Thanks....