I don't think that a therapist would put this upsetting subject out there, unless they were trying to spur a discussion about something that they may have heard you say during a session. An 'accusation' of sexual abuse is a very strong charge and probably not what was intended.
I find maternal comfort in the older women at work and out with friends, the grandmas with the demeanor and wisdom for when I am in need of assistance (I'm a male). Nothing wrong with it. My own mom is gone now.
A good therapist tries to dig deep and get to the heart of the matter. You might try asking why they would bring up such a subject. Certainly if there is nothing there, it is time to move on with other discussions. Anything is game and is worth talking about if it affects you so profoundly.
Statistically, probably about half of a therapist's patients may have experienced some sexual experimenting or trauma during their childhood. It may be a reasonable question if the therapist heard something from you that might have roots in abuse (or not). There are many patients who may not be able to discuss it without great difficulty or have repressed it deep inside. It may have been written off as a bad dream once or a distortion of a young story-telling imagination.
Therapy appointments are expensive and time-bound. They try to pack a lot of counseling energy into a short period of time. Keep working towards your answers.
Best of luck to you.