Partner talking to self with intrusive thoughts?

Hi,
My partner has been through a lot in his life, but for the past two years has developed something he says he can't control - he has intrusive sad memories and keeps saying things in a sad tone like "Please leave me alone", "I don't know", sometimes repeatedly. He doesn't seem to do it in front of friends, just when he's relaxed alone with me, or I hear him saying it when alone e.g. in the bathroom. I ask if he is okay, and he gets cross as I'm drawing his attention to unwanted thoughts. He'd rather I just comfort him with an "I love you", which I remember to do most of the time, but for example, just now he got angry because tired, I forgot what we had agreed, and asked if he is okay. When he got angry I explained I forgot the agreement and was just checking he is okay. I said I only asked because I care, and reminded him when I had issues I got help - so I didn't have to worry him, as it really worries me when he keeps saying to himself "please leave me alone", telling sad memories to go away. It upsets me a lot to hear, saddens me, makes me want to help. I've had an extremely difficult life, too, but I don't remind him every few hours, life is sad enough, I try to be present and focus on enjoying life now. I know life will never be perfect, but it would be good to just be happy once in a while. I think we're happy and laughing and relaxed for the evening, and then he says very sadly "please leave me alone" and I feel so down. I was abused by my family and need happiness. I've asked if he can keep his thoughts inside, he gets angry and says he can't control it. For the first 4 years of our relationship he didn't do this, only the last 2 years. What is it, and what should I do? Any thoughts gratefully received. Thank you.