Cheating on her with anyone sucks. I see two options. I'm not a therapist, doctor, or qualified in any way, this is all my own thought.
One, you could tell your girlfriend that you'd like to have a non-monogamous relationship. This would mean she gets to have sex with other people too, if she wants. You wouldn't need to be upfront about who you want to have sex with though it might help her decision-making to say that it's a man and not a woman. For some people, that feels like less risk of "emotional replacement", as whatever else happens, she can never be a man. But she might alternatively think that it's more of a "health risk". It would be her choice.
Two, you could be monogamous with her and not have sex with your brother, or anyone else, again. It's fine to be in love with people without making it sexual. You might then need to stop sharing a bed if you needed to decrease that temptation or take further steps if you needed to.
Whichever option you pick, it'd be good to talk to your brother about what you're doing and why. He has the same choice to make with his girlfriend too. You'll need to support each other with whatever decision you all make.
“We think too much and feel too little.
More than machinery, we need humanity.
More than cleverness, we need kindness and gentleness.”
Charlie Chaplain in The Great Dictator