Sometimes I find that looking at artistic or fictional depictions of guilt and remorse can help give me a little distance from my own emotions and allow me to consider them more objectively / philosophically. Also, there is some comfort in knowing that you're never alone, even if you feel that you deserve to be.
Anyway, I really like this comic by Junji Ito in which those with a criminal conscience become literally frozen in place. It's a good metaphor for how guilt stops someone from moving forwards and how, if someone has a conscience, even if they escape punishment, they'll always find a way to punish themselves. It's called 'The Earthbound' and it should have a *trigger warning* for mention of rape and murder and other crimes, though nothing explicit is depicted. It is unsettling though, but very clever IMO.
http://openawesome.com/junji-ito-horror-manga/theearthbound/2.html
Also, I think maybe everyone will find something to take away from it, as those who are remorseful will relate to the characters of the earthbound, while those who have only been harmed by others, but never harmed themselves, will be able to relate to the central protagonist and maybe take some comfort or pleasure from the justice that exists within the story.