I end up arguing with therapists because they make assumptions about me that are just not true. They think I'm being defensive or that they've gotten to the root of my problem, when in fact, it's just not even close to how I think or feel. I don't know if it makes sense or not.
It makes perfect sense to me. Therapy only works through the relationship with the therapist - if you have trouble with relationships in the rest of your life, how is it ever going to work with a therapist.
I am in a similar situation. I have never had what i would consider 'real' friends, and I never seem to connect with people generally - including my parents. I have been with a therapist for about 7 years now and it is hard going but my instincts tell me that this is the only way I will ever resolve anything. I don't kid myself into thinking I will ever be 'cured', but I am certain it is possible to live a life relatively free from anxiety.
I know everyone is different and I don't necessarily think your situation is the same as mine, but I do absolutely know what truly being isolated and alone feels like - and I would not recommend it to my worst enemy. I also know the paradox that feeling alone and disconnected is not something you can deal with alone. Its not an easy situation to be in by any stretch of the imagination.
Other people annoying you, a family that didn't seem to notice you, a father that you have no relationship with, the feeling that very little about your life is of your own choosing etc. all needs to be examined and talked about in detail. It needs to be brought out in the open and the emotional core of it needs to be experienced through to its natural conclusion - as hard as that may be - its the only way forward. Anything else is an illusion. At the very least start making a habit of writing about it in a journal and stick at it for the long term.
I still recommend therapy - Its expensive but I carry on with it because I honestly feel I cant afford not to. Its just a trick to find the right person. Also remember that arguing with them is important if you feel you need to - just make sure you can also discuss why you think the therapist doesn't understand and keep at it, and keep at it until you break through.
I hope you find your way.
All the best.