Based on the poll above (with 50% in the secularist/atheist/agnostic category), I wonder how much more that puts avoidants at odds with a general population that's mostly religious, at least in America. Human interaction is hard enough - trying to converse with those who actively believe in a "higher power," whether it's family, friends, or strangers, only makes life that much more complicated for an avoidant.
Only recently I've come out as an atheist to my parents (who are dyed in the wool Lutherans), and while they are a tolerant and accepting lot, it has only served to strain our relationship. I can only imagine how it would affect a potential friendship or romantic relationship, just to compound the other problems of AvPD.
If the other person is religious (particularly Christian), they may be able to accept your unbelief, but there's always that nagging problem that the two of you won't be in the same place in the afterlife. It automatically creates a divide, and opens up a new possibility of interpersonal judgment that I'd rather not deal with ("Atheist?! They must not have moral values, or believe in hope, or be happy, etc etc."). No amount of rational argument after disclosure of one's (dis)beliefs will do much to bring back the relationship as it existed previously, or so I've found.
There are more tolerant religious individuals who may not be so quick to judge (or do their best to refrain from judging), but their religion still condemns me in principle. When they believe in something that does that, I have to question them. Once again, another needless complication that humanity has not managed to outgrow from its infancy.