I just wrote a very long reply to this topic, but when I tried to submit it, the forum redirected me to the login page (altough I was already logged in!!! wtf?; I could't have written my post if I wasn't logged in in the first place!). So I lost everything I wrote
But anyways, to make (originally very) long post short.
There are people who are considered attractive (by majority).
And there are people who are considered unattractive (by majority).
And then there are people who are genuinly considered attractive by a notably large number of people and genuinly unattractive by a notably large number of people.
I belong to the third group. We are rare.
People in the third group create cognitive dissonance in other people. It's a cognitive experience of unease, surprise, disappointment, confusion and most importantly people feel a strong urge to solve the contradiction. In this case the contradiction is: "is that person unattractive or attractive?"
People want their everyday life to be simple and easy, so they can feel anxiety, anger and resentment towards an individual who causes cognitive dissonance for them.
There are people who are more sensitive to cognitive dissonance and hence react more emotionally or abruptly than others, but
nobody is immune to it.
There are people who even feel decepted when someone they first thought was attractive (because of, say, slim figure, shiny hair and high cheekbones) looks unattractive to them the next minute!
Like I said, I belong to the small third group myself, and I've experienced some strange things from complete strangers because of it. For example once a stranger came to me and made a gesture of wanting to shake hands with me, and when I responded to it, he all of a sudden pulled his hand away. And then he said: "That's how you made me feel" and then walked away. Only later did I realise what he meant.