Some people are very good at listening to their intuition about others. MY SIL without really knowing why never felt good around my partner, (a diagnosed NPD) although she's not met him much, she knows instinctively that she doesn't like him. This sounds like you.
In a lot of literatire about Narcissists I've read that people who meet one often walk away feeling either angry or hurt. This can be at a social gathering where the N played centre stage at their table perhaps, or a work colleague they have to deal with regularly, or someone who they attempted to start a relationship with.
I remember a lot of things my old boss used to do and can now see he had Narcissistic tendencies, I never liked him then, now I know why.
Some of us are very good at listening to our inner voice and acting upon it, while others suppress it, often to our own detriment. I believe that those who aren't afraid to let the narcissists they meet that they don't particularly like them, the N's can detect this and realising that you won't pamper to their needs will reject you. I wouldn't be insulted with this, rather the opposite!
If you believe that you can detect narcissists or narcissistic behaviour, why do you think it's a problem? It sounds more like a gift....