It's easy to distinguish between obsessions and delusions.
Just like some people fear they are having hallucinations because they hear a lot of "voices" in their heads, an internal voice is just a thought. A hallucination is something you see with your eyes or hear with your ears, an external voice.
Likewise, a delusion is something (unusual) you believe with your intellectual part of the brain. If you're asking the question "am I having delusions" you are most certainly not. And yes, OCD can provoke the weirdest thoughts. The relevant question is if you are able to intellectually recognize your thoughts as weird or unreasonable.
A minority of OCD sufferers are so confused with their obsessions that they are not able to understand the irrationality. This is known as "OCD with poor insight" but still, it's not delusions. People with delusions are 100% certain (uncorrectable) that their thoughts are telling the truth, no matter if the best doctor in the world told them otherwise.