I deal with this as well.
It's either all in your head (like are you "guessing" that they're staring at you? Do you just "feel" that they're staring? Or is it so blatant that there's no question about it?)
If it's real, then there must be something different about your appearance, or it's the vibe you give off.