I think it is more like the complete opposite. It doesn't have anything to do with attention for me at least. I care so much about what people think and if I am doing anything to hurt them that it makes things harder for me. I don't ask people if they want to hang out because I am worried that they would only say yes out of obligation. I worry that I wouldn't be a fun enough person to hang out with and think that they should hang out with better people.
Shouldn't you be happy then, that nobody hangs out with you? Since they are hanging out with people far more entertaining than yourself?
I don't understand. If you're so selfless, why bother with the people who already have friends to hang out with? Why not just help out at a soup kitchen and get even more fulfillment?
I don't want you to waste too long trying to answer those questions. My point is, be honest with yourself. You don't want to thanklessly enhance anyone's life you see as better than your own. You want that thanks, and that's the only reason you'd even consider the deed.
I'm not saying you wouldn't donate to charity or help people in need, because that is totally different than not-asking someone to hang out because you don't want to burden them. It's not that you don't want to burden them, it's that you don't want to feel like a burden... You want the opposite, and for your own sake.