The same thing can be said about any trait, really. People talk about confidence, self-love/acceptance, intelligence, "social skills", money, etc., the same way, but none of them guarantees any kind of social success.
I haven't been both attractive and unattractive, but I experience something analogous in that, as a man, I am considered merely plain, but as a woman, I am unequivocally very unattractive. Subjective preferences, at best, make you slightly more or less attractive to specific people.
I think it's safe to say that, all other things being equal, being attractive does provide significant advantages socially (eg. getting a job as a model; whereas people don't even want someone like me on a sales floor), but all other things are never equal, so there are always going to be exceptions.
Every trait has upsides and downsides. We know which traits have more upsides than downsides because those are the traits almost everyone wants more of (like being attractive). Most people have some direct experience of those advantages because: 1) they have "good" and "bad" days and can contrast their experiences on those days, and 2) they know what kind of traits they prefer in their partners. It seems very odd (and probably logically inconsistent) to argue that being attractive confers no advantage if you want an attractive partner yourself.
The best things in life are free, but so are the worst things. And you need money to avoid those.