I struggled with social anxiety on and off over the years, it comes up in certain circumstances as a response to trauma.
I organized a house party around 2010. I noticed 30 people went to one guy's house to watch a world cup game one Sunday morning. So for the next Sunday morning, I invited people, the same people. Not one person came. The same had happened for the Battlestar Galactica series finale in 2009, I invited people over, not one person came. This was in an environment of mostly nerds.
I moved in 2012, and from 2012 to most of 2015 I did not host a single dinner party or birthday party. I actually had bought nice furniture with the plan of having people come over, and I had become a good cook and baker, but I never invited people over. I never organized a birthday party. I was afraid I'd be there and nobody else would show up. My anxiety was well justified because as stated before, nobody had shown up to parties I had organized previously.
It's silly though, to have learned to cook, to own this "fancy" furniture, and to never have people over. I finally invited one set of friends over in late 2015 (I think), and another in early 2016. Both times there were specific occasions acting as enablers -- international people were in from out of town. I hosted dinner parties in their honour and it was a f*****g relief that people came.
It paid off in that I ended up making good friends this way. I met a friend of mine, his girlfriend, their friends at a restaurant one night after yoga. We all came to my house after the restaurant to eat dessert, they were all very impressed with me and then we ended up bonding over time and becoming close.
I think a big difference is partly that my social status had improved. I was seen as subhuman in 2009/2010 in part because i was not respected at work. If you're not good looking, and you're not good at your job, you not only have no value to society, but you have negative value. In contrast in the different job I held a few years later I was well-respected professionally, and thus people at least liked me as a friend. I could get along with people as human beings. That doesn't compensate for an absence of sex, but it's essential regardless for psychological well-being.
tldr --
1) If you're not attractive, you really have to be good at your job;
2) If you're in a position to face your fears on your own terms, do so;
