I've had some experience with social anxiety around boyfriends, and the thing I've learned is that every time you hang out it gets a little easier. Yes, it will be awkward at first, and yes, you will be anxious, but you should do it anyway because it will improve!
You said he knows you have social anxiety, so he will be more understanding of why you are awkward/anxious.
Trust me, I've been there. The more time you spend with him, the less anxious you will be. It just takes time. Eventually you will get to a point where you are much more comfortable around him and you will think it was silly how anxious you were before.
Courage is fear holding on a minute longer. ~George Smith Patton
A neurotic is a man who builds a castle in the air. A psychotic is the man who lives in it. A psychiatrist is the man who collects the rent. ~Jerome Lawrence
The better I get to know men, the more I find myself loving dogs. ~Charles de Gaulle