Hi all, nodding in agreement here. This was actually one of the anxiety symptoms that effected me so much at work in the past. I worked in sales and business, and once my anxiety flared up it made life hell. Phone conversations were fine, but sitting in a room doing a one-on-one eye-to-eye conversation was almost impossible. I came very close to bolting out of closed door meetings on more than one occasion because of the same factors you mentioned.
I think that avoiding these kind of encounters only hurts, in the long run. It's safe - but then we are more likely to create problems when they do occur. Now that I'm in need of a new job, it's terrible because a one-one-one interview is so scary to think about, that I'm terrified of even applying.
Oddly enough, after I quit I moved to Central America for 6 months. I had to speak Spanish most of the time, and in that case I had no anxiety problems whatsoever when dealing with meetings and long conversations. I think it is because that is my 2nd language, and I was thinking on a more basic level: "The sky is clear today. The food is good. I like this coffee" - simple types of conversations that did not allow my secondary brain to start yapping in the background.