Yes. I remember being very angry, and then someone who couldn't tell what state I was in tried to have a conversation with me about something related to it. I couldn't talk and felt anxious that they were speaking to me, so I hurried away.
If someone starts talking to me while I am walking, I generally keep walking and just kind of call a few things out before turning away instead of stopping to have a conversation. It's not necessarily because I wouldn't like to talk to the person, but more because I'm not sure how much of a conversation they want to have or whether I would be able to keep it going for an appropriate length of time and then make a non-awkward exit, in addition to the normal anxiety about having to talk to someone at all.
In anger, yes, but never because of SA.
I find it rude, even if I really want to, and I can't bring myself to do it. I would feel absolutely crushed if someone did that to me and I try to think of their feelings.
I usually wait for a good break in the conversation and then say I have to go or I just remembered something that I need to do. A few times, with guys I was interested in and they might have been interested in me, I just started acting bored and they took the hint and stopped talking and I always feel so bad about that because I definitely wanted to talk, but my SA was just screaming at me to 'get away'.