Don't think of a career change. You can do it!
I'm not a prof, but a teaching assistant. I've lectured to college class from 11 students to over 200 students.
I gave four lectures last week. What I noticed was that I did *so* much better when I "stepped out of the formality of the situation" so to speak - meaning, I let my personality come out. I don't have a very formal personality and lectures are a very formal situation, which always used to bother me. I noticed that I spoke better and got a better response when I deviated from my outline and made a sarcastic joke or tried to engage the class in the lecture. (Example: My lecture was on the Scientific Revolution and I did my best to get rid of any confusing math or physics concepts, since it's a history class. At one point, I stopped and asked the class if there were any science people in the class. A few nodded, and I apologized to them and said, "Sorry, this is what happens when social scientists get a hold of your ideas." And the whole class - about 80 people - laughed.)
Regarding authority, I was surprised, but because of the professor/student arrangement, it just comes naturally. They asked questions and seemed satisfied when I answered them. They asked me stuff about readings that my prof was supposed to post online, but had forgotten before he went out of town, etc. I think that if you have a problem in a class where you have to act as an authority, you'll be surprised, because oftentimes the student may protest at first, but will back down if you're just persistent.
Anyway, just my perspective. HTH.
Have a nice day,