What you feel definately isn't silly, and it doesn't make you a baby at all for feeling that way. I'll be 16 next month and I feel like you do.
When I visit my dad, he'll take us out to eat a lot if his girlfriend is working and can't cook. And he makes us order, so instead of ordering what I really want, I'll just say "I'll have what he's having." so there isn't much speaking involved. I walk with my head down, I can't look someone in the eye at all. I get nervous and queasy and start to shake. And if I get stuck in a room or mall with people my age, the major panic attacks come and I can't breathe for what seems like ages. I got tickets for my favorite band, front row, right up there in front of the stage as an early 16th birthday present and I know I'll have anxiety and be miserable the whole time so I sold them... And I know that really hurt my mother, who worked very hard to get them for me, but I just can't handle it.
My mom says to tell myself over and over to ignore it and realize that in reality, none of those people are even bothering to know what you're doing. They're busy doing what they're getting done and in reality, they don't really see me. Maybe that will help you. What helps me is to try and repeat all of that in my head and hang on to my brother's girlfriend (It helps to have someone you trust with you. At least in my case.) and try and focus on what we are doing and pretend the rest of the world is invisable.
You're not alone. If you ever want to talk, I'm here. I'm not a lot of help, but I'll do as much as I can.