It's super common and normal to feel anxious about going to a gym. I remember being a newbie and wanting to lift but feeling like I didn't belong in the weight area, that people were going to gawk at me, and that I'd look like a dumbass. (Reality check: I did belong, mostly nobody gawked, and probably sometimes I did look like a dumbass, but nobody died, so it's really fine.)
Here's a few ways I've coached my clients to feel more comfortable going into the weight area.
First, having a plan helps. Even if you just write on a piece of paper that you're going to do bicep curls, goblet squats, and an overhead press, then when you walk in there, you have an idea what you're going to do so you don't just wander about feeling awkward and self-conscious. (Side note: if you don't have a plan and want one, I'd love to talk more with you about that!)
Another strategy I suggest to reduce your gym anxiety is to look up the equipment you want to use on YouTube before you go. It helps if you know the specific model. You can find instructions for how to set up the equipment to your height/size, and details on how it works. This applies to everything from the simplest stuff (dumbbells, I guess?) to the most complicated pieces of machinery in a gym. It feels better to walk in with some idea of how to use the stuff there!
Most gyms also will offer some sort of tour or orientation. It might be free, or it might be free but kind of a sales pitch for personal training, but it can be a good opportunity to get a feel for what equipment is located where, and maybe how to use some of the equipment that might be unfamiliar.
Finally, I like to remind folks that you are paying the same membership as anyone else and have every bit as much right to be there, to take up space, to use equipment, as anyone else in that gym. You deserve to be there, and you are just as worthy of the equipment and space as anyone else. Everyone in there was new at some point. Some folks forget that, and that sucks, but everyone was a first-timer one time.
I hope these tips help you feel more comfortable going into the gym, especially the weight room. When I was newer to the gym, I would always gravitate to the elliptical or the bikes because it seemed like I couldn't embarrass myself there. Lifting at home is ALWAYS an option, and can be a really great option for a lot of people, but if you want to get into the gym, you belong there!