What are these bathroom mirror lamp holders called?
So, looking at the photos, what part is loose? The lamp socket, or the lamp base? Is the bulb loose in the screw base or the whole socket loose in the mirror? Here are some suggestions:

1) Bulb loose in socket (Lamp socket), breaker off, with a screw driver, gently pry up on the middle tab to bring it up into the socket further.

2) Lamp base loose in the mirror. You can remove the lamp base, and adjust the clamp on it, unless the clamp is broken. The clamp is the part that looks like a 2-prong fork on each side of the socket base (your last 2 photos.) Make sure that the prongs are on the mirror part and not falling into the hole. Sometimes those metal prongs become brittle and break. I had hallway lamps that used a metal "spring" that held the lamp shade on. After taking them down to clean them and do some painting, one of the "springs" broke and I was forced to replace the lamp or DIY a solution. The solution was to drill holes in the metal lamp base and use set screws. This doesn't look possible here, but you might be able to find suitable metal to create new clamps or build up the mirror surrounding the clamps.

If you can't do either, and the lamps are flush, you might have a hard time trying to find replacement sockets, and you might need to make a DIY solution that will work. You may need to convert to a keyless socket like this: https://www.1000bulbs.com/product/137319/SOCK-45389099.html?gclid=Cj0KCQiA5NPjBRDDARIsAM9X1GJrCuKLGaqEfPjjn9EjRkCRqzLa7R1FIDpdcu-ViyEzrbYVU28fnjYaAuyQEALw_wcB

This is just a sample and not intended to be the only solution, brand distributor or manufacturer.

A keyless socket would have either a fixed ring and adjustable ring, or 2 adjustable rings, that would allow you to adjust the depth of the socket.