Electrostatic Discharge on new Workbench Setup
You are missing the point here. If you are unfamiliar with how things work regarding protecting static sensitive devices you should use an ESD mat that is usually connected to ground via a discharge resistor (usually around 100 kΩ to 1 MΩ).
The point here is that when you place sensitive parts on the mat they remain at an ESD potential close to ground. And when you sit down to do work with those parts you firstly discharge yourself to the same potential as the mat by touching the mat and/or remaining in contact with the mat via a wrist-strap and curly wire. You are then at liberty to touch and handle the components in the full knowledge that in doing so, you won't cause an ESD event to damage them.
But, even before placing the components on the mat you have to be aware about handling. In other words, the sensitive components must be in an antistatic bag so that when you pick them up (with or without the wrist-strap), you won't cause them damage.
You then place them on the mat and only open up those parts from the packaging once you are connected to the mat and are absolutely ready to use them. The target circuit board should also be at &quot;mat&quot; potential so that when you place the components on the circuit board you don't create another possible ESD event.
In other words, it doesn't matter what potential the mat is really at because everything you do is relative to the mat potential. However, ground is a nice convenient point to connect the mat to.
When you grasp what is happening, you can learn how to handle components without a mat or wrist-strap but, until you grasp that, do the basics first and then learn how to handle stuff without mats and wrist-straps.

My question is, how do I test this? How can I be positive that when I
touch this lamp I'm connected to ground

Testing whether your lamp is earthed or not is largely irrelevant if you don't understand the basics and don't follow correct procedures. If you don't want to use a mat or wrist-strap, you better get to grips with how you handle components without them i.e. once you open the protective component package, you never release those components; you keep 'em in your sweaty palm/fingers and anything that you want to apply them to (like a PCB) you grasp with your spare hand to equalize potentials via your body.