Why is there a voltage between the mains ground and my radiator?
I measure 1VAC and 20mV DC between ground and my radiator.

I would consider that negligible. Very likely there is a connection somewhere between your radiator and ground. It might be a long path though, for example:

Radiator - pipes to central heating unit - cold water pipes (does your heater also provide warm tap water?) - cold water grounding connection.

In my house at the place where the cold water enters the house, the water pipe is grounded together with the mains ground.

This means your radiator probably is grounded only through a very long path. Due to the length of this path there are all kinds of opportunities for it to pick up a small voltage.

I recommend grounding your ESD mat through the mains earth. Do make sure however that there is a 1 Mohm (yes, 1 Mega-ohm) resistor between the ESD mat and the actual ground. This is needed for your safety.

Suppose you have one hand on the ESD mat and your other hand touches a mains live voltage, the current must somehow be limited for your safety. That's the function of that 1 Mohm resistor. If you bought  a "proper" ESD mat + ground connector, this 1 M resistor will be present already. But to be sure, I would measure it to confirm. For ESD that 1 Mohm resistor makes no difference at all, charges can still flow to ground through it.