In measuring relay contact resistance, should I use 4 wire measurement or calculate it using the voltage across the relay and current?
In the image below, from the document you mention, if you connect a voltmeter close to the contact you are measuring, you are performing a 4-wire measurement. The high current does not circulate through the voltmeter cables.

If you don't want to do this with this crude setup (variable resistor) you can improve it by implementing a current source (which is quite simple) or use a bench power supply with this feature.

By using any of these 2 options with current sources you can use your very precise bench meter to measure both the current and voltage.