Noise performance at cryogenic temperature
Resistors drop in noise proportional to the square root of temperature, so at 4.2K a resistor will have about 1/8 the noise as at 300K. 

Bipolar parts usually stop working entirely. 

Some GaAsFets apparently work okay, MOSFETs can work okay. 

The noise reduction will depend on the source of the noise. If it's white noise based on resistance it will decrease with sqrt(T). If it's shot noise it will depend on the barrier height. Flicker noise (1/f noise) may be a bit less predictable. 

Is it worth the cost? If you can't accomplish the goals any other way, then it may well be. It's possible to achieve measurements that are impossible any other way using superconducting quantum interference devices.