What would the units of thermal noise be?
Hertz is 1/second. So N in your formula is Watts. The formula is not for noise voltage figure but for noise power. Power is watts, so everything computes OK.

To fully answer your question, noise is measured either in absolute units or as a ratio. Relative unitless ratio is just a number. For each untiless number scale there is always some absolute value in physical units which is agreed upon. Say if reference number is milliwatt, then microwatt of noise is 0.001 unitless on milliwatt scale. Or logarithmically -30dBm or -60dBW, etc.

Side note: non zero thermal noise strictly speaking has no absolute maximum voltage amplitude for infinite period of observation. Every observed momentary value can always be exceeded in any of next observations. Saying that noise is 0.01V p-p (peak to peak) is oversimplification or misuse of terminology. Preferred quantification of noise is in any units but with note of the method of observation, like 0.5A rms (root mean square).