Does the noise voltage formula apply even if the impedance is not matched?
Thermal noise voltage applies always. The series resistance of the voltage source is R.
The power depends on the impedance you connect.