Resistor noise equation gives infinite noise for an open circuit?
The only way to develop any noise power from this infinite noise voltage source is to draw current from it. 

In other words, to connect it in parallel with a finite load resistance R.

And infinite resistance in parallel wth R is just ... R.

So the voltage noise produced by the overall network is just that of the load resistance R. 

Which happens any time you leave an amplifier input open-circuit (without an antenna wire).

Measuring (or listening to) the noise output of a high gain low noise amplifier with the input (a) open circuit, (b) terminated by the usual source impedance, (c) short circuited will show decreasing noise levels.