Noise Pick up in High Impedance nodes
May I know how the high gain circuit is capturing the 50 Hz line
frequency.

All around you are electricity wires in your walls and in your equipment. These produce an alternating electric field in the room where you are testing your amplifier. You don't feel it because the connection with your body is via capacitance from the building wiring and, that capacitance is very low (usually much less than 10 pF typically) but, nevertheless it is there.
So, if your amplifier input has a high input impedance (unlike the human body), an equivalent single digit pF of capacitance between the amplifier input and the room wiring is usually enough to generate a few microvolts of AC voltage at the amplifier input.
That gets amplified by 1000 and you see several millivolts of signal on the output.
You don't necessarily see that voltage on your o-scope when the gain is only 51 because the 50 Hz amplitude signal is hidden in the noise caused by your o-scope input.

Is it a radiated noise or conducted noise

Well, first of all, it's not noise because it is something you can see as a sine wave on your o-scope. However, it can be regarded as being interference. Probably that's a better phrase to use.
Secondly, it is loosely termed &quot;radiated&quot; in that it is not conductive but, the preferred mechanism is &quot;capacitively coupled&quot;.
Of course, if your input wires are long and unbalanced, you could be picking up magnetic induced signals due to the currents flowing in your room wires but, more likely, it's an electric/capacitive coupling.