Does a thermocouple need to be screwed in to work?
To be clear for future searchers: the nut is critical, it provides half of the electrical connection.

A thermocouple requires two wires to function: electricity is generated at the junction of two different metals and two wires are needed for current to flow.  While some thermocouples have two wires, the original poster is talking about a model with one wire.  The case of the thermocouple must firmly contact the frame of the furnace, to complete the circuit.  Consider buffing the furnace frame with steel wool to remove rust and scale.