Circuit Analysis of Op-Amp Offset Voltage Tester
Your calculation is correct - and as you say the exact gain is 1001.

The capacitor is to provide a high gain at DC for the second stage but a lower gain at higher frequency to keep the overall circuit stable.

If the capacitor was not present the gain of the second stage would be 1/10 so any output voltage would mean that there was a large non-zero output voltage from the first stage - in that case the offset voltage would not be nulled completely resulting in an error.

If the 1K resistor was not present the second stage would be an integrator and it is possible that the combined circuit again would be unstable, the resistor adds a zero in the feedback so that at medium frequencies the second stage has only a small phase shift.  Combined with the dominant pole in the first stage the combination is probably stable.