DFT of the same signal with different values of N
$$X_4[0] = x[0] + x[1]$$ 
$$X_4[2] = x[0] - x[1]$$

Now $$X_2[0] = x[0] + x[1]$$ 
$$X_2[1] = x[0] - x[1]$$

Therefore $$X_2[0] = X_4[0]$$
 $$X_2[1] = X_4[2]$$

An alternative explanation is the following, if we were to decimate in the frequency domain by 2 (the 4 point FFT), you would incur circularly shifted Aliasing in discrete time domain. Since we started with a two point sequence then zero padded it to get a 4 point sequence, this decimation will result in Aliasing at those zero stuffed places leaving your original sequence non aliased. So say if we were to decimate more than 2 the sequence would incur Aliasing in time domain. So the fact that we are still able to have the original 2 point sequence non aliased after the decimation, therefore the FFT values of this 4 point FFT still carries the information about the 2 point FFT

Ex: if say we would have a  4 point sequence and you zero pad to get a 16 point sequence and take the FFT, now if you decimate the FFT by a factor of 2 you would get Aliasing in samples 8 to 15 of the time domain sequence.