How to reconstruct an image from s-parameters in Matlab?
Each complex value can be described as $$A \cdot e^{i\cdot \phi}  $$. 
Where A is the magnitude and $\phi$ the angle.
With this knowlege you can use the ifft as you allready mentioned. 
If you consider that the image is a real signal you can take the real part by using real(). 

So overall it the part after reading the magnitude and angle would look like 

IMG = uint8(real(ifft2(mag .* exp(i * angle)))); 


Where IMG is the resulting image, mag the array of magnitudes (in linear scaling), angle the array of angles.

The only problem with that is, that you have to know the dimensions of the image. The spectrum of an image has (after FFT) the same dimensions as the picture (113 x 200 pixel picture -> 113 x 200 matrix after FFT).
So your mag and angle must have the dimension of the resulting picture (e.g. 113 x 200).