Apply the Discrete Fourier Transform as a Matrix Multiplication in MATLAB
Pay attention that by default MATLAB use DCT Type II hence the inverse is basically DCT Type III:
vX = [1 + 1i; 1 - 1i; -1 + 1i; 1 + 1i];    %Assume N = 4
vY = dct(vX); 
mD = dctmtx(length(vX));
vYY = mD * vX;

vYY ./ vY
max(abs(vY - vYY))

vY = idct(vX);
vYY = mD.' * vX;

vYY ./ vY
max(abs(vY - vYY))

The result:
ans =

   1.0000 + 0.0000i
   1.0000 + 0.0000i
   1.0000 - 0.0000i
   1.0000 + 0.0000i


ans =

   4.9651e-16


ans =

   1.0000 - 0.0000i
   1.0000 - 0.0000i
   1.0000 + 0.0000i
   1.0000 + 0.0000i


ans =

   4.5776e-16

So everything works as expected.
Pay attention that in your code using the transform you multiply by factor sqrt(4) and for the matrix form you divide by this factor.
So:
vY = sqrt(4) * idct(vX);
vYY = (mD' / sqrt(4)) * vX;

vY ./ vYY

ans =

   4.0000 + 0.0000i
   4.0000 + 0.0000i
   4.0000 + 0.0000i
   4.0000 - 0.0000i


So just don't divide and use the same factor and all is good.