why should I scale the fft using 1/N?
OK, let us go for a 2-point DFT. It should be noted that, depending on the software used, scaling can be different, and ought to be checked. 
The standard unscaled version does multiply the input vector by:

$$\left[\begin{matrix} 1 &amp; 1\\1 &amp; -1\end{matrix}\right]$$

We have two more options: preserve the average, thus we need:

$$\frac{1}{{2}}\left[\begin{matrix} 1 &amp; 1\\1 &amp; -1\end{matrix}\right]$$

as $\frac{1}{{2}}\left[\begin{matrix} 1 &amp; 1\\1 &amp; -1\end{matrix}\right]\left[\begin{matrix} x_1\\x_2\end{matrix}\right]$ has the same average as $\left[\begin{matrix} x_1\\x_2\end{matrix}\right]$, or 

$$\frac{1}{\sqrt{2}}\left[\begin{matrix} 1 &amp; 1\\1 &amp; -1\end{matrix}\right]$$

to preserve energy (thus orthogonality) in both domains. More generally, you have three standard options:


avoid scaling
scale by $\frac{1}{\sqrt{N}}$
or by $\frac{1}{{N}}$, 


depending on the invariance purpose. But you can scale columns with other quantities, as needed for your computational purpose. 
The following code aims at showing that the $1/\sqrt{N}$ scaling could work (if I am not too tired) to preserve Parseval-Plancherel or Rayleigh's energy theorem.

nSample = 2^1 ; % One can change the 2^* to other powers
data = randn(nSample,1) ; 
dataFFT = fft(data)/sqrt(nSample); 
ratioParseval = (norm(data)-norm(dataFFT))/norm(data)