PSD scaling due to the windowing I am struggling with the following problem for a while: If I have signal of duration N and I apply window (assume rectangular window) of size M &lt; N how will this influence PSD of that signal after doing fft? 

PSD of signal should be scaled (I assume) but by which amount and what is the mathematical explanation behind it?

Thank you for your answers in advance.
The shortest answer to that question is to divide your PSD by the squared sum of window values $w_i$:

$$PSD(n) = \dfrac{2|X(n)|^2}{S^2} $$

where:

$n=0\ldots N/2 $

$S = \sum_{i=0}^{N-1}w_i$ 

You can read more on that topic in the following article:


  Heinzel G., et al - Spectrum and spectral density estimation by the
  DFT, including a comprehensive list of window functions and some new
  flat-top windows.