python scipy fft on numpy hanning window smears peaks
Is this meant to happen?


Yes, that is absolutely a well known effect of using any window function.
Taking a look through the Wikipedia article on window functions, we find that the rectangular window has the sharpest peak.
This is quite simply due to the fact that the rectangular window acquires the most data, meaning that it can distinguish between slightly different frequencies the best.

The problem with the rectangular window is that it has large side lobes.
In other words, if you measure a signal which doesn't exactly match one of the discrete Fourier transform (DFT) frequencies (a.k.a. bins), then with the rectangular window you get a lot of leakage into neighboring bins.
The formula for this is actually rather simple.
Suppose your sampling interval is $\delta t$ and you measure $N$ points.
Then the DFT frequencies are
$$f_n = n / N \delta t, \quad n\in[0,1,\ldots,N-1] \, .$$

Now suppose you measure a sinusoid with frequency $\xi$ so that the sampled values are $x_n = \cos(2 \pi \xi n \delta t)$.
The mod squared of the DFT values are then
$$ |X_k|^2 = \left( \frac{\sin(\pi(\xi - k))}{\sin(\pi(\xi-k)/N} \right)^2 \, . \tag{$*$}$$

We plot this function in Figure 1 for $\xi=24.4$.
Note that when $\xi$ is not an integer the DFT values are nonzero for all bins, and you can see from equation $(*)$ that the values of $|X_k|^2$ only fall off as one over the square of frequency.


Figure 1: DFT values for a cosine of frequency 24.4.
The blue line indicates the underlying function given in equation $(*)$ and the black dots are the values of that function evaluated at the integers.
In other words, the black dots are the DFT bin values.

The point of a window function is to make the leakage of power into bins near the real frequency (in this case 24.4) fall off faster, but this comes at the cost that the main peak is broadened.$^{[a]}$


  Does anyone else get this problem?


It's not a problem.
It's how windows are supposed to work.


  For my data, it seems a rectangular window is fine, but to me it seems that the Hanning window must be wrong here, as the integral of the entire spectrum should be the same (after I multiplied by 2 to correct amplitude), but in this case it seems to be different.


Applying a window changes the total power in your signal, as you have observed.
This is not a mistake at all.
It's pretty obvious that this has to happen because the integral (sum) of the power in the frequency domain has to equal the power you would have computed in the time doman (Parseval's theorem).
It's clear that multiplying e.g. a cosine by a window function changes the integral of the time domain signal.
This is called "loss" and must be taken into account when using windows!

$[a]$: I'm not actually showing this because I didn't bother to compute the points with the window, but the Wikipedia article shows many examples.