Python FFT (average) Hanning spectrum size 4096 and 50% overlap
I can tell you what to do but I can't tell you how to do it in Python code. To average four spectra, do the following:

1) Multiply input samples x[0] –thru- x[4095] by a 4096-point Hanning sequence.

2) Compute a 4096-pt FFT of the above product sequence producing an X1[m] spectrum.

3) Multiply input samples x[2048] –thru- x[6143] by a 4096-point Hanning sequence.

4) Compute a 4096-pt FFT of the above product sequence producing an X2[m] spectrum.

5) Multiply input samples x[4096] –thru- x[8191] by a 4096-point Hanning sequence.

6) Compute a 4096-pt FFT of the above product sequence producing an X3[m] spectrum.

7) Multiply input samples x[6144] –thru- x[10239] by a 4096-point Hanning sequence.

8) Compute a 4096-pt FFT of the above product sequence producing an X4[m] spectrum.

9) Finally, average the magnitude-squared samples of the four spectra to produce a final |Xf[m]| magnitude spectrum.  That is, 

|Xf[0]| = (|X1[0]| + |X2[0]| + |X3[0]| + |X4[0]|)/4.

|Xf[1]| = (|X1[1]| + |X2[1]| + |X3[1]| + |X4[1]|)/4.

|Xf[2]| = (|X1[2]| + |X2[2]| + |X3[2]| + |X4[2]|)/4.

...

...

|Xf[4095]| = (|X1[4095]| + |X2[4095]| + |X3[4095]| + |X4[4095]|)/4.

If you desire a power spectrum, then merely square each of the |Xf[m]| magnitude samples. user25139, you're implementing Welch's Method of spectrum analysis. You can search the web for "Welch's Method" for more information.