How to window a time domain signal for 6 symbols with cyclic prefix length of 512 samples?
Assuming that it would suffice to have an average result of all six symbols, I suggest that you simply concatenate all symbols with cyclic prefix in time and then window and FFT the whole dataset. This would give you the same result as an average of the result you would get for each symbol + prefix independently. Therefore the window length would be (2048+512)*6 = 15,360. 

You would only consider overlap techniques when you need to continuously stream an fft process, or if you have a dataset that is not feasible to process with one fft. 15,360 is quite reasonable to do as one fft process.

Windowing involves multiplying the time domain data by the window weight sample for sample. After windowing the resulting windowed data is FFT'd. Purpose is to reduce spectral leakage from strong tones, thereby increasing the dynamic range and ability to see weaker signals together with stronger signals. This is at the expense of frequency resolution, or the ability to discern between two closely spaced tones that can be approximately the same power level (with decreased resolution, they would appear as one tone).

Note that the window will reduce the total power of the signal. For the measurement of power in bins where there are signals of similar power in adjacent bins (such as noise or waveforms that do not occupy single frequency bins), the non-coherent gain of the window would be used to compensate the measurement). For measurement of a single tone that occupies one bin (such as done with a calibration measurement), the coherent gain of the window would be used. The difference (in dB) between the coherent gain and non-coherent gain is $10Log10(RBW)$, where RBW is the resolution bandwidth. This makes sense as with a rectangular window (no further windowing other than selecting the data) the RBW is 1 bit, but any other window increases this bandwidth, thus if you sum square the bins to get the total power in the waveform, if you windowed other than the rectangular window you would be over-counting if there is similar power in adjacent bins since each bin includes the power in adjacent bins. The second link below goes into that further.

More on windowing:

How to apply Hamming Window?

How to calculate resolution of DFT with Hamming/Hann window?