Rectangular vs Hann windowing I'm trying to create a basic digital signal processing pipeline, to manipulate a WAV file. 
Nothing fancy, I'm just looking to apply some filtering.

My first attempt was to divide the input signal to N rectangular windows (which do not overlap), convert each window to the frequency domain, apply the filtering of my choice to the spectrum, convert it back to the time domain and insert the resulting window in place of the original one.

(By rectangular window, I mean rectangular in the time domain.)

This worked, however, the only downside was that there was a discontinuity at the edges of each window, severe enough to distort the overall signal and introduce harmonics of significant amplitude.

My attempt to resolve this was to replace the rectangular windows with Hann windows, and apply the same process. 
With non-overlapping windows, the obvious result was that the output signal became a set of Hann "pulses", which is as bad as the discontinuities.

I seem to be missing a very basic part of digital signal processing.

I understand the windows should overlap, but how exactly? Should the overlapping portions be averaged? How do we ensure that the amplitude of the resulting signal is unchanged despite Hann window overlapping/averaging?
When you filter a finite length signal, the filtered output will have a larger length, either you implement the filter in time domain or frequency domain. Your desired continuity comes from this excess part which must be added to the result of next segment or window.

For example when you take N samples of a signal and filter it with a filter of length M, the result has a length of N+M-1, where the M-1 excessive samples must be added to the next segment. To do so you have to find your filter response in time domain which has a length of M samples, then zero-pad both your signal and filter to N+M-1 samples then take N+M-1 point FT of both and multiply them.