Feature Extraction of Insect Sounds
Typically, for analyses like this, you do overlapping windows with a hop size that is fixed fraction of the window size.  So, for example, you might start by doing analyses of 1024 frames per window, and move the window forward 256 samples per analysis.  Depending on how the data looks then, you may or may not want to apply a window function (like Hamming or the like) to each window.

But since you're basically trying to remove noise, it might be simpler to take an example of the noise itself, and then try to reduce that noise from the original sample.  Take the FFT of the noise, subtract the frequency bins of the noise from the frequency bins of the sample, and run an inverse FFT to produce the final filtered result.  If you take this approach, make sure to only do FFTs at half the sample rate.