How were windows originally conceived?
This is only a partial answer, but there's a lecture online where Hamming talks about how he came up with his eponymous window. Starting at roughly 15:15 gives the full context.

With a reasonably entertaining story, he credits John Tukey with inventing the theory of windows (for spectrum analysis). However, he introduces the whole subject in the context of using Lanczos sigma factors to reduce Gibbs phenomenon. In addition, in The Art of Doing Science and Engineering (based on the same lectures), he describes how his window is a variation on the Hann window, which he claims was used by von Hann in economics (long before its application in signal processing). That suggests the history goes much further back, depending on how you want to define it.

The book where Tukey first named the Hamming window is The Measurement of Power Spectra from the Point of View of Communications Engineering. Given Hamming's assertion that Tukey invented the theory of windows, it would probably be a good place to start for a deeper understanding of how to design new ones. I think the book is just a reprint of Part I and Part II of his Bell System Technical Journal article, so it's available online.