Why is the Hamming and Hanning window the most widely used for speech spectograms? When taking the Short-Time Fourier Transform of the speech signal, windowing is applied to taper away the signal frame at both ends and mitigate introduced artifact at expense of the maximum obtainable resolution. Window design is then motivated controlling the trade-offs or spectral-time resolution, and various figures of merits, like the mainlobe width, sidelode width, sidelobe roll-off per octave is introduced in the literature.

In terms of a speech signal, usually a Hamming or Hanning window is employed, and the only justification I have found for it is that speech signals are narrow band signals. How could I demonstrate that a Hamming window is indeed better than, say, a rectangular window for a speech signal?

In a typical natural speech application, a speaker will exhibits frequencies between 0 and 200 Hz, a little higher perhaps for females. Utterances are usually a lot longer than that to produce required frequency, so we can afford to take more frames. I don't know a required resolution for speech. Thus, is it always better to use a Hamming/Hanning window on speech? Is there a clear reason I should tell someone using a rectangular window on producing spectograms, why he/she should not do it (apart from reproducibility of certain results in academic literature)?
any window (almost any) is better than a boxcar window, unless your spectrum is flat.  speech signals are not flat. 

I suspect that Hamming and Hann are commonly used because you can avoid scalloping loss for 50% overlap. One gets a sufficiently favorable tradeoff between time resolution and frequency domain dynamic (side lobe level) range. There is also the fact that these windows are cited in papers that go back to the 70s.  

You can also apply the widows with a 3 point convolution (3 coefficients instead of N) in the frequency domain, which might have had some advantages in hardware in the past when memory was a constraint.

Classical fixed windows are chosen according to characteristics like sidelobe level. They aren’t like adaptive windows that use the data itself.  It’s relatively pedestrian processing. 

in summary, boxcar windows are for flat spectra.