STFT window width vs. detectable Hz range
Regarding the first paragraph of your question: The number of samples used in a discrete Fourier transform (DFT) depends on your desired DFT bin spacing (the frequency difference between adjacent DFT bins) in the frequency domain. If that desired bin spacing is Fo Hz, and the time-domain signal sample rate is Fs samples/second, then the number time samples, N, is:

N = Fs/Fo samples.             (1)

For example, at Fs = 44100, if you select N = 4410 samples then the DFT bin spacing will be:

Fo = Fs/N = 44100/4410 = 10 Hz.        (2)

However, for the traditional radix-2 fast Fourier transform (FFT) N must be an integer power of two. So using the above Eq. (2) when N = 2^12 = 4096 then Fo = 44100/4096 = 10.766 Hz. 

Regarding the second paragraph of your question: I don't fully understand that paragraph but if Fs = 44100, someone performing 1024-point FFTs (N = 1024) produces spectra where the FFT bin spacing is:

Fo = Fs/N = 44100/1024 = 43.06 Hz.        (3)

Does that Fo = 43.06 Hz FFT bin spacing seem correct based on other information you see from the NMF chapter?