PyAudio recording is taking much longer than planned
First off, a small thing:

The sampling rate usually has to be one of the standard sampling rates. 
Closest to your 1024*40 would be 44100. 

The hardware has to support the sampling rate. Some drivers will resample to the given rate, but that will be somewhat slow as it is in software. Try using a proper sampling rate.

The real bottle neck is the plotting.  

Your are calculating the FFT for every block, and plotting it.

The FFT is blazing fast, no worries there.

Plotting on the other hand is horribly slow.  The plotting routine is not optimized for speed, and you are also rescaling and formatting the plot on every go around.

Comment out the lines that do the plotting.  Leave the FFT in.  You'll see a tremendous change in speed.



Edit:
Just noticed.  Besides plot being slower than molasses, you also have a pause in there.  It is only 1 millisecond, but I don't know how precise it really is.  Many timers under Windows use 15 millisecond steps, so that pause could be zero, or 1 millisecond, or 15 milliseconds.

In any case, it is not doing you any good.  Take it out.