In general, construction wise, what makes a speaker more or less sensitive to mid tones?
There's a LOT of factors in play for how a speaker behaves electro-acoustically. I took a course in school and we spent like 3 months describing how the electrical power translates to mechanical movement and into air movement. But to summarize, here are some factors:


First of all, audio pressure is proportional to the volume velocity of air times frequency. The lower the frequency, the more air you need to move around to make the same pressure (~approximately equal to volume).
You can move more air by having a larger speaker diaphragm (the thing that moves)
Problem is, the larger the speaker diaphragm, the larger the mass of the diaphragm and the more the air impedance upon it. This has the result of reducing its resonance point to a lower frequency.
Generally by controlling how big the diaphragm is, you control how efficient it is at what frequency. A large diaphragm is inherently more efficient at lower frequencies. 
At higher frequencies, one does not need to move the diaphragm that much to make the same sound volume, so (skipping lots of details) to get good performance you need a different set of speaker parameters to get similar performance. 
The most important parameter arguably is the diaphragm size. Making it smaller reduces the peak force requirements of mechanical parts (since there is less mass to move), at the cost of less air movement, but as explained earlier, you need less of it anyways so it works out.
Other things like the amount of windings matter, since you need to move the speaker less. More windings give you better coupling between the mechanical parts of the speaker with the electrical input, at the cost of increasing inductance. Since the midband signal requires less movement, and inductance is harmful, you want to decrease the amount of windings. 
A larger diaphragm also has the downside of acting as a directional radiator. Much like how a directional antenna works, as the "acoustic" size of the diaphragm increases (i.e. the frequency increases / size of the diaphragm increases wrt frequency), the more directional the acoustic "beam" becomes. This is akin to how dish antennas get more directivity the wider it becomes. 
This is of course bad because that means a large woofer might be able to produce sound at a higher frequency, but you get a narrow "beam" of audio directly in front of the woofer and it falls off as you go to the side.




I also want to mention that the frequency response (IIRC, it's been a while) of a closed boxed speaker falls off at the low end at -6dB/decade and -6dB/decade at the high end. The low end falloff is because you're riding on the low frequency falling edge of the resonance of the speaker system, and because there's a frequency dependence on volume velocity vs. pressure. The high end falloff is because of inductances in the speaker, which limit the amount of current passing through its coils and being on the high end edge of the resonance.