Male and female voice spectrum
in my opinion and limited experience, you can't just compare spectra of male and female voices to identify which is which.
2 decades ago i worked on two products called PurePitch and PitchDoctor that was able to shift the pitch of a voice independently of shifting the formants.  it also had the ability to increase or reduce the amount of pitch inflection of the voice (all the way to monotone).  the products had factory-supplied &quot;presets&quot;, one was called &quot;testosterone&quot; and the other was called &quot;estrogen&quot;, that could convincingly make a female voice sound masculine or a male voice sound feminine.
anyway, i don't remember the numerical parameters that were researched literally by my boss in usage of the alpha version of the product.  but it was something like 300 or 400 cents difference between the male and female voices and less than that for the difference between formants.
i think you might need a pitch detector and get lotsa words and do some statistics on the pitch to get a good machine idea of whether the pitch is or was more likely male or female.