Loudness detection for natural conversation
I would look into the EBU R-128 method of loudness measuring. Based on an ITU algorithm, it is used to determine loudness for TV which consist mostly of dialogue. 

ITU Docs

EBU R-128 Docs