Automatic Gain Control for voice
Two things come to mind.

First, let's address the low-amplitude noise problem. This one is a typical textbook case for a noise gate. To give you an extreme example, these are sometimes even used on guitar distortion effects which are essentially insane amounts of controlled overdrive. I have to admit that I failed to find any literature on creating digital noise gates, but they exist abundantly as plugins so maybe digging through some open-source projects will score you a good basis for learning how to implement them. In essence, you measure windowed energy of a signal, and if it falls below a certain threshold, you fade your gain to zero (usually decaying exponentials are used). In any case, I'm sure you know what a noise gate is and I'm beating around the bush here. Make one and use it.

Secondly, when you say roughly set the input gain for me automatically if it's way off, that sounds like a textbook case for using a fuzzy controller. I'm linking to the Wikipedia page because I'm feeling a bit lazy, but you can find tons of great papers out there just on the first ten pages of Google results. The concept is exactly as you described it. You'll have several (say five for the sake of a simple example) levels of how close the gain is to what you think it should be:


- Way too low
- Low
- About right
- High
- Way too high


Based on these you will have some sort of fuzzy response like:


- Increase at a high rate
- Increase at a low rate
- Do nothing
- Decrease at a low rate
- Decrease at a high rate


These are my thoughts on how I would do it. You still need some means of measuring your fuzzy control input (assessing how good current gain level is), but that's something for you to work out. I'd say that if you're riding very close to clipping, that's High, if you've clipped, that's Way too high, and so on. Depending on what the spec considers acceptable, fuzzy control can sometimes let you get away with very simple solutions.