How to segment phone call audio into silence/non silence?
There are a bunch of parameters that you can look at:


Overall energy
Short term spectrum: Speech has a fairly distinctive "pink-like" spectrum and noise (which is happening during the non-speech parts) tends to be white if it's electrically dominated or "red" (i.e. low frequency heavy) if it's acoustic background noise or microphone noise
Amplitude statistics. Most noise signals have a Gaussian distribution, speech is closer to a Laplace distribution


I think a combination of these three should give a fairly robust detection scheme.