How do I completely mute the sound in phone calls?
Indeed there is a minimum output volume for the audio coming from the call.
There might be a hidden settings, although it might be difficult to find or to apply to some phone models or Android verions.
There is also an option that is probably applicable to all phones: you can plug any headphones using the audio jack (assuming your phone has an audio jack, which perhaps all Android phones still have).  This redirects the audio to the headphones, and if you don't wear the headphones you will not hear it.  Which is what you were asking for, isn't it?
Notice: if you plug a headset (headphones with included microphone), your phone will most certainly use the headset's microphone, which may or may not be what you want.  If you plug headphones (without microphone) your phone will continue to use its internal microphone.