When retreating from a compromised territory should you always leave one infantry?
In answer to your subquestion of "why does it make sense to leave one infantry in a territory you're retreating from?"

The core of the analysis around this hinges on the assumption that you will be trading back and forth over the contested territory with small numbers of units- that you have a large, constantly growing stack in A, your opponent has a large, constantly growing stack in C, and you keep trading off B between the two territories with as small stacks as you can manage. Each player has a steady supply chain providing new troops from the factory. The reason why you do this is because if you move your large stack into the territory, it will be missing the current turn's set of reinforcements, and be unable to use your planes for defense, and your opponent can attack with their whole stack and use all their planes for offense. This is the whole idea of Dead Zones. You want to control them because they're worth IPCs, but with as few units as possible because those units will be destroyed by a counterattack next turn.

So why leave a lone infantry there? Because if it's empty, your opponent can take it with the bare minimum of units, and their planes are free to attack elsewhere. If it isn't, they are forced to commit some of their planes to it, and it's economically in their interest to send 2 infantry. Even if that is a good trade for him (you will lose your 1 infantry for only a 33% chance of killing one of his) the next turn, you will be able to kill both his infantry, an even better trade for you! 

So then what about the title question: Should you always leave one infantry? 

No. Almost always, but not always. If your opponent will move in with his whole stack next turn (because it's big enough that yours is no longer a threat) the lone infantry is no longer economically in your favor. You will be trading an infantry for a 33% chance to kill an infantry, which is not effective. You might still do it to prevent a blitz but it's no longer automatically good.

Also, if the territory is only worth 1IPC. Generally, the territories you have these big-stack standoffs over are worth more than 1, so this doesn't come up. For a 1IPC territory, it no longer makes your opponent want to send in 2 infantry. The expected value for him attacking your lone infantry with 1 infantry + 1 fighter will be approximately 7/6, while the value he'll get from just waltzing into the territory uncontested will be 6/6. You might still do it to block a blitz or just to force the opportunity cost of him using his planes. But if he has more planes than he knows what to do with and no blitz threat, you wouldn't want leave the guy there.