What makes primary batteries leak on very deep discharge?
In the case of the carbon-zinc cell, the case of the cell is one of the electrodes.  The process of generating electricity uses up the zinc, so it corrodes from the inside out.  Eventually holes will develop, and the electrolyte can leak out.