Zemach was born in 1886 in Płońsk, Poland, then part of the Russian Empire, and was a descendant of Rabbi Avraham Gombiner (known as the "Magen Avraham") and his descendant, Rabbi Zvi Hirsch HaLevi of Plonsk (known as the "Plant (Zemach) of Avraham", from where Zemach's surname originates).
From 1909 to 1914, Zemach studied agriculture, literature and philosophy in France.