 Carlebach descended from a well known German rabbinical family.He was the son of Simson Carlebach (1875–1942), a banker, and his wife Resi née Graupe.His grandfather Salomon Carlebach (1845–1919), who married Esther Carlebach née Adler (1853–1920), was already a Rabbi in Lübeck.