Herter was born in Paris, France, to American artist and expatriate parents, Albert Herter and Adele McGinnis, and attended the École Alsacienne there (1901-1904) before moving to New York City, where he attended the Browning School (1904-1911).
Herter married the wealthy heiress Mary Caroline Pratt (1895-1980) in 1917.