Duruy was born in Paris, the son of a factory worker, and at first intended for his father's trade.
He devoted himself to secondary school education, holding his chair in the College Henri IV at Paris for over a quarter of a century.