He was born in Sogndal as a son of vicar Jens Ludvig Carl Olsen (1816-1866) and Vally Marie Caroline Juell (1832-1906).
His wife Vally was a daughter of Sogndal's mayor Fredrik Christian Juell and sister-in-law of Carl August Gulbranson.