 Born in Detroit and raised in Manhattan, Osborn was the son of Major General Frederick Osborn, a military officer who helped to develop the G.I.Bill.He was a descendant of John Jay and Cornelius Vanderbilt.