 In 1879 he married Isobel Osbourne, the daughter of Fanny Vandegrift and step daughter of the writer Robert Louis Stevenson.Stevenson described Joseph in "The Silverado Squatters" as a great omelet maker.The couple traveled to the Kingdom of Hawaii in 1882, where they lived for several years.