 He was the son of Wisconsin State Senator Charles S. Kelsey (born 1822) and Lucretia Parson (née Bacon) Kelsey (died 1868).Congressman William H. Kelsey and Wisconsin State Senator Edwin B. Kelsey were his uncles.He became a printer, then studied law, was admitted to the bar in 1875, and practiced law in Geneseo, Livingston County, New York.