Nelson Baker was born in Buffalo, New York on February 16, 1842 to Lewis Becker (later Baker) and Caroline Donnellan, parents who were ethnic German and Irish, during a period when the rate of immigration was increasing from Europe.
During the Civil War, Baker enlisted at age 21 as a Union soldier in early July 1863 as part of the 74th regiment of the New York State Militia.