 Born in 1797, Smith was the youngest of five daughters of Hannah Hadassah (Hickock) Smith and Zephaniah Hollister Smith, a Nonconformist clergyman turned farmer.Smith's mother authored one of the earliest anti-slavery petitions, presented to the United States Congress by John Quincy Adams.The family was united in support of her advocacy of education, abolition and women's rights.