 He was the son of Cornelius Grinnell (1758–1850) and Sylvia (née Howland) Grinnell (1765–1837).His siblings included Joseph Grinnell and Moses Hicks Grinnell.After graduating from the New Bedford Academy, Grinnell moved to New York City in 1818, where he became a clerk in the commission house of H.D.