Born in London, he was the second of the eight children of Robert Smith F.R.S., a solicitor, and his wife Mary Bogle; his sister Maria was the mother of Maria Abdy.
Smith entered his father's office and succeeded him as solicitor to the Board of Ordnance in 1812.