Palmer was born in Long Sutton in Somerset, the eldest son of William Palmer and his wife, Mary, the daughter of William Isaac of Sturminster Newton in Dorset [REF].
His father died in 1826, and he was educated at Sidcot School near Weston-super-Mare, before becoming an apprentice to his uncle, who was a miller and confectioner.