Williamson was born the son of James Williamson in a croft in Hirnlay near Aboyne.
Though little acknowledged in history, there was in those days a thriving slave trade in stolen children, most of whom were taken to North America.