Spence was born on the island of Eday in the Orkney Islands, Scotland, the son of a stonemason, and migrated to Australia with his family as a six-year-old child.
Since the Australian economy was expanding rapidly at this time and there was an acute shortage of labour, the unions were in a strong bargaining position and were able to secure great improvements in the living standards of Australia's rural working class.