During World War I Kemp sat as a Unionist in the coalition government formed under Robert Borden.
On November 4, 1921, Kemp was appointed to the Canadian Senate on the recommendation of Prime Minister Arthur Meighen.