The son of William Wylie, a block calico printer of Kilmarnock, by his wife Agnes, daughter of John Howie of Lochgoin, he was born at Kilmarnock on 24 February 1833.
In 1859 Wylie was president of the University Dialectic Society, and soon afterwards became a student at Regent's Park College, London, under Joseph Angus.