The family lived in poverty and the young Riddell was subjected to hard labour during his upbringing.
He finished his education in 1830 and became a licentiate of the Church of Scotland, moving in with his eldest brother, William, at his cottage at Teviothead.