 His mother was Anna Maria, daughter of Richard Stout, judge and member of the governor's council on Cape Breton Island.Harris was descended from Sir Amias Poulett, English ambassador to France in the reign of Queen Elizabeth I and afterwards keeper of Mary, Queen of Scots.Harris was educated from 1837 at the Manchester Free Grammar School and from 1839 Trinity College, Cambridge where he graduated B.A.