Harrison was related by marriage to a prominent citizen of early Newtown, the miller Nathaniel Armstrong.Members of Armstrong's family later owned a carding mill in Newtown, and years after Harrison's residence at the house, it was known as the "Armstrong Mill".