 When the troubles between the military and Governor Bligh began, Hayes took the side of the governor and was sent to the coal mines at Newcastle.Bligh would have pardoned him if he could have obtained possession of the great seal, and after Macquarie came Hayes was pardoned in 1812.He then sailed to Europe in the same vessel, the "Isabella", with Joseph Holt; an account of their shipwreck can be found in the "Memoirs of Joseph Holt".