In 1660 Fisher held a dispute with Thomas Danson at Sandwich, Kent, and later that year was in Newgate Prison.
Shortly after his discharge he was again arrested at Charlwood, Surrey, and committed to the White Lion Prison, Southwark, where he was confined for about two years.