The town was first settled around 1788 by members of a religious group, led by Jemima Wilkinson.
The town was named "Torrey" in order to secure the support of Henry Torrey to enable the creation of the new town over the objections of the two towns losing their territory.