The town was first settled in 1799 when four brothers named Andrew, John, Samuel, and Daniel Millet left their homes in Connecticut, traveled west and became the first settlers in what is now the hamlet of Walworth.
Theron Yeomans of Walworth introduced Holstein cattle to the United States and was the first president of the Holstein-Friesian Association of America.