The Oneida tribe claim to be the original inhabitants of the area.
Many of the northern towns of the county were established, or partly established, from parts of Brownville, including Le Ray (1806), Lyme (1818), Pamelia (1819), Orleans (1821), and Alexandria (1821).