The town is in the Catskill Mountains and borders Sullivan County, New York. It is rugged and heavily wooded. Most settlement is concentrated around the hamlet of Sundown on Rondout Creek, along the two branches of the Neversink River or in the plateau between the two valleys northeast of Red Hill, at elevations 2,500 feet (762 m) above sea level and higher.