Granville is located in the eastern corner of Addison County. The villages of Granville and Lower Granville are located in a valley in the center of the town, with the Northfield Mountains rising to the east and the main crest of the Green Mountains rising to the west. Vermont Route 100 runs north-south through the main valley in the town, passing through Granville Notch in the northern part of the town before beginning its descent to the north into Warren. Route 100 leads south to Hancock and Rochester. The eastern portion of town, including the village of East Granville, is notable in that there is no direct road connection from there to either the main settlement of Granville, or anywhere else in Addison County. Vermont Route 12A runs north-south through this portion of town and connects to Roxbury (in Washington County, to the north) and Braintree (in Orange County, to the south). The Long Trail passes briefly through the western part of the town, crossing the summit of Little Hans Peak ().