The highest point in Gorham is along its western border, on a spur of Mount Madison, where the elevation reaches above sea level. While Gorham itself is surrounded by larger mountains, a small prominent peak within the town is high Pine Mountain.