Crawford Depot is located a short way north of the height of land of Crawford Notch, a glacially formed gorge in New Hampshire's White Mountains carrying the Saco River southward from its headwaters at Saco Lake. The depot is set with United States Route 302 to the east, at the northwestern end of Saco Lake, and the railroad tracks formerly of the Maine Central Railroad Mountain Division to the west. The Appalachian Mountain Club's Crawford Notch Highland Center is located just to the north.