As with most other peaks in the Mill Brook Ridge range, there was very little human impact on Graham for most of its recorded history. That changed drastically after the construction of Turner Hollow Road in the late 19th century, connecting Seager with the end of Beaver Kill Road, allowing anglers to reach private fishing clubs on the Beaver Kill headwaters from the Arkville Station on the Ulster and Delaware Railroad. Around 1890 George Gould bought the Furlow Lodge tract, including most of the mountain, since his father had enjoyed fishing at the lake so much.