Most of Smith Mountain is located within Hancock with lower east slopes within Pittsfield. West Mountain, , a spur of Smith Mountain, is located along the mountain's south ridgeline. The Taconic ridgeline continues north from Smith Mountain as Pine Mountain and Tower Mountain, and south as Doll Mountain. It is bordered by West Hill to the west across the Wyomanock Creek valley. The west side of the mountain drains into Wyomanock Creek, then into Kinderhook Creek, thence into the Hudson River and Long Island Sound. The east side drains into Smith Brook, thence to the Housatonic River and Long Island Sound.