Nisbet's grandson Douglas Robert Hadow (1846-1865) was killed in a mountaineering accident a few hours after taking part in the first ascent of the Matterhorn. Another grandson, Frank Hadow, won the Wimbledon championship in 1878.