His paternal grandfather, Aaron Atherton (1745-1821), was part of a group of settlers who travelled through the Cumberland Gap, led by Samuel Goodwin, who in 1780 founded Goodin or Goodwin Fort, as a frontier settlement of Virginia. It was a time of sieges and skirmishes with local tribes, with the settlement being raised in July 1781. Following the American Revolutionary War more settlers arrived. By the time Kentucky established statehood over this area, Aaron and his son, Peter Atherton (1771-1844) had been operating a small distillery on the banks of Rolling Fork River at Knob Creek for over thirty years, since around 1790 making them one of the first whiskey pioneers of Kentucky