In 1893, Major Dooley had a large stone mansion built on a large estate overlooking the James River in [Henrico County, west of Richmond], which he and his wife named Maymont. By 1912 the Dooleys also completed an enormous mountain retreat, Swannanoa, in the Blue Ridge Mountains at Rockfish Gap near Waynesboro, Virginia in Nelson County.