Shepherd was born at Mintonville, near Suffolk, Virginia, the son of Thomas Swepson Shepherd (1811-1860) and his first wife Ann Eliza Browne (1813-1852).
At the end of summer in 1862, Lee's Army of Northern Virginia moved into Maryland during its first invasion of the North.