 With the outbreak of the Civil War, Hillyer raised a Walton County company known as the "Hillyer Rifles" in the late spring of 1861.The men were mustered into the Confederate Army on June 13, with Hillyer elected as the captain of what became Company C of the 9th Georgia Volunteer Infantry.He and the regiment were sent by train to Virginia and assigned to the newly created brigade of George "Tige" Anderson in the newly organized Army of Northern Virginia.