Following the Kentucky Campaign, Frazer again resigned from the Confederate Army in late 1862. His most notable military service occurred after he was appointed a brigadier general on May 19, 1863, and given command of the 5th Brigade of the Army of Tennessee that July. His force consisted of about 2,300 men in three regiments and a battery of artillery, and was ordered to defend the Cumberland Gap at all hazards by area commander Maj. Gen. Simon B. Buckner, who was himself ordered to rejoin the Army of Tennessee just prior to the Battle of Chickamauga. Frazer began strengthening the fortifications already there to try to block any Federal advance into East Tennessee.