[85]
Johnson returned home when his state took up the issue of secession. His successor as governor, Isham G. Harris, and the legislature organized a referendum on whether to have a constitutional convention to authorize secession; when that failed, they put the question of leaving the Union to a popular vote. Despite threats on Johnson's life, and actual assaults, he campaigned against both questions, sometimes speaking with a gun on the lectern before him. Although Johnson's eastern region of Tennessee was largely against secession, the second referendum passed, and in June 1861, Tennessee joined the Confederacy. Believing he would be killed if he stayed, Johnson fled through the Cumberland Gap, where his party was in fact shot at. He left his wife and family in Greeneville.[86][87]
As the only member from a seceded state to remain in the Senate and the most prominent Southern Unionist, Johnson had Lincoln's ear in the early months of the war.[88] With most of Tennessee in Confederate hands, Johnson spent congressional recesses in Kentucky and Ohio, trying in vain to convince any Union commander who would listen to conduct an operation into East Tennessee.