Most Southerners elected to Congress were former Confederates, with the most prominent being Georgia Senator-designate and former Confederate vice president Alexander Stephens. Congress assembled in early December 1865; Johnson's conciliatory annual message to them was well received. Nevertheless, Congress refused to seat the Southern legislators and established a committee to recommend appropriate Reconstruction legislation.[126]
Northerners were outraged at the idea of unrepentant Confederate leaders, such as Stephens, rejoining the federal government at a time when emotional wounds from the war remained raw. They saw the Black Codes placing African Americans in a position barely above slavery. Republicans also feared that restoration of the Southern states would return the Democrats to power.[127][128] In addition, according to David O. Stewart in his book on Johnson's impeachment, "the violence and poverty that oppressed the South would galvanize the opposition to Johnson".[129]

Break with the Republicans: 1866
Congress was reluctant to confront the President, and initially only sought to fine-tune Johnson's policies towards the South.[130] According to Trefousse, "If there was a time when Johnson could have come to an agreement with the moderates of the Republican Party, it was the period following the return of Congress.