 By mid-1863, Johnson was under pressure from Lincoln to hold elections in the state.He continuously hesitated, however, as he was concerned that Unionists who opposed the Proclamation (known as "Conservative Unionists") would prevail.Conservative Unionists thus grew increasingly frustrated with Johnson over his emancipation stance and his refusal to hold elections.