Brown was constantly frustrated by bureaucratic delays, and found Russian censorship of mail and media appalling. He was also puzzled by the rise of slavophilia, which he believed could not compete with western culture. In spite of his misgivings about Russian society, Brown urged the United States to compete in the vast Russian market. He helped convince the Emperor to abandon an increase in duties on cotton imports, and secured Russian approval of the Clayton–Bulwer Treaty between the U.S. and Britain. In his last weeks in office, Brown expressed concern over the build-up to the Crimean War, and the effect such a conflict might have on U.S. commercial interests. Secession crisis, the Civil War and later life Upon his return from Russia, Brown resumed his role as a leader of Tennessee's Whigs. After the collapse of the national Whig Party, he aligned himself with the American Party ("Know Nothings"), and in 1855 was once again elected to the state House of Representatives, this time representing Davidson County. He was selected as speaker shortly afterward. In the mid-1850s, Brown campaigned across the state on behalf of presidential candidate Millard Fillmore, but was largely outshone by rising Democratic politician and future Confederate governor Isham Harris, who was campaigning on behalf of James Buchanan. Brown attended the Constitutional Union Party convention in 1860, and helped nominate fellow Tennessean John Bell as the party's presidential candidate. On April 18, 1861, following the attack on Fort Sumter in South Carolina, Brown and several other Whig leaders published a letter reaffirming their pro-Union stance. While they dismissed secession, "both as a constitutional right and as a remedy for existing evils," they nevertheless stated that Tennessee and other southern states were obliged to defend themselves should federal troops invade. In May 1861, Oliver Perry Temple, a Whig attorney from Knoxville, met with Brown and several other Whig leaders in Nashville. Temple later recalled that Brown seemed "paralyzed" by the rapid rise of secessionist sentiment in the city, but nevertheless delivered an "eloquent" denunciation of secession. Shortly after this meeting, however, Brown abandoned his support for the Union, and marched with a parade in Nashville in support of the new Confederate States of America. When Union soldiers occupied Nashville in 1862, Brown was jailed and his house was burned. Following the war, Tennessee's radical governor, William Brownlow, ordered the arrest of Isham Harris, and offered a $5,000 reward