However, his popularity soon eroded because of his response to the Panic of 1837, which centered on his Independent Treasury system, a plan under which the federal government of the United States would store its funds in vaults rather than in banks; more conservative Democrats and Whigs in Congress ultimately delayed his plan from being implemented until 1840. His presidency was further marred by the costly Second Seminole War (a result of continuing Jackson's Indian removal policy); and his refusal to admit Texas to the Union as a slave state as an attempt to avoid heightened sectional tensions. In 1840, Van Buren lost his re-election bid to William Henry Harrison, the nominee of the anti-Jacksonian Whig Party.
Van Buren was initially the leading candidate for the Democratic Party's nomination again in 1844, but his continued opposition to the annexation of Texas angered Southern Democrats, leading to the nomination of James K. Polk. Van Buren was the newly formed Free Soil Party's presidential nominee in 1848, and his candidacy helped Whig nominee Zachary Taylor defeat Democrat Lewis Cass. Van Buren returned to the Democratic Party after 1848, but grew increasingly opposed to slavery, and became one of the party's outspoken abolitionists. 