He was a rival for the state party leadership with the editor Thurlow Weed and Weed's protégé, William H. Seward. Throughout his career, Fillmore declared slavery an evil but that it was beyond the powers of the federal government. Seward was openly hostile to slavery and argued that the federal government had a role to play in ending it. Fillmore was an unsuccessful candidate for Speaker of the U.S. House of Representatives when the Whigs took control of the chamber in 1841, but he was made the chairman of the Ways and Means Committee. Defeated in bids for the Whig nomination for vice president in 1844 and for New York governor the same year, Fillmore was elected Comptroller of New York in 1847, the first to hold that post by direct election.
As vice president, Fillmore was largely ignored by Taylor, and even in the dispensing of patronage in New York, Taylor consulted Weed and Seward. In his capacity as president of the Senate, however, Fillmore presided over the Senate's angry debates, as the 31st Congress decided whether to allow slavery in the Mexican Cession. 