[191] By the time he entered the Senate in 1827, he had resigned himself to spending part of the year away from his family. Still, he sought to remain close to his children through letters.[192]
Tyler was a slaveholder, at one point keeping 40 slaves at Greenway.[193] Although he regarded slavery as an evil, and did not attempt to justify it, he never freed any of his slaves. Tyler considered slavery a part of states' rights, and therefore the federal government lacked the authority to abolish it. The living conditions of his slaves are not well documented, but historians surmise that he cared for their well-being and abstained from physical violence against them.[193] In December 1841, Tyler was attacked by the abolitionist publisher Joshua Leavitt, with the unsubstantiated allegation that Tyler had fathered several sons with his slaves, and later sold them. A number of black families today maintain a belief in their descent from Tyler, but there is no evidence of such genealogy.[194]
Tyler's wealth exceeded $50 million (2020 peak value), but he became indebted during the Civil War and died much poorer.