Eliza Monroe Hay (also his cousin, then a widow). Gouverneur started work on publishing the papers or a book on Monroe, but it was never finished. After Mrs. Hay died in 1840, the Gouverneurs moved to Washington, DC where he worked in the consular bureau of the U.S. Department of State from 1844 to 1849. After congress agreed to buy the papers of President Madison, Gouverneur proposed a similar arrangement, which was concluded in 1850. Some personal papers would be retained for a few generations. Personal life On March 9, 1820, Gouverneur was married to Maria Hester Monroe, his first cousin and the daughter of President Monroe. The wedding was officiated by the Rev. William Dickinson Hawley and was the first wedding held in the White House for a child of a president. The wedding was small, with only 42 guests and no cabinet members invited, and General Thomas Jesup served as groomsman for Gouverneur. The couple went on a brief one-week honeymoon, and upon their return, Commodore and Mrs. Stephen Decatur gave them a reception at the Decatur House on May 20, 1820. Another ball was planned, but was cancelled due to Decatur's death two days later in a duel. After moving to New York, the Gouverneur's bought and lived at 63 and 65 Prince Street at Lafayette Street in Manhattan. Together, Samuel and Maria were the parents of four children: a daughter, who died in infancy (d. September 4, 1821) James Monroe Gouverneur (1822–1885), a deaf-mute who died at the Spring Grove Asylum in Baltimore, Maryland; Elizabeth Kortright Gouverneur (1824–1868), who married Dr. Henry Lee Heishell, James M. Bibby, and Colonel G. D. Sparrier. Samuel Laurence Gouverneur, Jr. (1826–1880), who married Marian Campbell (1821–1914), and became the first U.S. consul in Fuzhou, China (then spelled Foo Chow). In 1832, the Gouverneurs' sold their Prince Street residence to Miles R. Burke. On June 20, 1850, his wife Maria died at the Oak Hill estate, which was sold two years later in 1852. In September 1851, the widower Gouverneur married Mary Digges Lee (1810–1898), a granddaughter of Thomas Sim Lee (1745–1819). They retired to the Lee estate called "Needwood", near Frederick, Maryland and Harpers Ferry, West Virginia. This stressed family relations during the American Civil War, with Gouverneur associated with the Union government, while his in-laws had deep roots in the Confederate states. Death Gouverneur died at his Needwood estate