 His paternal grandparents were Peter Jay, a wealthy trader in furs, wheat, timber, and other commodities, and Mary (née Van Cortlandt) Jay (daughter of New York mayor Jacobus Van Cortlandt).He graduated from Yale College in 1808.After his graduation, he took up the management of his father's large estate in Westchester County, New York, and also studied law at Albany.