He was the first ( and longest-serving ) Governor of New York , and then Vice President of the United States under Thomas Jefferson and James Madison . He was the first Governor of New York , and then the fourth Vice President of the United States ( 1805-1812 ) , serving under Presidents Thomas Jefferson and James Madison .