According to the United States Census Bureau, the town has a total area of , of which is land and , or 1.94%, is water. Situated to the west of the Taconic Mountains, Poultney is drained by the Poultney River, a tributary of Lake Champlain.