According to the United States Census Bureau, the town has a total area of 9.6 square miles (24.8 km2), of which 9.0 square miles (23.4 km2) is land and 0.6 square mile (1.4 km2) (5.75%) is water.
Ayer borders the following towns: Shirley, Groton, Littleton, and Harvard.