1. A method, comprising: receiving, at a computing system, first data sensed at a first time by a sensor system of an autonomous vehicle, the first data being representative of a first object of a plurality of objects on or proximate a road surface in a region in an environment the autonomous vehicle has autonomously navigated; comparing the first data with reference data associated with a plurality of reference semantic classifications; based at least in part on the comparing, determining that the first object does not match any of the plurality of reference semantic classifications; identifying additional objects having additional object data similar to the first data, the first object and the additional objects comprising a subset of the objects; receiving, at the computing system, second data sensed subsequent to the first time, the second data representing a behavior of at least one of the additional objects; based at least in part on determining that the first object does not match any of the plurality of reference semantic classifications, determining, based on the first data and the second data, a probability that the subset of the objects conforms to a behavior; generating, at the computing system, an inferred semantic classification associated with the subset of the objects when the probability indicates a pattern of objects in the subset of the objects conforming to the behavior; associating the inferred semantic classification with the plurality of reference semantic classifications, the inferred semantic classification being different from each of the plurality of reference semantic classifications; updating, at the computing system, map data associated with the environment to include information about the inferred semantic classification; and transmitting the updated map data to the autonomous vehicle and at least one additional autonomous vehicle.