1. A method, comprising: receiving, at a computing system, first data sensed at a first time by a sensor system of an autonomous vehicle, the first data being representative of a first object of a plurality of objects on or proximate a road surface in a region in an environment the autonomous vehicle has autonomously navigated; comparing a semantic classification associated with the first object with data representing reference semantic classifications; based at least in part on the comparing, determining that the semantic classification does not match any of the reference semantic classifications; identifying additional objects having the semantic classification, the first object and the additional objects comprising a subset of the objects; receiving, at the computing system, second data sensed subsequent to the first time, the second data representing a behavior of at least one of the additional objects; based at least in part on determining that the semantic classification does not match any of the reference semantic classifications, determining, based on the first data and the second data, a probability that the subset of the objects conforms to a behavior; generating, at the computing system, data comprising an inferred semantic classification associated with the subset of the objects when the probability indicates a pattern of objects in the subset of the objects conforming to the behavior; updating, at the computing system, map data associated with the environment to include information about the inferred semantic classification; and transmitting the updated map data to the autonomous vehicle and at least one additional autonomous vehicle.