1. A computer-implemented method, comprising: receiving observations for multiple robots in an environment over a period of time, wherein the observations each include: an object observation generated by a corresponding robot of the robots that defines, for a corresponding time in the period of time: a corresponding identifier of a corresponding object of multiple objects of the environment and a measured object pose for the corresponding object, the measured object pose generated based on at least one sensor of the corresponding robot, and a localization observation for the corresponding robot that defines, for the corresponding time: a measured source pose for the sensor utilized to generate the measured object pose; generating, for each of the observations, a pose value for the corresponding object of the observation with respect to a reference frame, the generating based on the measured object pose of the observation; generating, for each of the pose values, an uncertainty measure based on both the object observation and the localization observation of the observation utilized to generate the pose value, wherein the uncertainty measures are each indicative of a degree of uncertainty in a corresponding one of the pose values; generating, based at least in part on the uncertainty measures, a spatio-temporal model that defines the pose values and the corresponding times for a plurality of the objects of the environment; wherein generating the spatio-temporal model based at least in part on the uncertainty measures comprises one or both of: filtering a plurality of the pose values and the corresponding times from the spatio-temporal model based on the uncertainty measures; and assigning the uncertainty measures to the corresponding pose values in the spatio-temporal model; accessing the generated spatio-temporal model to determine a pose for an object of the objects of the spatio-temporal model at a target time; identifying one or more pose values for the object in the spatio-temporal model based on proximity of the target time to the corresponding measurement times for the pose values; and determining a pose for the object for the target time based on at least one of the identified pose values.