1. A computer-implemented method for updating virtual machine images, comprising: under the control of one or more computer systems configured with executable instructions, receiving an update to be applied to a virtual machine and a time to apply the update; instantiating a representative virtual machine; applying the update to the representative virtual machine; determining one or more changes to the representative virtual machine corresponding to applying the update; generating, based at least in part on the determined changes, an offline update and an indirection map containing information mapping computing resources of a virtual machine image requiring the update to logical equivalents of physical resources of the representative virtual machine; as a result of determining that the time to apply the update has expired with at least one virtual machine image still requiring the update, applying the update to generate an updated virtual machine by at least: enabling a monitoring service access to the virtual machine image requiring the update; and applying the offline update to the virtual machine image requiring the update based at least in part on the indirection map; and causing an image of the updated virtual machine to be persistently stored.