1. A non-transitory computer-readable storage medium having computer-executable instructions stored thereupon which, when executed by a computer, cause the computer to: perform a deployment of an application to a plurality of hosts by creating one or more new hosts, deploying the application to the one or more new hosts, adding the one or more new hosts to a deployment group, and removing one or more hosts from the deployment group; maintain the removed hosts for a user-specified period of time following the deployment; determine whether one or more user-defined failure conditions occurred during the user-specified period of time following the deployment; and return the removed hosts to the deployment group and remove the new hosts from the deployment group, in response to determining that one or more user-defined failure conditions occurred during the user-specified period of time.