1. A computer-implemented method for selecting and deploying updates to a programmable execution service (“PES”) application, the method comprising performing computer-implemented operations for: maintaining an update pool comprising one or more updates to a PES application; selecting one or more updates to the PES application from the update pool; creating a test environment for use in determining whether the selected updates to the PES application are suitable for deployment by programmatically replicating a production network and one or more production virtual machine instances executing the PES application; deploying the selected updates to the PES application to the virtual machine instances in the test environment; determining based upon the operation of the virtual machine instances in the test environment whether the selected updates to the PES application are suitable for deployment to the PES application; and deploying the selected updates to the PES application to the one or more production virtual machine instances executing the PES application in response to determining that the one or more updates to the PES application are suitable for deployment to the PES application.