1. A method, comprising: monitoring, by a processing device, a load measurement of containers associated with an application executed on virtual machines (VMs) in a multi-tenant Platform-as-a-Service (PaaS) system; comparing, by the processing device, the load measurement with a criteria; and when the load measurement exceeds the criteria, causing resources to be added to execute functionality of the application in the VMs of the multi-tenant PaaS system, wherein the load measurement and the criteria comprise at least one of: the load measurement corresponding to an amount of work requests in a first container of the containers and the criteria comprises a maximum amount of work requests that the first container can handle; the load measurement corresponding to an overall load of the containers and the criteria comprises a maximum load of the containers; the load measurement corresponding to a load at the first container and the criteria comprises a plurality of thrash prevention criteria that are satisfied before removing the application; or the load measurement corresponding to the load at the first container and the criteria comprises a first criteria for adding resources liberally and a second criteria for conservatively removing resources.