1. At a computer system, the computer system including a processor, a method for identifying a computer system problem during execution of an application, instances of the application load balanced on a plurality of devices, the method comprising: identifying a set of performance indicators that are to be monitored for the application; for each of the plurality of devices, the processor collecting values for the set of performance indicators, the values collected over different periods when there is varied levels of demand for the application; for each of the performance indicators, calculating an average value based on said collected values; and identifying a device as an outlying device when a value for a monitored performance indicator for the device varies from the calculated average for the monitored performance indicator, the identified device identified from among the plurality of devices.