1. A method for improving the performance of applications in a production environment based on observed application behavior, the method comprising: while executing applications in a runtime production environment, collecting raw production profile data about the applications by randomly sampling machines in a datacenter where the sampling produces sequences of instructions which provide insight into an individual application's execution, the raw production profile data being a mapping from an instructions address to the instruction's execution count; converting the raw production profile data into symbolized profiles by restricting the raw production profile data by a particular time range, build, cell, job, or user and mapping build id and instruction address pairs of the raw production profile data into symbolized triplets of function, file, and line; storing the symbolized profiles in a database; aggregating, into a single aggregated profile per application, the stored symbolized profiles collected for each application by executing a mapping/reduction process on the stored symbolized profiles to consolidate information; and using each aggregated profile as a compilation input when compiling other versions of an application's binary to alter the application's performance for observed application behavior.