1. A computer-implemented method for optimizing a program executing on plurality of compute nodes of a massively parallel computing system, comprising: identifying a plurality of blocks of code included in the program; selecting, for at least a first block of code, a plurality of sets of optimization parameters to use in compiling the first block of code into a set of native instructions; transmitting one of the sets of optimization parameters to each of the plurality of compute nodes; compiling, by each of the plurality of compute nodes, the first block of code using the set of optimization parameters received by a respective compute node; and monitoring at least a performance characteristic of the program when the set of native instructions generated by compiling the block of code according to the set of optimization parameters are executed by one of the compute nodes.