1. A computer-implemented method for optimizing a program executing on plurality of compute nodes of a massively parallel computing system, comprising: identifying a plurality of blocks of code included in the program; selecting, for at least a first block of code, a plurality of sets of optimization parameters to use in compiling the first block of code into a set of native instructions; transmitting one of the sets of optimization parameters to each of the plurality of compute nodes, wherein each compute node includes a processor and a memory, and wherein the plurality of compute nodes are connected via at least one data network; compiling, by each of the plurality of compute nodes, the first block of code using the set of optimization parameters received by a respective compute node; monitoring, by a service node, at least a performance characteristic of the program when the set of native instructions generated by compiling the first block of code according to the set of optimization parameters are executed by one of the compute nodes; transmitting, by the service node, to at least a first compute node of the plurality of compute nodes, an indication to recompile the first block of code using a selected one of the sets of optimization parameters selected by evaluating the monitored performance characteristic; and recompiling, by the first compute node, the selected block of code using the selected set of optimization parameters.