1. A computer-implemented method, comprising: in a multi-thread runtime system configured to run on a parallel-processing computer system that includes multiple processing elements, at least two of which have different instruction set architectures, performing the following at run time: receiving one or more operation requests issued by an application; generating a first compute kernel for a first subset of the operation requests and a second compute kernel for a second subset of the operation requests, wherein the first subset of the operation requests is dynamically chosen by the runtime system for generating the first compute kernel and the second subset of the operation requests is pre-specified in the application for generating the second compute kernel, wherein the first compute kernel is configured to be executed on a first processing element and the second compute kernel is configured to be executed on a second processing element, the first and second processing elements having different instruction set architectures; storing the first compute kernel in a first program cache for later reuse by the runtime system in lieu of generating a corresponding compute kernel; storing the second compute kernel in a second program cache for later reuse by the runtime system in lieu of generating a corresponding compute kernel; and executing the first and second compute kernels on the corresponding first and second processing elements of the parallel-processing computer system.