1. A computer-implemented method configured to be performed by a runtime system at a parallel-processing computer system that includes multiple types of processing elements, comprising: at runtime: receiving one or more operation requests issued by an application, wherein the one or more operation requests correspond to a code reuse section of the application; generating one or more intermediate representation entries for the one or more operation requests; dynamically choosing a respective type of processing element for the one or more intermediate representation entries; identifying, among a plurality of compute kernels previously compiled by the runtime system for the application, one or more previously compiled compute kernels for the one or more intermediate representation entries, wherein the identified previously compiled compute kernels are configured to execute on the dynamically chosen type of processing element; and scheduling execution of the identified one or more previously compiled compute kemels at the respective type of processing element, wherein the multiple types of processing elements include two or more of: single-core central processing units, multi-core central processing units, graphics processing units, single-core co-processors and multi-core coprocessors.