1. A parallel computing system comprising: a host processor; one or more compute units coupled to the host processor; a memory coupled to at least one of the host processor and the one or more compute units, wherein a parallel computing program is stored in the memory, wherein the parallel computing program is executable by the host processor, the parallel computing program specifying a source code and processing requirements for a data processing task based on the source code, wherein the source code is not yet compiled before runtime when the parallel computing program is executed, the parallel computing program including calls to one or more APIs (application programming interface) for the host processor to perform during the runtime of the parallel computing program: selecting the compute units based on the processing requirements which include a multi-dimensional value specifying a number of threads to perform the data processing task, at least one of the APIs is called with the processing requirements; generating executable codes from the source code for the one or more compute units, at least one of the APIs is called with a specification of the source code; and loading the executable codes to be executed concurrently via the number of threads among the one or more compute units to perform the data processing task.