1. A parallel computing system comprising: a host processor; one or more compute units coupled to the host processor; a memory coupled to at least one of the host processor and the one or more compute units, wherein a parallel computing program is stored in the memory, the parallel computing program including calls to one or more APIs (application programming interface) for the host processor to perform: selecting the compute units based on processing requirements, at least one of the calls to the APIs specifying the processing requirements for the selection; generating a program object corresponding to a source code for a data processing task, at least one of the calls to the APIs specifying the source code for the generation of the program object; generating executable codes from the program object for the one or more compute units, at least one of the calls to the APIs specifying the program object for the generation of the executable codes; and loading the executable codes to be executed concurrently among the one or more compute units to perform the data processing task.