1. A parallel computing system comprising: a host processor; one or more compute units coupled to the host processor; a memory coupled to at least one of the host processor and the one or more compute units, wherein a parallel computing program is stored in the memory, the parallel computing program including calls to one or more APIs (application programming interface) for the host processor to perform: generating a program object corresponding to a source code for a data processing task; generating executable codes from the program object for the one or more compute units; and loading the executable codes to be executed concurrently among the one or more compute units to perform the data processing task.