1. A computer implemented method comprising: identifying a dimension of a data parallel task comprising one or more executable codes compiled from a single source, wherein a total number of threads to concurrently execute the one or more executable codes in one or more target processing units is based on a global thread number, the global thread number comprising a multi-dimensional value, and wherein the global thread number is specified in an API (Application Programming Interface) request from an application running in a host processing unit; determining a thread group size for each target processing unit based on one of the identified dimension and a local thread group number for the respective target processing unit, the local thread group number comprising a multi-dimensional value, wherein the total number of threads is partitioned into thread groups for the one or more target processing units based on the thread group sizes for the one or more target processing units, and wherein threads in a thread group for a target processing unit are executed concurrently; and loading the one or more executable codes in the one or more target processing units, wherein each executable code loaded in target processing unit executes in the thread group for the respective target processing unit.