1. A computer implemented method comprising: loading, by a host processing unit in response to receiving a second API (application program interface) request from an application running on the host processing unit, an executable previously compiled for a processing unit from a source in the application to perform a data processing task in a target processing unit, the target processing unit identified by a compute device identifier specified in the second API request and previously selected by the application from one or more compute device identifiers sent by the host processing system in response to receiving a first API request from the application to identify processing units that support a processing requirement for the data processing task; detecting, by the host processing unit, a difference between the processing unit and the target processing unit; and compiling, by the host processing unit in response to the detecting the difference, the source for the target processing unit based on the difference.