1. A programming language system for a parallel computing architecture, the programming language system implemented by a parallel computing program stored in memory in a system having parallel processors, the system comprising: a host processor; a graphics processing unit (GPU) coupled to the host processor; a memory coupled to at least one of the host processor and the GPU, the memory storing predetermined capabilities for each of the parallel processors, the parallel processors including the host processor and the GPU, the parallel computing program being stored in the memory to allocate threads between the host processor and the GPU and wherein the programming language includes an API to allow an application to make calls using the API to allocate execution of the threads between the host processor and the GPU, wherein the application is associated with a program source code in the programming language, the program source code including a first source code and a second source code, wherein the first source code includes a plurality of APIs for the host processor, the APIs including a first API specifying processing capability requirement for a processing task, the APIs including a second API specifying the second source code for the processing task, wherein the program source code further comprises at least one API call including at least one processing capability requirement passed to the first API, wherein the application calls the first API to cause the host processor to match the processing capability requirement specified in the first API and the predetermined capabilities for the parallel processors to select one or more of the parallel processors for the processing task, wherein said matching selects the at least one processor based on at least one of: a) the predetermined capabilities of the at least one processor satisfying all of the processing capability requirements or b) the predetermined capabilities for the selected processor scoring a highest matching score indicating highest compatibility with the processing capability requirements, wherein the application calls the second API to cause the host processor to build an executable dynamically to be executed in a selected parallel processor for the processing task, each executable being compiled from the second source code, and wherein the execution of the threads is based on the executables dynamically built.