1. At a computer system, the computer system including a compiler for generating lower level code from program source code, a method for generating lower level code so that kernel execution can be optimized at runtime, the method comprising: an act of accessing program source code, the program source code including an annotation that identifies part of the program source code as a kernel that is to be executed on a co-processor, the program source code also declaring properties of the kernel; an act of compiling the program source code into lower level code, including: an act of detecting the code annotation within the program source code; in response to detecting the code annotation, compiling the program source code into proxy code and into separate stub code, the proxy code for execution on a central processing unit and the stub code for execution on a co-processor, compilation including: an act of generating the proxy code, the proxy code configured to invoke the stub code in accordance with the declared properties of the kernel, the proxy code including a descriptor for referencing any runtime optimization objects stored during compilation; an act of analyzing an intermediate representation of the stub code to derive usage information about the declared properties of the kernel; an act of generating the stub code in accordance with the derived usage information; an act of storing the derived usage information in one or more runtime optimization objects alongside the stub code; and an act of linking the descriptor to the one or more runtime optimization objects stored alongside the stub code to provide the proxy code with access to the derived usage information for making kernel optimization decisions at runtime.