the functions that are applied to each element in the stream. In the GPUs, vertices and fragments are the elements in streams and vertex and fragment shaders are the kernels to be run on them. For each element we can only read from the input, perform operations on it, and write to the output. It is permissible to have multiple inputs and multiple outputs, but never a piece of memory that is both readable and writable. Arithmetic intensity is defined as the number of operations performed per word of memory transferred. It is important for GPGPU applications to have high arithmetic intensity else the memory access latency will limit computational speedup. Ideal GPGPU applications have large data sets, high parallelism, and minimal dependency between data elements. GPU programming concepts Computational resources There are a variety of computational resources available on the GPU: Programmable processors – vertex, primitive, fragment and mainly compute pipelines allow programmer to perform kernel on streams of data Rasterizer – creates fragments and interpolates per-vertex constants such as texture coordinates and color Texture unit – read-only memory interface Framebuffer – write-only memory interface In fact, a program can substitute a write only texture for output instead of the framebuffer. This is done either through Render to Texture (RTT), Render-To-Backbuffer-Copy-To-Texture (RTBCTT), or the more recent stream-out. Textures as stream The most common form for a stream to take in GPGPU is a 2D grid because this fits naturally with the rendering model built into GPUs. Many computations naturally map into grids: matrix algebra, image processing, physically based simulation, and so on. Since textures are used as memory, texture lookups are then used as memory reads. Certain operations can be done automatically by the GPU because of this. Kernels Compute kernels can be thought of as the body of loops. For example, a programmer operating on a grid on the CPU might have code that looks like this: // Input and output grids have 10000 x 10000 or 100 million elements. void transform_10k_by_10k_grid(float in[10000][10000], float out[10000][10000]) { for (int x = 0; x < 10000; x++) { for (int y = 0; y < 10000; y++) { // The next line is executed 100 million times out[x][y] = do_some_hard_work(in[x][y]); } } } On the GPU, the programmer only specifies the body of the loop as the kernel and what data to loop over by invoking geometry processing. Flow control