1. A method for creating a graph that defines a parallel processing job, comprising: reading code for building said graph, said code includes syntax for replication, pointwise connect, cross connect and merge, said pointwise connect connects point to point outputs from a first sub-graph to inputs of a second sub-graph, said pointwise connect further includes connecting a first output of said first sub-graph only to a first input of said second sub-graph, and connecting a second output of said first sub-graph only to a second input of said second sub-graph; building said graph based on said code, building said graph including creating a data structure storing a set of identifications of vertices corresponding to a set of program units to be executed, a set of identifications of edges corresponding to flow of data between the set of program units, a set of identifications of inputs and a set of identifications of outputs; populating one or more vertices into a vertex queue; determining at least one vertex in the vertex queue that are ready to be executed based on inputs associated with the at least one vertex; and creating an execution record associated with the at least one vertex, the execution record associated with the at least one vertex including a ready state associated with the at least one vertex.