. Fork . Fork . Fork . Join (wait for parallel forks to complete). . Deallocate . Procedure adds into , element-wise: Base case: if , set (or do a short loop, perhaps unrolled). Otherwise: Partition into , , , . Partition into , , , . In parallel: Fork . Fork . Fork . Fork . Join. Here, fork is a keyword that signal a computation may be run in parallel with the rest of the function call, while join waits for all previously "forked" computations to complete. achieves its goal by pointer manipulation only. This algorithm has a critical path length of steps, meaning it takes that much time on an ideal machine with an infinite number of processors; therefore, it has a maximum possible speedup of on any real computer. The algorithm isn't practical due to the communication cost inherent in moving data to and from the temporary matrix , but a more practical variant achieves speedup, without using a temporary matrix. Communication-avoiding and distributed algorithms On modern architectures with hierarchical memory, the cost of loading and storing input matrix elements tends to dominate the cost of arithmetic. On a single machine this is the amount of data transferred between RAM and cache, while on a distributed memory multi-node machine it is the amount transferred between nodes; in either case it is called the communication bandwidth. The naïve algorithm using three nested loops uses communication bandwidth. Cannon's algorithm, also known as the 2D algorithm, is a communication-avoiding algorithm that partitions each input matrix into a block matrix whose elements are submatrices of size by , where is the size of fast memory. The naïve algorithm is then used over the block matrices, computing products of submatrices entirely in fast memory. This reduces communication bandwidth to , which is asymptotically optimal (for algorithms performing computation). In a distributed setting with processors arranged in a by 2D mesh, one submatrix of the result can be assigned to each processor, and the product can be computed with each processor transmitting words, which is asymptotically optimal assuming that each node stores the minimum elements. This can be improved by the 3D algorithm, which arranges the processors in a 3D cube mesh, assigning every product of two input submatrices to a single processor. The result submatrices are then generated by performing a reduction over each row. This algorithm transmits words per processor,