1. A computing unit for accelerating tensor computations, comprising: a first memory bank having a first data width for storing at least one of input activations or output activations; a second memory bank having a second data width that is larger than the first data width for storing one or more parameters used in performing computations; a direct memory access traversal unit configured to: access at least one memory location of the first memory bank to obtain at least one input activation, or at least one output activation, based on instructions received from a source external to the computing unit; and access at least one memory location of the second memory bank to obtain the one or more parameters based on the received instructions; at least one cell comprising at least one multiply accumulate (“MAC”) operator that receives parameters obtained from the second memory bank and performs the computations; a tensor traversal unit in data communication with at least the first memory bank, the tensor traversal unit configured to provide a control signal to the first memory bank to cause an input activation to be provided to a data bus accessible by the at least one MAC operator; and wherein the computing unit performs one or more computations associated with at least one element of a data array, the one or more computations being performed by the at least one MAC operator and comprising, in part, a multiply operation of the input activation received from the data bus and a parameter received from the second memory bank.