1. A method comprising: receiving a request to process network inputs to a neural network using an integrated circuit that performs neural network computations in hardware using a matrix computation unit, the neural network comprising a depth concatenation neural network layer that specifies a concatenation of an input tensor having dimensions x 1 by y 1 by z 1 and an input tensor having dimensions x 1 by y 1 by z 2 along a depth dimension to generate an output tensor having dimensions x 1 by y 1 by (z 1 +z 2 ); and generating instructions that, when executed by the integrated circuit, cause the integrated circuit to, during processing of a network input by the neural network, generate a layer output tensor that satisfies the specification of the depth concatenation neural network layer by performing operations comprising: for each spatial location in a first input tensor to the depth concatenation layer and a second input tensor to the depth concatenation layer: multiplying, using the matrix computation unit, a second depth vector for the spatial location in the second input tensor by a shift weight matrix for the depth concatenation layer to generate a shifted second depth vector that has zeroes as the first z 1 entries and entries of the second depth vector as the last z 2 entries; and adding the shifted second depth vector and a first input depth vector for the spatial location in the first input tensor to generate a concatenated depth vector, the first input depth vector having entries of the first input depth vector as the first z 1 entries of the first input depth vector and zeroes as the last z 2 entries of the first input depth vector.