1. An apparatus, comprising: a first memory that holds rows of N weight words indexed as 0 through N−1 and that receives an address, wherein the address has log 2 W bits and an extra bit; a second memory that holds rows of N data words indexed as 0 through N−1, wherein N is at least 512; an array of N processing units (PUs) indexed as 0 through N−1, each PUs of index J having: first and second registers, an accumulator having an output, and an arithmetic unit having first, second and third inputs and that performs an operation thereon to generate a result to store in the accumulator, the first input receives the output of the accumulator, the second input receives a weight word output by the first register, the third input receives a data word output by the second register; first multiplexing logic that, for the PUs 0 through (N/2)−1 receives weight words J and J+(N/2) output by the first memory and for PUs N/2 through N−1 receives weight words J and J−(N/2) output by the first memory, and that outputs a selected weight word to the first register; and second multiplexing logic that receives data word J output by the second memory and receives a data word output by the second register of PU J−1 and that outputs a selected data word to the second register, wherein PU J−1 is PU N−1 for PU 0 ; and wherein the second multiplexing logic of PU 0 also receives the data word output by the second register of PU (N/2)−1, and the second multiplexing logic of PU N/2 also receives the data word output by the second register of PU N−1.