1. A pipelined hardware implementation of a neural network circuit, comprising: an input stage for receiving and storing input values; a first processing stage coupled to the input stage, the first processing stage including a plurality of first processing units, each first processing unit including: (a) a weight store for storing a plurality of weighted values; (b) a plurality of multipliers each for multiplying an input value by a respective weighted value; (c) an adder for adding a product outputted from one of the multipliers with at least one product outputted from a respective multiplier of another one of the plurality of first processing units; (d) a function circuit for receiving a sum outputted by the adder and for generating therefrom a processing unit value; and (e) a register for storing the processing unit value generated by the function circuit; an additional processing stage coupled to an upstream processing stage and including a plurality of additional stage processing units, each additional stage processing unit including: (i) a weight store for storing a plurality of weighted values; (ii) a plurality of multipliers each for multiplying a processing unit value received from a processing unit of the upstream processing stage by a respective weighted value; (iii) an adder for adding a product outputted from one of the multipliers of the respective additional stage processing unit with at least one product outputted from a respective multiplier of another one of the plurality of additional stage processing units; (iv) a function circuit for receiving a sum outputted by the adder of the respective additional stage processing unit and for generating therefrom a processing unit value; and (v) a register for storing the processing unit value generated by the function circuit of the respective additional stage processing unit; and an output stage formed from output ports of the registers of the additional processing stage.