1. A system for emulating a neural network, said neural network to be emulated having architecture, operation mode and input data which are defined by a set of instructions, said system comprising: a host computer for receiving, storing and transmitting said set of instructions defining the neural network and the neural network's input data; a neural engine coupled to said host computer for processing said instructions and for performing neural network computations; said neural engine including a control circuit comprising a microprocessor slice means for receiving, storing and transmitting said set of instructions received from said host computer defining the architecture and operation mode of said neural network; each of said instructions having three fields, including opcode, source register and destination register fields, said instructions including register load and store instructions for initializing and storage of data in the microprocessor slice means; said set of instructions also including neural network instructions for causing the neural engine to process the neural network specified in the microprocessor slice means; a plurality of multiply and accumulate modules coupled to said control circuit for performing multiply and accumulate functions of said neural network; and said multiply and accumulate modules each including a plurality of weight storage means for storing a set of interconnection weights and the multiply and accumulate modules also including a plurality of multiplier and accumulate devices for generating the product of said input data times said interconnection weights and summing up all products for each neuron in said neural network, each multiplier and accumulate device being coupled to one of said weight storage means, wherein said multiplier and accumulate devices simultaneously perform multiply and accumulate operations during a single clock cycle of said system.