1. A neural network computing system comprising: a Field Programmable Gate Array (FPGA) configured to have a hardware logic performing computations associated with a neural network training algorithm by receiving streamed data directly from a host computer device, the hardware logic including a processing element that performs computations related to a hidden layer of the neural network training algorithm, the processing element including a plurality of arithmetic logic units each representing a hidden node of the hidden layer; and an interface for connecting the FPGA to the host computing device and receiving the streamed data.