What makes the animal brain so special?
One probable hardware limiting factor is internal bandwidth.  A human brain has $10^{15}$ synapses.  Even if each is only exchanging a few bits of information per second, that's on the order of $10^{15}$ bytes/sec internal bandwidth.  A fast GPU (like those used to train neural networks) might approach $10^{11}$ bytes/sec of internal bandwidth.  You could have 10,000 of these together to get something close to the total internal bandwidth of the human brain, but the interconnects between the nodes would be relatively slow, and would bottleneck the flow of information between different parts of the &quot;brain.&quot;
Another limitation might be raw processing power.  A modern GPU has maybe 5,000 math units.  Each unit has a cycle time of ~1 ns, and might require ~1000 cycles to do the equivalent processing work one neuron does in ~1/10 second (this value is totally pulled from the air; we don't really know the most efficient way to match brain processing in silicon).  So, a single GPU might be able to match $5 \times 10^8$ neurons in real-time.  You would optimally need 200 of them to match the processing power of the brain.
This back-of-the-envelope calculation shows that internal bandwidth is probably a more severe constraint.