Why did the developement of neural networks stop between 50s and 80s?
I will first address your main question &quot;Why did the development of neural networks stop between 50s and 80s?&quot;
In 40-50s there was a lot of progress (McCulloch and Pitts); the perceptron was invented (Rosenblatt). That gave rise to an AI hype giving many promises (exactly like today)!
However, Minsky and Papert have proved in 1969 that a single-layer architecture is not enough to build a universal approximating machine (see e.g. Minsky, M. &amp; Papert, S. Perceptrons: An Introduction to Computational Geometry, vol. 165 (1969)). That led to the first disappointment in &quot;AI&quot;. Which lasted until several major breakthroughs in 1980s: the proof of universal approximating capabilities of multi-layer perceptron by Cybenko, the popularisation of backpropagation algorithm (Hinton and colleagues), etc.
I agree with LeCun that using continuous activation functions have enabled the backpropagation algorithm at the time. It is only recently that we have learned to backpropagate in networks with binary activation functions (2016!).