What did Geoffrey Hinton mean when he said this? I was going through some lectures from the Deep Learning course that Geoffrey Hinton taught on Coursera and I came across this statement:

"RNNs could potentially learn to implement lots of small programs that each capture a nugget of knowledge and run in parallel, interacting to produce very complicated effects."

I have no idea what he meant nor seen/figured out any examples of this idea (and I'm sure other people are wondering too.) So if somebody would care to demonstrate, it would be much appreciated.
Creating a model is having the computer write a small program. If you have a multi-layer network, then multiple nuggets might be interacting to produce complicated results.

That its pretty much how things like face detection work - you start with a series of really dumb "detectors" that look like basic rectangles and they combine in ways so that at the top, you're recognizing Brad Pitt.