How to transfer declarative knowledge into neural networks
One way to look at intelligence is it's the way to compress the universe. That means we have a short mental representation of meaningful concepts.
For example, if I would say &quot;there is a red swan in your building, it's dangerous and can kill you&quot;, you already have concepts of &quot;red&quot;, &quot;swan&quot;, &quot;danger&quot; and this easy allows you to add the bird to your classifier &quot;should I beware it or not&quot;. (and you probably have tried to imagine the red swan by now)
There are similar properties in deep networks when deeper layers could respond to more abstract representation concepts. For example, if you have a convolutional network classifying the faces, the first layers would detect simple shapes like lines and circles, then more complex shapes and then parts of the face like eyes, nose and ears (see example here)
Now, let's imagine you have a classification task &quot;is the thing dangerous or not&quot;. You have a trained convolutional model and it has &quot;red&quot; and &quot;swan&quot; abstract features on some level. You could freeze all the preliminary weights and make a single backpropagation with a high learning rate only for the subnetwork which is of interest to you.
That could work in theory, but the key challenge here is that most of the neural network representations are not interpretable at all and you would need to solve this problem before (there is a whole research direction with workshops and conferences on the topic with varying success)