Knowledge Transfer Between Artificial Intelligence Systems
  We consider the fundamental question: how a legacy "student" Artificial
Intelligent (AI) system could learn from a legacy "teacher" AI system or a
human expert without complete re-training and, most importantly, without
requiring significant computational resources. Here "learning" is understood as
an ability of one system to mimic responses of the other and vice-versa. We
call such learning an Artificial Intelligence knowledge transfer. We show that
if internal variables of the "student" Artificial Intelligent system have the
structure of an $n$-dimensional topological vector space and $n$ is
sufficiently high then, with probability close to one, the required knowledge
transfer can be implemented by simple cascades of linear functionals. In
particular, for $n$ sufficiently large, with probability close to one, the
"student" system can successfully and non-iteratively learn $k\ll n$ new
examples from the "teacher" (or correct the same number of mistakes) at the
cost of two additional inner products. The concept is illustrated with an
example of knowledge transfer from a pre-trained convolutional neural network
to a simple linear classifier with HOG features.
