Journal
FRONTIERS IN NEUROROBOTICS
Volume 12, Issue -, Pages -Publisher
FRONTIERS MEDIA SA
DOI: 10.3389/fnbot.2018.00049
Keywords
stochastic separation theorems; concentration of measure; knowledge transfer in artificial intelligence systems; error correction; supervised learning; neural networks
Funding
- Ministry of Education and Science of Russia [14.Y26.31.0022]
- Innovate UK (Knowledge Transfer Partnership grant) [KTP010522]
Ask authors/readers for more resources
We consider the fundamental question: how a legacy student Artificial Intelligent (AI) system could learn from a legacy teacher AI system or a human expert without re-training and, most importantly, without requiring significant computational resources. Here learning is broadly understood as an ability of one system to mimic responses of the other to an incoming stimulation and vice-versa. We call such learning an Artificial Intelligence knowledge transfer. We show that if internal variables of the student Artificial Intelligent system have the structure of an n-dimensional topological vector space and n is sufficiently high then, with probability close to one, the required knowledge transfer can be implemented by simple cascades of linear functionals. In particular, for n sufficiently large, with probability close to one, the student system can successfully and non-iteratively learn k << n new examples from the teacher (or correct the same number of mistakes) at the cost of two additional inner products. The concept is illustrated with an example of knowledge transfer from one pre-trained convolutional neural network to another.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available