Title: The Case for Meta-Cognitive Machine Learning: On Model Entropy and Concept Formation in Deep Learning,
Abstract: Machine learning is usually defined in behaviourist terms, where external
validation is the primary mechanism of learning. In this paper, I argue for a
more holistic interpretation in which finding more probable, efficient and
abstract representations is as central to learning as performance. In other
words, machine learning should be extended with strategies to reason over its
own learning process, leading to so-called meta-cognitive machine learning. As
such, the de facto definition of machine learning should be reformulated in
these intrinsically multi-objective terms, taking into account not only the
task performance but also internal learning objectives. To this end, we suggest
a "model entropy function" to be defined that quantifies the efficiency of the
internal learning processes. It is conjured that the minimization of this model
entropy leads to concept formation. Besides philosophical aspects, some initial
illustrations are included to support the claims.