Title: Using Human Brain Activity to Guide Machine Learning,
Abstract: Machine learning is a field of computer science that builds algorithms that
learn. In many cases, machine learning algorithms are used to recreate a human
ability like adding a caption to a photo, driving a car, or playing a game.
While the human brain has long served as a source of inspiration for machine
learning, little effort has been made to directly use data collected from
working brains as a guide for machine learning algorithms. Here we demonstrate
a new paradigm of "neurally-weighted" machine learning, which takes fMRI
measurements of human brain activity from subjects viewing images, and infuses
these data into the training process of an object recognition learning
algorithm to make it more consistent with the human brain. After training,
these neurally-weighted classifiers are able to classify images without
requiring any additional neural data. We show that our neural-weighting
approach can lead to large performance gains when used with traditional machine
vision features, as well as to significant improvements with already
high-performing convolutional neural network features. The effectiveness of
this approach points to a path forward for a new class of hybrid machine
learning algorithms which take both inspiration and direct constraints from
neuronal data.