Title: A General Theory for Training Learning Machine,
Abstract: Though the deep learning is pushing the machine learning to a new stage,
basic theories of machine learning are still limited. The principle of
learning, the role of the a prior knowledge, the role of neuron bias, and the
basis for choosing neural transfer function and cost function, etc., are still
far from clear. In this paper, we present a general theoretical framework for
machine learning. We classify the prior knowledge into common and
problem-dependent parts, and consider that the aim of learning is to maximally
incorporate them. The principle we suggested for maximizing the former is the
design risk minimization principle, while the neural transfer function, the
cost function, as well as pretreatment of samples, are endowed with the role
for maximizing the latter. The role of the neuron bias is explained from a
different angle. We develop a Monte Carlo algorithm to establish the
input-output responses, and we control the input-output sensitivity of a
learning machine by controlling that of individual neurons. Applications of
function approaching and smoothing, pattern recognition and classification, are
provided to illustrate how to train general learning machines based on our
theory and algorithm. Our method may in addition induce new applications, such
as the transductive inference.