The Relativity of Induction
  Lately there has been a lot of discussion about why deep learning algorithms
perform better than we would theoretically suspect. To get insight into this
question, it helps to improve our understanding of how learning works. We
explore the core problem of generalization and show that long-accepted Occam's
razor and parsimony principles are insufficient to ground learning. Instead, we
derive and demonstrate a set of relativistic principles that yield clearer
insight into the nature and dynamics of learning. We show that concepts of
simplicity are fundamentally contingent, that all learning operates relative to
an initial guess, and that generalization cannot be measured or strongly
inferred, but that it can be expected given enough observation. Using these
principles, we reconstruct our understanding in terms of distributed learning
systems whose components inherit beliefs and update them. We then apply this
perspective to elucidate the nature of some real world inductive processes
including deep learning.
