Why Artificial Intelligence Needs a Task Theory --- And What It Might
  Look Like
  The concept of "task" is at the core of artificial intelligence (AI): Tasks
are used for training and evaluating AI systems, which are built in order to
perform and automatize tasks we deem useful. In other fields of engineering
theoretical foundations allow thorough evaluation of designs by methodical
manipulation of well understood parameters with a known role and importance;
this allows an aeronautics engineer, for instance, to systematically assess the
effects of wind speed on an airplane's performance and stability. No framework
exists in AI that allows this kind of methodical manipulation: Performance
results on the few tasks in current use (cf. board games, question-answering)
cannot be easily compared, however similar or different. The issue is even more
acute with respect to artificial *general* intelligence systems, which must
handle unanticipated tasks whose specifics cannot be known beforehand. A *task
theory* would enable addressing tasks at the *class* level, bypassing their
specifics, providing the appropriate formalization and classification of tasks,
environments, and their parameters, resulting in more rigorous ways of
measuring, comparing, and evaluating intelligent behavior. Even modest
improvements in this direction would surpass the current ad-hoc nature of
machine learning and AI evaluation. Here we discuss the main elements of the
argument for a task theory and present an outline of what it might look like
for physical tasks.
