Machine Number Sense: A Dataset of Visual Arithmetic Problems for
  Abstract and Relational Reasoning
  As a comprehensive indicator of mathematical thinking and intelligence, the
number sense (Dehaene 2011) bridges the induction of symbolic concepts and the
competence of problem-solving. To endow such a crucial cognitive ability to
machine intelligence, we propose a dataset, Machine Number Sense (MNS),
consisting of visual arithmetic problems automatically generated using a
grammar model--And-Or Graph (AOG). These visual arithmetic problems are in the
form of geometric figures: each problem has a set of geometric shapes as its
context and embedded number symbols. Solving such problems is not trivial; the
machine not only has to recognize the number, but also to interpret the number
with its contexts, shapes, and relations (e.g., symmetry) together with proper
operations. We benchmark the MNS dataset using four predominant neural network
models as baselines in this visual reasoning task. Comprehensive experiments
show that current neural-network-based models still struggle to understand
number concepts and relational operations. We show that a simple brute-force
search algorithm could work out some of the problems without context
information. Crucially, taking geometric context into account by an additional
perception module would provide a sharp performance gain with fewer search
steps. Altogether, we call for attention in fusing the classic search-based
algorithms with modern neural networks to discover the essential number
concepts in future research.
