Black-box Testing of First-Order Logic Ontologies Using WordNet
  Artificial Intelligence aims to provide computer programs with commonsense
knowledge to reason about our world. This paper offers a new practical approach
towards automated commonsense reasoning with first-order logic (FOL)
ontologies. We propose a new black-box testing methodology of FOL SUMO-based
ontologies by exploiting WordNet and its mapping into SUMO. Our proposal
includes a method for the (semi-)automatic creation of a very large benchmark
of competency questions and a procedure for its automated evaluation by using
automated theorem provers (ATPs). Applying different quality criteria, our
testing proposal enables a successful evaluation of a) the competency of
several translations of SUMO into FOL and b) the performance of various
automated ATPs. Finally, we also provide a fine-grained and complete analysis
of the commonsense reasoning competency of current FOL SUMO-based ontologies.
