Benchmarking Knowledge-driven Zero-shot Learning
  External knowledge (a.k.a. side information) plays a critical role in
zero-shot learning (ZSL) which aims to predict with unseen classes that have
never appeared in training data. Several kinds of external knowledge, such as
text and attribute, have been widely investigated, but they alone are limited
with incomplete semantics. Some very recent studies thus propose to use
Knowledge Graph (KG) due to its high expressivity and compatibility for
representing kinds of knowledge. However, the ZSL community is still in short
of standard benchmarks for studying and comparing different external knowledge
settings and different KG-based ZSL methods. In this paper, we proposed six
resources covering three tasks, i.e., zero-shot image classification (ZS-IMGC),
zero-shot relation extraction (ZS-RE), and zero-shot KG completion (ZS-KGC).
Each resource has a normal ZSL benchmark and a KG containing semantics ranging
from text to attribute, from relational knowledge to logical expressions. We
have clearly presented these resources including their construction,
statistics, data formats and usage cases w.r.t. different ZSL methods. More
importantly, we have conducted a comprehensive benchmarking study, with two
general and state-of-the-art methods, two setting-specific methods and one
interpretable method. We discussed and compared different ZSL paradigms w.r.t.
different external knowledge settings, and found that our resources have great
potential for developing more advanced ZSL methods and more solutions for
applying KGs for augmenting machine learning. All the resources are available
at https://github.com/China-UK-ZSL/Resources_for_KZSL.
