HetMAML: Task-Heterogeneous Model-Agnostic Meta-Learning for Few-Shot
  Learning Across Modalities
  Existing gradient-based meta-learning approaches to few-shot learning assume
that all tasks have the same input feature space. However, in the real world
scenarios, there are many cases that the input structures of tasks can be
different, that is, different tasks may vary in the number of input modalities
or data types. Existing meta-learners cannot handle the heterogeneous task
distribution (HTD) as there is not only global meta-knowledge shared across
tasks but also type-specific knowledge that distinguishes each type of tasks.
To deal with task heterogeneity and promote fast within-task adaptions for each
type of tasks, in this paper, we propose HetMAML, a task-heterogeneous
model-agnostic meta-learning framework, which can capture both the
type-specific and globally shared knowledge and can achieve the balance between
knowledge customization and generalization. Specifically, we design a
multi-channel backbone module that encodes the input of each type of tasks into
the same length sequence of modality-specific embeddings. Then, we propose a
task-aware iterative feature aggregation network which can automatically take
into account the context of task-specific input structures and adaptively
project the heterogeneous input spaces to the same lower-dimensional embedding
space of concepts. Our experiments on six task-heterogeneous datasets
demonstrate that HetMAML successfully leverages type-specific and globally
shared meta-parameters for heterogeneous tasks and achieves fast within-task
adaptions for each type of tasks.
