Few-shot learning focuses on learning a new visual concept with very limited labelled examples. A successful approach to tackle this problem is to compare the similarity between examples in a learned metric space based on convolutional neural networks. However, existing methods typically suffer from meta-level overfitting due to the limited amount of training tasks and do not normally consider the importance of the convolutional features of different examples within the same channel. To address these limitations, we make the following two contributions: (a) We propose a novel meta-learning approach for aggregating useful convolutional features and suppressing noisy ones based on a channel-wise attention mechanism to improve class representations. The proposed model does not require fine-tuning and can be trained in an end-to-end manner. The main novelty lies in incorporating a shared weight generation module that learns to assign different weights to the feature maps of different examples within the same channel. (b) We also introduce a simple meta-level dropout technique that reduces meta-level overfitting in several few-shot learning approaches. In our experiments, we find that this simple technique significantly improves the performance of the proposed method as well as various state-of-the-art meta-learning algorithms. Applying our method to few-shot image recognition using Omniglot and miniImageNet datasets shows that it is capable of delivering a state-of-the-art classification performance.
@cite_13: An efficient learner is one who reuses what they already know to tackle a new problem. For a machine learner, this means understanding the similarities amongst datasets. In order to do this, one must take seriously the idea of working with datasets, rather than datapoints, as the key objects to model. Towards this goal, we demonstrate an extension of a variational autoencoder that can learn a method for computing representations, or statistics, of datasets in an unsupervised fashion. The network is trained to produce statistics that encapsulate a generative model for each dataset. Hence the network enables efficient learning from new datasets for both unsupervised and supervised tasks. We show that we are able to learn statistics that can be used for: clustering datasets, transferring generative models to new datasets, selecting representative samples of datasets and classifying previously unseen classes. We refer to our model as a neural statistician, and by this we mean a neural network that can learn to compute summary statistics of datasets without supervision.
@cite_12: In this paper, we propose a conceptually simple and general framework called MetaGAN for few-shot learning problems. Most state-of-the-art few-shot classification models can be integrated with MetaGAN in a principled and straightforward way. By introducing an adversarial generator conditioned on tasks, we augment vanilla few-shot classification models with the ability to discriminate between real and fake data. We argue that this GAN-based approach can help few-shot classifiers to learn sharper decision boundary, which could generalize better. We show that with our MetaGAN framework, we can extend supervised few-shot learning models to naturally cope with unsupervised data. Different from previous work in semi-supervised few-shot learning, our algorithms can deal with semi-supervision at both sample-level and task-level. We give theoretical justifications of the strength of MetaGAN, and validate the effectiveness of MetaGAN on challenging few-shot image classification benchmarks.
@cite_17: Feature augmentation attacks low-shot learning problem by easing the catastrophic forgetting and over-fitting issues in low data regime. However, previous works either erroneously assume that any intra-class variances of base classes are sharable with any novel classes, or employ naive generation methods to hallucinate finite examples without modeling their latent distributions. In this work, we propose Covariance-Preserving Adversarial Augmentation Network for low-shot learning to overcome existing limits. We regulate generation within base-novel class pairs that are semantically adaptable. For better feature generation quality, a new Generative Adversarial Network is designed to model the latent distribution of each novel class given its base counterparts. Since direct estimation on novel classes can be inductively biased, we explicitly preserve covariance information as the variability'' of base examples during generation process. Empirical evidences show that our model can generate realistic yet diverse examples, leading to substantial improvements on the ImageNet benchmark over the state-of-the-art.
These approaches learn to generate artificial examples to compensate the lack of training data. The Neural Statistician approach learns to produce statistics of a dataset, such as mean or variance, which are used to specify a Gaussian distribution for generating data @cite_13 . Other methods introduce generative adversarial networks to learn sharper decision boundaries (MetaGAN) @cite_12 or model the latent distribution of novel classes @cite_17 . These meta-learners generate fake examples to assist few-shot learning tasks. However, these examples could be non-informative when the few training examples are not representative. Conversely, our method learns to aggregate useful information and suppress noisy information, which can be more stable.