Question: Represent the text to find a very short summary (1-2 sentences max)

Question: While tasks could come with varying the number of instances and classes in realistic settings, the existing meta-learning approaches for few-shot classification assume that number of instances per task and class is fixed. Due to such restriction, they learn to equally utilize the meta-knowledge across all the tasks, even when the number of instances per task and class largely varies. Moreover, they do not consider distributional difference in unseen tasks, on which the meta-knowledge may have less usefulness depending on the task relatedness. To overcome these limitations, we propose a novel meta-learning model that adaptively balances the effect of the meta-learning and task-specific learning within each task. Through the learning of the balancing variables, we can decide whether to obtain a solution by relying on the meta-knowledge or task-specific learning. We formulate this objective into a Bayesian inference framework and tackle it using variational inference. We validate our Bayesian Task-Adaptive Meta-Learning (Bayesian TAML) on two realistic task- and class-imbalanced datasets, on which it significantly outperforms existing meta-learning approaches. Further ablation study confirms the effectiveness of each balancing component and the Bayesian learning framework. Despite the success of deep learning in many real-world tasks such as visual recognition and machine translation, such good performances are achievable at the availability of large training data, and many fail to generalize well in small data regimes. To overcome this limitation of conventional deep learning, recently, researchers have explored meta-learning approaches, whose goal is to learn a model that generalizes well over distribution of tasks, rather than instances from a single task, in order to utilize the obtained meta-knowledge across tasks to compensate for the lack of training data for each task. However, so far, most existing meta-learning approaches (; ; ; ; ; have only targeted an artificial scenario where all tasks participating in the multi-class classification problem have equal number of training instances per class. Yet, this is a highly restrictive setting, as in real-world scenarios, tasks that arrive at the model may have different training instances (task imbalance), and within each task, the number of training instances per class may largely vary (class imbalance). Moreover, the new task may come from a distribution that is different from the task distribution the model has been trained on (out-of-distribution task) (See (a) of Figure 1 ). Under such a realistic setting, the meta-knowledge may have a varying degree of utility to each task. Tasks with small number of training data, or close to the tasks trained in meta-training step may want to rely mostly on meta-knowledge obtained over other tasks, whereas tasks that are out-of-distribution or come with more number of training data may obtain better solutions when trained in a task-specific manner. Furthermore, for multi-class classification, we may want to treat the learning for each class differently to handle class imbalance. Thus, to optimally leverage meta-learning under various imbalances, it would be beneficial for the model to task-and class-adaptively decide how much to use from the meta-learner, and how much to learn specifically for each task and class. Head class Figure 1: Concept. (a) To handle task imbalance (Task Imbal.), class imbalance (Class Imbal.) and outof-distribution tasks (OOD) for each task τ, we introduce task-specific balancing variables γ τ, ω τ and z τ, respectively. (b) With those variables, we learn to balance between the meta-knowledge θ and task-specific update to handle imbalances and distributional discrepancies. To this end, we propose a novel Bayesian meta-learning framework, which we refer to as Bayesian Task-Adaptive Meta-Learning (Bayesian TAML), that learns variables to adaptively balance the effect of meta-and task-specific learning. Specifically, we first obtain set-representations for each task, which are learned to convey useful statistics about the task or class distribution, such as mean, variance, tailedness (kurtosis), and skewness, and then learn the distribution of three balancing variables as the function of the set: 1) task-dependent learning rate decay, which decides how far away to deviate from the meta-knowledge, when performing task-specific learning. Tasks with higher shots could benefit from taking gradient steps afar, while tasks with few shots may need to stay close to the initial parameter. 2) class-dependent learning rate, which decides how much information to use from each class, to automatically handle class imbalance where the number of instances per class can largely vary. 3) task-dependent attention mask, which modifies the shared parameter for each task by learning a set-dependent attention mask to it, such that the task can decide how much and what to use from the initial shared parameter and what to ignore based on its set representation. This is especially useful when handling out-of-distribution task, which may need to ignore some of the meta-knowledge. We validate our model on Omniglot and mini-ImageNet dataset, as well as a new dataset that consists of heterogeneous datasets, under a scenario where every class in each episode can have any number of shots, that leads to task and class imbalance, and where the dataset at meta-test time is different from that of meta-training time. The experimental show that our Bayesian TAML significantly improves the performance over the existing approaches under these realistic scenarios. Further analysis of each component reveals that the improvement is due to the effectiveness of the balancing terms for handling task and class imbalance, and out-of-distribution tasks. To summarize, our contribution in this work is threefold: • We consider a novel problem of meta-learning under a realistic task distribution, where the number of instances across classes and tasks could largely vary, or the unseen task at the meta-test time is largely different from the seen tasks. • For effective meta-learning with such imbalances, we propose a Bayesian task-adaptive meta-learning (Bayesian TAML) framework that can adaptively adjust the effect of the meta-learner and the task-specific learner, differently for each task and class. • We validate our model on realistic imbalanced few-shot classification tasks with a varying number of shots per task and class and show that it significantly outperforms existing meta-learning models. Meta-learning Meta-learning is an approach to learn a model to generalize over a distribution of task. The approaches in general can be categorized into either memory-based, metric-based, and optimization-based methods. A memory-based approach learns to store correct instance and label into the same memory slot and retrieve it later, in a task-generic manner. Metric-based approaches learn a shared metric space . defines the distance between the instance and the class prototype, such that the instances are closer to their correct prototypes than to others. As for optimization-based meta-learning, learns a shared initialization parameter that is optimal for any tasks within few gradient steps from the initial parameter. Meta-SGD improves upon MAML by learning the learning rate differently for each parameter. For effective learning of a meta-learner, meta-learning approaches adopt the episodic training strategy which trains and evaluates a model over a large number of tasks, which are called meta-training and meta-test phase, respectively. However, existing approaches only consider an artificial scenario which samples the classification of classes with exactly the same number of training instances, both within each episode and across episodes. On the other hand, we consider a more challenging scenario where the number of shots per class and task could vary at each episode, and that the task given at meta-test time could be an out-of-distribution task. Task-adaptive meta-learning The goal of learning a single meta-learner that works well for all tasks may be overly ambitious and leads to suboptimal performances for each task. Thus recent approaches adopt task-adaptively modified meta-learning models. proposed to learn the temperature scaling parameter to work with the optimal similarity metric. also suggested a model that generates task-specific parameters for the network layers, but it only trains with many-shot classes, and implicitly expects generalization to few-shot cases. proposed a network type task-specific parameter producer, and proposed to differentiate the network weights into task-shared and task-specific weights. Our model also aims to obtain task-specific parameter for each task, but is rather focused on learning how to balance between the meta-learning and task-/class-specific learning. To our knowledge, none of the existing approaches explicitly tackle this balancing problem since they only consider few-shot learning with the fixed number of instances for each class and task. Probabilistic meta-learning Recently, a probabilistic version of MAML has been proposed , where they interpret a task-specific gradient update as a posterior inference process under variational inference framework. proposed Bayesian MAML with a similar motivation but with a stein variational inference framework and chaser loss. proposed a probabilistic meta-learning framework where the paramter for a novel task is rapidly estimated under decision theoretic framework, given a set representation of a task. The motivation behind these works is to represent the inherent uncertainty in few-shot classification tasks. Our model also uses Bayesian modeling, but it focuses on leveraging the uncertainties of the meta-learner and the gradient-direction in order to balance between meta-and task-or class-specific learning. We first introduce notations and briefly recap the model-agnostic meta-learning (MAML) by. Suppose a task distribution p(τ) that randomly generates task τ consisting of a training set Then, the goal of MAML is to meta-learn the initial model parameter θ as a meta-knowledge to generalize over the task distribution p(τ), such that we can easily obtain the task-specific predictor θ τ in a single (or a few) gradient step from the initial θ. Toward this goal, MAML optimizes the following gradient-based meta-learning objective: where α denotes stepsize and L denotes empirical loss such as negative log-likelihood of observations. Note that by meta-learning the initial point θ, the task-specific predictor θ even with D τ which only contains few samples. We can easily extend the Eq., such that we obtain θ τ with more than one inner-gradient steps from the initial θ. However, the existing MAML framework has the following limitations that prevent the model from efficiently solving real-world problems involving task/class imbalance and out-of-distribution tasks. 1. Task imbalance. MAML has a fixed number of inner-gradient steps and stepsize α across all tasks, which prevents the model from adaptively deciding how much to use from the meta-knowledge depending on the number of the training examples per task. 2. Class imbalance. The model does not provide any framework to handle class imbalance within each task. Therefore, classes with large number of training instances (head classes) may dominate the task-specific learning during the inner-gradient steps, yielding low performance on classes with fewer shots (tail classes). 3. Out-of-distribution tasks. The model assumes that the meta-knowledge will be equally useful for the unseen tasks, but for unseen tasks that are out-of-distribution, the metaknowledge may be less useful. As shown in Figure 1 for the concepts, we introduce three balancing variables γ τ, ω τ, z τ to tackle each problem mentioned above. How to compute these variables will be described in Section 4. In order to learn with realistic scenarios, we assume that the task distribution p(τ) samples some fixed number of C classes ("way"), and then sample uniform-random number of instances for each class ("shots"), thereby simulating both task and class imbalance at the same time. Tackling task imbalance. To control whether to stay close to the initial parameter or deviate far from it, we introduce a clipping function f (·) = max(0, min(·, 1)) and a task-dependent learning-rate decaying factor f (γ τ), such that the learning rate exponentially decays as to be large for large tasks, such that they rely more on task-specific updates, while small tasks use small f (γ τ) to benefit from the meta-knowledge. Tackling class imbalance. To handle class imbalance, we vary the learning rate of classspecific gradient update for each task-specific gradient update step. Specifically, for class c = 1,..., C, we introduce a non-negative activation function g(·) = SoftPlus(·) and a set of classspecific non-negative scalars g(ω is the set of instances and labels for class c. We expect g(ω τ c) to be large for tail-classes to consider them more in task-specific gradient updates. Tackling out-of-distribution tasks. Lastly, we introduce an additional task-dependent variable z τ with the non-negative activation function g(·) which weights the initial parameter θ according to the usefulness for each task. We expect the variable g(z τ) to heavily emphasize the meta-knowledge θ when D τ is similar to the trained dataset, and use less of it when D τ is unfamilar. This behavior can be implemented with Bayesian modeling on the latent z τ, which we introduce in the next subsection. A unified framework. Finally, we assemble all these components together into a single unified framework. The update rule for the task-specific θ τ is recursively defined as follows: where the last step θ K corresponds to the task-specific predictor θ τ and α is a multi-dimensional global learning rate vector that is learned such as. As previously mentioned, we need a Bayesian framework for modeling z τ, since it needs a prior in order to prevent the posterior of z τ from overly utilizing the meta-knowledge θ when the task is out-ofdistribution. Moreover, for the learning of balancing variables γ τ and ω τ, Bayesian modeling improve the quality of the inference on them, which we empirically verified through extensive experiments. We allow the three variables to share the same inference network pipeline to minimize the computational cost, and thereby effectively amortize the inference rule across variables as well. for training, and for test. Let φ τ denote the collection of three latent variables, γ τ, ω τ and z τ for uncluttered notation. Then, the generative process is as follows for each task τ (See Figure 2): for the complete data likelihood. Note that the deterministic θ is shared across all the tasks. The goal of learning for each task τ is to maximize the log-likelihood of the joint datasetD τ and. However, solving it involves the true posterior p(φ τ |D τ,D τ), which is intractable. Thus, we resort to amortized variational inference with a tractable form of approximate posterior q(φ τ |D τ,D τ ; ψ) parameterized by ψ. Further, similarly to , we drop the dependency on the test datasetD τ for the approximate posterior, in order to make the two different pipelines consistent; one for meta-training where we observe the whole test dataset, and the other for meta-testing where the test labels are unknown. The form of our approximate posterior is now q(φ τ |D τ ; ψ). It greatly simplifies the inference framework, while ensuring that the following objective is still a valid lower bound of the log evidence. Also, considering that performing the inner-gradient steps with the training dataset D τ automatically maximizes the training log-likelihood in MAML framework, we slightly modify the objective so that the expected loss term only involves the test examples. The ant form of the lower bound that suits for our meta-learning purpose is as follows: We assume q(φ τ |D τ ; ψ) fully factorizes for each variable and also for each dimension as well: where we assume that each single dimension of q(φ τ |D τ ; ψ) follows univariate gaussian having trainable mean and variance. We also let each dimension of prior p(φ τ) factorize into N. The KL-divergence between two univariate gaussians has a simple closed form , thereby we obtain the low-variance estimator for the lower bound L τ θ,ψ. The final form of the meta-training minimization objective with Monte-Carlo approximation for the expection in is as follows:, and T is the number of tasks. We implicitly assume the reparameterization trick for φ τ to obtain stable and unbiased gradient estimate w.r.t. ψ . We set the number of MC samples to S = 1 for meta-training for computational efficiency. When meta-testing, we can set S = 10 or naively approximate the expectation by taking the expectation inside:, which works well in practice. The main challenge in modeling our variational distribution q(φ τ |D τ ; ψ) is how to refine the training dataset D τ into informative representation capturing the dataset as a distribution, which is not trivial. This inference network should capture all the necessary statistical information in the dataset D τ to solve both imblanace and out-of-distribution problems. DeepSets is frequently used as a practical set-encoder, where each instance in the set is transformed by the shared nonlinearity, and then summed together to generate a single vector summarizing the set. However, for the classification dataset D τ which is the set of (class) sets, we cannot use DeepSets directly as it will completely ignore the label information. Therefore, we need to stack the structure of DeepSets twice according to the hierarchical set of sets structure of classification dataset. However, there exists additional limitation of DeepSets with sum-pooling when describing the distribution. Suppose that we have a set containing a replication of single instance. Then, its representation will change based on the number of replications, although distribution-wise all sets should be the same. Mean-pooling may alleviate the problem; however, it does not recognize the number of elements in the set, which is a critical limitation in encoding imbalance. To overcome the limitations of the two pooling methods, we propose to use higher-order statistics in addition to the sample mean, namely element-wise sample variance, skewness and kurtosis. For instance, the sample variance could capture task imbalance and skewness will capture class imbalance (imbalance in the number of instances per class). Based on this intuition, we propose the following encoder network StatisticsPooling(·) that generates the concatenation of those statistics (See Figure 3): for classes c = 1,..., C, and X τ c is the collection of class c examples in task t. NN 1 and NN 2 are some appropriate neural networks parameterized by ψ. The vector v τ finally summarizes the whole classification dataset D τ and our balancing variables γ τ, ω τ and z τ are generated from it with an additional affine transformation. See Appendix B for the justification. We validate our method in imbalanced scenarios, where each task, or every class within a task can have different shots, and the tasks at the evaluation time could come from a different task distribution from the seen task distribution. , we use 4-block convolutional neural networks with 64 channels for each layer for Omniglot and MNIST. We reduce the number of channels into 32 for other datasets. Imbalanced Omniglot. This dataset consists of 1623 hand-written character classes, with 20 training instances per class. We consider 10-way classification problems, where we have 5 queries per each class. To generate imbalanced tasks, we randomly set the number of training instances to be sampled within the range of 1 to 15. We set 5 inner-gradient steps for all gradient-based models and train all models on Omniglot, and evaluate on both Omniglot and MNIST, where the latter is used to evaluate the performance on out-of-distribution task. Imbalanced tiered-ImageNet. This is sub-sampled ImageNet dataset including 608 classes . As similarly with the Imbalanced Omniglot dataset, we consider 5-way classification problems, while randomly setting the number of training instances per class within the range of 1 to 50, and use 15 instances per class for test (queries). We set 5 inner-gradient steps for all gradient-based models and train all models on the tiered-ImageNet dataset, and evaluate the model on the test split of tiered-ImageNet and mini-ImageNet, where the latter is used to evaluate on out-of-distribution task. See the Appendix A for more details of the experimental setup. Analysis. Multi-Dataset OVD. We further test our model under a more challenging setting where tasks could come from a highly heterogeneous dataset. To this end, we combine Omniglot, VGG flower , DTD into a single dataset OVD, and randomly sample each class from the combined dataset for every task. We train all models with 10-way any-shot tasks with 3 inner gradient steps and test on OVD and FasionMNIST , where the latter is used to generate out-of-distribution tasks at evaluation time. The in Table 2 shows that under this challenging multi-dataset setting, our Bayesian TAML outperforms all baselines, especially with larger gains on the out-of-distribution tasks (Fashion MNIST) consistent with the of Table 1. We now validate the effectiveness of each balancing variable. For all of the ablations, we set the meta-training condition to Omniglot 5-way any-shot (i.e. 1-to 15-shot) classification with 5 innergradient steps. For the meta-testing condition for each ablation, see each of the tables below. Further, in order to correctly evaluate each variable, we add in each individual component to Meta-SGD, one at a time, dropping all other balancing variables, for both meta-training and meta-testing. We report mean accuracies over 1000 random episodes with 95% confidence intervals. g(z τ) for handling distributional discrepancy. g(z τ) modulates the initial model parameter θ, deciding what and how much to use from the meta-knowledge θ based on the relatedness between θ and the task at hand. Table 3 shows that by adding in Bayesian g(z τ) component to Meta-SGD, we can effectively handle out-of-distribution tasks (MNIST) by significant margin. The histogram in Figure 4 shows that the actual distribution of mask g(z τ) of OOD tasks is more skewed toward zero than the distribution of ID tasks, which agrees with our expectation. f (γ τ) for handling task imbalance. f (γ τ), which is a decaying factor for inner gradient steps, handles inter-task imbalance where each task has different number of examples. Figure 5 shows f (γ τ) values w.r.t. the task size, where it increases monotonically with the number of instances allowing the model to stay close to the initial parameter for few-shot cases and deviate far from it for many-shot cases. Table 4 shows that the larger gains of our model for 1-shot than 5 or 15 shots support that relying on meta-knowledge is useful for improving the performance on smaller tasks. g(ω τ) for handling class imbalance. g(ω τ) rescales the class-specific gradients to handle class imbalance where the number of instances per class largely varies. Table 5 shows the under the varying degree of class imbalance across the task distribution. The degree of class imbalance (×N) means that the maximum number of shots is N times larger than the minimum number of shots within the given task. We observe that our model significantly outperforms baselines especially under the high degree of class imbalance (×5 and ×15). Figure 6 shows that g(ω τ) actually increases the gradient scale of tail-classes (classes with fewer instances), so that we obtain the big improvements on those tail classes. Effectiveness of Bayesian modeling We further demonstrate the effectiveness of Bayesian modeling by comparing it with the deterministic version of our model (Deterministic TAML), where three balancing variables are no longer stochastic and we apply 2 regularization of 10 −3 on them instead of KL-divergence in Eq. 5. Table 6 shows the under the same setting of Table 3. The clearly show that the Bayesian modeling greatly contribute to addressing the imbalance problem, especially for the OOD tasks (MNIST). Figure 7 further shows that our balancing variables g(ω τ) and f (γ τ), that are responsible for handling class/task imbalance, more sensitively react to the actual imbalance conditions with the Bayesian modeling (Bayesian TAML) than without Bayesian (Deterministic TAML). Dataset encoding Lastly, we perform an ablation study to validate the effectiveness of the proposed dataset encoding scheme, Set of Sets, for generating the balancing variables. Table 7 shows the performance of various encoding schemes on the imbalanced tiered-ImageNet 5-way classification, the same setting as in Table 1. We see that Set of Sets, equipped with higher-order statistics and hierarchical set encoding, is far more effective than simple mean-pooling method (; ;). We propose Bayesian TAML that learns to balance the effect of meta-learning and task-adaptive learning, to consider meta-learning under a more realistic task distribution where each task and class can have varying number of instances. Specifically, we encode the dataset for each task into hierarchical set-of-sets representations, and use it to generate attention mask for the original parameter, learning rate decay, and the class-specific learning rate. We use a Bayesian framework to infer the posterior of these balancing variables, and propose an effective variational inference framework to solve for them. Our model outperforms existing meta-learning methods when validated on imbalanced few-shot classification tasks. Further analysis of each balancing variable shows that each variable effectively handles task imbalance, class imbalance, and out-of-distribution tasks respectively. We believe that our work makes a meaningful step toward application of meta-learning to real-world problems. A EXPERIMENTAL SETUP A.1 BASELINES AND NETWORK ARCHITECTURE. We describe baseline models and our task-adaptive learning to balance model. Note that all gradientbased models can be extended to take K inner-gradient steps for both meta-training and meta-testing. 1) Meta-Learner LSTM. A meta-learner that learns optimization algorithm with LSTM . The model performs few-shot classification using cosine similarities between the embeddings generated from a shared convolutional network. 2) Prototypical Networks. A metric-based few-shot classification model proposed by . The model learns the metric space based on Euclidean distance between class prototypes and query embeddings. 3) MAML. The Model-Agnostic Meta-Learning (MAML) model by , which aims to learn the global initial model parameter, from which we can take a few gradient steps to get task-specific predictors. A base MAML with the learnable learning-rate vector (without any restriction on sign) element-wisely multiplied to each step inner-gradient. A gradient-based meta-learning model proposed by. The model obtains a task-specific parameter only w.r.t. a subset of the whole dimension (M-Net), followed by a linear transformation to learn a metric space (T-Net). 6) Probabilistic MAML. A probabilistic version of MAML by , where they model task-adaptive inner-gradient steps as a posterior inference process under hierarchical Bayesian framework. This model also interprets MAML under hierarchical Bayesian framework, but they propose to share and amortize the inference rules across both global initial parameters as well as the task-specific parameters. 8) Bayesian TAML. Our learning to balance model that can adaptively balance between meta-and task-specific learners for each task and class. A.2 REALISTIC ANY-SHOT CLASSIFICATION. We describe more detailed settings for realistic any-shot classification. Imbalanced Omniglot. We modified the episode generating strategy of C-way classification, which selects the number of shots randomly between 1 to 15 for each of the classes. The metalearning rate β and the total number of iterations are set to 1e-3 and 60000, respectively for all models, and the inner gradient step size α is set to 0.05 for MAML, MT-NET, and ABML and is set to learnable parameters for Meta-SGD and our model. The number of inner-gradient steps is 5 for all gradient-based models and all other components are the same as reported for fixed-way and fixed-shot few-shot classification in the paper for each model. We keep the meta-batch as 1 for all experiments to clearly see the effect of imbalance scenario. We trained models in 10-way 5 inner-gradient steps on the Omniglot, and evaluated with the test split of Omniglot and MNIST. Imbalanced tiered-ImageNet. We modified the episode generating strategy of C-way classification, which selects the number of shots randomly between 1 to 50 for each of the classes. We set the number of query points as 15 and the meta learning rate β is set to 1e-4. Other components are set to the same as referred in Imbalanced Omniglot. We trained models on the tiered-ImageNet, and evaluated with the test split of tiered-ImageNet and mini-ImageNet. We describe the network architecture of the inference network that takes a classification dataset as an input and generates three balancing variables as output. We additionally used two average pooling with 2 × 2 strides before the shared encoder NN 1 with large inputs such as tiered-ImageNet and mini-ImageNet. We empirically found that attaching average pooling reduces computation cost while improving performance. Based on the previous justification of DeepSets , we can easily justify the Set-ofSets structure proposed in the main paper as well, in terms of the two-level permutation invariance properties required for any classification dataset. The main theorem of DeepSets is: Theorem 1. A function f operating on a set X ∈ X is a valid set function (i.e. permutation invariant), iff it can be decomposed as f (X) = ρ 2 (x∈X ρ 1 (x)), where ρ 1 and ρ 2 are appropriate nonlinearities. See for the proof. Here we apply the same argument twice as follows. 1. A function f operating on a set of representations {s 1, . . ., s C} (we assume each s c is an output from a shared function g) is a valid set function (i.e. permutation invariant w.r.t. the order of {s 1, . . ., s C}), iff it can be decomposed as f ({s 1, . . ., s C}) = ρ 2 (C c=1 ρ 1 (s c)) with appropriate nonlinearities ρ 1 and ρ 2. 2. A function g operating on a set of examples {x c,1, . . ., x c,N} is a valid set function (i.e. permutation invariant w.r.t. the order of {x c,1, . . ., x c,N}) iff it can be decomposed as ) with appropriate nonlinearities ρ 3 and ρ 4. Inserting s c = g({x c,1, . . ., x c,N}) into the expression of f, we arrive at the following valid composite function operating on a set of sets: Let F denote the composite of f and (multiple) g and let NN 2 denote the composite of ρ 1 and ρ 4. Further define NN 1:= ρ 3 and NN 3:= ρ 2. Then, we have where C is the number of classes and N is the number of examples per class. See Section A.3 for the correspondence between Eq. and the actual encoder structure. We provide the comparison between the two approximation schemes for evaluating the expectation of the test example predictions at meta-testing time. Naive approximation means that we take the expectation inside (i.e. we do not sample) and MC approximation means that we perform MonteCarlo integration with sample size S = 10 1. We see from the Table 8 that MC ingetration performs better than the naive approximation, especially with OOD tasks (e.g. MNIST, mini-ImageNet). This is because the predictive distributions involve higher uncertainty for OOD tasks, hence there exists more benefit from considering the large variance than simply ignoring it. and tiered-ImageNet(right) with imbalanced setting. All reported are average performances over 1000 for Omniglot and MNIST and 600 for tiered-ImageNet and mini-ImageNet randomly selected episodes with standard errors for 95% confidence interval over tasks. We further compare our model with the existing meta-learning approaches on conventional few-shot classification with fixed-way and fixed-shot. Omniglot. We report the 20-way classification performance for this dataset. , we use 4-block convolutional neural network architecture with 64 channels for each layer. We set the number of inner-gradient steps K to 5 for both meta-training and meta-testing. Mini-ImageNet. We report the 5-way classification performance with the meta-batch 4 and 2 for 1-and 5-shot, respectively. We reduce the convolution filter size of the 4-block CNN network into 32 to prevent overfitting. We set K = 5 for multi-step models for both meta-training and meta-testing. 95.80 ± 0.30 98.90 ± 0.20 48.70 ± 1.84 63.11 ± 0.92 Meta-SGD 95. 97.60 ± 0.20 99.10 ± 0.10 50.44 ± 0.82 65.32 ± 0.70 MT-NET 96.20 ± 0.40 -51.70 ± 1.84 -Probabilistic MAML --50.13 ± 1.86 -Reptile 89.43 ± 0.14 97.12 ± 0.32 49.97 ± 0.32 65.99 ± 0.58 BMAML --53.80 ± 1.46 -VERSA 97. We first compare our method on conventional fixed-way fixed-shot classification task against existing meta-learning methods. Though the classification with uniformly distributed instances is not the task we aim to tackle, we find that Bayesian TAML outperforms most baseline models, except for the mini-imagenet 5-way 1-shot experiment where BMAML works the best. Table 9: Classification to θ0 inititalization with g(z). θ 0 inititalization with g(z). Applying g(z) to the shared initial parameter θ may set some of the initial parameters to smaller values. Thus, we further compare our model with applying the linear interpolation between θ and a random initialization vector θ random with coefficient g(z) as θ 0 = g(z) • θ + (1 − g(z) • θ random ) to Bayesian TAML (Bayesian g(z) TAML + Interpolation). As shown in the Table 9, those models has no significant difference in the performance. 1 At meta-training time, we perform MC approximation with a single sample for computational efficiency 2 We adopt the accuracies of the Prototypical Network in the setting which the number of shot and way are the same for training and testing phase for the consistency with other methods. In the "higher way" setting where 20-way is used during training for 5-way testing, the reported performance of the model is 68.20 ± 0.66%.

represent the tldr for finding the corresponding scientific article.

A novel meta-learning model that adaptively balances the effect of the meta-learning and task-specific learning, and also class-specific learning within each task.

Represent the TLDR for finding the corresponding scientific article!

We present a new approach, SNIP, that is simple, versatile and interpretable; it prunes irrelevant connections for a given task at single-shot prior to training and is applicable to a variety of neural network models without modifications.


Question: Represent the article for finding a single-sentence TLDR

Question: Many tasks in artificial intelligence require the collaboration of multiple agents. We exam deep reinforcement learning for multi-agent domains. Recent research efforts often take the form of two seemingly conflicting perspectives, the decentralized perspective, where each agent is supposed to have its own controller; and the centralized perspective, where one assumes there is a larger model controlling all agents. In this regard, we revisit the idea of the master-slave architecture by incorporating both perspectives within one framework. Such a hierarchical structure naturally leverages advantages from one another. The idea of combining both perspective is intuitive and can be well motivated from many real world systems, however, out of a variety of possible realizations, we highlights three key ingredients, i.e. composed action representation, learnable communication and independent reasoning. With network designs to facilitate these explicitly, our proposal consistently outperforms latest competing methods both in synthetics experiments and when applied to challenging StarCraft micromanagement tasks. Reinforcement learning (RL) provides a formal framework concerned with how an agent takes actions in one environment so as to maximize some notion of cumulative reward. Recent years have witnessed successful application of RL technologies to many challenging problems, ranging from game playing [17; 21] to robotics BID8 and other important artificial intelligence (AI) related fields such as BID19 etc. Most of these works have been studying the problem of a single agent. However, many important tasks require the collaboration of multiple agents, for example, the coordination of autonomous vehicles BID1, multi-robot control BID12, network packet delivery BID31 and multi-player games BID24 to name a few. Although multi-agent reinforcement learning (MARL) methods have historically been applied in many settings [1; 31], they were often restricted to simple environments and tabular methods. Motivated from the success of (single agent) deep RL, where value/policy approximators were implemented via deep neural networks, recent research efforts on MARL also embrace deep networks and target at more complicated environments and complex tasks, e.g. [23; 19; 4; 12] etc. Regardless though, it remains an open challenge how deep RL can be effectively scaled to more agents in various situations. Deep RL is notoriously difficult to train. Moreover, the essential state-action space of multiple agents becomes geometrically large, which further exacerbates the difficulty of training for multi-agent deep reinforcement learning (deep MARL for short).From the viewpoint of multi-agent systems, recent methods often take the form of one of two perspectives. That is, the decentralized perspective where each agent has its own controller; and the centralized perspective where there exists a larger model controlling all agents. As a consequence, learning can be challenging in the decentralized settings due to local viewpoints of agents, which perceive non-stationary environment due to concurrently exploring teammates. On the other hand, under a centralized perspective, one needs to directly deal with parameter search within the geometrically large state-action space originated from the combination of multiple agents. BID0 StarCraft and its expansion StarCraft: Brood War are trademarks of Blizzard Entertainment TM In this regard, we revisit the idea of master-slave architecture to combine both perspectives in a complementary manner. The master-slave architecture is a canonical communication architecture which often effectively breaks down the original challenges of multiple agents. Such architectures have been well explored in multi-agent tasks [18; 28; 15; 16]. Although our designs vary from these works, we have inherited the spirit of leveraging agent hierarchy in a master-slave manner. That is, the master agent tends to plan in a global manner without focusing on potentially distracting details from each slave agent and meanwhile the slave agents often locally optimize their actions with respect to both their local state and the guidance coming from the master agent. Such idea can be well motivated from many real world systems. One can consider the master agent as the central control of some organized traffic systems and the slave agents as each actual vehicles. Another instantiation of this idea is to consider the coach and the players in a football/basketball team. However, although the idea is clear and intuitive, we notice that our work is among the first to explicitly design master-slave architecture for deep MARL.Specifically, we instantiate our idea with policy-based RL methods and propose a multi-agent policy network constructed with the master-slave agent hierarchy. For both each slave agent and the master agent, the policy approximators are realized using recurrent neural networks (RNN). At each time step, we can view the hidden states/representations of the recurrent cells as the "thoughts" of the agents. Therefore each agent has its own thinking/reasoning of the situation. While each slave agent takes local states as its input, the master agent takes both the global states and the messages from all slave agents as its input. The final action output of each slave agent is composed of contributions from both the corresponding slave agent and the master agent. This is implemented via a gated composition module (GCM) to process and transform "thoughts" from both agents to the final action. We test our proposal (named MS-MARL) using both synthetic experiments and challenging StarCraft micromanagement tasks. Our method consistently outperforms recent competing MARL methods by a clear margin. We also provide analysis to showcase the effectiveness of the learned policies, many of which illustrate interesting phenomena related to our specific designs. In the rest of this paper, we first discuss some related works in Section 2. In Section 3, we introduce the detailed proposals to realize our master-slave multi-agent RL solution. Next, we move on to demonstrate the effectiveness of our proposal using challenging synthetic and real multi-agent tasks in Section 4. And finally Section 5 concludes this paper with discussions on our findings. Before proceeding, we summarize our major contributions as follows• We revisit the idea of master-slave architecture for deep MARL. The proposed instantiation effectively combines both the centralized and decentralized perspectives of MARL.• Our observations highlight and verify that composable action representation, independent master/slave reasoning and learnable communication in-between are key factors to be successful in MS-MARL.• Our proposal empirically outperforms recent state-of-the-art methods on both synthetic experiments and challenging StarCraft micromanagement tasks, rendering it a novel competitive MARL solution in general. Current main stream RL methods apply conventional wisdoms such as Q-learning, policy gradient, actor-critic etc. BID23. Recent progress mainly focuses on practical adaptations especially when applying deep neural networks as value/policy approximators. BID9 provides a recent review on deep RL.Although MARL has been studied in the past, they have been focused on simple tasks BID0. Only until recently, with the encouragement from the successes of deep RL, deep MARL has become a popular research area targeting at more complex and realistic tasks, see e.g. [3; 23; 7; 19; 4; 12] etc. and BID6 are among the first to propose learnable communications via back-propagation in deep Qnetworks. However, due to their motivating tasks, both works focused on a decentralized perspective and usually applies to only a limited number of agents., BID4 and BID18 all proposed practical network structure or training strategies from a centralized perspective of MARL. Specifically, BID18 proposed a bidirectional communication channel among all agents to facilitate effective communication and many interesting designs toward the StarCraft micromanagement tasks. BID26 proposed episodic exploration strategy for deterministic policy search and BID4 proposed the concept of stabilizing experience replay for MARL.Note that the above works take only one of the two perspectives and are then inherently missing out the advantages of the other. Perhaps the most related works are from BID2, BID3 and BID11. BID22 proposed the "CommNet", where a broadcasting communication channel among all agents was set up to share global information realized as summation of the output from all individual agents. This design represents an initial version of the proposed master-slave framework, however it does not facilitate an independently reasoning master agent which takes in messages from all agents step by step and processes such information in an recurrent manner. In BID3 and BID11, a global critic was proposed, which could potentially work at a centralized level, however since critics are basically value networks, they do not provide explicit policy guidance. Therefore they tend to work more like a commentator of a game who job is to analyze and criticize the play, rather than a coach coaching the game. As discussed above, the master-slave architecture has already been studied in several multi-agent scenarios. BID17 utilized the master-slave architecture to resolve conflicts between multiple soccer agents; while [28; 15; 16] explored master-slave hierarchy in RL applied to load-balancing and distributed computing environments. Our proposal can be viewed as a revisit to similar ideas for deep MARL. With the proposed designs, we facilitate independent master reasoning at a global level and each slave agent thinking at a local but focused scale, and collectively achieve optimal rewards via effective communication learned with back propagation. Compared with existing works, we emphasize such independent reasoning, the importance of which are well justified empirically in the experiments. We consistently outperform existing MARL methods and achieve state-of-the-art performance on challenging synthetic and real multi-agent tasks. Since the master-slave architecture constructs agent hierarchy by definition, another interesting related field is hierarchical RL, e.g. [8; 29]. However, such hierarchical deep RL methods studies the hierarchy regarding tasks or goals and are usually targeting at sequential sub-tasks where the meta-controller constantly generates goals for controllers to achieve. Master-slave architecture, on the other hand, builds up hierarchy of multiple agents and mainly focuses on parallel agent-specific tasks instead, which is fundamentally different from the problems that hierarchical RL methods are concerned with. We start by reiterating that the key idea is to facilitate both an explicit master controller that takes the centralized perspective and organize agents in a global or high level manner and all actual slave controllers work as the decentralized agents and optimize their specific actions relatively locally while depending on information from the master controller. Such an idea can be realized using either value-based methods, policy-based methods or actor-critic methods. Hereafter we focus on introducing an instantiation with policy gradient methods as an example, which also represents the actual solution in all our experiments. In particular, our target is to learn a mapping from states to actions π θ (a t |s t) at any given time t, where s = {s m, s 1, ..., s C} and a = {a 1, ..., a C} are collective states and actions of C agents respectively and θ = {θ m, θ 1, ..., θ C} represents the parameters of the policy function approximator of all agents, including the master agent θ m. Note that we have explicitly formulated s m to represent the independent state to the master agent but have left out a corresponding a m since the master's action will be merged with and represented by the final actions of all slave agents. This design has two benefits: 1) one can now input independent and potentially more global states to the master agent; and meanwhile 2) the whole policy network can be trained end-to-end with signals directly coming from actual actions. In FIG0, we illustrate the whole pipeline of our master-slave multi-agent architecture. Specifically, we demonstrate the network structure unfolded at two consecutive time steps. In the left part, for example, at time step t, the state s consists of each s i of the i-th slave agent and s m = o t of the master agent. Each slave agent is represented as a blue circle and the master agent is represented as a yellow rectangle. All agents are policy networks realized with RNN modules such as LSTM BID5 cells or a stack of RNN/LSTM cells. Therefore, besides the states, all agents also take the hidden state of RNN h t−1 as their inputs, representing their reasoning along time. Meanwhile the master agent also take as input some information from each slave agent c i and broadcasts back its action output to all agents to help forming their final actions. These communications are represented via colored connections in the figure. To merge the actions from the master agent and those from the slave agents, we propose a gated composition module (GCM), whose behavior resembles LSTM. Figure 4 illustrates more details. Specifically, this module takes the "thoughts" or hidden states of the master agent h, which later will be added to independent action proposals from the corresponding slave agents a i t. Since such a module depends on both the "thoughts" from the master agent and that from certain slave agent, it facilitates the master to provide different action proposals to individual slave agents. Moreover, we demonstrate in 4 that the GCM module naturally facilitates our model to generalize to cases involving heterogeneous agents. In certain cases, one may also want the master to provide unified action proposals to all agents. This could easily be implemented as a special case where the gate related to the slave's "thoughts" shuts down, which is denoted as regular MS-MARL. As mentioned above, due to our design, learning can be performed in an end-to-end manner by directly applying policy gradient in the centralized perspective. Specifically, one would update all parameters following the policy gradient theorem BID23 as DISPLAYFORM0 where data samples are collected stochastically from each episode {s 1, a 1, r 2, ..., s T −1, a T −1, r T} ∼ π θ and v t = t j=1 r t. Note that, for discrete action space, we applied softmax policy on the top layer and for continous action space, we adopted Gaussian policy instead. We summarize the whole algorithm in Algorithm 1 as follows. DISPLAYFORM1 Update network parameter θ m, θ 1,..., θ C using; As stated above, our MS-MARL proposal can leverage advantages from both the centralized perspective and the decentralized perspective. Comparing with the latter, we would like to argue that, not only does our design facilitate regular communication channels between slave agents as in previous works, we also explicitly formulate an independent master agent reasoning based on all slave agents' messages and its own state. Later we empirically verify that, even when the overall information revealed does not increase per se, an independent master agent tend to absorb the same information within a big picture and effectively helps to make decisions in a global manner. Therefore compared with pure in-between-agent communications, MS-MARL is more efficient in reasoning and planning once trained. On the other hand, when compared with methods taking a regular centralized perspective, we realize that our master-slave architecture explicitly explores the large action space in a hierarchical way. This is so in the sense that if the action space is very large, the master agent can potentially start searching at a coarse scale and leave the slave agents focus their efforts in a more fine-grained domain. This not only makes training more efficient but also more stable in a similar spirit as the dueling Q-network design BID29, where the master agent works as base estimation while leaving the slave agents focus on estimating the advantages. And of course, in the perspective of applying hierarchy, we can extend master-slave to master-master-slave architectures etc. To justify the effectiveness of the proposed master-slave architecture, we conducted experiments on five representative multi-agent tasks and environments in which multiple agents interact with BID21 and originally proposed in BID22, (c)-(f) are mini-games from StarCraft that is included in the TorchCraft platform BID24 each other to achieve certain goals. These tasks and environments have been widely used for the evaluation of popular MARL methods such as [23; 12; 19; 27; 4; 5].The first two tasks are the traffic junction task and the combat task both originally proposed in BID22.(Examples are shown in FIG2 (a)(b)) These two tasks are based on the MazeBase environment BID21 and are discrete in state and action spaces. Detailed descriptions are given below:The traffic junction task As originally designed in BID22, there are three versions of this task corresponding to different hard levels. In our experiments, we choose the hardest version which consists of four connected junctions of two-way roads (as shown in FIG2). During every time step, new cars enter the grid area with a certain probability from each of the eight directions. Each car occupies a single-cell in the grid. All the cars keep to the right-hand side of the road. Note that there are only three possible routes for each car. A car is allowed to perform two kinds of actions at each time step: advances by one cell while keeping on its route or stay still at the current position. Once a car moves outside of the junction area, it will be removed. A collision happens when two cars move to the same location. In our experiments, following the literature, we apply the same state and reward originally designed in BID22.The combat task This environment simulates a simple battle between two opposing teams. The entire area is a 15 × 15 grid as shown in FIG2. Both teams have 5 members and will be born at a random position. Possible actions for each agent are as follow: move one step towards four directions; attack an enemy agent within its firing range; or keep idle. One attack will cause an agent lose 1 point of health. And the initial health point of each agent is set to 3. If the health point reduces to 0, the agent will die. One team will lose the battle if all its members are dead, or win otherwise. Default settings from BID22 are used in our experiments following the literature. The following StarCraft micromanagement tasks originate from the well-known real-time strategy (RTS) game: StarCraft and was originally defined in [25; 27]. Instead of playing a full StarCraft game, the micromanagement tasks involve a local battle between two groups of units. Similar to the combat task, the possible actions also include move and attack. One group wins the game when all the units of the other group are eliminated. However, one big difference from the combat task is that the StarCraft environment is continuous in state and action space, which means a much larger search space for learning. In other words, the combat task can also be considered as a simplified version of StarCraft micromanagement task with discrete states and actions. FIG2 (c)-(e)). All the three tasks are categorized as "hard combats" in BID18. Thus it is quite hard for an AI bot to win the combat without learning a smart policy. On all the three tasks, independent policy methods have been proved to be less effective. Instead, MARL methods must learn effective collaboration among its agents to win these tasks e.g. [19; 27]. In this task, one needs to control 15 Terran marines to defeat a builtin AI that controls 16 Terran marines (showcased in FIG2). Note that the Marines are ranged ground units. Among the "mXvY" tasks defined in BID24, such as m5v5 (5 Marines vs. 5 Marines), m10v10 (10 Marines vs. 10 Marines) m18v18 (18 Marines vs. 18 Marines), the chosen combat "15 Marines vs. 16 Marines" represents a more challenging version in the sense that the total number of agents is high and that the controlled team has less units than the enemies. Therefore a model need to learn very good strategies to win the battle. As described in BID26, the key to winning this combat is to focus fire while avoiding "overkill". Besides this, another crucial policy -"Spread Out" is also captured in our experiments.10 Marines vs. 13 Zerglings While we still control Marines in this scenario, the enemies belong to another type of ground unit -Zerglings. Unlike Terran Marines, Zerglings can only attack by direct contact, despite their much higher moving speed. Useful strategies also include "focus fire without overkill", which is similar to the "15 Marines vs. 16 Marines" task. Interestingly, unlike the "15M vs. 16M" task, the "Spread Out" strategy is not effective anymore in this case according to our experiments. (An example of this task is given in FIG2 (d))15 Wraiths vs. 17 Wraiths As a contrast to the above two tasks, this one is about Flying units. 15 Wraiths (ranged flying unit) are controlled to fight against 17 opponents of the same type. An important difference from the ground units is that the flying units will not collide with each other. Hence it is possible for flying units to occupy the same tile. In this case, the "Spread Out" strategy may not be important anymore. But it is essential to avoid "overkill" in this task since Wraiths have much longer "cooldown" time and much higher attack damage. (This task is shown in FIG2 (e))Dragoons and Zealots Unlike the previous three StarCraft tasks, this one involves a heterogeneous multi-agent task where 2 Dragoons(attack from remote) and 3 Zealots(attack by direct contact) need to fight against a group of enemies of the same unit type and number. Typical successful policy for this task is to focus fire on enemy zealots (low attack and high defense) at first and then eliminate the Dragoons (high attack and low defense), which was analyzed in detail by BID26. (Illustrated in FIG2 (f))As presented in BID18, the state-of-the-art performance on these tasks are still quite low compared with others. Amazingly, the proposed MS-MARL method achieves much higher winning rates, as demonstrated in later section. We will refer to these tasks as {15M vs. 16M, 10M vs. 13Z, 15W vs. 17W and 2D3Z} for brevity in the rest of this paper. In order to help reproduce our , we hereby provide our very implementation details. Model Architecture A detailed illustration of our implementation is displayed in Figure 4. As introduced in section 3, in our MS-MARL model, both the master agents and the slave agents are implemented using RNN or LSTM to incorporate temporal recurrence. Here in Figure 4, we use LSTM for the master module, and RNN for the slave module. The dimension of the hidden states in RNN or LSTM (including cell states) are all set to 50 for both the master and slave agents. The GCM module in Figure 4 (c) is noted as the "Gated Composition Module", which is introduced in section 3.1 in detail. Note that the action output is different for discrete and continuous tasks. For the traffic junction task and the combat task, the output of the network is designed as the probability of a number of actions since the action space is discrete (actually a Softmax policy). As a contrast, for "15M vs. 16M" "10M vs. 13Z" and "15W vs. 17W", our network directly generates a continuous action following Gaussian policy as described in section 3. The Softmax/Gaussian action modules are illustrated by Figure 4 D). State Features For the traffic junction task and the combat task, we just apply the original state features designed in BID22. For "15M vs. 16M", "10M vs. 13Z" and "15W vs. 17W", we adopt one representation similar to existing methods [19; 4; 5]. To be specific, the feature of an agent is a map of a certain-sized circle around it, as illustrated in Figure 3. If there is an agent in a cell, a feature vector shown in Fig. 3 will be included in the final observation map. A slightly different state definition is applied for the "2D3Z" task, where unit type is critical for learning correct collaborative policy. In practice, we directly add the type id as one component to the input state feature for the "2D3Z" task. Note that the proposed GCM module facilitates the learning of different strategy for different unit type, which is essential for winning this task. Action Definition For action definitions of the first two tasks, we use the default ones from BID22.Since "15M vs. 16M", "10M vs. 13Z" and "15W vs. 17M" needs a continuous action space. We apply the design from BID18. It contains a 3-dimension vector, each of which is of the range from -1 to 1, representing action type, relative angle and distance to the target respectively. Reward Definition The reward definition of the traffic junction task and the combat task is the same as in BID22. Here we define the reward for "15M vs. 16M", "10M vs. 13Z" and "15W vs. 17W"by simulating the design of that in the combat task. For each time step, there is a reward formulated in 2 to evaluate how the current action of agent i works. DISPLAYFORM0 Note that here ∆n t j∈Nm(i) = n t j∈Nm(i) − n t−1 j∈Nm(i) (the definition of ∆n t k∈Ne(i) is similar) which is actually the change of the number of our units in the neighboring area. N m (i) and N e (i) refers to the sets containing our units and enemy units within a certain range of the current agent i. In experiments we define the range as a circle of a radius of 7 around agent i, which is exactly the circular range as 3 shows. Besides, we also define a terminal reward (as formulated in ) to add to the final reward at the end of each episode. DISPLAYFORM1 Training Procedure The entire training process is shown in Algorithm 1. Note that we follow the softmax policy as proposed in BID22 for the first two discrete tasks. As for the cotinuous tasks {15M vs. 16M, 10M vs. 13Z, 15W vs. 17W}, a Gaussian policy is adopted for stochastic policy parameterization. Specifically, the output of the network is considered as the mean of a Gaussian policy distribution. By fixing the variance to σ = 0.05, the final actions are sampled as follows DISPLAYFORM2 And the score function in can be computed as DISPLAYFORM3 For the traffic junction task, we use a batch size of 16. For the combat task, a larger batch size of 144 is adopted for the training of both CommNet and our master-slave models. And for {15M vs. 16M, 10M vs. 13Z, 15W vs. 17W}, we find that a small batch size of 4 is good enough to guarantee a successful training. The learning rate is set to 0.001 for the first two tasks and 0.0005 for {15M vs. 16M, 10M vs. 13Z, 15W vs. 17W}. The action variance for {15M vs. 16M, 10M vs. 13Z, 15W vs. 17W} is initialized as 0.01 and drops down gradually as the training proceeds. For all tasks, the number of batch per training epoch is set to 100.Baselines For comparison, we select three state-of-the-art MARL methods that have been tested on these tasks of interest. A brief introduction of these baselines are given as follow:• CommNet: This method exploits a simple strategy for multi-agent communication. The idea is to gather a message output from all agents and compute the mean value. Then the mean value is spread to all agents again as an input signal of the next time step. In this way, a communication channel is established for all agents. Such a simple strategy has been shown to work reasonably well for the first two tasks in BID22 and for {15M vs. 16M, 10M vs. 13Z, 15W vs. 17W} when explored in BID18.• GMEZO: The full name of this method is GreedyMDP with Episodic Zero-Order Optimization. And it was proposed particularly to solve StarCraft micromanagement tasks in BID26. The main idea is to employ a greedy update over MDP for inter-agent communications, while combining direct exploration in the policy space and backpropagation.• BiCNet: A bidrectional RNN is proposed to enable communication among multiple agents in BID18. The method adopts an Actor-Critic structure for reinforcement learning. It has been shown that their model is able to handle several different micromanagement tasks successfully as well as learn meaningful polices resembling skills of expert human players. TAB2 demonstrate the performance improvement of our method when compared with the baselines. For CommNet we directly run the released code on the traffic junction task and the combat task using hyper-parameters provided in BID22. We compute the mean winning rates in TAB2 by testing the trained models for 100 rounds. However, since the code of GMEZO and BiCNet is not released yet, there is no report of their performance on traffic junction and combat tasks. Therefore we only compare with CommNet on these two tasks. And it can be seen that our MS-MARL model performs better than CommNet on both of the two tasks. To further clarify the contribution of the proposed master-slave scheme, we add another baseline "CommNet + Occupancy Map" which takes its original state as well as the occupancy map explicitly. As is shown, CommNet performs better with global information. However, with an explicit global planner (the "master" agent) designed in our model, such information seems better utilized to facilitate learning of more powerful collaborative polices. The on the more challenging StarCraft micromanagement tasks {15M vs. 16M, 10M vs. 13Z, 15W vs. 17W, 2D3Z} are displayed in TAB3. Obviously, on all the tasks, our MS-MARL method achieves consistently better performance compared to the available of the three baselines. FIG4 shows the training process of CommNet and our MS-MARL method by plotting win rate curves for the first two tasks as well as "15M vs. 16M". As analyzed above, one key difference between our MS-MARL model and CommNet is that we explicitly facilitate an independent master agent reasoning with its own state and messages from all slave agents. From this plot, our MS-MARL model clearly enjoys better, faster and more stable convergence, which highlights the importance of such a design facilitating independent thinking. In the setting of the combat task, we further analyze how different components of our proposal contribute individually. Specifically, we compare the performance among the CommNet model, our MS-MARL model without explicit master state (e.g. the occupancy map of controlled agents in this case), and our full model with an explicit occupancy map as a state to the master agent. As shown in FIG7 (a)(b), by only allowing an independently thinking master agent and communication among agents, our model already outperforms the plain CommNet model which only supports broadcasting the same message to all agents. Further more, by providing the master agent with its unique state, our full model finally achieves a significant improvement over the CommNet model. Note here that, every information revealed from the extra occupancy map is by definition already included to each agents state as their locations. Another interesting observation is how the master agent and each of the slave agents contribute to the final action choices (as shown in FIG6). We observe that the master agent does often learn an effective global policy. For example, the action components extracted from the master agent lead the whole team of agents move towards the enemies regions. Meanwhile, all the slave agents adjust their positions locally to gather together. This interesting phenomenon showcases different functionality of different agent roles. Together, they will gather all agents and move to attack the enemies, which seems to be quite a reasonable strategy in this setting. Our method combines the advantages of both centralized and decentralized perspectives. To better demonstrate this point, we compare a set of baselines by modifying the structure of our model. FIG6 (c) we draw the training curve of the first 300 steps for all the baselines. It is evident that our MS-MARL performs better than the other baselines with a quite clear margin(more than 20 percent higher than the second best one). Also we find that it is very hard for a single master agent to control all units (unfortunately it earns a 0% winning chance in this case). While independent agents do learn some winning policy, their performance is far lower from MS-MARL but comparable to CommNet. From these , we can see that a stronger global planner as well as a more powerful gating mechanism for message collection/dispatching is necessary for multi-agent coordination. Note that the CommNet model has already been verified to have learned meaningful policies in these tasks according to [23; 19]. However, in our experiments, we often observe more successful policies learned by our method which may not be captured very well by the CommNet model. For example, in the case of combat task, we often observe that some agents of the CommNet model just fail to find the enemies (potentially due to their limited local views and ineffective communications) and therefore lose the battle in a shorthanded manner, see e.g. FIG7 (a). As a contrast, the learned model of our MS-MARL method usually gathers all agents together first and then moves them to attack the enemies. Another support case is in the task of "15M vs. 16M". In this task, our model learns a particular policy of spreading the agents into a half-moon shape ("Spread Out") to focus fire and attacking the frontier enemies before the others enter the firing range, as demonstrated in FIG9. Actually, this group behavior is similar to the famous military maneuver "Pincer Movement" which is widely exploited in representative battles in the history. Although CommNet sometimes also follows such kind of policy, it often fails to spread to a larger "pincer" to cover the enemies and therefore loses the battle. FIG9 (a) shows one of such examples. The size of the "pincer" seems especially important for winning the t ask of "15M vs. 16M" where we have less units than the enemies. From the above analysis, we see that the master module does contribute to a higher performance. To further explore the key ingredients, we generate a visualization of the hidden state from the master agent (projected to 2D with t-SNE BID10, shown in FIG7 (c)). Specifically, we plot the hidden state of two different versions of our MS-MARL model: the blue points version is coupled with a GCM module as well as occupancy map as master's input (with a 61% win rate), while the red points are just from the original version (with only a 55% win rate). Although an explicit explanation of the meanings of the hidden states cannot be certified at this point, what excites us here is that the distribution of master's message does correspond to several representative successful polices. This interesting phenomenon indicates that the hidden states of the master agent effectively carry meaningful messages, according to which, the distribution of successful policies stands out from that of relatively poor policies. In this paper, we revisit the master-slave architecture for deep MARL where we make an initial stab to explicitly combine a centralized master agent with distributed slave agents to leverage their individual contributions. With the proposed designs, the master agent effectively learns to give high-level instructions while the local agents try to achieve fine-grained optimality. We empirically demonstrate the superiority of our proposal against existing MARL methods in several challenging mutli-agent tasks. Moreover, the idea of master-slave architecture should not be limited to any specific RL algorithms, although we instantiate this idea with a policy gradient method, more existing RL algorithms can also benefit from applying similar schemes.

represent the tldr for finding the corresponding scientific article
------
Fewshot example: "Show that age confounds cognitive impairment detection + solve with fair representation learning + propose metrics and models." == "One of the most prevalent symptoms among the elderly population, dementia, can be detected by classifiers trained on linguistic features extracted from narrative transcripts. However, these linguistic features are impacted in a similar but different fashion by the normal aging process. Aging is therefore a confounding factor, whose effects have been hard for machine learning classifiers to isolate. In this paper, we show that deep neural network (DNN) classifiers can infer ages from linguistic features, which is an entanglement that could lead to unfairness across age groups. We show this problem is caused by undesired activations of v-structures in causality diagrams, and it could be addressed with fair representation learning. We build neural network classifiers that learn low-dimensional representations reflecting the impacts of dementia yet discarding the effects of age. To evaluate these classifiers, we specify a model-agnostic score $\Delta_{eo}^{(N)}$ measuring how classifier are disentangled from age. Our best models outperform baseline neural network classifiers in disentanglement, while compromising accuracy by as little as 2.56\% and 2.25\% on DementiaBank and the Famous People dataset respectively. One in three seniors die of Alzheimer's and other types of dementia in the United States . Although its causes are not yet fully understood, dementia impacts people's cognitive abilities in a detectable manner. This includes different syntactic distributions in narrative descriptions BID28, more pausing BID29, higher levels of difficulty in recalling stories BID21, and impaired memory generally BID20. Fortunately, linguistic features can be used to train classifiers to detect various cognitive impairments. For example, BID8 detected primary progressive aphasia with up to 100% accuracy, and classified subtypes of primary progressive aphasia with up to 79% accuracy on a set of 40 participants using lexical-syntactic and acoustic features. BID7 classified dementia from control participants with 82% accuracy on narrative speech. However, dementia is not the only factor causing such detectable changes in linguistic features of speech. Aging also impairs cognitive abilities BID11, but in subtly different ways from dementia. For example, aging inhibits fluid cognitive abilities (e.g., cognitive processing speed) much more than the consolidated abilities (e.g., those related to cumulative skills and memories) BID4. In other words, the detected changes of linguistic features, including more pauses and decreased short-term memories, could attribute to just normal aging process instead of dementia. Unfortunately, due to the high correlation between dementia and aging, it can be difficult to disentangle symptoms are caused by dementia or aging BID24. Age is therefore a confounding factor in detecting dementia. The effects of confounding factors are hard for traditional machine learning algorithms to isolate, and this is largely due to sampling biases in the data. For example, some algorithms predict higher risk of criminal recidivism for people with darker skin colors BID15, others identify images of smiling Asians as blinking BID19, and GloVe word embeddings can project European-American names significantly closer to the words like'pleasant' than African-American names BID3. It is preferable for classifiers to make decisions without biasing too heavily on demographic factors, and therefore to isolate the effects of confounding factors. However, as we will show in Experiments, traditional neural network classifiers bias on age to infer dementia; this can lead to otherwise avoidable false positives and false negatives that are especially important to avoid in the medical domain. Graphically, if both age A and dementia D cause changes in a feature X, the is a v-structure BID17 A → X ← D which is activated upon observing X. In other words, the confounder A affects P (D|X) if we train the classifier in traditional ways, which is to collect data points {(X, D) (i) } and to learn an inference model P (D|X) approximating the affected P (D|X).Traditionally, there are several ways to eliminate the effects of confounding factors A.Controlling A gives a posterior distribution P (D|X, A)P (A). This is unfortunately unrealistic for small, imbalanced clinical datasets, in which sparsity may require stratification. However, the stratified distributions P (D|X, A) can be far from a meaningful representation of the real world (as we will show, e.g., in FIG3). Moreover, a discrepancy in the sizes of age groups can skew the age prior P (A), which would seriously inhibit the generalizability of a classifier. Controlling X Conducting a randomized control trial (RCT) on X removes all causal paths leading "towards" the variable X, which gives a de-confounded dataset P (D|do(X)) according to the notation in BID27. However, RCTs on X are even less practical because simultaneously controlling multiple features produces exponential number of scenarios, and doing this to more than 400 features require far more data points than any available dataset. Pre-adjusting X according to a pre-trained model X = f (A) per feature could also approximately generate the dataset P (D|do(X)). However, such a model should consider participant differences, otherwise interpolating using a fixed age A would give exactly the same features for everybody. The participant differences, however, are best characterized via X, which are the values you want to predict. To overcome the various problems with these methods, we let our classifiers be aware of cognitive impairments while actively filtering out any information related to aging. This is a fair representation learning framework that protects age as a "sensitive attribute".Fair representation learning frameworks can be used to train classifiers to equally consider the subjects with different sensitive attributes. A sensitive attribute (or "protected attribute") can be race, age, or other variables whose impact should be ignored. In the framework proposed by BID32, classifiers were penalized for the differences in classification probabilities among different demographic groups. After training, the classifiers produced better demographic similarities while compromising only a little overall accuracy. To push the fair representation learning idea further, adversarial training can be incorporated. BID9 introduced generative adversarial networks, in which a generator and a discriminator are iteratively optimized against each other. Incorporating adversarial training, BID22 proposed a framework to learn a latent representation of data in order to limit its adversary's ability to classify based on the sensitive attributes. However, these approaches to fair representation learning only handle binary attributes. E.g., BID22 binarized age. To apply to cognitive impairments detection, we want to represent age on a continuous scale (with some granularity if necessary). We formulate a fairness metric for evaluating the ability of a classifier to isolate a continuous-valued attribute. We also propose four models that compress high-dimensional feature vectors into low-dimensional representations which encrypt age from an adversary. We show empirically that our models achieve better fairness metrics than baseline deep neural network classifiers, while compromising accuracies by as little as 2.56% and 2.25% on our two empirical datasets, respectively. There are many measures of entanglement between classifier outcomes and specific variables. We briefly review some relevant metrics, and then propose ours. Correlation (Pearson, Spearman, etc.) is often used to compare classification outputs with component input features. To the extent that these variables are stochastic, several information theoretic measures could be applied, including Kullback-Leibler divergence and Jensen-Shannon divergence. These can be useful to depict characteristics of two distributions when no further information about available data is given. Mutual information can depict the extent of entanglement of two random variables. If we treat age (A) and dementia (D) as two random variables, then adopting the approach of BID18 gives an estimation of I(A, D). However, given the size of clinical datasets, it can be challenging to give precise estimations. An alternative approach is to assume that these variables fit into some probabilistic models. For example, we might assume the age variable A, dementia indicator variable D, and multi-dimensional linguistic feature X fit into some a priori model (e.g., the v-structure mentioned above, A → X ← D), then the mutual information between A and D is: DISPLAYFORM0 where the entropy of age H A and of cognitive impairment H D remain constant with respect to the input data X, and DISPLAYFORM1 However, this marginalized probability is difficult to approximate well, because the accuracy of the term p(A|X) relies on the ability of our model to infer age from features, and FORMULA20 it is hard to decide on a good prior distribution on linguistic features p(X). We want to make the model agnostic to age, leading to a meaningless mutual information in the'ideal' case. In our frameworks, we do not assume specific graphical models that correlate confounds and outcomes, and we propose more explainable metrics than the traditional statistical ones. The literature in fairness representation learning offers several metrics for evaluating the extent of bias in classifiers. Generally, the fairer the classifier is, the less entangled the are with respect to some protected features. Demographic parity BID32 stated that the fairest scenario is reached when the composition of the classifier outcome for the protected group is equal to that of the whole population. While generally useful, this does not apply to our scenario, in which there really are more elderly people suffering from cognitive impairments than younger people (see FIG3).Cross-entropy loss BID5 used the binary classification loss of an adversary that tried to predict sensitive data from latent representations, as a measure of fairness. This measure can only apply to those models containing an adversary component, not traditional classifiers. Moreover, this loss also depends on the ability of the adversary network. For example, a value of this loss could indicate confusing representations (so sensitive information are protected well), but it could also indicate a weak adversary. Equalized odds BID12 proposed a method in which false positive rates should be equal across groups in the ideal case. BID22 defined fairness distance as the absolute difference in false positive rates between two groups, plus that of the false negative rates: DISPLAYFORM0 where p a and n a correspond to the false positive rate and false negative rate, respectively, with sensitive attribute a = 0 (a = 1). We propose an extension of the metric used by BID22 to continuous sensitive attributes, suitable for evaluating an arbitrary two-class classifier., and classifier C. In age-indep-autoencoder and age-indep-entropy FIG0, a reconstructor R tries to reconstruct input data from the hidden representation. In age-indep-consensus-nets FIG0 ), a discriminator D tells apart from which modality the representation originates. First, groups of age along a scale are divided so that each group has multiple participants with both positive and negative diagnoses, respectively. Let a be the age group each participant is in. Then, we aim for the expected false positive (FP) rates of the classifier to be as constant as possible across age groups. This applies likewise to the false negative (FN) rates. To measure their variability, we use their sum of differences against the mean. DISPLAYFORM0 wherex represents the mean of variable x. Special cases To illustrate the nature of our metric, we apply it to several special cases, i.e.:1. When there is only one age group, our fairness metric has its best possible value: ∆ eo = 0. 2. When there are only two age groups, our metric equals that of BID22. 3. In the extreme case where there are as many age groups as there are sample points (assuming there are no two people with identical ages but with different diagnoses), our metric becomes less informative, because the empirical expected false positive rates of that group is either 0 or 1. This is a limitation of our metric, and is the reason that we limit the number of age groups to accommodate the size of the training dataset. Bounds Our metric is bounded. The lower bound, 0, is reached when all false positive rates are equal and when all false negative rates are equal across age groups. Letting N a be the number of age groups divided, an upper bound for ∆ (Na) eo is N a for any better-than-trivial binary classifier. The detailed proof is included in the Appendix. Disentanglement Our fairness metric illustrates disentanglement. A higher ∆ (N) eo corresponds to a higher variation of incorrect predictions by the classifier across different age groups. Therefore, a lower value of ∆ (N) eo is desired for classifiers isolating the effects of age to a better extent. Throughout this paper, we use the terms'fairness','disentanglement', and'isolation' interchangeably. We explain a few design choices here, namely linearity and indirect optimization. eo to be as linear as possible, for explainability of the fairness score itself. This eliminates possible scores consisting of higher order terms of FP / FN rates. eo, we encourage the hidden representations to be age-agnostic (we will explain how to set up age agnostic models in the following section). On the other hand, FP / FN rates are not differentiable after all. In this section, we describe four different ways of building representation learning models, which we call age-indep-simple, age-indep-autoencoder, age-indep-consensus-net, and age-indep-entropy. The simplest model consists of an interpreter network I to compress high-dimensional input data, x, to low-dimensional representations: z = I(x) An adversary A tries to predict the exact age from the representation: DISPLAYFORM0 A classifier C estimated the probability of label (diagnosis) based on the representation: for minibatch x in training data X do 4: DISPLAYFORM1 DISPLAYFORM2 Calculate L a, L c 6: DISPLAYFORM3 For optimization, we set up two losses: the classification negative log likelihood loss L c and the adversarial (L2) loss L a, where: DISPLAYFORM4 We want to train the adversary to minimize the L2 loss, to train the interpreter to maximize it, and to train the classifier (and interpreter) to minimize classification loss. Overall, DISPLAYFORM5 The training steps are taken iteratively, as in previous work BID9. The age-indep-autoencoder structure is similar to BID22, and can be seen as an extension from the age-indep-simple structure. Similar to age-indep-simple, there is an interpreter I, an adversary A, and a classifier C network. The difference is that there is a reconstructor network R that attempts to recover input data from hidden representations: DISPLAYFORM0 The loss functions are set up as: DISPLAYFORM1 Overall, we want to train both the interpreter and the reconstructor to minimize the reconstruction loss term, in additional to all targets mentioned in the age-indep-simple network. DISPLAYFORM2 The detailed algorithm is similar to Algorithm 1 and is in the Appendix. This is another extension from the age-indep-simple structure, borrowing an idea from consensus networks BID33, i.e., that agreements between multiple modalities can in representations that are beneficial for classification. By examining the performance of age-indepconsensus-net, we would like to see whether agreement between multiple modalities of data can be trained to be disentangled from age. Similar to age-indep-simple structures, there is also an adversary A and a classifier C. The interpreter, however, is replaced with several interpreters I 1..M, each compressing a subset of the input data ("modality") into a low-dimensional representation. The key of age-indep-consensusnetwork models is that these representations are encouraged to be indistinguishable. For simplicity, we randomly divide the input features into three modalities (M = 3) with equal (±1) features. A discriminator D tries to identify the modality from which the representation comes: DISPLAYFORM0 The loss functions are set up as: DISPLAYFORM1 Overall, we want to iteratively optimize the networks: DISPLAYFORM2 L c and max DISPLAYFORM3 The detailed algorithm is in the Appendix. Note that we do not combine the consensus network with the reconstructor because they do not work well with each other empirically. In one of the experiments by BID34, each interpreter I m is paired with a reconstructor R m and the performance decreases dramatically. The reconstructor encourages hidden representations to retain the fidelity of data, while the consensus networks urges hidden representations to keep only the information common among modalities, which prohibits the reconstructor and consensus mechanism to function together. The fourth model we apply to fair representation learning is motivated by categorical GANs , where information theoretic metrics characterizing the confidences of predictions can be optimized. This motivates an additional loss function term; i.e., we want to encourage the interpreter to increase the uncertainty (i.e., to minimize the entropy) while letting the adversary become more confident in predicting ages from representations. Age-indep-entropy models have the same network structures as age-indep-autoencoder, except that instead of predicting the exact age, the adversary network outputs the probability of the sample age being larger than the mean: DISPLAYFORM0 This enables us to define the empirical entropy H[p] = E x plog 1 p, which describes the uncertainty of predicting age. Formally, the loss functions are set up as follows: DISPLAYFORM1 where λ H is a hyper-parameter. For comparison, we also include two variants, namely the ageindep-entropy (binary) and age-indep-entropy (Honly) variants, each keeping only one of the two terms in L a. In our experiments, we show that these two terms in L a are better applied together. Overall, the training procedure is the same as age-indep-autoencoder and algorithm pseudocode is in the Appendix: min DISPLAYFORM2 All models are implemented in PyTorch BID26, optimized with Adam BID16 with initial learning rate of 3 × 10 −4, and L2 weight decay 10. For simplicity, we use fully connected networks with ReLU activations BID25 and batch normalization BID14 before output layers, for all interpreter, adversary, classifier, and discriminator networks. Our frameworks can be applied to other types of networks in the future. DementiaBank DementiaBank 1 is the largest available public dataset for assessing cognitive impairments using speech, containing 473 narrative picture descriptions from subjects aged between 45 and 90 BID2. In each sample, a participant talks about what is happening in a clinically validated picture. There is no time limit in each session, but the average description lasts about a minute. 79 samples are excluded due to missing age information. In the remaining data samples, 182 are labeled'control', and 213 are labeled'dementia'. All participants have mini-mental state estimation (MMSE) scores BID6 ) between 1 and 30 2. Of all data samples containing age information, the mean is 68.26 and standard deviation is 9.00. The Famous People dataset BID1 contains 252 transcripts from 17 people (8 with dementia including Gene Wilder, Ronald Reagan and Glen Campbell, and 9 healthy controls including Michael Bloomberg, Woody Allen, and Tara VanDerveer), collected and transcribed from publicly available speech data (e.g., press conferences, interviews, debatse, talk shows). Seven data samples are discarded due to missing age information. Among the remaining samples, there are 121 labeled as control and 124 as impaired. Note that the data samples were gathered across a wide range of ages (mean 59.25, standard deviation 13.60). For those people diagnosed with dementia, there are data samples gathered both before and after the diagnosis, and all of which are labeled as'dementia'. The Famous People dataset permits for early detection several years before diagnosis, which is a more challenging classification task than DementiaBank. Older participants in both DementiaBank FIG3 ) and the Famous People dataset FIG3 ) are more likely to have cognitive impairments. eo and ∆eo ) of several traditional classifiers. DNN is the baseline used to benchmark our neural network based representation learning models. We extract 413 linguistic features from the narrative descriptions and their transcripts. These features were previously identified as the most useful for this task BID28 BID7 BID21 BID13. Each feature is z-score normalized. Acoustic: mean, variance, skewness, and kurtosis of the first 42 cepstral coefficients. Speech fluency: pause-word ratio, utterance length, number and lengths of filled/unfilled pauses. Lexical: cosine similarity between pairs of utterances, word lengths, lexical richness (movingaverage type-token ratio, Brunet's index, and Honoré's statistics BID10).PoS: Number of occurrences of part-of-speech tags, tagged by SpaCy 3.Syntactic and semantic: occurrences of context-free grammar phrase types, parsed by Stanford CoreNLP BID23, and Yngve depth statistics BID31. As part of expository data analysis, we show that these linguistic features contain information indicating age. Simple fully connected neural networks can predict age with mean absolute error of 15.5 ± 1.3 years (on DementiaBank 4) and 14.3 ± 2.5 years (on the Famous People dataset 5). This indicates that even simple neural networks are able to infer information about age from linguistic features. Neural classifiers can therefore also easily bias on age, given the utility of age in downstream tasks. We first set up benchmarks for our classifiers. We evaluate several traditional classifiers with our fairness metrics (∆ eo, corresponding to dividing ages into N = 2 and N = 5 groups respectively). The 6 are listed in Table 1. A DNN is used as the baseline because all our models are based on neural networks, and DNN classifiers have had the best (or statistically indistinguishable from the best) accuracy on the DementiaBank and Famous People datasets. We evaluate the performances of our four proposed neural networks against the DNN baseline. As an additional ablation study, two variants of age-indep-entropy are also evaluated. TAB1: Evaluation of our representation learning models. The "age-indep" prefix are replaced with "*" in model names. age-indep-simple and age-indep-autoencoder have better disentanglement scores, while the rest two models could have better accuracy. Accuracy The fair representation learning models compromise accuracy, in comparison to DNN baselines. This confirms that part of the classification power of DNNs come from biasing with regards to age. On DementiaBank, the age-indep-autoencoder reduces accuracy the least (only 2.56% in comparison to the DNN baseline). On the Famous People data, age-indep-consensus and age-indep-entropy models compromise accuracies by only 2.25% and 2.75% respectively, which are not statistically different from the DNN baseline 7.Disentanglement In comparison to DNN baselines, our fair representation learning models improve disentanglement/fairness 8, the improvements are mostly significant when measured by the two-group scores ∆eo. Also, the five-group scores ∆eo are less stable for both datasets, and the scores in the Famous People have higher variances than in DementiaBank. Following is an explanation. DementiaBank has ∼400 data samples. In 5-fold cross validation, each of the five age groups has only ∼16 samples during evaluation. Famous People data contains ∼250 samples, which increases the variance. When the number of groups, N of ∆ (N) eo, is kept small (e.g., ∼100 samples per label per group, as in DementiaBank N = 2), the fairness metrics are stable. The model age-indep-entropy is best used with a loss function containing both the binary classification term and the uncertainty minimization term. As shown in TAB1, although having similar fairness metrics 9, the two variants with only one term could have lower accuracy than age-indep-entropy. In general, age-indep-simple and age-indep-autoencoder achieve the best fairness metrics. Noticeably, the better of them surpass traditional classifiers in both ∆ Here, we identify the problem of entangling age in the detection of cognitive impairments. After explaining this problem with causality diagrams, we formulate it into a fair representation learning task, and propose a fairness score to measure the extent of disentanglement. We put forward four fair representation learning models that learn low-dimensional representations of data samples containing as little age information as possible. Our best model improves upon the DNN baseline in our fairness metrics, while compromising as little accuracy as 2.56% (on DementiaBank) and 2.25% (on the Famous People dataset).7 p = 0.20, 0.16 on 38-DoF one-tailed t-tests, respectively. 8 On DementiaBank, p = 0.01 and 0.03 for age-indep-simple and age-indep-entropy on ∆ eo respectively; these are significant. p = 0.08 and 0.09 on age-indep-autoencoder and age-indep-consensus-net on ∆ eo respectively; these are marginally significant. However, these differences are not as significant on ∆ Proof of Theorem For each of the age groups: |p a −p| + |n a −n| ≤ max{|p a − 0| + |n a − 0|, |p a − 0.5| + |n a − 0.5|} ≤ max{0.5, 1} = 1 Summing up the N a age groups in our upper bound N a for non-trivial classifiers. Following are the pseudo-code algorithms for our remaining three models; age-indep-AutoEncoder, age-indep-ConsensusNetworks, and age-indep-Entropy. for minibatch x in training data X do for minibatch x in training data X do"

We revisit the idea of the master-slave architecture in multi-agent deep reinforcement learning and outperforms state-of-the-arts.

Represent the TLDR for finding the corresponding scientific article

We identify and formalize the memorization problem in meta-learning and solve this problem with novel meta-regularization method, which greatly expand the domain that meta-learning can be  applicable to and effective on.