Question: Represent the article for finding a single-sentence TLDR.

Question: In this paper, we consider the problem of detecting object under occlusion. Most object detectors formulate bounding box regression as a unimodal task (i.e., regressing a single set of bounding box coordinates independently). However, we observe that the bounding box borders of an occluded object can have multiple plausible configurations. Also, the occluded bounding box borders have correlations with visible ones. Motivated by these two observations, we propose a deep multivariate mixture of Gaussians model for bounding box regression under occlusion. The mixture components potentially learn different configurations of an occluded part, and the covariances between variates help to learn the relationship between the occluded parts and the visible ones. Quantitatively, our model improves the AP of the baselines by 3.9% and 1.2% on CrowdHuman and MS-COCO respectively with almost no computational or memory overhead. Qualitatively, our model enjoys explainability since we can interpret the ing bounding boxes via the covariance matrices and the mixture components. Figure 1: We observe that an occluded bounding box usually exhibits multiple modes in most detection datasets, no matter whether the ground truth annotation is visible box or full box: (a) visible bounding box annotation (b) full object bounding box labeled by different annotators (c) visible bounding box annotated accurately (d) visible bounding box annotated inaccurately Object detectors based on deep convolutional neural networks (CNNs) are the backbone of many real-world applications like self-driving cars , robotics grasping and video surveillance . Most object detectors learn to detect an object in two folds : categorization of the candidate bounding box regress each coordinate of the candidate box towards the ground truth one independently. Currently, there are two styles of bounding box annotation among the large-scale object detection datasets: visible box that only contains visible parts (e.g., MS-COCO and PASCAL VOC ) full box that contains both visible and occluded parts (e.g., CrowdHuman and VehicleOcclusion ). For full box annotation, regressing a single set of bounding box coordinates works well for fully visible objects, since it is a unimodal problem. However, when an object is occluded, we observe that its occluded parts can have several plausible configurations (e.g., Figure 1 (b)), which is a multimodal problem. Even for visible box annotation, an object sometimes still exhibits multiple modes due to inaccurate labeling (e.g., Figure 1 (c) vs. (d)). We argue that an object detector robust to occlusion should learn a multimodal distribution with the capability of proposing more than one plausible hypothesis for the configuration of an occluded part. Besides, we also observe that the bounding box coordinates have correlations by nature. Take Figure 1 (c) as an example, by knowing the position of the car's roof, we can easily infer the location of the left border even without looking at it. Therefore, an object detector robust to occlusion also needs to be capable of inferring the correlations between the occluded bounding box borders and the visible ones. Motivated by these two observations, we propose a deep multivariate mixture of Gaussians model for object detection under occlusion. Concretely, instead of regressing a single set of bounding box coordinates, our model regresses several sets of coordinates, which are the means of the Gaussians. Moreover, we learn a covariance matrix for the coordinates of each Gaussian mixture component. These components are summed together as the prediction for the distribution of plausible bounding box configurations. At inference time, we choose the expectation of our model's distribution as the final predicted bounding box. To demonstrate the generalizability of our proposed model, we conduct experiments on four datasets: CrowdHuman, MS-COCO, VehicleOcclusion, and PASCAL VOC 2007. Quantitatively, our model improves the AP (Average Precision of the baselines by 3.9% and 1.2% on CrowdHuman and MS-COCO respectively (Table 1 and Table 2). Qualitatively, our model enjoys explainability since the ing bounding boxes can be interpreted using the covariance matrices and the Gaussian mixture components (Figure 5 and Figure 4). More importantly, our model is almost computation and memory free, since predicting the mixture components only requires a fully-connected layer, and we can discard the covariance matrices at inference time (Table 5). Object Detection: Deep convolutional neural networks were first introduced to object detection in R-CNN and Fast R-CNN . Currently, there are mainly two types of object detectors: one-stage object detectors and two-stage object detectors. One-stage detectors like YOLO , SSD and RetinaNet are fast in general. Two-stage detectors (; ;) are accurate however sacrificing speed. In this paper, although we conduct experiments based on the Faster R-CNN heads of Faster R-CNN and Mask R-CNN, our method is not limited to two-stage detectors. Object Detection Under Occlusion: Occlusion-aware R-CNN (b) proposes to divide pedestrian detection into five parts and predict the visibility scores respectively, which are integrated with the prior structure information of the human body into the network to handle occlusion. Zhang et al. (2018a) proposes an attention network with self or external guidance. These methods are specifically designed for pedestrian detection task. By contrast, our method is designed for general object detection. Deep Voting (c) proposes to utilize spatial information between visual cues and semantic parts and also learn visual cues from the context outside an object. However, detecting semantic parts needs manual labels, which our approach does not require. Besides, our approach does not introduce additional computation during the inference (Table 5). Amodal instance segmentation considers the task of predicting the region encompassing both visible and occluded parts of an object. The authors propose to add synthetic occlusion to visible objects and retain their original masks, then employ a CNN to learn on the generated composite images, which resembles the VehicleOcclusion in our experiments. proposes bounding box regression with uncertainty, which is a degradation case of our model (Gaussian). Datasets for Detection under Occlusion: Currently, there are three categories of annotation for an occluded object: visible bounding box that contains the visible parts full box that contains both visible and occluded parts of an object annotated by human full box by synthesizing occluders on a visible object. MS-COCO, PASCAL VOC, ImageNet Figure 2: Faster R-CNN head architecture for our approach: We extended the existing Faster R-CNN head to predict the parameters of multivariate mixture of Gaussian µ, φ and Σ Cityscapes fall into the first category. CrowdHuman and Semantic Amodal Segmentation dataset require the annotators to label the invisible parts. VehicleOcclusion instead synthesizes the occluders for visible objects. We conduct experiments on MS-COCO, PASCAL VOC 2007, CrowdHuman, and VehicleOcclusion, covering all these categories. We observe that when an object is partially occluded, the occluded bounding box border can usually be inferred to some extent by other visible parts of the object (e.g., it is easy to infer the left border of the car given the car roof position in Figure 1 (c)). Besides, the occluded bounding box exhibits multiple modes. For example, the left arm of the teddy bear could have several possible configurations in Figure 1 (b). Motivated by these two observations, we propose to estimate the bounding box coordinates as a probability distribution during bounding box regression instead of a set of deterministic coordinates. Specifically, we propose to estimate a multivariate mixture of Gaussians distribution with a deep network. Multivariate Gaussian helps the case where bounding box borders have correlations, and a mixture of Gaussians helps the case where an occluded bounding box border exhibits multiple modes. Formally, we predict the distribution p θ (x|I) given the feature maps I of a region of interest (RoI). The distribution is parameterized by θ, which is a neural network (e.g., Faster R-CNN head, Figure 2). The distribution has T, which is the most probable bounding box coordinates relative to the RoI, estimated by the component: Σ is the covariance matrix, which is a symmetric semi-positive definite matrix in general. To be able to compute the inverse Σ −1, we constrain the covariance matrix to be a symmetric positive definite matrix. In this case, the precision matrix Σ −1 is also a symmetric positive definite matrix. During training, the model estimates the precision matrix Σ −1 instead of the covariance matrix Σ, so that we do not need to compute the inverse every time during training which we also find more stable in our experiments. To ensure the properties of the precision matrix Σ −1, we parameterize it using the Cholesky decomposition: where U is an upper triangular matrix with strictly positive diagonal entries, such that Cholesky decomposition is guaranteed to be unique. We parameterize the mixture weights φ i using Softmax, so that they range from 0 to 1 and sum to 1: z i, u ii and µ i are outputs produced by a fully-connected layer on top of the final fully-connected layer fc7 on the Faster R-CNN head. Take Faster R-CNN with RPN as an example, Figure 2 shows the architecture of our model. Since we only modify a small part of the architecture, our approach might also be applied to other object detectors than Faster R-CNN, like one-stage object detectors YOLO and RetinaNet. Learning: Our model parameterizes the distribution over bounding boxes using a neural network which depends on RoI features. During training, we estimate the parameters θ with maximum likelihood estimation on a given dataset {I, µ * | = 1, 2, ..., N}, where µ * represents the ground truth coordinates for RoI feature maps I and N is the number of observations: In practice, N is the number of samples in a mini-batch. We use momentum stochastic gradient descent (SGD) to minimize the localization loss L loc and the classification loss L cls: Note that we use different parameters θ for different classes in practice. For simplicity, the formulation above only considers the regression problem for a single class. Inference: During testing, we use the expectation of our mixture module as prediction: Notice that the covariance matrix Σ i is not involved in inference. In practice, we discard the neurons that produce the covariance matrix to speed up inference. In our experiments (Table 5), our model has almost the same inference latency and memory consumption as the baseline network. Multivariate Gaussian: When the number of mixture components K = 1, our model degrades into a multivariate Gaussian model. And the localization loss can be rewritten as follow (for simplicity, we only illustrate the loss for a single sample): where 2 ln 2π is a constant which can be ignored during training. where (U i) jj is the jth diagonal element of the matrix U i. Multimodality is helpful under occlusion because an occluded object usually has multiple modes. Gaussian: When the number of mixture components K = 1 and the covariance is constrained to be a diagonal matrix, it becomes a simple Gaussian model where different variables are independent: We argue that this simple model helps detection in most cases. Here (U) jj behaves like a balancing term. When the bounding box regression is inaccurate (large (µ * j − µ j) 2 /2), the variance 1/(U) 2 jj tends to be larger. Therefore smaller gradient will be provided to bounding box regression (U) 2 /2 in this case, which might help training the network (Table 1 and Table 2). If bounding box regression is perfect, U tend to infinity (i.e., the variance should be close 0). However, regression is not that accurate in practice, U will be punished for being too large. Euclidean Loss: When all the diagonal elements (U) jj are one (u jj = 0), our model degenerates to the standard euclidean loss: We initialize the weights of µ i, z i and u ii layers (Figure 2) using random Gaussian initialization with standard deviations 0.0001 and biases 0, −1 and 0 respectively. So that at the start of training, bounding box coordinate µ i is at an unbiased position, U i is an identity matrix and φ i treats each mixture component equally. Our model can be trained end-to-end. Unless specified, we follow settings in Detectron and those original papers. To demonstrate the generalizability of our method, we conduct experiments on four datasets: CrowdHuman is a large, rich-annotated and highly diverse dataset for better evaluation of detectors in crowd scenarios. Its training and validation sets contain a total of 470k human instances, and around 22.6 persons every image under various kinds of occlusions. The annotations for occluded bounding boxes are full boxes (Figure 1 (b) ) instead of visible boxes (Figure 1 (a) ). The experiments are in Table 1. VehicleOcclusion is a synthetic dataset designed for object detection under occlusion . Same as above, the annotations are full boxes. The occlusion annotations are more accurate since the occluders (occluding objects) are randomly placed on the annotated visible object. It contains six types of vehicles and occluded instances at various difficulty levels. Specifically, it consists of four occlusion levels: No occlusion (0%), L1 (20% ∼ 40%), L2 (40% ∼ 60%), L3 (60% ∼ 80%). The percentages are computed by pixels. At level L1, L2 and L3, there are two, three, and four occluders placed on the object, respectively (Table 4). is a large-scale object detection dataset containing 80 object categories, 330k images (> 200k labeled) and 1.5 million object instances. Compared with the two datasets above, MS-COCO has fewer occlusion cases. For example, the IoU (intersection over union) between overlapped human bounding boxes in MS-COCO are less than 0.7 . We use train2017 for training and val2017 for testing (Table 2). Different from above, the annotations are visible boxes. PASCAL VOC 2007 has 9,963 images and 20 classes in total, containing 24,640 annotated objects (Everingham et al.). Similar with MS-COCO, this dataset has less occlusion cases than the first two datasets. We use voc_2007_train and voc_2007_val for training and voc_2007_test for testing (Table 3). The annotations are visible boxes. Number of Mixture Components: Shown in Figure 3, we test our mixture of Gaussians model by varying the number of mixture components. The baseline is ResNet-50 FPN Faster R-CNN on CrowdHuman. As the number of components increases from 1, 4 to 8, we observe consistent performance improvement. The mixture of eight Gaussians model (Eq. 8) outperforms Gaussian model (Eq. 9) by 1% AP. However, the performance goes down when there are more than 16 components. This might be because the objects in the dataset might not have as many as 16 modes when occluded. Besides, the more components we have, the higher the chance of over-fitting. Unless specified, we use eight components for the mixture of Gaussians model. Mixture of Gaussian vs. Multivariate Gaussian: Shown in Table 1 and 2, we compare the degradation cases of our complete model (Eq. 1): Gaussian (Eq. 9), mixture of Gaussians (Eq. 8) and multivariate Gaussian (Eq. 7) on CrowdHuman and MS-COCO. For CrowdHuman, we use ResNet-50 FPN Faster R-CNN as the baseline. For MS-COCO, we use ResNet-50 FPN Mask R-CNN. On CrowdHuman which has a lot of crowded scenes, our model greatly improves the baseline. Gaussian improves the baseline by 1.2% AP. A mixture of eight Gaussians improves 2.3% AP, and multivariate Gaussians improves 2.9% AP. The complete model improves the performance by 3.9% AP. The improvements indicate all these assumptions are helpful under heavy occlusion. Gaussian helps training the regression network by learning to decrease the gradients for high variance cases. Multivariate Gaussian helps to learn the correlations between an occluded border and the visible borders. Mixture of Gaussians helps to learn a multimodal model for the occluded cases which have multiple modes. Soft-NMS modifies classification scoring, while our approach improves localization. Though it achieves comparable performance (1.7% AP improvement), it can be applied together with our method. With soft-NMS, the AP of mixture of 8 Gaussian, multivariate Gaussian and the complete model further improves 1.7%, 1.5% and 1.5% respectively. On MS-COCO, the bounding box annotations are visible boxes instead of full boxes used in CrowdHuman. Gaussian still works here which improves the baseline by 0.4% AP, since there are variances in the dataset caused by inaccurate annotation (e.g., Figure 1 (d) ). Gaussian helps to reduce the gradients for these ambiguous cases. A mixture of eight Gaussians improves 0.6% AP, and multivariate Gaussians improves 0.7% AP. The complete model improves the performance by 1.2% AP. The improvements are noticeable, however less significant than on CrowdHuman. On the one hand, there are fewer occluded instances in MS-COCO, multimodality and covariances might not as helpful as in CrowdHuman. On the other hand, predicting full boxes require guessing the invisible parts where multimodality and covariances are more useful. We further conduct experiments on PASCAL VOC 2007, shown in Table 3. VGG-CNN-M-1024 Faster R-CNN is the baseline. Similar to MS-COCO, the bounding box annotations are visible boxes instead of full boxes used in CrowdHuman. We observe that Gaussian improve the mAP (mean Average Precision) by 1.5%. The complete model improves the mAP by 2.0%. Multimodality and multivariate Gaussian do not substantially improve the performance. These observations coincide with the observations on MS-COCO. Comparison with State-of-the-art: Shown in Table 4, we compare multivariate mixture of eight Gaussians model to DeepVoting Zhang et al. (2018c) on VehicleOcclusion. Similar to CrowdHuman, the bounding box annotations are full boxes. The baseline is VGG-16 Faster R-CNN. Our multivariate mixture of eight Gaussians model outperforms DeepVoting by a large margin at different occlusion levels. Without occlusion, our model also helps to learn a better detector, coinciding the experiments above. We argue that our model considers multiple modes of an object and the correlations between each border of a bounding box, which helps detection under occlusion. Model Size and Inference Speed: We measure the inference speed of our models using ResNet-50 FPN Mask R-CNN with a TITAN Xp, CUDA 10.1 and cuDNN 7.5.0 on MS-COCO val2017. Shown in Table 5, Gaussian (Eq. 9) and multivariate Gaussian (Eq. 7) neither slow down the inference nor increase the number of parameters, since we can discard the covariance Σ at inference time (Section 3.1). The complete model, multivariate mixture of eight Gaussians (Eq. 1), only increases 2M parameters and sacrifices 0.9 FPS on GPU. Our models outperform the baselines by large margins (Table 1, 2 and 4), while requires almost no additional computation and memory. Note that we measure the inference latency on MS-COCO where there are 80 classes, such that the number of parameters for µ is 1024 × 80 × K (1024 is the number of output channels of fc7, Figure 2). On CrowdHuman where there is only one class (human), the number of parameters for µ is only 1024 × K, which will consume even fewer computation and memory resources. Figure 4 shows the visualization of our mixture of Gaussian prediction on CrowdHuman. When the object is not occluded, our model usually only exhibits a single mode. In Figure 4 (a), the predictions of the mixture components for the athlete are almost the same. When the object is occluded, the occluded bounding box border usually exhibits multiple modes. For example, the left arm of the man can have several reasonable poses in Figure 4 (b). Figure 5 shows the visualization of our multivariate Gaussian prediction on CrowdHuman. When the object is not occluded, like in Figure 5 (a), most terms in the covariance matrix are usually almost zeros. When a border of the object is occluded, like in Figure 5 (b), the variance term for that border tends to be very high. Sometimes our model learns the covariance between bounding box borders. For example, in Figure 5 (c), x 1 and x 2 has a positive correlation, which suggests if the left border moves right, the right border might also move right. When the object is heavily occluded, most of its variance terms are usually very high, shown in Figure 5 (d). We propose a multivariate mixture of Gaussians model for object detection under occlusion. Quantitatively, it demonstrates consistent improvements over the baselines among MS-COCO, PASCAL VOC 2007, CrowdHuman, and VehicleOcclusion. Qualitatively, our model enjoys explainability as the detection can be diagnosed via the covariance matrices and the mixture components.

a deep multivariate mixture of Gaussians model for bounding box regression under occlusion

Represent the TLDR for finding the corresponding scientific article.


Question: represent the text to find a very short summary (1-2 sentences max):

Question: Adversarial examples are a pervasive phenomenon of machine learning models where seemingly imperceptible perturbations to the input lead to misclassifications for otherwise statistically accurate models. Adversarial training, one of the most successful empirical defenses to adversarial examples, refers to training on adversarial examples generated within a geometric constraint set. The most commonly used geometric constraint is an $L_p$-ball of radius $\epsilon$ in some norm. We introduce adversarial training with Voronoi constraints, which replaces the $L_p$-ball constraint with the Voronoi cell for each point in the training set. We show that adversarial training with Voronoi constraints produces robust models which significantly improve over the state-of-the-art on MNIST and are competitive on CIFAR-10. Deep learning at scale has led to breakthroughs on important problems in computer vision , natural language processing , and robotics . Shortly thereafter, the interesting phenomena of adversarial examples was observed. A seemingly ubiquitous property of machine learning models where perturbations of the input that are imperceptible to humans reliably lead to confident incorrect classifications . What has ensued is a standard story from the security literature: a game of cat and mouse where defenses are proposed only to be quickly defeated by stronger attacks . This has led researchers to develop methods which are provably robust under specific attack models;;; ) as well as empirically strong heuristics; ). As machine learning proliferates into society, including security-critical settings like health care or autonomous vehicles , it is crucial to develop methods that allow us to understand the vulnerability of our models and design appropriate counter-measures. Adversarial training has been one of the few heuristic methods which has not been defeated by stronger attacks. In this paper, we propose a modification to the standard paradigm of adversarial training. We replace the L p -ball constraint with the Voronoi cells of the training data, which have several advantages detailed in Section 3. In particular, we need not set the maximum perturbation size as part of the training procedure. The Voronoi cells adapt to the maximum allowable perturbation size locally on the data distribution. We show how to construct adversarial examples within the Voronoi cells and how to incorporate Voronoi constraints into standard adversarial training. In Section 5 we show that adversarial training with Voronoi constraints gives state-of-the-art robustness on MNIST and competitive on CIFAR-10. Adversarial training, the process of training on adversarial examples generated in L p -balls around the training data, is a very natural approach to constructing robust models and was originally proposed by. formalized the adversarial training objective and highlighted the importance of a strong adversary for constructing adversarial examples in the inner training loop. Their approach to adversarial training, which utilized a projected gradient descent adversary, produced some of the first empirically robust models which were not later broken by stronger attacks. There's was the only approach surveyed by which was not either fully circumvented by or in a later paper . More recently, the celebrated algorithm TRADES ) has been proposed, which attempts to provide a principled way to trade off between robustness and natural accuracy. The analysis that inspires TRADES decomposes the robust error into two terms: natural error and error near the decision boundary. The yields an objective function with two terms, one which encourages accuracy and another which pushes the decision boundary away from the data distribution. Constructing a decision boundary that is far from the data distribution is explored in other heuristic works such as;;. Our approach falls into this class of defenses and so we will compare exclusively against such defenses. The frequency with which heuristic defenses have been defeated by stronger attacks has led to a line of work on certifiable robustness, which can guarantee that there exists no perturbation within an L pball of radius which causes the classifier to change its classification. One of the first works by proposed to approximate the set of possible activations of every L ∞ -bounded perturbation by propagating upper and lower bounds for each activation through the network. These upper and lower bounds are used to construct a convex outer approximation to the set of possible activations in the final layer, and a linear program is used to certify that this convex approximation does not intersect the decision boundary. This initial work had several notable drawbacks, and several subsequent works have attempted to improve upon these initial (; ; ; ;). However the fundamental problems have remained: these approaches do not scale to larger networks despite considerable effort, they often depend crucially on the specific details of the architecture, and the size of which can be certified is often considerably smaller than what we observe to be empirically robust. A different approach to certified robustness which addresses some of these concerns, called randomized smoothing , has recently been proposed. Randomized smoothing leverages the ability of any classifier f to perform well on Gaussian noise to construct a new classifier g which is certifiably robust under adversarial L 2 perturbations. Unlike prior approaches to certified robustness, randomized smoothing is a simple approach which does not depend on the architecture details of the classifier. Its main drawback is that it is currently, and possibly fundamentally, limited to L 2. We also note that more recent work has combined randomized smoothing with adversarial training to produce even more certifiably robustness classifiers in L 2 . Since the goal and limitations of these method are often different from heuristic approaches we do not compare our method against these approaches. Finally there has been a long line of work on the theory of adversarial examples. explore the sample complexity required to produce robust models. They demonstrate a simple setting, a mixture of two Gaussians, in which a linear classifier with near perfect natural accuracy can be learned from a single sample, but any algorithm that produces any binary classifier requires Ω(√ d) samples to produce a robust classifier. Followup work by suggests that adversarial examples may arise from computational constraints. They exhibit pairs of distributions that differ only in a k-dimensional subspace, and are otherwise standard Gaussians, and show that while it is information-theoretically possible to distinguish these distributions, it requires exponentially many queries in the statistical query model of computation. We note that both of these constructions produce distributions whose support is the entirety of R d. Additionally there is a line work that attempts to explain the pervasiveness of adversarial examples through the lens of high-dimensional geometry. The work of experimentally evaluated the setting of two concentric under-sampled 499-spheres embedded in R 500, and concluded that adversarial examples occur on the data manifold. suggest that adversarial examples may be an unavoidable consequence of the high-dimensional geometry of data. Their depends upon the use of an isopermetric inequality. The main drawback of these works, as well as the constructions in the previous paragraph, is that they assume that the support of the data distribution has full or nearly full dimension. We do not believe this to be the case in practice, instead we believe that the data distribution is often supported on a very low-dimensional subset of R d. This case is addressed in , where they consider the problem of adversarial robustness in the case where data is drawn from a low-dimensional manifold embedded in R d. They highlight the role of co-dimension, the difference between the dimension of the embedding space and the dimension of the data manifold, as a key source of the pervasiveness of adversarial vulnerability. Said differently, it is the low-dimensional structure of features embed-ded in high-dimensional space that contributes, at least in part, to adversarial examples. This idea is also explored in , but with emphasis on the cross-entropy loss. We build on the work of , specifically their on adversarial training in high-codimensions, which make clear several drawbacks of the L p -ball formulation. In Section 5.1 we show that our approach improves robustness in high-codimension settings. 3 originally proposed adversarial training where adversarial examples were constructed inside of an L p -ball of radius. The use of the L p -ball was meant to represent a simple notion of similarity between two images, delaying the complicated question of what is an adversarial image in favor of a tractable research problem. However it was never meant to be the final say on the threat model of the adversary and recent work has begun to explore alternative adversaries . describe a number of issues associated with the use of L p -balls. Their are formalized in the manifold setting, where samples from each class are sampled from one of C class manifolds M 1,..., M C, and the data manifold show that the L 2 -balls centered on a dense sample of M covers a negligible fraction of the neighborhood around M. Thus, when constructing adversarial examples in the inner training loop, the adversary is restricted to constructing adversarial examples in a negligible fraction of the neighborhood around the data manifold. This vulnerability increases with the codimension d − k of M. Furthermore they show that, for any p, a nearest neighbor classifier more effectively covers the neighborhood around M than a robust empirical risk minimization oracle, which outputs a classifier that is guaranteed to be correct in the L p -balls centered on the data. The Voronoi diagram for a dense sample drawn from a low-dimensional distribution with two classes, one in red and one in black. The Voronoi cells, shown in green, vary in size depending on how close a sample is to samples in the other class. The Voronoi edges that are adjacent to two samples from two different classes are shown in solid green, and approach a decision boundary which is as far from the data distribution as possible. To remedy these shortcomings, we replace the L p -ball constraint with a different geometric constraint, namely the Voronoi cell at each sample x, defined as In words, the Voronoi cell Vor p x of x is the set of all points in R d that are closer to x than to any other sample in X. The Voronoi diagram is defined as the collection of Voronoi cells, and their lower dimensional faces, for each sample in X. Figure 1 shows the Voronoi diagram for a dense sample from a dataset with two classes of data. The Voronoi cell constraint has many advantages over the L p -ball constraint. First the Voronoi cells partition the entirety of R d and so the interiors of Voronoi cells generated by samples from different classes do not intersect. This is in contrast to L p -balls which may intersect for sufficiently large. In particular the Voronoi cells partition the neighborhood around M and, for dense samples, are elongated in the directions normal to the data manifold . Thus the Voronoi cells are well suited for high codimension settings. Second, the size of the Voronoi cells adapts to the data distribution. A Voronoi cell generated by a sample which is close to samples from a different class manifold is smaller, while those further away are larger. See Figure 1. Thus we do not need to set a value for in the optimization procedure. The constraint naturally adapts to the largest value of possible locally on the data manifold. Note that the maximum perturbation size possible will often vary as we move along the data manifold, and cannot be captured by a single number which, by necessity, is upper bounded by the smallest distance to a different class. In summary, the Voronoi constraint gives the adversary the freedom to explore the entirety of the neighborhood around M. At each iteration of standard adversarial training, we must solve the inner optimization problem max δ∈B(0,) L(x + δ, y; θ) to generate an adversarial example. solve this problem using the fast gradient sign method (FGSM), while use projected gradient descent. To incorporate Voronoi constraints, at each iteration of the outer training loop we must solve the inner optimization problem maximizê x L(x, y; θ) When p = 2 the Voronoi cells are convex and so we can project a point onto a Voronoi cell by solving a quadratic program. Thus we can solve Problem 2 using projected gradient descent, as in. When p = 2 the Voronoi cells are not necessarily convex. In this setting there are many approaches, such as barrier and penalty methods, one might employ to approximately solve Problem 2 . However we found that the following heuristic is both fast and works well in practice. At each iteration of the outer training loop, for each training sample x in a batch, we generate adversarial examples by taking iterative steps in the direction of the gradient starting from x. Instead of projecting onto a constraint after each iterative step, we instead check if any of the Voronoi constraints of x shown in Equation 1 are violated. If no constraint is violated we perform the iterative update, otherwise we simply stop performing updates for x. Figure 2 illustrates the procedure. Figure 2: To construct an adversarial example within a Voronoi cell, we repeatedly take steps in the direction of the gradient of the loss, shown in blue. After each iteration we check if any of the Voronoi constraints are violated. We take the last iteration before a constraint is violated as our adversarial example. Problem 2 has n − 1 constraints, one for each sample in X\{x}. In practice however very few samples contribute to the Voronoi cell of x. Even fewer contribute to the faces of the Voronoi cell that are shared by samples in different classes, as shown in Figure 1. At each iteration, we perform a nearest neighbor search query to find the m nearest samples to x in each other class. That is we search for m(C − 1) samples where C is the number of classes. We do not impose constraints from samples in the same class as x; there is no benefit to restricting the adversary's movement with the neighborhood around the class manifold of x. In our experiments we set m = 10. 4 ADVERSARIAL TRAINING WITH VORONOI CONSTRAINTS formalize adversarial training by introducing the robust objective L(x, y; θ) where D is the data distribution and B is a L p -ball centered at x with radius. Their main contribution was the use of a strong adversary which used projected gradient descent to solve the inner optimization problem. To incorporate Voronoi constraints, we replace the L p -ball constraint in Equation 3 with the Voronoi cell at x. That is, we formalize the adversarial training objective as where we use the optimization procedure described in Section 3 to solve the inner optimization problem. Datasets. introduce a synthetic dataset, PLANES, to investigate how the codimension (low-rank features) of a dataset influences robustness. The PLANES dataset consists of two 2-dimensional planes, the first in the x d = 0 and the second in x d = 2. The first two axis of both planes are bounded as −10 ≤ x 1, x 2 ≤ 10, while x 3 =... = x d−1 = 0. The training set is sampled at the vertices of a regular grid with side length √ 2, and the test set at the centers of the grid cubes. This sampling is chosen so that the L 2 -balls of radius 1 cover the 2-dimensional planes, and so a classifier that does well inside these balls also has perfect natural accuracy. The spacing along the axis x d is chosen so the maximum perturbation size is 1. The codimension of this dataset is d − 2. We also evaluate on MNIST and CIFAR-10. Our controlled experiments on synthetic data consider a fully connected network with 1 hidden layer, 100 hidden units, and ReLU activations. We set the learning rate for Adam as α = 0.1. Our experimental are averaged over 20 retrainings. For a fair comparison to adversarial training, our experiments on MNIST and CIFAR-10 use the same model architectures as in. We train the MNIST model using Adam for 100 epochs and the CIFAR-10 model using SGD for 250 epochs. On MNIST we apply 300-step projected gradient descent (PGD), with step sizes {0.05, 0.07, 0.1, 0.15, 0.17, 0.2}. On CIFAR-10 we apply 20-step PGD with step sizes {2.0, 3.0, 4.0}. For both datasets we also apply the fast gradient sign method (FGSM) to uncover possible gradient masking as recommended in. We evaluate these attacks per sample, meaning that if any attack successfully constructs an adversarial example for a sample x at a specific, it reduces the robust accuracy of the model at that. Accuracy measures. We plot the robust classification accuracy as a function of, for each of our datasets. Since one of the primary advantages of Voronoi constraints is that we do not need to set, we need a measure of robustness that considers the total robustness of the model. Thus we report the normalized area under the curve (NAUC) defined as where acc: [0, max] → measures the classification accuracy and max is the largest perturbation considered. Note that NAUC ∈ with higher values corresponding to more robust models. Implementation Details. Constructing adversarial examples within the Voronoi cells, as described in Section 3, requires a nearest neighbor search query to find the m nearest samples to x in each other class. When the dataset remains constant throughout the course of training, this search can be performed once before training begins and reused at each iteration. However when the dataset is augmented during training, as in the case of data augmentation on CIFAR-10, the nearest neighbor search query must be computed at each iteration. Since this computation is performed on the CPU, we create 16 threads, each with a copy of a k-d tree, which constantly pull mini-batches of samples from a queue and perform nearest neighbor queries. With 16 threads running in parallel, the bottleneck for training became the construction of adversarial examples on the GPU, and so adversarial training with Voronoi constraints ran in time similar to standard adversarial training. showed that as the codimension of the PLANES dataset increases, the adversarial training approach of with training = 1 became less robust. They suggested that this was because the L 2 -balls with radius 1 around the dataset covered an increasingly smaller fraction of the neighborhood around the data manifold. To explore the performances of adversarial training with Voronoi constraints on more realistic datasets, we evaluate on MNIST and CIFAR-10 and compare against the robust pretrained models of. 12. We include the recently proposed Jacobian regularization algorithm of with λ jr = 1.0 as an additional baseline. Figure 4 (Left) shows that our model maintains near identical robustness to the Madry model on MNIST up to = 0.3, after which our model significantly outperforms the Madry model. The Madry model was explicitly trained for = 0.3 perturbations. We emphasize that one advantage of our approach is that we did not need to set a value for the maximum perturbation size. The Voronoi cells adapt to the maximum size allowable locally on the data distribution. Our model maintains 76.3% accuracy at = 0.4 compared to 2.6% accuracy for the Madry model. Furthermore our model achieves NAUC of 0.81, while the Madry model achieves NAUC of 0.67, an improvement of 20.8% and over the baseline. To our knowledge, this is the most robust MNIST model to L ∞ attacks. In particular, our model maintains 76.3% accuracy at = 0.4, compared to 2.6% accuracy for the Madry model. Right: On CIFAR-10, both models achieve NAUC of 0.29, but our model trades natural accuracy for robustness to larger perturbations. A natural approach to improving the robustness of models produced by the adversarial training paradigm of is to simply increase the maximum allowable perturbation size of the norm ball constraint. As shown in Figure 5, increasing the size of to 0.4, from the 0.3 with which originally trained, and training for only 100 epochs produces a model which exhibits significantly worse robustness in the range [0, 0.3] than the pretrained model. If we increase the number of training epochs to 150, the approach of We emphasize that our approach does not require us to set, which is particularly important in practice where the maximum amount of robustness achievable may not be known a-priori. The L p -ball constraint for describing adversarial perturbations has been a productive formalization for designing robust deep networks. However, the use of L p -balls has significant drawbacks in highcodimension settings and leads to sub-optimal in practice. Adversarial training with Voronoi constraints improves robustness by giving the adversary the freedom to explore the neighborhood around the data distribution.

Represent the TLDR for finding the corresponding scientific article

We replace the Lp ball constraint with the Voronoi cells of the training data to produce more robust models.

Represent this text to find a much much longer article about it with say hundreds-thousands of words

We study deep ensembles through the lens of loss landscape and the space of predictions, demonstrating that the decorrelation power of random initializations is unmatched by subspace sampling that only explores a single mode.


Question: Represent the text to find a very short summary (1-2 sentences max)

Question: Recent research efforts enable study for natural language grounded navigation in photo-realistic environments, e.g., following natural language instructions or dialog. However, existing methods tend to overfit training data in seen environments and fail to generalize well in previously unseen environments. In order to close the gap between seen and unseen environments, we aim at learning a generalizable navigation model from two novel perspectives: we introduce a multitask navigation model that can be seamlessly trained on both Vision-Language Navigation (VLN) and Navigation from Dialog History (NDH) tasks, which benefits from richer natural language guidance and effectively transfers knowledge across tasks; we propose to learn environment-agnostic representations for navigation policy that are invariant among environments, thus generalizing better on unseen environments. Extensive experiments show that our environment-agnostic multitask navigation model significantly reduces the performance gap between seen and unseen environments and outperforms the baselines on unseen environments by 16% (relative measure on success rate) on VLN and 120% (goal progress) on NDH, establishing the new state of the art for NDH task. Navigation in visual environments by following natural language guidance is a fundamental capability of intelligent robots that simulate human behaviors, because humans can easily reason about the language guidance and navigate efficiently by interacting with the visual environments. Recent efforts (b; ;) empower large-scale learning of natural language grounded navigation that is situated in photorealistic simulation environments. Nevertheless, the generalization problem commonly exists for these tasks, especially indoor navigation: the agent usually performs poorly on unknown environments that have never been seen during training. One of the main causes for such behavior is data scarcity as it is expensive and time-consuming to extend either visual environments or natural language guidance. The number of scanned houses for indoor navigation is limited due to high expense and privacy concerns. Besides, unlike vision-only navigation tasks (; ; Manolis Savva* et al., 2019;) where episodes can be exhaustively sampled in simulation, natural language grounded navigation is supported by human demonstrated interaction and communication in natural language. It is impractical to fully collect and cover all the samples for individual tasks. Therefore, it is essential though challenging to efficiently learn a more generalized policy for natural language grounded navigation tasks from existing data (a; b). In this paper, we study how to resolve the generalization and data scarcity issues from two different angles. First, previous methods are trained for one task at the time, so each new task requires training a brand new agent instance that can only solve the one task it was trained on. In this work, we propose a generalized multitask model for natural language grounded navigation tasks such as Vision-Language Navigation (VLN) and Navigation from Dialog History (NDH), aiming at efficiently transferring knowledge across tasks and effectively solving both tasks with one agent simultaneously. Moreover, although there are thousands of trajectories paired with language guidance, the underlying house scans are restricted. For instance, the popular Matterport3D dataset contains only 61 unique house scans in the training set. The current models perform much better in seen environments by taking advantage of the knowledge of specific houses they have acquired over multiple task completions during training, but fail to generalize to houses not seen during training. Hence we propose an environment-agnostic learning method to learn a visual representation that is invariant to specific environments but still able to support navigation. Endowed with the learned environment-agnostic representations, the agent is further prevented from the overfitting issue and generalizes better on unseen environments. To the best of our knowledge, we are the first to introduce natural language grounded multitask and environment-agnostic training regimes and validate their effectiveness on VLN and NDH tasks. Extensive experiments demonstrate that our environment-agnostic multitask navigation model can not only efficiently execute different language guidance in indoor environments but also outperform the single-task baseline models by a large margin on both tasks. Besides, the performance gap between seen and unseen environments is significantly reduced. We also set a new state of the art on NDH with over 120% improvement in terms of goal progress. Vision-and-Language Navigation. Vision-and-Language Navigation (b;) task requires an embodied agent to navigate in photo-realistic environments to carry out natural language instructions. The agent is spawned at an initial pose p 0 = (v 0, φ 0, θ 0), which includes the spatial location, heading and elevation angles. Given a natural language instruction X = {x 1, x 2, ..., x n}, the agent is expected to perform a sequence of actions {a 1, a 2, ..., a T} and arrive at the target position v tar specified by the language instruction X, which describes stepby-step instructions from the starting position to the target position. In this work, we consider VLN task defined for Room-to-Room (R2R) (b) dataset which contains instructiontrajectory pairs across 90 different indoor environments (houses). Previous VLN methods have studied various aspects to improve the navigation performance, such as planning, data augmentation , cross-modal alignment (; b), progress estimation (a), error correction (b;), interactive language assistance Nguyen & Daumé ) etc. This work tackles VLN via multitask learning and environmentagnostic learning, which is orthogonal to all these prior arts. Navigation from Dialog History. Different from Visual Dialog which involves dialog grounded in a single image, the recently introduced Cooperative Vision-and-Dialog Navigation (CVDN) dataset includes interactive language assistance for indoor navigation, which consists of over 2,000 embodied, human-human dialogs situated in photo-realistic home environments. The task of Navigation from Dialog History (NDH) is defined as: given a target object t 0 and a dialog history between humans cooperating to perform the task, the embodied agent must infer navigation actions towards the goal room that contains the target object. The dialog history is denoted as < t 0, Q 1, A 1, Q 2, A 2,..., Q i, A i >, including the target object t 0, the questions Q and answers A till the turn i (0 ≤ i ≤ k, where k is the total number of Q-A turns from the beginning to the goal room). The agent, located in p 0, is trying to move closer to the goal room by inferring from the dialog history that happened before. Multitask Learning. The basis of Multitask (MT) learning is the notion that tasks can serve as mutual sources of inductive bias for each other . When multiple tasks are trained jointly, MT learning causes the learner to prefer the hypothesis that explains all the tasks simultaneously, hence leading to more generalized solutions. MT learning has been successful in natural language processing , speech recognition , computer vision , drug discovery , and Atari games . The deep reinforcement learning methods that have become very popular for training models on natural language grounded navigation tasks (; a; b;) are known to be data inefficient. In this work, we introduce multitask reinforcement learning for such tasks to improve data efficiency by positive transfer across related tasks. Environment-agnostic Learning. A few studies on agnostic learning have been proposed recently. For example, Model-Agnostic Meta-Learning (MAML) aims to train a model on a variety of learning tasks and solve a new task using only a few training examples. proposes a unified feature disentangler that learns domain-invariant representation across multiple domains for image translation. Other domain-agnostic techniques are also proposed for supervised and unsupervised domain adaption . In this work, we pair the environment classifier with a gradient reversal layer to learn an environment-agnostic representation that can be better generalized on unseen environments in a zero-shot fashion where no adaptation is involved. Distributed Actor-Learner Navigation Learning Framework. To train models for the various language grounded navigation tasks like VLN and NDH, we develop a distributed actor-learner learning infrastructure 1. The framework design is inspired by IMPALA and uses its off-policy correction method called V-trace to efficiently scale reinforcement learning methods to thousands of machines. The framework additionally supports a variety of supervision strategies important for navigation tasks such as teacher-forcing (b), studentforcing (b) and mixed supervision . The framework is built using TensorFlow and supports ML accelerators (GPU, TPU). 3.1 OVERVIEW Our environment-agnostic multitask navigation model is illustrated in Figure 1. First, we adapt the reinforced cross-modal matching (RCM) model and make it seamlessly transfer across tasks by sharing all the learnable parameters for both NDH and VLN, including joint word embedding layer, language encoder, trajectory encoder, cross-modal attention module (CM-ATT), and action predictor. Furthermore, to learn the environment-agnostic representation z t, we equip the navigation model with an environment classifier whose objective is to predict which house the agent is. But note that between trajectory encoder and environment classifier, a gradient reversal layer is introduced to reverse the gradients backpropagated to the trajectory encoder, making it learn representations that are environment-agnostic and thus more generalizable in unseen environments. During training, the environment classifier is minimizing the environment classification loss L env, while the trajectory encoder is maximizing L env and minimizing the navigation loss L nav. The other modules are optimized with the navigation loss L nav simultaneously. Below we introduce multitask reinforcement learning and environmentagnostic representation learning. A more detailed model architecture is presented in Section 4. Interleaved Multitask Data Sampling. To avoid overfitting to individual tasks, we adopt an interleaved multitask data sampling strategy to train the model. Particularly, each data sample within a mini-batch can be from either task, so that the VLN instruction-trajectory pairs and NDH dialogtrajectory pairs are interleaved in a mini-batch though they may have different learning objectives. Reward Shaping. Following prior art, we first implement a discounted cumulative reward function R for the VLN and NDH tasks: where γ is the discounted factor, d(s t, v tar) is the distance between state s t and the target location v tar, and d th is the maximum distance from v tar that the agent is allowed to terminate for success. Different from VLN, NDH is essentially room navigation instead of point navigation because the agent is expected to reach a room that contains the target object. Suppose the goal room is occupied by a set of nodes {v i} N 1, we replace the distance function d(s t, v tar) in Equation 1 with the minimum distance to the goal room Navigation Loss. Since human demonstrations are available for both VLN and NDH tasks, we use behavior cloning to constrain the learning algorithm to model state-action spaces that are most relevant to each task. Following previous works , we also use reinforcement learning to aid the agent's ability to recover from erroneous actions in unseen environments. During multitask navigation model training, we adopt a mixed training strategy of reinforcement learning and behavior cloning, so the navigation loss function is: where we use REINFORCE policy gradients and supervised learning gradients to update the policy π. b is the estimated baseline to reduce the variance and a * t is the human demonstrated action. To further improve the generalizability of the navigation policy, we propose to learn a latent environment-agnostic representation that is invariant among seen environments. We would like to get rid of the environment-specific features that are irrelevant to general navigation (e.g. unique house appearances), preventing the model from overfitting to specific seen environments. We can reformulate the navigation policy as where z t is a latent representation. As shown in Figure 1, p(a t |z t, s t) is modeled by the policy module (including CM-ATT and action predictor) and p(z t |s t) is modeled by the trajectory encoder. In order to learn the environmentagnostic representation, we employ an environment classifier and a gradient reversal layer . The environment classifier is parameterized to predict the identity of the house where the agent is, so its loss function L env is defined as where y * is the ground-truth house label. The gradient reversal layer has no parameters. It acts as an identity transform during forward-propagation, but multiplies the gradient by −λ and passes it to the trajectory encoder during back-propagation. Therefore, in addition to minimizing the navigation loss L nav, the trajectory encoder is also maximizing the environment classification loss L env, trying to increase the entropy of the classifier in an adversarial learning manner where the classifier is minimizing the classification loss conditioned on the latent representation z t. Language Encoder. The natural language guidance (instruction or dialog) is tokenized and embedded into n-dimensional space X = {x 1, x 2, ..., x 3} where the word vectors x i are initialized randomly. The vocabulary is restricted to tokens that occur at least five times in the training instructions (The vocabulary used when jointly training VLN and NDH tasks is the union of the two tasks' vocabularies.). All out-of-vocabulary tokens are mapped to a single out-of-vocabulary identifier. The token sequence is encoded using a bi-directional LSTM to create H X following: where − → h X t and ← − h X t are the hidden states of the forward and backward LSTM layers at time step t respectively, and the σ function is used to combine − → h X t and Similar to benchmark models (; ; b), at each time step t, the agent perceives a 360-degree panoramic view at its current location. The view is discretized into k view angles (k = 36 in our implementation, 3 elevations by 12 headings at 30-degree intervals). The image at view angle i, heading angle φ and elevation angle θ is represented by a concatenation of the pre-trained CNN image features with the 4-dimensional orientation feature [sin φ; cos φ; sin θ; cos θ] to form v t,i. The visual input sequence V = {v 1, v 2, ..., v m} is encoded using a LSTM to create H V following: is the attention-pooled representation of all view angles using previous agent state h t−1 as the query. We use the dot-product attention hereafter. Policy Module. The policy module comprises of cross-modal attention (CM-ATT) unit as well as an action predictor. The agent learns a policy π θ over parameters θ that maps the natural language instruction X and the initial visual scene v 1 to a sequence of actions [a 1, a 2, ..., a n]. The action space which is common to VLN and NDH tasks consists of navigable directions from the current location. The available actions at time t are denoted as u t,1..l, where u t,j is the representation of the navigable direction j from the current location obtained similarly to v t,i. The number of available actions, l, varies per location, since graph node connectivity varies. As in , the model predicts the probability p d of each navigable direction d using a bilinear dot product: where c Environment Classifier. The environment classifier is a two-layer perceptron with a SoftMax layer as the last layer. Given the latent representation z t (which is h V t in our setting), the classifier generates a probability distribution over the house labels. Implementation Details. In the experiments, we use a 2-layer bi-directional LSTM for the instruction encoder where the size of LSTM cells is 256 units in each direction. The inputs to the encoder are 300-dimensional embeddings initialized randomly. For the visual encoder, we use a 2-layer LSTM with a cell size of 512 units. The encoder inputs are image features derived as mentioned in Section 4. The cross-modal attention layer size is 128 units. The environment classifier has one hidden layer of size 128 units followed by an output layer of size equal to the number of classes. During training, some episodes in the batch are identical to available human demonstrations in the training dataset where the objective is to increase the agent's likelihood of choosing human actions (behavioral cloning ). The rest of the episodes are constructed by sampling Figure 2: Selected tokens from the vocabulary for VLN (left) and NDH (right) tasks which gained more than 40 additional occurrences in the training dataset due to joint-training. from agent's own policy. In the experiments, unless otherwise stated, we use entire dialog history from NDH task for model training. All the reported in subsequent studies are averages of at least 3 independent runs. Evaluation Metrics. The agents are evaluated on two datasets, namely Validation Seen that contains new paths from the training environments and Validation Unseen that contains paths from previously unseen environments. The evaluation metrics for VLN task are as follows: Path Length (PL) measures the total length of the predicted path; Navigation Error (NE) measures the distance between the last nodes in the predicted and the reference paths; Success Rate (SR) measures how often the last node in the predicted path is within some threshold distance of the last node in the reference path; Success weighted by Path Length (SPL) (a) measures Success Rate weighted by the normalized Path Length; and Coverage weighted by Length Score (CLS) measures predicted path's conformity to the reference path weighted by length score. For NDH task, the agent's progress is defined as reduction (in meters) from the distance to the goal region at agent's first position versus at its last position . Table 1 shows the of training the navigation model using environment-agnostic learning (EnvAg) as well as multitask learning (MT-RCM). First, both learning methods independently help the agent learn more generalized navigation policy as is evidenced by significant reduction in agent's performance gap between seen and unseen environments. For instance, performance gap for agent's goal progress on NDH task drops from 3.85m to 0.92m using multitask learning and agent's success rate on VLN task between seen and unseen datasets drops from 9.26% to 8.39% using environmentagnostic learning. Second, the two techniques are complementary-the agent's performance when trained with both the techniques simultaneously improves on unseen environments compared to when trained separately. Finally, we note here that MT-RCM + EnvAg outperforms the state-of-theart goal progress of 2.10m on NDH validation unseen dataset by more than 120%. At the same time, it outperforms the equivalent RCM baseline of 40.6% success rate by more than 16% (relative measure) on VLN validation unseen dataset. Next, we conduct studies to examine cross-task transfer using multitask learning alone. One of the main advantages of multitask learning is that under-represented tokens in each of the individual tasks get a significant boost in the number of training samples. Figure 2 illustrates that tokens with less than 40 occurrences end up with sometimes more than 300 occurrences during joint-training. To examine the impact of dialog history in NDH task, we conduct studies with access to different parts of the dialog-the target object t o, the last oracle answer A i, the prefacing navigator question Q i and the full dialog history. Table 2 shows the of jointly training MT-RCM model on VLN and NDH tasks. MT-RCM model learns a generalized policy that consistently outperforms the competing model with access to similar parts of the dialog on previously unseen environments. As noted before, multitask learning significantly reduces the gap between the agent's performance on previously seen and unseen environments for both tasks. Furthermore, we see a consistent and gradual increase in the success rate of MT-RCM on VLN task as it is trained on paths with richer dialog history from the NDH task. This shows that the agent benefits from more complete information about the path implying the importance given by the agent to the language instructions in the task. We also investigate the impact of parameter sharing of the language encoder for both tasks. As shown in Table 3, the model with shared language encoder for NDH and VLN tasks outperforms the model that has separate language encoders for the two tasks, hence demonstrating the importance of parameter sharing during multitask learning. A more detailed analysis can be found in the Appendix. From Table 1, it can be seen that both VLN and NDH tasks benefit from environment-agnostic learning independently. To further examine the generalization property due to environment-agnostic objective, we train a model with the opposite objective-learn to correctly predict the navigation environments by removing the gradient reversal layer (environment-aware learning). Interesting Figure 3: t-SNE visualization of trajectory encoder's output (1000 random paths across 11 different color-coded environments) for models trained with environment-aware objective (left) versus environment-agnostic objective (right). are observed in Table 4 that environment-aware learning leads to overfitting on the training dataset (performance on environments seen during training consistently increases for both tasks), while environment-agnostic learning leads to more generalizable policy which performs better on previously unseen environments. Figure 3 further shows that due to environment-aware objective, the model learns to represent visual inputs from the same environment closer to each other while the representations of different environments are farther from each other ing in a clustering learning effect. On the other hand, the environment-agnostic objective leads to more general representation across different environments which in better performance on unseen environments. As discussed in Section 3.2, we conducted studies to shape the reward for NDH task. The in Table 5 indicate that incentivizing the agent to get closer to the goal room is better than to the exact goal location, because it is aligned with the objective of NDH task, which is to reach the room containing the goal object. Detailed ablation is presented in Appendix showing that the same holds true consistently as the agent is provided access to different parts of the dialog history. In this work, we show that the model trained using environment-agnostic multitask learning approach learns a generalized policy for the two natural language grounded navigation tasks. It closes down the gap between seen and unseen environments, learns more generalized environment representations and effectively transfers knowledge across tasks outperforming baselines on both the tasks simultaneously by a significant margin. At the same time, the two approaches independently benefit the agent learning and are complementary to each other. There are possible future extensions to our work-the MT-RCM can further be adapted to other language-grounded navigation datasets, such as those using Street View (e.g., Touchdown Table 6 presents a more detailed ablation of Table 5 using different parts of dialog history. The prove that agents rewarded for getting closer to the goal room consistently outperform agents rewarded for getting closer to the exact goal location. Table 7 presents a more detailed analysis from Table 3 with access to different parts of dialog history. The models with shared language encoder consistently outperform those with separate encoders. Figure 4: Visualizing performance gap between seen and unseen environments for VLN and NDH tasks. For VLN, the plotted metric is agent's success rate while for NDH, the metric is agent's progress. As mentioned in Section 5.2, both multitask learning as well as environment-agnostic learning methods reduce the agent's performance gap between seen and unseen environments which is demonstrated in Figure 4.

Represent the short summary for finding the corresponding scientific article (long)

We propose to learn a more generalized policy for natural language grounded navigation tasks via environment-agnostic multitask learning.

Represent the TLDR for finding the corresponding scientific article

We propose a novel Intrinsically Motivated Goal Exploration architecture with unsupervised learning of goal space representations, and evaluate how various implementations enable the discovery of a diversity of policies.