Question: Represent text

Question: Convolutional neural networks (CNNs) are inherently equivariant to translation. Efforts to embed other forms of equivariance have concentrated solely on rotation. We expand the notion of equivariance in CNNs through the Polar Transformer Network (PTN). PTN combines ideas from the Spatial Transformer Network (STN) and canonical coordinate representations. The is a network invariant to translation and equivariant to both rotation and scale. PTN is trained end-to-end and composed of three distinct stages: a polar origin predictor, the newly introduced polar transformer module and a classifier. PTN achieves state-of-the-art on rotated MNIST and the newly introduced SIM2MNIST dataset, an MNIST variation obtained by adding clutter and perturbing digits with translation, rotation and scaling. The ideas of PTN are extensible to 3D which we demonstrate through the Cylindrical Transformer Network. Whether at the global pattern or local feature level BID8, the quest for (in/equi)variant representations is as old as the field of computer vision and pattern recognition itself. State-of-the-art in "hand-crafted" approaches is typified by SIFT . These detector/descriptors identify the intrinsic scale or rotation of a region BID19 BID1 and produce an equivariant descriptor which is normalized for scale and/or rotation invariance. The burden of these methods is in the computation of the orbit (i.e. a sampling the transformation space) which is necessary to achieve equivariance. This motivated steerable filtering which guarantees transformed filter responses can be interpolated from a finite number of filter responses. Steerability was proved for rotations of Gaussian derivatives BID6 and extended to scale and translations in the shiftable pyramid BID31. Use of the orbit and SVD to create a filter basis was proposed by BID26 and in parallel, BID29 proved for certain classes of transformations there exists canonical coordinates where deformation of the input presents as translation of the output. Following this work, BID25 and BID10; Teo & BID33 proposed a methodology for computing the bases of equivariant spaces given the Lie generators of a transformation. and most recently, BID30 proposed the scattering transform which offers representations invariant to translation, scaling, and rotations. The current consensus is representations should be learned not designed. Equivariance to translations by convolution and invariance to local deformations by pooling are now textbook BID17, p.335) but approaches to equivariance of more general deformations are still maturing. The main veins are: Spatial Transformer Network (STN) BID13 which similarly to SIFT learn a canonical pose and produce an invariant representation through warping, work which constrains the structure of convolutional filters BID36 and work which uses the filter orbit BID3 to enforce an equivariance to a specific transformation group. In this paper, we propose the Polar Transformer Network (PTN), which combines the ideas of STN and canonical coordinate representations to achieve equivariance to translations, rotations, and dilations. The three stage network learns to identify the object center then transforms the input into logpolar coordinates. In this coordinate system, planar convolutions correspond to group-convolutions in rotation and scale. PTN produces a representation equivariant to rotations and dilations without http://github.com/daniilidis-group//polar-transformer-networks Figure 1: In the log-polar representation, rotations around the origin become vertical shifts, and dilations around the origin become horizontal shifts. The distance between the yellow and green lines is proportional to the rotation angle/scale factor. Top rows: sequence of rotations, and the corresponding polar images. Bottom rows: sequence of dilations, and the corresponding polar images.the challenging parameter regression of STN. We enlarge the notion of equivariance in CNNs beyond Harmonic Networks BID36 and Group Convolutions BID3 by capturing both rotations and dilations of arbitrary precision. Similar to STN; however, PTN accommodates only global deformations. We present state-of-the-art performance on rotated MNIST and SIM2MNIST, which we introduce. To summarize our contributions:• We develop a CNN architecture capable of learning an image representation invariant to translation and equivariant to rotation and dilation.• We propose the polar transformer module, which performs a differentiable log-polar transform, amenable to backpropagation training. The transform origin is a latent variable.• We show how the polar transform origin can be learned effectively as the centroid of a single channel heatmap predicted by a fully convolutional network. One of the first equivariant feature extraction schemes was proposed by BID25 who suggested the discrete sampling of 2D-rotations of a complex angle modulated filter. About the same time, the image and optical processing community discovered the Mellin transform as a modification of the Fourier transform BID39 BID0. The Fourier-Mellin transform is equivariant to rotation and scale while its modulus is invariant. During the 80's and 90's invariances of integral transforms were developed through methods based in the Lie generators of the respective transforms starting from one-parameter transforms BID5 and generalizing to Abelian subgroups of the affine group BID29.Closely related to the (in/equi)variance work is work in steerability, the interpolation of responses to any group action using the response of a finite filter basis. An exact steerability framework began in BID6, where rotational steerability for Gaussian derivatives was explicitly computed. It was extended to the shiftable pyramid BID31, which handle rotation and scale. A method of approximating steerability by learning a lower dimensional representation of the image deformation from the transformation orbit and the SVD was proposed by BID26.for the largest Abelian subgroup and incrementally steering for the remaining subgroups. Cohen & Welling (2016a); recently combined steerability and learnable filters. The most recent "hand-crafted" approach to equivariant representations is the scattering transform BID30 which composes rotated and dilated wavelets. Similar to SIFT this approach relies on the equivariance of anchor points (e.g. the maxima of filtered responses in (translation) space). Translation invariance is obtained through the modulus operation which is computed after each convolution. The final scattering coefficient is invariant to translations and equivariant to local rotations and scalings. BID15 achieve transformation invariance by pooling feature maps computed over the input orbit, which scales poorly as it requires forward and backward passes for each orbit element. Within the context of CNNs, methods of enforcing equivariance fall to two main veins. In the first, equivariance is obtained by constraining filter structure similarly to Lie generator based approaches BID29 BID10. Harmonic Networks BID36 use filters derived from the complex harmonics achieving both rotational and translational equivariance. The second requires the use of a filter orbit which is itself equivariant to obtain group equivariance. BID3 convolve with the orbit of a learned filter and prove the equivariance of group-convolutions and preservation of rotational equivariance in the presence of rectification and pooling. BID4 process elements of the image orbit individually and use the set of outputs for classification. BID7 produce maps of finite-multiparameter groups, BID38 and BID21 use a rotational filter orbit to produce oriented feature maps and rotationally invariant features, and BID18 propose a transformation layer which acts as a group-convolution by first permuting then transforming by a linear filter. Our approach, PTN, is akin to the second vein. We achieve global rotational equivariance and expand the notion of CNN equivariance to include scaling. PTN employs log-polar coordinates (canonical coordinates in BID29) to achieve rotation-dilation group-convolution through translational convolution subject to the assumption of an image center estimated similarly to the STN. Most related to our method is BID11, which achieves equivariance by warping the inputs to a fixed grid, with no learned parameters. When learning features from 3D objects, invariance to transformations is usually achieved through augmenting the training data with transformed versions of the inputs BID37, or pooling over transformed versions during training and/or test BID22 BID27. BID28 show that a multi-task approach, i.e. prediction of both the orientation and class, improves classification performance. In our extension to 3D object classification, we explicitly learn representations equivariant to rotations around a family of parallel axes by transforming the input to cylindrical coordinates about a predicted axis. This section is divided into two parts, the first offers a review of equivariance and groupconvolutions. The second offers an explicit example of the equivariance of group-convolutions through the 2D similarity transformations group, SIM, comprised of translations, dilations and rotations. Reparameterization of SIM to canonical coordinates allows for the application of the SIM group-convolution using translational convolution. Equivariant representations are highly sought after as they encode both class and deformation information in a predictable way. Let G be a transformation group and L g I be the group action applied to an image I. A mapping Φ: E → F is said to be equivariant to the group action DISPLAYFORM0 where L g and L g correspond to application of g to E and F respectively and satisfy DISPLAYFORM1 Invariance is the special case of equivariance where L g is the identity. In the context of image classification and CNNs, g ∈ G can be thought of as an image deformation and Φ a mapping from the image to a feature map. The inherent translational equivariance of CNNs is independent of the convolutional kernel and evident in the corresponding translation of the output in response to translation of the input. Equivariance to other types of deformations can be achieved through application of the group-convolution, a generalization of translational convolution. Letting f (g) and φ(g) be real valued functions on G DISPLAYFORM2 A slight modification to the definition is necessary in the first CNN layer since the group is acting on the image. The group-convolution reduces to translational convolution when G is translation in R n with addition as the group operator, DISPLAYFORM3 Group-convolution requires integrability over a group and identification of the appropriate measure dg. It can be proved that given the measure dg, group-convolution is always group equivariant: DISPLAYFORM4 This is depicted in response of an equivariant representation to input deformation (Figure 2 (left)). A similarity transformation, ρ ∈ SIM, acts on a point in x ∈ R 2 by DISPLAYFORM0 where SO is the rotation group. To take advantage of the standard planar convolution in classical CNNs we decompose a ρ ∈ SIM into a translation, t in R 2 and a dilated-rotation r in SO×R +.Equivariance to SIM FORMULA2 is achieved by learning the center of the dilated rotation, shifting the original image accordingly then transforming the image to canonical coordinates. In this reparameterization the standard translational convolution is equivalent to the dilated-rotation group-convolution. The origin predictor is an application of STN to global translation prediction BID13, the centroid of the output is taken as the origin of the input. Transformation of the image L t I = I(t − t 0) (canonization in Soatto FORMULA0) reduces the SIM deformation to a dilated-rotation if t o is the true translation. After centering, we perform SO × R + convolutions on the new image I o = I(x − t o): DISPLAYFORM1 and the feature maps f in subsequent layers DISPLAYFORM2 where r, s ∈ SO × R +. We compute this convolution through use of canonical coordinates for Abelian Lie-groups BID29. The centered image I o (x, y) 1 is transformed to logpolar coordinates, I(e ξ cos(θ), e ξ sin(θ)) hereafter written λ(ξ, θ) with (ξ, θ) ∈ SO × R + for Figure 2: Left: Group-convolutions in SO. The images in the left most column differ by 90• rotation, the filters are shown in the top row. Application of the rotational group-convolution with an arbitrary filter is shown to produce an equivariant representation. The inner-product each of filter orbit (rotated from 0 − 360 •) and the image is plotted in blue for the top image and red for the bottom image. Observe how the filter response is shifted by 90•. Right: Group-convolutions in SO × R +. Images in the left most column differ by a rotation of π/4 and scaling of 1.2. Careful consideration of the ing heatmaps (shown in canonical coordinates) reveals a shift corresponding to the deformation of the input image.notational convenience. The shift of the dilated-rotation equivariant representation in response to input deformation is shown in Figure 2 (right) using canonical coordinates. In canonical coordinates s −1 r = ξ r − ξ, θ r − θ and the SO × R + group-convolution 2 can be expressed and efficiently implemented as a planar convolution DISPLAYFORM3 To summarize, we construct a network of translational convolutions, take the centroid of the last layer, shift the original image to accordingly, convert to log-polar coordinates, and apply a second network 3 of translational convolutions. The is a feature map equivariant to dilated-rotations around the origin. PTN is comprised of two main components connected by the polar transformer module. The first part is the polar origin predictor and the second is the classifier (a conventional fully convolutional network). The building block of the network is a 3 × 3 × K convolutional layer followed by batch normalization, an ReLU and occasional subsampling through strided convolution. We will refer to this building block simply as block. Figure 3 shows the architecture. The polar origin predictor operates on the original image and comprises a sequence of blocks followed by a 1 × 1 convolution. The output is a single channel feature map, the centroid of which is taken as the origin of the polar transform. There are some difficulties in training a neural network to predict coordinates in images. Some approaches BID35 attempt to use fully connected layers to directly regress the coordinates with limited success. A better option is to predict heatmaps BID34 BID24, and take their argmax. However, this can be problematic since backpropogation gradients are zero in all but one point, which impedes learning.1 we abuse the notation here and momentarily we use x as the x-coordinate instead of x ∈ R 2. 2 abuse of the term, SO × R + is not a group because the dilation ξ is not compact. 3 the network employs rectifier and pooling which have been shown to preserve equivariance BID3 Figure 3: Network architecture. The input image passes through a fully convolutional network, the polar origin predictor, which outputs a heatmap. The centroid of the heatmap (two coordinates), together with the input image, goes into the polar transformer module, which performs a polar transform with origin at the input coordinates. The obtained polar representation is invariant with respect to the original object location; and rotations and dilations are now shifts, which are handled equivariantly by a conventional classifier CNN.The usual approach to heatmap prediction is evaluation of a loss against some ground truth. In this approach the argmax gradient problem is circumvented by supervision. In PTN the the gradient of the output coordinates must be taken with respect to the heatmap since the polar origin is unknown and must be learned. Use of argmax is avoided by using the centroid of the heatmap as the polar origin. The gradient of the centroid with respect to the heatmap is constant and nonzero for all points, making learning possible. The polar transformer module takes the origin prediction and image as inputs and outputs the logpolar representation of the input. The module uses the same differentiable image sampling technique as STN BID13, which allows output coordinates V i to be expressed in terms of the input U and the source sample point coordinates (x s i, y s i). The log-polar transform in terms of the source sample points and target regular grid (x t i, y t i) is: DISPLAYFORM0 DISPLAYFORM1 where (x 0, y 0) is the origin, W, H are the output width and height, and r is the maximum distance from the origin, set to 0.5 √ H 2 + W 2 in our experiments. To maintain feature map resolution, most CNN implementations use zero-padding. This is not ideal for the polar representation, as it is periodic about the angular axis. A rotation of the input in a vertical shift of the output, wrapping at the boundary; hence, identification of the top and bottom most rows is most appropriate. This is achieved with wrap-around padding on the vertical dimension. The top most row of the feature map is padded using the bottom rows and vice versa. Zero-padding is used in the horizontal dimension. TAB5 shows a performance evaluation. To improve robustness of our method, we augment the polar origin during training time by adding a random shift to the regressed polar origin coordinates. Note that this comes for little computational cost compared to conventional augmentation methods such as rotating the input image. TAB5 quantifies the performance gains of this kind of augmentation. We briefly define the architectures in this section, see A for details. CCNN is a conventional fully convolutional network; PCNN is the same, but applied to polar images with central origin. STN is our implementation of the spatial transformer networks BID13. PTN is our polar transformer networks, and PTN-CNN is a combination of PTN and CCNN. The suffixes S and B indicate small and big networks, according to the number of parameters. The suffixes + and ++ indicate training and training+test rotation augmentation. We perform rotation augmentation for polar-based methods. In theory, the effect of input rotation is just a shift in the corresponding polar image, which should not affect the classifier CNN. In practice, interpolation and angle discretization effects in slightly different polar images for rotated inputs, so even the polar-based methods benefit from this kind of augmentation. TAB1 shows the . We divide the analysis in two parts; on the left, we show approaches with smaller networks and no rotation augmentation, on the right there are no restrictions. Between the restricted approaches, the Harmonic Network BID36 outperforms the PTN by a small margin, but with almost 4x more training time, because the convolutions on complex variables are more costly. Also worth mentioning is the poor performance of the STN with no augmentation, which shows that learning the transformation parameters is much harder than learning the polar origin coordinates. Between the unrestricted approaches, most variants of PTN-B outperform the current state of the art, with significant improvements when combined with CCNN and/or test time augmentation. Finally, we note that the PCNN achieves a relatively high accuracy in this dataset because the digits are mostly centered, so using the polar transform origin as the image center is reasonable. Our method, however, outperforms it by a high margin, showing that even in this case, it is possible to find an origin away from the image center that in a more distinctive representation. FORMULA0 6 Test time performance is 8x slower when using test time augmentation We also perform experiments in other MNIST variants. MNIST R, RTS are replicated from BID13. We introduce SIM2MNIST, with a more challenging set of transformations from SIM. See B for more details about the datasets. TAB2 shows the . We can see that the PTN performance mostly matches the STN on both MNIST R and RTS. The deformations on these datasets are mild and data is plenty, so the performance may be saturated. On SIM2MNIST, however, the deformations are more challenging and the training set 5x smaller. The PCNN performance is significantly lower, which reiterates the importance of predicting the best polar origin. The HNet outperforms the other methods (except the PTN), thanks to its translation and rotation equivariance properties. Our method is more efficient both in number of parameters and training time, and is also equivariant to dilations, achieving the best performance by a large margin. BID13 0 DISPLAYFORM0.28 (0.05) 44k 31.42 TI-Pooling BID15 0.8 DISPLAYFORM1 No augmentation is used with SIM2MNIST, despite the + suffixes 2 Our modified version, with two extra layers with subsampling to account for larger input We visualize network activations to confirm our claims about invariance to translation and equivariance to rotations and dilations. Figure 4 (left) shows some of the predicted polar origins and the of the polar transform. We can see that the network learns to reject clutter and to find a suitable origin for the polar transform, and that the representation after the polar transformer module does present the properties claimed. We proceed to visualize if the properties are preserved in deeper layers. FIG1 (right) shows the activations of selected channels from the last convolutional layer, for different rotations, dilations, and translations of the input. The reader can verify that the equivariance to rotations and dilations, and the invariance to translations are indeed preserved during the sequence of convolutional layers. We extend our model to perform 3D object classification from voxel occupancy grids. We assume that the inputs are transformed by random rotations around an axis from a family of parallel axes. Then, a rotation around that axis corresponds to a translation in cylindrical coordinates. In order to achieve equivariance to rotations, we predict an axis and use it as the origin to transform to cylindrical coordinates. If the axis is parallel to one of the input grid axes, the cylindrical transform amounts to channel-wise polar transforms, where the origin is the same for all channels and each channel is a 2D slice of the 3D voxel grid. In this setting, we can just apply the polar transformer layer to each slice. We use a technique similar to the anisotropic probing of BID27 to predict the axis. Let z denote the input grid axis parallel to the rotation axis. We treat the dimension indexed by z as channels, and run regular 2D convolutional layers, reducing the number of channels on each layer, eventually collapsing to a single 2D heatmap. The heatmap centroid gives one point of the axis, and the direction is parallel to z. In other words, the centroid is the origin of all channel-wise polar transforms. We then proceed with a regular 3D CNN classifier, acting on the cylindrical representation. The 3D convolutions are equivariant to translations; since they act on cylindrical coordinates, the learned representation is equivariant to input rotations around axes parallel to z. We run experiments on ModelNet40 BID37, which contains objects rotated around the gravity direction (z). FIG2 shows examples of input voxel grids and their cylindrical coordinates representation, while table 3 shows the classification performance. To the best of our knowledge, our method outperforms all published voxel-based methods, even with no test time augmentation. However, the multi-view based methods generally outperform the voxel-based. BID27.Note that we could also achieve equivariance to scale by using log-cylindrical or log-spherical coordinates, but none of these change of coordinates would in equivariance to arbitrary 3D rotations. Cylindrical Transformer (Ours) 86.5 89.9 3D ShapeNets BID37 77.3 -VoxNet BID22 83 -MO-SubvolumeSup BID27 86.0 89.2 MO-Aniprobing BID27 85.6 89.9 We have proposed a novel network whose output is invariant to translations and equivariant to the group of dilations/rotations. We have combined the idea of learning the translation (similar to the spatial transformer) but providing equivariance for the scaling and rotation, avoiding, thus, fully connected layers required for the pose regression in the spatial transformer. Equivariance with respect to dilated rotations is achieved by convolution in this group. Such a convolution would require the production of multiple group copies, however, we avoid this by transforming into canonical coordinates. We improve the state of the art performance on rotated MNIST by a large margin, and outperform all other tested methods on a new dataset we call SIM2MNIST. We expect our approach to be applicable to other problems, where the presence of different orientations and scales hinder the performance of conventional CNNs. We implement the following architectures for comparison,• Conventional CNN (CCNN), a fully convolutional network, composed of a sequence of convolutional layers and some rounds of subsampling.• Polar CNN (PCNN), same architecture as CCNN, operating on polar images. The logpolar transform is pre-computed at the image center before training, as in BID11. The fundamental difference between our method and this is that we learn the polar origin implicitly, instead of fixing it.• Spatial Transformer Network (STN), our implementation of BID13, replacing the localization network by four blocks of 20 filters and stride 2, followed by a 20 unit fully connected layer, which we found to perform better. The transformation regressed is in SIM, and a CCNN comes after the transform.• Polar Transformer Network (PTN), our proposed method. The polar origin predictor comprises three blocks of 20 filters each, with stride 2 on the first block (or the first two blocks, when input is 96 × 96). The classification network is the CCNN.• PTN-CNN, we classify based on the sum of the per class scores of instances of PTN and CCNN trained independently. The following suffixes qualify the architectures described above:• S, "small" network, with seven blocks of 20 filters and one round of subsampling (equivalent to the Z2CNN in Cohen & Welling (2016b)).• B, "big" network, with 8 blocks with the following number of filters: 16, 16, 32, 32, 32, 64, 64, 64. Subsampling by strided convolution is used whenever the number of filters increase. We add up to two 2 extra blocks of 16 filters with stride 2 at the beginning to handle larger input resolutions (one for 42 × 42 and two for 96 × 96).• +, training time rotation augmentation by continuous angles.• ++, training and test time rotation augmentation. We input 8 rotated versions the the query image and classify using the sum of the per class scores. The axis prediction part of the cylindrical transformer network is composed of four 2D blocks, with 5 × 5 kernels and 32, 16, 8, and 4 channels, no subsampling. The classifier is composed of eight 3D convolutional blocks, with 3 × 3 × 3 kernels, the following number of filters: 32, 32, 32, 64, 64, 64, 128, 128, and subsampling whenever the number of filters increase. Total number of params is approximately 1M. • Rotated MNIST The rotated MNIST dataset BID16 is composed of 28 × 28, 360• rotated images of handwritten digits. The training, validation and test sets are of sizes 10k, 2k, and 50k, respectively.• MNIST R, we replicate it from BID13. It has 60k training and 10k testing samples, where the digits of the original MNIST are rotated between [−90 DISPLAYFORM0 It is also know as half-rotated MNIST BID15 .• MNIST RTS, we replicate it from BID13 . It has 60k training and 10k testing samples, where the digits of the original MNIST are rotated between [−45 DISPLAYFORM1 •], scaled between 0.7 and 1.2, and shifted within a 42 × 42 black canvas.• SIM2MNIST, we introduce a more challenging dataset, based on MNIST, perturbed by random transformations from SIM. The images are 96 × 96, with 360 • rotations; the scale factors range from 1 to 2.4, and the digits can appear anywhere in the image. The training, validation and test set have size 10k, 5k, and 50k, respectively. In order to demonstrate the efficacy of PTN on real-world RGB images, we run experiments on the Street View House Numbers (SVHN) dataset BID23, and a rotated version that we introduce (ROTSVHN). The dataset contains cropped images of single digits, as well as the slightly larger images from where the digits are cropped. Using the latter, we can extract the rotated digits without introducing artifacts. FIG3 shows some examples from the ROTSVHN.We use a 32 layer Residual Network BID9 as a baseline (ResNet32). The PTN-ResNet32 has 8 residual convolutional layers as the origin predictor, followed by a ResNet32.In contrast with handwritten digits, the 6s and 9s in house numbers are usually indistinguishable. To remove this effect from our analysis, we also run experiments removing those classes from the datasets (which is denoted by appending a minus to the dataset name). TAB4 shows the . The reader will note that rotations cause a significant performance loss on the conventional ResNet; the error increases from 2.09% to 5.39%, even when removing 6s and 9s from the dataset. With PTN, on the other hand, the error goes from 2.85% to 3.96%, which shows our method is more robust to the perturbations, although the performance on the unperturbed datasets is slightly worse. We expect the PTN to be even more advantageous when large scale variations are also present. We quantify the performance boost obtained with wrap around padding, polar origin augmentation, and training time rotation augmentation. Results are based on the PTN-B variant trained on Rotated MNIST. We remove one operation at a time and verify that the performance consistently drops, which indicates that all operations are indeed helpful. TAB5 shows the .

Represent the short summary for finding the corresponding scientific article (long).

We learn feature maps invariant to translation, and equivariant to rotation and scale.

Represent this text to find a much much longer article about it with say hundreds-thousands of words

We propose a differentiable family of "kaleidoscope matrices," prove that all structured matrices can be represented in this form, and use them to replace hand-crafted linear maps in deep learning models.


Question: Represent the text to find a very short summary (1-2 sentences max)

Question: Meta-learning allows an intelligent agent to leverage prior learning episodes as a basis for quickly improving performance on a novel task. Bayesian hierarchical modeling provides a theoretical framework for formalizing meta-learning as inference for a set of parameters that are shared across tasks. Here, we reformulate the model-agnostic meta-learning algorithm (MAML) of as a method for probabilistic inference in a hierarchical Bayesian model. In contrast to prior methods for meta-learning via hierarchical Bayes, MAML is naturally applicable to complex function approximators through its use of a scalable gradient descent procedure for posterior inference. Furthermore, the identification of MAML as hierarchical Bayes provides a way to understand the algorithm’s operation as a meta-learning procedure, as well as an opportunity to make use of computational strategies for efficient inference. We use this opportunity to propose an improvement to the MAML algorithm that makes use of techniques from approximate inference and curvature estimation. A remarkable aspect of human intelligence is the ability to quickly solve a novel problem and to be able to do so even in the face of limited experience in a novel domain. Such fast adaptation is made possible by leveraging prior learning experience in order to improve the efficiency of later learning. This capacity for meta-learning also has the potential to enable an artificially intelligent agent to learn more efficiently in situations with little available data or limited computational resources BID45 BID4 BID37.In machine learning, meta-learning is formulated as the extraction of domain-general information that can act as an inductive bias to improve learning efficiency in novel tasks (; BID52 . This inductive bias has been implemented in various ways: as learned hyperparameters in a hierarchical Bayesian model that regularize task-specific parameters BID18, as a learned metric space in which to group neighbors BID7, as a trained recurrent neural network that allows encoding and retrieval of episodic information BID43, or as an optimization algorithm with learned parameters BID45 BID3 .The model-agnostic meta-learning (MAML) of BID12 is an instance of a learned optimization procedure that directly optimizes the standard gradient descent rule. The algorithm estimates an initial parameter set to be shared among the task-specific models; the intuition is that gradient descent from the learned initialization provides a favorable inductive bias for fast adaptation. However, this inductive bias has been evaluated only empirically in prior work BID12.In this work, we present a novel derivation of and a novel extension to MAML, illustrating that this algorithm can be understood as inference for the parameters of a prior distribution in a hierarchical Bayesian model. The learned prior allows for quick adaptation to unseen tasks on the basis of an implicit predictive density over task-specific parameters. The reinterpretation as hierarchical Bayes gives a principled statistical motivation for MAML as a meta-learning algorithm, and sheds light on the reasons for its favorable performance even among methods with significantly more parameters. More importantly, by casting gradient-based meta-learning within a Bayesian framework, we are able to improve MAML by taking insights from Bayesian posterior estimation as novel augmentations to the gradient-based meta-learning procedure. We experimentally demonstrate that this enables better performance on a few-shot learning benchmark. The goal of a meta-learner is to extract task-general knowledge through the experience of solving a number of related tasks. By using this learned prior knowledge, the learner has the potential to quickly adapt to novel tasks even in the face of limited data or limited computation time. Formally, we consider a dataset D that defines a distribution over a family of tasks T. These tasks share some common structure such that learning to solve a single task has the potential to aid in solving another. Each task T defines a distribution over data points x, which we assume in this work to consist of inputs and either regression targets or classification labels y in a supervised learning problem (although this assumption can be relaxed to include reinforcement learning problems; e.g., see BID12 . The objective of the meta-learner is to be able to minimize a task-specific performance metric associated with any given unseen task from the dataset given even only a small amount of data from the task; i.e., to be capable of fast adaptation to a novel task. In the following subsections, we discuss two ways of formulating a solution to the meta-learning problem: gradient-based hyperparameter optimization and probabilistic inference in a hierarchical Bayesian model. These approaches were developed orthogonally, but, in Section 3.1, we draw a novel connection between the two. A parametric meta-learner aims to find some shared parameters θ that make it easier to find the right task-specific parameters φ when faced with a novel task. A variety of meta-learners that employ gradient methods for task-specific fast adaptation have been proposed (e.g., BID2 BID26 BID57 . MAML BID12 is distinct in that it provides a gradient-based meta-learning procedure that employs a single additional parameter (the meta-learning rate) and operates on the same parameter space for both meta-learning and fast adaptation. These are necessary features for the equivalence we show in Section 3.1.To address the meta-learning problem, MAML estimates the parameters θ of a set of models so that when one or a few batch gradient descent steps are taken from the initialization at θ given a small sample of task data x j 1,..., x j N ∼ p T j (x) each model has good generalization performance on another sample x j N +1,..., x j N +M ∼ p T j (x) from the same task. The MAML objective in a maximum likelihood setting is DISPLAYFORM0 where we use φ j to denote the updated parameters after taking a single batch gradient descent step from the initialization at θ with step size α on the negative log-likelihood associated with the task T j. Note that since φ j is an iterate of a gradient descent procedure that starts from θ, each φ j is of the same dimensionality as θ. We refer to the inner gradient descent procedure that computes φ j as fast adaptation. The computational graph of MAML is given in Figure 1 (left). An alternative way to formulate meta-learning is as a problem of probabilistic inference in the hierarchical model depicted in Figure 1 (right). In particular, in the case of meta-learning, each task-specific parameter φ j is distinct from but should influence the estimation of the parameters {φ j | j = j} from other tasks. We can capture this intuition by introducing a meta-level parameter θ on which each task-specific parameter is statistically dependent. With this formulation, the mutual dependence of the task-specific parameters φ j is realized only through their individual dependence DISPLAYFORM0 The computational graph of the MAML BID12 algorithm covered in Section 2.1.Straight arrows denote deterministic computations and crooked arrows denote sampling operations. (Right) The probabilistic graphical model for which MAML provides an inference procedure as described in Section 3.1. In each figure, plates denote repeated computations (left) or factorization (right) across independent and identically distributed samples.on the meta-level parameters θ. As such, estimating θ provides a way to constrain the estimation of each of the φ j.Given some data in a multi-task setting, we may estimate θ by integrating out the task-specific parameters to form the marginal likelihood of the data. Formally, grouping all of the data from each of the tasks as X and again denoting by x j 1,..., x j N a sample from task T j, the marginal likelihood of the observed data is given by DISPLAYFORM1 Maximizing FORMULA2 as a function of θ gives a point estimate for θ, an instance of a method known as empirical Bayes BID5 BID15 due to its use of the data to estimate the parameters of the prior distribution. Hierarchical Bayesian models have a long history of use in both transfer learning and domain adaptation (e.g., BID25 BID58 BID14 BID8 BID56 . However, the formulation of meta-learning as hierarchical Bayes does not automatically provide an inference procedure, and furthermore, there is no guarantee that inference is tractable for expressive models with many parameters such as deep neural networks. In this section, we connect the two independent approaches of Section 2.1 and Section 2.2 by showing that MAML can be understood as empirical Bayes in a hierarchical probabilistic model. Furthermore, we build on this understanding by showing that a choice of update rule for the taskspecific parameters φ j (i.e., a choice of inner-loop optimizer) corresponds to a choice of prior over task-specific parameters, p(φ j | θ). In general, when performing empirical Bayes, the marginalization over task-specific parameters φ j in FORMULA2 is not tractable to compute exactly. To avoid this issue, we can consider an approximation that makes use of a point estimateφ j instead of performing the integration over φ in. Usingφ j as an estimator for each φ j, we may write the negative logarithm of the marginal likelihood as FORMULA3 recovers the unscaled form of the one-step MAML objective in. This tells us that the MAML objective is equivalent to a maximization with respect to the meta-level parameters θ of the marginal likelihood p(X | θ), where a point estimate for each task-specific parameter φ j is computed via one or a few steps of gradient descent. By taking only a few steps from the initialization at θ, the point estimateφ j trades off DISPLAYFORM0 DISPLAYFORM1 DISPLAYFORM2 Algorithm 2: Model-agnostic meta-learning as hierarchical Bayesian inference. The choices of the subroutine ML-· · · that we consider are defined in Subroutine 3 and Subroutine 4. DISPLAYFORM3 Subroutine 3: Subroutine for computing a point estimateφ using truncated gradient descent to approximate the marginal negative log likelihood (NLL).minimizing the fast adaptation objective − log p(x j 1, . . ., x j N | θ) with staying close in value to the parameter initialization θ. We can formalize this trade-off by considering the linear regression case. Recall that the maximum a posteriori (MAP) estimate of φ j corresponds to the global mode of the posterior DISPLAYFORM4 In the case of a linear model, early stopping of an iterative gradient descent procedure to estimate φ j is exactly equivalent to MAP estimation of φ j under the assumption of a prior that depends on the number of descent steps as well as the direction in which each step is taken. In particular, write the input examples as X and the vector of regression targets as y, omit the task index from φ, and consider the gradient descent update DISPLAYFORM5 for iteration index k and learning rate α ∈ R +. shows that, starting from φ = θ, φ (k) in solves the regularized linear least squares problem DISPLAYFORM6 with Q-norm defined by z Q = z T Q −1 z for a symmetric positive definite matrix Q that depends on the step size α and iteration index k as well as on the covariance structure of X. We describe the exact form of the dependence in Section 3.2. The minimization in can be expressed as a posterior maximization problem given a conditional Gaussian likelihood over y and a Gaussian prior over φ. The posterior takes the form DISPLAYFORM7 Since φ (k) in maximizes, we may conclude that k iterations of gradient descent in a linear regression model with squared error exactly computes the MAP estimate of φ, given a Gaussian-noised observation model and a Gaussian prior over φ with parameters µ 0 = θ and Σ 0 = Q. Therefore, in the case of linear regression with squared error, MAML is exactly empirical Bayes using the MAP estimate as the point estimate of φ. In the nonlinear case, MAML is again equivalent to an empirical Bayes procedure to maximize the marginal likelihood that uses a point estimate for φ computed by one or a few steps of gradient descent. However, this point estimate is not necessarily the global mode of a posterior. We can instead understand the point estimate given by truncated gradient descent as the value of the mode of an implicit posterior over φ ing from an empirical loss interpreted as a negative log-likelihood, and regularization penalties and the early stopping procedure jointly acting as priors (for similar interpretations, see BID47 BID6 BID9 .The exact equivalence between early stopping and a Gaussian prior on the weights in the linear case, as well as the implicit regularization to the parameter initialization the nonlinear case, tells us that every iterate of truncated gradient descent is a mode of an implicit posterior. In particular, we are not required to take the gradient descent procedure of fast adaptation that computesφ to convergence in order to establish a connection between MAML and hierarchical Bayes. MAML can therefore be understood to approximate an expectation of the marginal negative log likelihood (NLL) for each task T j as DISPLAYFORM8 The algorithm for MAML as probabilistic inference is given in Algorithm 2; Subroutine 3 computes each marginal NLL using the point estimate ofφ as just described. Formulating MAML in this way, as probabilistic inference in a hierarchical Bayesian model, motivates the interpretation in Section 3.2 of using various meta-optimization algorithms to induce a prior over task-specific parameters. From Section 3.1, we may conclude that early stopping during fast adaptation is equivalent to a specific choice of a prior over task-specific parameters, p(φ j | θ). We can better understand the role of early stopping in defining the task-specific parameter prior in the case of a quadratic objective. Omit the task index from φ and x, and consider a second-order approximation of the fast adaptation objective (φ) = − log p(x 1 . . ., x N | φ) about a minimum φ *: DISPLAYFORM0 where the Hessian H = ∇ 2 φ (φ *) is assumed to be positive definite so that˜ is bounded below. Furthermore, consider using a curvature matrix B to precondition the gradient in gradient descent, giving the update DISPLAYFORM1 If B is diagonal, we can identify as a Newton method with a diagonal approximation to the inverse Hessian; using the inverse Hessian evaluated at the point φ (k−1) recovers Newton's method itself. On the other hand, meta-learning the matrix B matrix via gradient descent provides a method to incorporate task-general information into the covariance of the fast adaptation prior, p(φ | θ). For instance, the meta-learned matrix B may encode correlations between parameters that dictates how such parameters are updated relative to each other. Formally, taking k steps of gradient descent from φ = θ using the update rule in gives a φ (k) that solves DISPLAYFORM2 The minimization in corresponds to taking a Gaussian prior p(φ | θ) with mean θ and covariance BID44 where B is a diagonal matrix that from a simultaneous diagonalization of H and B as O T HO = diag(λ 1, . . ., λ n) = Λ and DISPLAYFORM3 DISPLAYFORM4.., n (Theorem 8.7.1 in BID16 . If the true objective is indeed quadratic, then, assuming the data is centered, H is the unscaled covariance matrix of features, X T X. Identifying MAML as a method for probabilistic inference in a hierarchical model allows us to develop novel improvements to the algorithm. In Section 4.1, we consider an approach from Bayesian parameter estimation to improve the MAML algorithm, and in Section 4.2, we discuss how to make this procedure computationally tractable for high-dimensional models. We have shown that the MAML algorithm is an empirical Bayes procedure that employs a point estimate for the mid-level, task-specific parameters in a hierarchical Bayesian model. However, the use of this point estimate may lead to an inaccurate point approximation of the integral in if the posterior over the task-specific parameters, p(φ j | x j N +1, . . ., x j N +M, θ), is not sharply peaked at the value of the point estimate. The Laplace approximation BID24 BID29 a) is applicable in this case as it replaces a point estimate of an integral with the volume of a Gaussian centered at a mode of the integrand, thereby forming a local quadratic approximation. We can make use of this approximation to incorporate uncertainty about the task-specific parameters into the MAML algorithm at fast adaptation time. In particular, suppose that each integrand in has a mode φ * j at which it is locally well-approximated by a quadratic function. The Laplace approximation uses a second-order Taylor expansion of the negative log posterior in order to approximate each integral in the product in as DISPLAYFORM0 where H j is the Hessian matrix of second derivatives of the negative log posterior. Classically, the Laplace approximation uses the MAP estimate for φ * j, although any mode can be used as an expansion site provided the integrand is well enough approximated there by a quadratic. We use the point estimateφ j uncovered by fast adaptation, in which case the MAML objective in becomes an appropriately scaled version of the approximate marginal likelihood DISPLAYFORM1 The term log p(φ j | θ) from the implicit regularization imposed by early stopping during fast adaptation, as discussed in Section 3.1. The term 1 /2 log det(H j), on the other hand, from the Laplace approximation and can be interpreted as a form of regularization that penalizes model complexity. Using as a training criterion for a neural network model is difficult due to the required computation of the determinant of the Hessian of the log posterior H j, which itself decomposes into a sum of the Hessian of the log likelihood and the Hessian of the log prior as DISPLAYFORM0 In our case of early stopping as regularization, the prior over task-specific parameters p(φ j | θ) is implicit and thus no closed form is available for a general model. Although we may use the quadratic approximation derived in Section 3.2 to obtain an approximate Gaussian prior, this prior is not diagonal and does not, to our knowledge, have a convenient factorization. Therefore, in our experiments, we instead use a simple approximation in which the prior is approximated as a diagonal Gaussian with precision τ. We keep τ fixed, although this parameter may be cross-validated for improved performance. DISPLAYFORM1 Subroutine 4: Subroutine for computing a Laplace approximation of the marginal likelihood. Similarly, the Hessian of the log likelihood is intractable to form exactly for all but the smallest models, and furthermore, is not guaranteed to be positive definite at all points, possibly rendering the Laplace approximation undefined. To combat this, we instead seek a curvature matrixĤ that approximates the quadratic curvature of a neural network objective function. Since it is well-known that the curvature associated with neural network objective functions is highly non-diagonal (e.g., BID32, a further requirement is that the matrix have off-diagonal terms. Due to the difficulties listed above, we turn to second order gradient descent methods, which precondition the gradient with an inverse curvature matrix at each iteration of descent. The Fisher information matrix BID13 has been extensively used as an approximation of curvature, giving rise to a method known as natural gradient descent BID1 . A neural network with an appropriate choice of loss function is a probabilistic model and therefore defines a Fisher information matrix. Furthermore, the Fisher information matrix can be seen to define a convex quadratic approximation to the objective function of a probabilistic neural model BID38 BID31 . Importantly for our use case, the Fisher information matrix is positive definite by definition as well as non-diagonal. However, the Fisher information matrix is still expensive to work with. BID33 developed Kronecker-factored approximate curvature (K-FAC), a scheme for approximating the curvature of the objective function of a neural network with a block-diagonal approximation to the Fisher information matrix. Each block corresponds to a unique layer in the network, and each block is further approximated as a Kronecker product (see BID54 of two much smaller matrices by assuming that the second-order statistics of the input activation and the back-propagated derivatives within a layer are independent. These two approximations ensure that the inverse of the Fisher information matrix can be computed efficiently for the natural gradient. For the Laplace approximation, we are interested in the determinant of a curvature matrix instead of its inverse. However, we may also make use of the approximations to the Fisher information matrix from K-FAC as well as properties of the Kronecker product. In particular, we use the fact that the determinant of a Kronecker product is the product of the exponentiated determinants of each of the factors, and that the determinant of a block diagonal matrix is the product of the determinants of the blocks BID54 . The determinants for each factor can be computed as efficiently as the inverses required by DISPLAYFORM2 We make use of the Laplace approximation and K-FAC to replace Subroutine 3, which computes the task-specific marginal NLLs using a point estimate forφ. We call this method the Lightweight Laplace Approximation for Meta-Adaptation (LLAMA), and give a replacement subroutine in Subroutine 4. The goal of our experiments is to evaluate if we can use our probabilistic interpretation of MAML to generate samples from the distribution over adapted parameters, and futhermore, if our method can be applied to large-scale meta-learning problems such as miniImageNet. and amplitudes, and the interpretation of the method as hierarchical Bayes makes it practical to directly sample models from the posterior. In this figure, we illustrate various samples from the posterior of a model that is meta-trained on different sinusoids, when presented with a few datapoints (in red) from a new, previously unseen sinusoid. Note that the random samples from the posterior predictive describe a distribution of functions that are all sinusoidal and that there is increased uncertainty when the datapoints are less informative (i.e., when the datapoints are sampled only from the lower part of the range input, shown in the bottom-right example). The connection between MAML and hierarchical Bayes suggests that we should expect MAML to behave like an algorithm that learns the mean of a Gaussian prior on model parameters, and uses the mean of this prior as an initialization during fast adaptation. Using the Laplace approximation to the integration over task-specific parameters as in assumes a task-specific parameter posterior with mean at the adapted parametersφ and covariance equal to the inverse Hessian of the log posterior evaluated at the adapted parameter value. Instead of simply using this density in the Laplace approximation as an additional regularization term as in FORMULA0, we may sample parameters φ j from this density and use each set of sampled parameters to form a set of predictions for a given task. To illustrate this relationship between MAML and hierarchical Bayes, we present a meta-dataset of sinusoid tasks in which each task involves regressing to the output of a sinusoid wave in Figure We observe in FIG0 that our method allows us to directly sample models from the task-specific parameter distribution after being presented with 10 datapoints from a new, previously unseen sinusoid curve. In particular, the column on the right of FIG0 demonstrates that the sampled models display an appropriate level of uncertainty when the datapoints are ambiguous (as in the bottom right). We evaluate LLAMA on the miniImageNet BID40 1-shot, 5-way classification task, a standard benchmark in few-shot classification. miniImageNet comprises 64 training classes, 12 validation classes, and 24 test classes. Following the setup of BID55, we structure the N -shot, J-way classification task as follows: The model observes N instances of J unseen classes, and is evaluated on its ability to classify M new instances within the J classes. We use a neural network architecture standard to few-shot classification (e.g., BID55 BID40, consisting of 4 layers with 3 × 3 convolutions and 64 filters, followed by batch normalization (BN) BID19, a ReLU nonlinearity, and 2 × 2 max-pooling. For the scaling variable β and centering variable γ of BN (see BID19, we ignore the fast adaptation update as well as the Fisher factors for K-FAC. We use Adam BID20 as the meta-optimizer, and standard batch gradient descent with a fixed learning rate to update the model BID53 49.82 ± 0.78 BID12 48.70 ± 1.84 LLAMA (Ours) DISPLAYFORM0 49.40 ± 1.83 during fast adaptation. LLAMA requires the prior precision term τ as well as an additional parameter η ∈ R + that weights the regularization term log detĤ contributed by the Laplace approximation. We fix τ = 0.001 and selected η = 10 −6 via cross-validation; all other parameters are set to the values reported in BID12.We find that LLAMA is practical enough to be applied to this larger-scale problem. In particular, our TensorFlow implementation of LLAMA trains for 60,000 iterations on one TITAN Xp GPU in 9 hours, compared to 5 hours to train MAML. As shown in TAB1, LLAMA achieves comparable performance to the state-of-the-art meta-learning method by BID53. While the gap between MAML and LLAMA is small, the improvement from the Laplace approximation suggests that a more accurate approximation to the marginalization over task-specific parameters will lead to further improvements. Meta-learning and few-shot learning have a long history in hierarchical Bayesian modeling (e.g., BID51 BID11 BID25 BID58 BID14 BID8 BID56 . A related subfield is that of transfer learning, which has used hierarchical Bayes extensively (e.g., BID39 . A variety of inference methods have been used in Bayesian models, including exact inference BID22, sampling methods BID41, and variational methods BID10 . While some prior works on hierarchical Bayesian models have proposed to handle basic image recognition tasks, the complexity of these tasks does not yet approach the kinds of complex image recognition problems that can be solved by discriminatively trained deep networks, such as the miniImageNet experiment in our evaluation BID30 .Recently, the Omniglot benchmark Lake et al. FORMULA0 has rekindled interest in the problem of learning from few examples. Modern methods accomplish few-shot learning either through the design of network architectures that ingest the few-shot training samples directly (e.g., BID21 BID55 BID48 BID17 BID53, or formulating the problem as one of learning to learn, or meta-learning (e.g., BID45 BID4 BID46 BID3 . A variety of inference methods have been used in Bayesian models, including exact inference , sampling methods BID42, and variational methods BID10.Our work bridges the gap between gradient-based meta-learning methods and hierarchical Bayesian modeling. Our contribution is not to formulate the meta-learning problem as a hierarchical Bayesian model, but instead to formulate a gradient-based meta-learner as hierarchical Bayesian inference, thus providing a way to efficiently perform posterior inference in a model-agnostic manner. We have shown that model-agnostic meta-learning (MAML) estimates the parameters of a prior in a hierarchical Bayesian model. By casting gradient-based meta-learning within a Bayesian framework, our analysis opens the door to novel improvements inspired by probabilistic machinery. As a step in this direction, we propose an extension to MAML that employs a Laplace approximation to the posterior distribution over task-specific parameters. This technique provides a more accurate estimate of the integral that, in the original MAML algorithm, is approximated via a point estimate. We show how to estimate the quantity required by the Laplace approximation using Kroneckerfactored approximate curvature (K-FAC), a method recently proposed to approximate the quadratic curvature of a neural network objective for the purpose of a second-order gradient descent technique. Our contribution illuminates the road to exploring further connections between gradient-based metalearning methods and hierarchical Bayesian modeling. For instance, in this work we assume that the predictive distribution over new data-points is narrow and well-approximated by a point estimate. We may instead employ methods that make use of the variance of the distribution over task-specific parameters in order to model the predictive density over examples from a novel task. Furthermore, it is known that the Laplace approximation is inaccurate in cases where the integral is highly skewed, or is not unimodal and thus is not amenable to approximation by a single Gaussian mode. This could be solved by using a finite mixture of Gaussians, which can approximate many density functions arbitrarily well BID49 BID0. The exploration of additional improvements such as this is an exciting line of future work.

represent this text to find a much much longer article about it with say hundreds-thousands of words

------

To give you a sense - "We propose a method that can make use of the multiple passages information for open-domain QA." should be close to "Very recently, it comes to be a popular approach for answering open-domain questions by first searching question-related passages, then applying reading comprehension models to extract answers. Existing works usually extract answers from single passages independently, thus not fully make use of the multiple searched passages, especially for the some questions requiring several evidences, which can appear in different passages, to be answered. The above observations raise the problem of evidence aggregation from multiple passages. In this paper, we deal with this problem as answer re-ranking. Specifically, based on the answer candidates generated from the existing state-of-the-art QA model, we propose two different re-ranking methods, strength-based and coverage-based re-rankers, which make use of the aggregated evidences from different passages to help entail the ground-truth answer for the question. Our model achieved state-of-the-arts on three public open-domain QA datasets, Quasar-T, SearchQA and the open-domain version of TriviaQA, with about 8\% improvement on the former two datasets. Open-domain question answering (QA) aims to answer questions from a broad range of domains by effectively marshalling evidence from large open-domain knowledge sources. Such resources can be Wikipedia, the whole web BID12, structured knowledge bases BID2 or combinations of the above (Baudiš &Šedivỳ, 2015).Recent work on open-domain QA has focused on using unstructured text retrieved from the web to build machine comprehension models BID9. These studies adopt a two-step process: an information retrieval (IR) model to coarsely select passages relevant to a question, followed by a reading comprehension (RC) model BID26 to infer an answer from the passages. These studies have made progress in bringing together evidence from large data sources, but they predict an answer to the question with only a single retrieved passage at a time. However, answer accuracy can often be improved by using multiple passages. In some cases, the answer can only be determined by combining multiple passages. In this paper, we propose a method to improve open-domain QA by explicitly aggregating evidence from across multiple passages. Our method is inspired by two notable observations from previous open-domain QA analysis:• First, compared with incorrect answers, the correct answer is often suggested by more passages repeatedly. For example, in FIG0 (a), the correct answer "danny boy" has more passages providing evidence relevant to the question compared to the incorrect one. This observation can be seen as multiple passages collaboratively enhancing the evidence for the correct answer.• Second, sometimes the question covers multiple answer aspects, which spreads over multiple passages. In order to infer the correct answer, one has to find ways to aggregate those multiple passages in an effective yet sensible way to try to cover all aspects. In FIG0 the correct answer "Galileo Galilei" at the bottom has passages P1, "Galileo was a physicist..." and P2, "Galileo discovered the first 4 moons of Jupiter", mentioning two pieces of evidence to match the question. In this case, the aggregation of these two pieces of evidence can help entail the ground-truth answer "Galileo Galilei". In comparison, the incorrect answer "Isaac Newton" has passages providing partial evidence on only "physicist, mathematician and astronomer". This observation illustrates the way in which multiple passages may provide complementary evidence to better infer the correct answer to a question. To provide more accurate answers for open-domain QA, we hope to make better use of multiple passages for the same question by aggregating both the strengthened and the complementary evidence from all the passages. We formulate the above evidence aggregation as an answer re-ranking problem. Re-ranking has been commonly used in NLP problems, such as in parsing and translation, in order to make use of high-order or global features that are too expensive for decoding algorithms BID6 BID27 BID16 BID11. Here we apply the idea of re-ranking; for each answer candidate, we efficiently incorporate global information from multiple pieces of textual evidence without significantly increasing the complexity of the prediction of the RC model. Specifically, we first collect the top-K candidate answers based on their probabilities computed by a standard RC/QA system, and then we use two proposed re-rankers to re-score the answer candidates by aggregating each candidate's evidence in different ways. The re-rankers are:• A strength-based re-ranker, which ranks the answer candidates according to how often their evidence occurs in different passages. The re-ranker is based on the first observation if an answer candidate has multiple pieces of evidence, and each passage containing some evidence tends to predict the answer with a relatively high score (although it may not be the top score), then the candidate is more likely to be correct. The passage count of each candidate, and the aggregated probabilities for the candidate, reflect how strong its evidence is, and thus in turn suggest how likely the candidate is the corrected answer.• A coverage-based re-ranker, which aims to rank an answer candidate higher if the union of all its contexts in different passages could cover more aspects included in the question. To achieve this, for each answer we concatenate all the passages that contain the answer together. The is a new context that aggregates all the evidence necessary to entail the answer for the question. We then treat the new context as one sequence to represent the answer, and build an attention-based match-LSTM model between the sequence and the question to measure how well the new aggregated context could entail the question. Overall, our contributions are as follows: 1) We propose a re-ranking-based framework to make use of the evidence from multiple passages in open-domain QA, and two re-rankers, namely, a strengthbased re-ranker and a coverage-based re-ranker, to perform evidence aggregation in existing opendomain QA datasets. We find the second re-ranker performs better than the first one on two of the three public datasets. 2) Our proposed approach leads to the state-of-the-art on three different datasets (Quasar-T BID9, SearchQA BID10 and TriviaQA BID17) and outperforms previous state of the art by large margins. In particular, we achieved up to 8% improvement on F1 on both Quasar-T and SearchQA compared to the previous best . Given a question q, we are trying to find the correct answer a g to q using information retrieved from the web. Our method proceeds in two phases. First, we run an IR model (with the help of a search engine such as google or bing) to find the top-N web passages p 1, p 2,..., p N most related to the question. Then a reading comprehension (RC) model is used to extract the answer from these passages. This setting is different from standard reading comprehension tasks (e.g. BID24), where a single fixed passage is given, from which the answer is to be extracted. When developing a reading comprehension system, we can use the specific positions of the answer sequence in the given passage for training. By contrast, in the open-domain setting, the RC models are usually trained under distant supervision BID9 BID17. Specifically, since the training data does not have labels indicating the positions of the answer spans in the passages, during the training stage, the RC model will match all passages that contain the ground-truth answer with the question one by one. In this paper we apply an existing RC model called R 3 to extract these candidate answers. After the candidate answers are extracted, we aggregate evidence from multiple passages by reranking the answer candidates. Given a question q, suppose we have a baseline open-domain QA system that can generate the top-K answer candidates a 1,..., a K, each being a text span in some passage p i. The goal of the re-ranker is to rank this list of candidates so that the top-ranked candidates are more likely to be the correct answer a g. With access to these additional features, the re-ranking step has the potential to prioritize answers not easily discoverable by the base system alone. We investigate two re-ranking strategies based on evidence strength and evidence coverage. An overview of our method is shown in FIG1. In open-domain QA, unlike the standard RC setting, we have more passages retrieved by the IR model and the ground-truth answer may appear in different passages, which means different answer spans may correspond to the same answer. To exploit this property, we provide two features to further re-rank the top-K answers generated by the RC model. This method is based on the hypothesis that the more passages that entail a particular answer, the stronger the evidence for that answer and the higher it should be ranked. To implement this we count the number of occurrences of each answer in the top-K answer spans generated by the baseline QA model and return the answer with the highest count. Measuring Strength by Probability Since we can get the probability of each answer span in a passages based on the RC model, we can also sum up the probabilities of the answer spans that are referring to the same answer. In this method, the answer with the highest probability is the final prediction 1. In the re-ranking scenario, it is not necessary to exhaustively consider all the probabilities of all the spans in the passages, as there may be a large number of different answer spans and most of them are irrelevant to the ground-truth answer. Remark: Note that neither of the above methods require any training. Both just take the candidate predictions from the baseline QA system and perform counting or probability calculations. At test time, the time complexity of strength-based re-ranking is negligible. Consider FIG0 where the two answer candidates both have evidence matching the first half of the question. Note that only the correct answer has evidence that could also match the second half. In this case, the strength-based re-ranker will treat both answer candidates the same due to the equal amount of supporting evidence, while the second answer has complementary evidence satisfying all aspects of the question. To handle this case, we propose a coverage-based re-ranker that ranks the answer candidates according to how well the union of their evidence from different passages covers the question. In order to take the union of evidence into consideration, we first concatenate the passages containing the answer into a single "pseudo passage" then measure how well this passage entails the answer for the question. As in the examples shown in FIG0 (b), we hope the textual entailment model will reflect (i) how each aspect of the question is matched by the union of multiple passages; and (ii) whether all the aspects of the question can be matched by the union of multiple passages. In our implementation an "aspect" of the question is a hidden state of a bi-directional LSTM BID15. The match-LSTM BID31 model is one way to achieve the above effect in entailment. Therefore we build our coverage-based re-ranker on top of the concatenated pseudo passages using the match-LSTM. The detailed method is described below. Passage Aggregation We consider the top-K answers, a 1,..., a K, provided by the baseline QA system. For each answer a k, k ∈ [1, K], we concatenate all the passages that contain a k, {p n |a k ∈ p n, n ∈ [1, N]}, to form the union passagep k. Our further model is to identify which union passage, e.g.,p k, could better entail its answer, e.g., a k, for the question. Measuring Aspect(Word)-Level Matching As discussed earlier, the first mission of the coverage-based re-ranker is to measure how each aspect of the question is matched by the union of multiple passages. We achieve this with word-by-word attention followed by a comparison module. First, we write the answer candidate a, question q and the union passagep of a as matrices A, Q,P, with each column being the embedding of a word in the sequence. We then feed them to the bidirectional LSTM as follows: DISPLAYFORM0 where H a ∈ R l×A, H q ∈ R l×Q and H p ∈ R l×P are the hidden states for the answer candidate, question and passage respectively; l is the dimension of the hidden states, and A, Q and P are the length of the three sequences, respectively. Next, we enhance the question representation H q with H a: DISPLAYFORM1 where [·; ·] is the concatenation of two matrices in row and H aq ∈ R l×(A+Q). As most of the answer candidates do not appear in the question, this is for better matching with the passage and finding more answer-related information from the passage.2 Now we can view each aspect of the question as a column vector (i.e. a hidden state at each word position in the answer-question concatenation) in the enhanced question representation H aq. Then the task becomes to measure how well each column vector can be matched by the union passage; and we achieve this by computing the attention vector Parikh et al. FORMULA0 for each hidden state of sequences a and q as follows: DISPLAYFORM2 where α ∈ R P ×(A+Q) is the attention weight matrix which is normalized in column through softmax. H aq ∈ R l×(A+Q) are the attention vectors for each word of the answer and the question by weighted summing all the hidden states of the passagep. Now in order to see whether the aspects in the question can be matched by the union passage, we use the following matching function: DISPLAYFORM3 where · ⊗ e (A+Q) is to repeat the vector (or scalar) on the left A + Q times; (· ·) and (· − ·) are the element-wise operations for checking whether the word in the answer and question can be matched by the evidence in the passage. We also concatenate these matching representations with the hidden state representations H aq and H aq, so that the lexical matching representations are also integrated into the the final aspect-level matching representations 3 M ∈ R 2l×(A+Q), which is computed through the non-linear transformation on four different representations with parameters DISPLAYFORM4 Measuring the Entire Question Matching Next, in order to measure how the entire question is matched by the union passagep by taking into consideration of the matching at each aspect, we add another bi-directional LSTM on top of it to aggregate the aspect-level matching information 4: DISPLAYFORM5 where H m ∈ R l×(A+Q) is to denote all the hidden states and h s ∈ R l, the max of pooling of each dimension of H m, is the entire matching representation which reflects how well the evidences in questions could be matched by the union passage.2 Besides concatenating H q with H a, there are other ways to make the matching process be aware of an answer's positions in the passage, e.g. replacing the answer spans in the passage to a special token like in. We tried this approach, which gives similar but no better , so we keep the concatenation in this paper. We leave the study of the better usage of answer position information for future work. 4 Note that we use LSTM here to capture the conjunction information (the dependency) among aspects, i.e. how all the aspects are jointly matched. In comparison simple pooling methods will treat the aspects independently. Low-rank tensor inspired neural architectures (e.g., BID21) could be another choice and we will investigate them in future work. Re-ranking Objective Function Our re-ranking is based on the entire matching representation. For each candidate answer a k, k ∈ [1, K], we can get a matching representation h s k between the answer a k, question q and the union passagep k through Eqn.. Then we transform all representations into scalar values followed by a normalization process for ranking: DISPLAYFORM6 where we concatenate the match representations for each answer in row through [·; ·], and do a non-linear transformation by parameters W r ∈ R l×l and b r ∈ R l to get hidden representation R ∈ R l×K. Finally, we map the transformed matching representations into scalar values through parameters w o ∈ R l and w o ∈ R. o ∈ R K is the normalized probability for the candidate answers to be ground-truth. Due to the aliases of the ground-truth answer, there may be multiple answers in the candidates are ground-truth, we use KL distance as our objective function: DISPLAYFORM7 where y k indicates whether a k the ground-truth answer or not and is normalized by K k=1 y k and o k is the ranking output of our model for a k. Although the coverage-based re-ranker tries to deal with more difficult cases compared to the strength-based re-ranker, the strength-based re-ranker works on more common cases according to the distributions of most open-domain QA datasets. We can try to get the best of both worlds by combining the two approaches. The full re-ranker is a weighted combination of the outputs of the above different re-rankers without further training. Specifically, we first use softmax to re-normalize the top-5 answer scores provided by the two strength-based rankers and the one coverage-based reranker; we then weighted sum up the scores for the same answer and select the answer with the largest score as the final prediction. We conduct experiments on three publicly available open-domain QA datasets, namely, Quasar-T BID9, SearchQA BID10 and TriviaQA BID17. These datasets contain passages retrieved for all questions using a search engine such as Google or Bing. We do not retrieve more passages but use the provided passages only. The statistics of the three datasets are shown in Table 1. 5 BID9 ) is based on a trivia question set. The data set makes use of the "Lucene index" on the ClueWeb09 corpus. For each question, 100 unique sentence-level passages were collected. The human performance is evaluated in an open-book setting, i.e., the human subjects had access to the same passages retrieved by the IR model and tried to find the answers from the passages. 6 BID10 ) is based on Jeopardy! questions and uses Google to collect about 50 web page snippets as passages for each question. The human performance is evaluated in a similar way to the Quasar-T dataset. 7 BID17 Our baseline models 9 include the following: GA BID8 b), a reading comprehension model with gated-attention; BiDAF BID26 ), a RC model with bidirectional attention flow; AQA BID4, a reinforced system learning to aggregate the answers generated by the re-written questions; R 3 ), a reinforced model making use of a ranker for selecting passages to train the RC model. As R 3 is the first step of our system for generating candidate answers, the improvement of our re-ranking methods can be directly compared to this baseline. TriviaQA does not provide the leaderboard under the open-domain setting. As a , there is no public baselines in this setting and we only compare with the R 3 baseline. We first use a pre-trained R 3 model, which gets the state-of-the-art performance on the three public datasets we consider, to generate the top 50 candidate spans for the training, development and test datasets, and we use them for further ranking. During training, if the groundtruth answer does not appear in the answer candidates, we will manually add it into the answer candidate list. For the coverage-based re-ranker, we use Adam BID19 to optimize the model. Word embeddings are initialized by GloVe BID23 and are not updated during training. We set all the words beyond Glove as zero vectors. We set l to 300, batch size to 30, learning rate to 0.002. We tune the dropout probability from 0 to 0.5 and the number of candidate answers for re-ranking (K) in 11. In this section, we present and analysis of our different re-ranking methods on the three different public datasets.8 Despite the open-domain QA data provided, the leaderboard of TriviaQA focuses on evaluation of RC models over filtered passages that is guaranteed to contain the correct answers (i.e. more like closed-domain setting). The evaluation is also passage-wise, different from the open-domain QA setting.9 Most of the of different models come from the public paper. While we re-run model R 3 based on the authors' source code and extend the model to the datasets of SearchQA and TriviaQA datasets. 10 To demonstrate that R 3 serves as a strong baseline on the TriviaQA data, we generate the R 3 following the leaderboard setting. The showed that R 3 achieved F1 56.0, EM 50.9 on Wiki domain and F1 68.5, EM 63.0 on Web domain, which is competitive to the state-of-the-arts. This confirms that R 3 is a competitive baseline when extending the TriviaQA questions to open-domain setting.11 Our code will be released under https://github.com/shuohangwang/mprc. DISPLAYFORM0 GA BID8 26.4 26.4 ----BiDAF BID26 25.9 28.5 28.6 34.6 --AQA BID4 The performance of our models is shown in TAB3. We use F1 score and Exact Match (EM) as our evaluation metrics 12. From the , we can clearly see that the full re-ranker, the combination of different re-rankers, significantly outperforms the previous best performance by a large margin, especially on Quasar-T and SearchQA. Moreover, our model is much better than the human performance on the SearchQA dataset. In addition, we see that our coverage-based re-ranker achieves consistently good performance on the three datasets, even though its performance is marginally lower than the strength-based re-ranker on the SearchQA dataset. In this subsection, we analyze the benefits of our re-ranking models. BM25 as an alternative coverage-based re-ranker We use the classical BM25 retrieval model BID25 ) to re-rank the aggregated passages the same way as the coveragebased re-ranker, where the IDF values are first computed from the raw passages before aggregation. From the in TAB3, we see that the BM25-based re-ranker improves the F1 scores compared with the R 3 model, but it is still lower than our coverage-based re-ranker with neural network models. Moreover, with respect to EM scores, the BM25-based re-ranker sometimes gives lower performance. We hypothesize that there are two reasons behind the relatively poor performance of BM25. First, because BM25 relies on a bag-of-words representation, context information is not taken into consideration and it cannot model the phrase similarities. Second, shorter answers tend to be preferred by BM25. For example, in our method of constructing pseudo-passages, when an answer sequence A is a subsequence of another answer sequence B, the pseudo passage of A is always a superset of the pseudo passage of B that could better cover the question. Therefore the F1 score could be improved but the EM score sometimes becomes worse. Re-ranking performance versus answer lengths and question types FIG3 decomposes the performance according to the length of the ground truth answers and the types of questions on TriviaQA and Quasar-T. We do not include the analysis on SearchQA because, for the Jeopardy! style questions, it is more difficult to distinguish the questions types, and the range of answer lengths is narrower. Our show that the coverage-based re-ranker outperforms the baseline in different lengths of answers and different types of questions. The strength-based re-ranker (counting) also gives improvement but is less stable across different datasets, while the strength-based re-ranker (probability) Table 3: The upper bound (recall) of the Top-K answer candidates generated by the baseline R 3 system (on dev set), which indicates the potential of the coverage-based re-ranker.tends to have and trends that are close to the baseline curves, which is probably because the method is dominated by the probabilities predicted by the baseline. The coverage-based re-ranker and the strength-based re-ranker (counting) have similar trends on most of the question types. The only exception is that the strength-based re-ranker performs significantly worse compared to the coverage-based re-ranker on the "why" questions. This is possibly because those questions usually have non-factoid answers, which are less likely to have exactly the same text spans predicted on different passages by the baseline. Potential improvement of re-rankers Table 3 shows the percentage of times the correct answer is included in the top-K answer predictions of the baseline R 3 method. More concretely, the scores are computed by selecting the answer from the top-K predictions with the best EM/F1 score. Therefore the final top-K EM and F1 can be viewed as the recall or an upper bound of the top-K predictions. From the , we can see that although the top-1 prediction of R 3 is not very accurate, there is high probability that a top-K list with small K could cover the correct answer. This explains why our re-ranking approach achieves large improvement. Also by comparing the upper bound performance of top-5 and our re-ranking performance in TAB3, we can see there is still a clear gap of about 10% on both datasets and on both F1 and EM, showing the great potential improvement for the re-ranking model in future work. Effect of the selection of K for the coverage-based re-ranker As shown in Table 3, as K ranges from 1 to 10, the recall of top-K predictions from the baseline R 3 system increases significantly. Table 5: Results of running strength-based re-ranker (counting) on different number of top-K answer candidates on Quasar-T (dev set).Ideally, if we use a larger K, then the candidate lists will be more likely to contain good answers. At the same time, the lists to be ranked are longer thus the re-ranking problem is harder. Therefore, there is a trade-off between the coverage of rank lists and the difficulty of re-ranking; and selecting an appropriate K becomes important. Table 4 shows the effects of K on the performance of coveragebased re-ranker. We train and test the coverage-based re-ranker on the top-K predictions from the baseline, where K ∈ {3, 5, 10}. The upper bound are the same ones from Table 3. The show that when K is small, like K=3, the performance is not very good due to the low coverage (thus low upper bound) of the candidate list. With the increase of K, the performance becomes better, but the top-5 and top-10 are on par with each other. This is because the higher upper bound of top-10 counteracts the harder problem of re-ranking longer lists. Since there is no significant advantage of the usage of K=10 while the computation cost is higher, we report all testing with K=5.Effect of the selection of K for the strength-based re-ranker Similar to Table 4, we conduct experiments to show the effects of K on the performance of the strength-based re-ranker. We run the strength-based re-ranker (counting) on the top-K predictions from the baseline, where K ∈ {10, 50, 100, 200}. We also evaluate the upper bound for these Ks. Note that the strength-based re-ranker is very fast and the different values of K do not affect the computation speed significantly compared to the other QA components. The are shown in Table 5, where we achieve the best when K=50. The performance drops significantly when K increases to 200. This is because the ratio of incorrect answers increases notably, making incorrect answers also likely to have high counts. When K is smaller, such incorrect answers appear less because statistically they have lower prediction scores. We report all testing with K=50.Examples Table 6 shows an example from Quasar-T where the re-ranker successfully corrected the wrong answer predicted by the baseline. This is a case where the coverage-based re-ranker helped: the correct answer "Sesame Street" has evidence from different passages that covers the aspects "Emmy Award" and "children's television shows". Although it still does not fully match all the facts in the question, it still helps to rank the correct answer higher than the top-1 prediction "Great Dane" from the R 3 baseline, which only has evidence covering "TV" and "1969" in the question. Table 6: An example from Quasar-T dataset. The ground-truth answer is "Sesame Street". Q: question, A: answer, P: passages containing corresponding answer. Open Domain Question Answering The task of open domain question answering dates back to as early as BID13 and was popularized by TREC-8 BID30. The task is to produce the answer to a question by exploiting resources such as documents BID30, webpages BID20 or structured knowledge bases BID2 BID3.Recent efforts BID10 BID9 benefit from the advances of machine reading comprehension (RC) and follow the search-and-read QA direction. These deep learning based methods usually rely on a document retrieval module to retrieve a list of passages for RC models to extract answers. As there is no passage-level annotation about which passages entail the answer, the model has to find proper ways to handle the noise introduced in the IR step. uses bi-gram passage index to improve the retrieval step; BID10; BID9 propose to reduce the length of the retrieved passages. focus more on noise reduction in the passage ranking step, in which a ranker module is jointly trained with the RC model with reinforcement learning. To the best of our knowledge, our work is the first to improve neural open-domain QA systems by using multiple passages for evidence aggregation. Moreover, we focus on the novel problem of "text evidence aggregation", where the problem is essentially modeling the relationship between the question and multiple passages (i.e. text evidence). In contrast, previous answer re-ranking research did not address the above problem: traditional QA systems like BID12 have similar passage retrieval process with answer candidates added to the queries. The retrieved passages were used for extracting answer scoring features, but the features were all extracted from single-passages thus did not utilize the information of union/co-occurrence of multiple passages. KB-QA systems BID0; BID34 sometimes use text evidence to enhance answer re-ranking, where the features are also extracted on the pair of question and a single-passage but ignored the union information among multiple passages. We are the first to introduce re-ranking methods to neural open-domain QA and multi-passage RC. Meanwhile, our two-step approach shares some similarities to the previous multi-step approaches proposed for standard single-passage RC, in terms of the purposes of either using additional information or re-fining answer predictions that are not easily handled by the standard answer extraction models for RC.On cloze-test tasks BID14, relates to our work in the sense that it is a two-step extractor-reasoner model, which first extracts K most probable singletoken answer candidates and then constructs a hypothesis by combining each answer candidate to the question and compares the hypothesis with all the sentences in the passage. Their model differs from ours in several aspects: (i) Epireader matches a hypothesis to every single sentence, including all the "noisy" ones that does not contain the answer, that makes the model inappropriate for open-domain QA setting; (ii) The sentence matching is based on the sentence embedding vectors computed by a convolutional neural network, which makes it hard to distinguish redundant and complementary evidence in aggregation; (iii) Epireader passes the probabilities predicted by the extractor to the reasoner directly to sustain differentiability, which cannot be easily adapted to our problem to handle phrases as answers or to use part of passages. Similarly, BID7 ) also combined answer candidates to the question to form hypotheses, and then explicitly use language models trained on documents to re-rank the hypotheses. This method benefits from the consistency between the documents and gold hypotheses (which are titles of the documents) in cloze-test datasets, but does not handle multiple evidence aggregation like our work. S-Net BID28 proposes a two-step approach for generative QA. The model first extracts an text span as the answer clue and then generates the answer according to the question, passage and the text span. Besides the different goal on answer generation instead of re-ranking like this work, their approach also differs from ours on that it extracts only one text span from a single selected passage. We have observed that open-domain QA can be improved by explicitly combining evidence from multiple retrieved passages. We experimented with two types of re-rankers, one for the case where evidence is consistent and another when evidence is complementary. Both re-rankers helped to significantly improve our individually, and even more together. Our considerably advance the state-of-the-art on three open-domain QA datasets. Although our proposed methods achieved some successes in modeling the union or co-occurrence of multiple passages, there are still much harder problems in open-domain QA that require reasoning and commonsense inference abilities. In future work, we will explore the above directions, and we believe that our proposed approach could be potentially generalized to these more difficult multipassage reasoning scenarios. This work was partially supported by DSO grant DSOCL15223.We thank Mandar Joshi for testing our model on the unfiltered TriviaQA hidden test dataset."

A specific gradient-based meta-learning algorithm, MAML, is equivalent to an inference procedure in a hierarchical Bayesian model. We use this connection to improve MAML via methods from approximate inference and curvature estimation.

We show that posterior collapse in linear VAEs is caused entirely by marginal log-likelihood (not ELBO). Experiments on deep VAEs suggest a similar phenomenon is at play.