Question: Represent the article for finding a single-sentence TLDR

Question: In this paper we propose to view the acceptance rate of the Metropolis-Hastings algorithm as a universal objective for learning to sample from target distribution -- given either as a set of samples or in the form of unnormalized density. This point of view unifies the goals of such approaches as Markov Chain Monte Carlo (MCMC), Generative Adversarial Networks (GANs), variational inference. To reveal the connection we derive the lower bound on the acceptance rate and treat it as the objective for learning explicit and implicit samplers. The form of the lower bound allows for doubly stochastic gradient optimization in case the target distribution factorizes (i.e. over data points). We empirically validate our approach on Bayesian inference for neural networks and generative models for images. Bayesian framework and deep learning have become more and more interrelated during recent years. Recently Bayesian deep neural networks were used for estimating uncertainty BID6, ensembling BID6 and model compression BID20. On the other hand, deep neural networks may be used to improve approximate inference in Bayesian models BID13.Learning modern Bayesian neural networks requires inference in the spaces with dimension up to several million by conditioning the weights of DNN on hundreds of thousands of objects. For such applications, one has to perform the approximate inference -predominantly by either sampling from the posterior with Markov Chain Monte Carlo (MCMC) methods or approximating the posterior with variational inference (VI) methods. MCMC methods provide the unbiased (in the limit) estimate but require careful hyperparameter tuning especially for big datasets and high dimensional problems. The large dataset problem has been addressed for different MCMC algorithms: stochastic gradient Langevin dynamics BID28, stochastic gradient Hamiltonian Monte Carlo, minibatch MetropolisHastings algorithms BID15 BID1. One way to address the problem of high dimension is the design of a proposal distribution. For example, for the Metropolis-Hastings (MH) algorithm there exists a theoretical guideline for scaling the variance of a Gaussian proposal BID24 BID25. More complex proposal designs include adaptive updates of the proposal distribution during iterations of the MH algorithm BID12 BID7. Another way to adapt the MH algorithm for high dimensions is combination of adaptive direction sampling and the multiple-try Metropolis algorithm as proposed in BID17. Thorough overview of different extensions of the MH algorithm is presented in BID18.Variational inference is extremely scalable but provides a biased estimate of the target distribution. Using the doubly stochastic procedure BID27 BID11 VI can be applied to extremely large datasets and high dimensional spaces, such as a space of neural network weights BID14 BID5. The bias introduced by variational approximation can be mitigated by using flexible approximations BID22 and resampling BID9.Generative Adversarial Networks BID8 ) (GANs) is a different approach to learn samplers. Under the framework of adversarial training different optimization problems could be solved efficiently BID0 BID21. The shared goal of "learning to sample" inspired the connection of GANs with VI BID19 and MCMC BID26.In this paper, we propose a novel perspective on learning to sample from a target distribution by optimizing parameters of either explicit or implicit probabilistic model. Our objective is inspired by the view on the acceptance rate of the Metropolis-Hastings algorithm as a quality measure of the sampler. We derive a lower bound on the acceptance rate and maximize it with respect to parameters of the sampler, treating the sampler as a proposal distribution in the Metropolis-Hastings scheme. We consider two possible forms of the target distribution: unnormalized density (density-based setting) and a set of samples (sample-based setting). Each of these settings reveals a unifying property of the proposed perspective and the derived lower bound. In the density-based setting, the lower bound is the sum of forward and reverse KL-divergences between the true posterior and its approximation, connecting our approach to VI. In the sample-based setting, the lower bound admit a form of an adversarial game between the sampler and a discriminator, connecting our approach to GANs. The closest work to ours is of BID26. In contrast to their paper our approach is free from hyperparameters; is able to optimize the acceptance rate directly; avoids minimax problem in the density based setting. Our main contributions are as follows:1. We introduce a novel perspective on learning to sample from the target distribution by treating the acceptance rate in the Metropolis-Hastings algorithm as a measure of sampler quality. 2. We derive the lower bound on the acceptance rate allowing for doubly stochastic optimization of the proposal distribution in case when the target distribution factorizes (i.e. over data points). 3. For sample-based and density-based forms of target distribution we show the connection of the proposed algorithm to variational inference and GANs. The rest of the paper is organized as follows. In Section 2 we introduce the lower bound on the AR. Special forms of target distribution are addressed in Section 3. We validate our approach on the problems of approximate Bayesian inference in the space of high dimensional neural network weights and generative modeling in the space of images in Section 4. We discuss and directions of the future work in Section 5. In MH algorithm we need to sample from target distribution p(x) while we are only able to sample from proposal distribution q(x | x). One step of the MH algorithm can be described as follows. DISPLAYFORM0 If the proposal distribution q(x | x) does not depend on x, i.e. q(x | x) = q(x), the algorithm is called independent MH algorithm. The quality of the proposal distribution is measured by acceptance rate and mixing time. Mixing time defines the speed of convergence of the Markov chain to the stationary distribution. The acceptance rate of the MH algorithm is defined as DISPLAYFORM1 where DISPLAYFORM2 In case of independent proposal distribution we show that the acceptance rate defines a semimetric in distribution space between p and q (see Appendix A.2). Although, we can maximize the acceptance rate of the MH algorithm (Eq. 1) directly w.r.t. parameters φ of the proposal distribution q φ (x | x), we propose to maximize the lower bound on the acceptance rate. As our experiments show (see Section 4) the optimization of the lower bound compares favorably to the direct optimization of the acceptance rate. To introduce this lower bound we first express the acceptance rate in terms of total variation distance. DISPLAYFORM0 where TV is the total variation distance. The proof of Theorem 1 can be found in Appendix A.1. This reinterpretation in terms of total variation allows us to lower bound the acceptance rate through the Pinsker's inequality DISPLAYFORM1 The maximization of this lower bound can be equivalently formulated as the following optimization problem DISPLAYFORM2 In the following sections, we show the benefits of this optimization problem in two different settings -when the target distribution is given in a form of unnormalized density and as a set of samples. In Appendix C.5 and C.1 we provide the empirical evidence that maximization of the proposed lower bound in the maximization of the acceptance rate. From now on we consider only optimization problem Eq. 5 but the proposed algorithms can be also used for the direct optimization of the acceptance rate (Eq. 1).To estimate the loss function (Eq. 5) we need to evaluate the density ratio. In the density-based setting unnormalized density of the target distribution is given, so we suggest to use explicit proposal distribution to compute the density ratio explicitly. In the sample-based setting, however, we cannot compute the density ratio, so we propose to approximate it via adversarial training BID8. The brief summary of constraints for both settings is shown in TAB0. DISPLAYFORM0 The following subsections describe the algorithms in detail. In the density-based setting, we assume the proposal to be an explicit probabilistic model, i.e. the model that we can sample from and evaluate its density at any point up to the normalization constant. We also assume that the proposal is reparameterisable BID13 BID23 BID4.During the optimization of the acceptance rate we might face a situation when proposal collapses to the delta-function. This problem usually arises when we use Markov chain proposal, for example, q φ (x | x) = N (x | x, σ). For such proposal we can obtain arbitrary high acceptance rate, making the σ small enough. However, this is not the case for the independent proposal distribution q φ (x | x) = q φ (x). In Appendix B.1 we provide more details and intuition on this property of acceptance rate maximization. We also provide empirical evidence in Section 4 that collapsing to the delta-function does not happen for the independent proposal. In this paper, we consider two types of explicit proposals: simple parametric family (Section 4.2) and normalizing flows BID22 BID3 ) (Section 4.1). Rich family of normalizing flows allows to learn expressive proposal and evaluate its density in any point of target distribution space. Moreover, an invertible model (such as normalizing flow) is a natural choice for the independent proposal due to its ergodicity. Indeed, choosing the arbitrary point in the target distribution space, we can obtain the corresponding point in the latent space using the inverse function. Since every point in the latent space has positive density, then every point in the target space also has positive density. Considering q φ (x) as the proposal, the objective of optimization problem 5 takes the form DISPLAYFORM0 Explicit form of the proposal q φ (x) and the target p(x) distributions allows us to obtain density ratios q φ (x)/q φ (x) and p(x)/p(x) for any points x, x. But to estimate the loss in Eq. 6 we also need to obtain samples from the target distribution x ∼ p(x) during training. For this purpose, we use the current proposal q φ and run the independent MH algorithm. After obtaining samples from the target distribution it is possible to perform optimization step by taking stochastic gradients w.r.t. φ. Pseudo-code for the obtained procedure is shown in Algorithm 1.Algorithm 1 Optimization of proposal distribution in density-based case DISPLAYFORM1 approximate loss with finite number of samples DISPLAYFORM2 perform gradient descent step end while return optimal parameters φ Algorithm 1 could also be employed for the direct optimization of the acceptance rate (Eq. 1). Now we apply this algorithm for Bayesian inference problem and show that during optimization of the lower bound we can use minibatches of data, while it is not the case for direct optimization of the acceptance rate. We consider Bayesian inference problem for discriminative model on dataset DISPLAYFORM3, where x i is the feature vector of ith object and y i is its label. For the discriminative model we know likelihood p(y i | x i, θ) and prior distribution p(θ). In order to obtain predictions for some object x i, we need to evaluate the predictive distribution DISPLAYFORM4 To obtain samples from posterior distribution p(θ | D) we suggest to learn proposal distribution q φ (θ) and perform independent MH algorithm. Thus the objective 6 can be rewritten as DISPLAYFORM5 Note that due to the usage of independent proposal, the minimized KL-divergence splits up into the sum of two KL-divergences. DISPLAYFORM6 Minimization of the first KL-divergence corresponds to the variational inference procedure. DISPLAYFORM7 The second KL-divergence has the only term that depends on φ. Thus we obtain the following optimization problem DISPLAYFORM8 The first summand here contains the sum over all objects in dataset D. We follow doubly stochastic variational inference and suggest to perform unbiased estimation of the gradient in problem 11 using only minibatches of data. Moreover, we can use recently proposed techniques BID15 BID1 that perform the independent MH algorithm using only minibatches of data. Combination of these two techniques allows us to use only minibatches of data during iterations of algorithm 1. In the case of the direct optimization of the acceptance rate, straightforward usage of minibatches in biased gradients. Indeed, for the direct optimization of the acceptance rate (Eq. 1) we have the product over the all training data inside min function. In the sample-based setting, we assume the proposal to be an implicit probabilistic model, i.e. the model that we can only sample from. As in the density-based setting, we assume that we are able to perform the reparameterization trick for the proposal. In this subsection we consider only Markov chain proposal q φ (x | x), but everything can be applied to independent proposal q φ (x) by simple substitution q φ (x | x) with q φ (x). From now we will assume our proposal distribution to be a neural network that takes x as its input and outputs x. Considering proposal distribution parameterized by a neural network allows us to easily exclude delta-function from the space of solutions. We avoid learning the identity mapping by using neural networks with the bottleneck and noisy layers. For the detailed description of the architectures see Appendix C.8.The set of samples from the true distribution X ∼ p(x) allows for the Monte Carlo estimation of the loss DISPLAYFORM0 To compute the density ratio DISPLAYFORM1 we suggest to use well-known technique of density ratio estimation via training discriminator network. Denoting discriminator output as D(x, x), we suggest the following optimization problem for the discriminator. DISPLAYFORM2 Speaking informally, such discriminator takes two images as input and tries to figure out which image is sampled from true distribution and which one is generated by the one step of proposal distribution. It is easy to show that optimal discriminator in problem 13 will be DISPLAYFORM3 Note that for optimal discriminator we have D(x, x) = 1 − D(x, x). In practice, we have no optimal discriminator and these values can differ significantly. Thus, we have four ways for density ratio estimation that may differ significantly. DISPLAYFORM4 To avoid the ambiguity we suggest to use the discriminator of a special structure. Let D(x, x) be a convolutional neural network with scalar output. Then the output of discriminator D(x, x) is defined as follows. DISPLAYFORM5 In other words, such discriminator can be described as the following procedure. For single neural network D(·, ·) we evaluate two outputs D(x, x) and D(x, x). Then we take softmax operation for these values. Summing up all the steps, we obtain algorithm 2.Algorithm 2 Optimization of proposal distribution in sample-based case DISPLAYFORM6 approximate loss with finite number of samples DISPLAYFORM7 perform gradient descent step end for return parameters φ Algorithm 2 could also be employed for direct optimization of the acceptance rate (Eq. 1). But, in Appendix B.2 we provide an intuition for this setting that the direct optimization of the acceptance rate may struggle from vanishing gradients. In this section, we provide experiments for both density-based and sample-based settings, showing the proposed procedure is applicable to high dimensional target distributions. Code for reproducing all of the experiments will be published with the camera-ready version of the paper. To demonstrate performance of our approach we reproduce the experiment from BID26. For target distributions we use synthetic 2d distributions (see Appendix C.3 for densities): ring (a ring-shaped density), mog2 (a mixture of 2 Gaussians), mog6 (a mixture of 6 Gaussians), ring5 (a mixture of 5 distinct rings). We measure performance of learned samplers using Effective Sample Size (see Appendix C.4 for formulation). Since the unnormalized densities of target distributions are given, we can learn proposals as suggested in the density-based setting (Section 3.1).To learn the independent proposal we use RealNVP model BID3 ) (see details in Appendix C.2) and compare the performance of proposals after optimization of different objectives: the acceptance rate (AR), our lower bound on the acceptance rate (ARLB), evidence lower bound that corresponds to the variational inference (VI). We also compare the performance of obtained independent proposals with the performance of Markov chain proposals: A-NICE-MC BID26, Hamiltonian Monte Carlo (HMC).In Tables 2, 3 we see that our approach has comparable performance with A-NICE-MC BID26. However, comparison between A-NICE-MC and learning independent proposal is not the main subject of interest, since A-NICE-MC learns Markov chain proposal. On the one hand, Markov chain proposal uses more information while generating a new sample, hence can learn more expressive stationary distribution, on the other hand, usage of previous sample increase autocorrelation between samples and reduces ESS. Thus, the main point of interest is the comparison of two independent proposals: one is learned by maximization of the acceptance rate (or its lower bound), and the second is learned by variational inference procedure, i.e. maximization of evidence lower bound. In TAB1 we see that both maximization of the acceptance rate and its lower bound outperform variational inference for all target distributions. Moreover, in FIG0 we show that variational inference fails to cover all the modes of mog6 in contrast to proposals learned via maximization of acceptance rate or its lower bound. Densities of learned proposals and histograms for all distributions are presented in Appendix C.6. In density-based setting, we consider Bayesian inference problem for the weights of a neural network. In our experiments we consider approximation of predictive distribution (Eq. 7) as our main goal. To estimate the goodness of the approximation we measure negative log-likelihood and accuracy on the test set. In subsection 3.1 we show that lower bound on acceptance rate can be optimized more efficiently than acceptance rate due to the usage of minibatches. But other questions arise.1. Does the proposed objective in Eq. 11 allow for better estimation of predictive distribution compared to the variational inference?2. Does the application of the MH correction to the learned proposal distribution allow for better estimation of the predictive distribution (Eq. 7) than estimation via raw samples from the proposal?To answer these questions we consider reduced LeNet-5 architecture (see Appendix C.7) for classification task on 20k images from MNIST dataset (for test data we use all of the MNIST test set). Even after architecture reduction we still face a challenging task of learning a complex distribution in 8550-dimensional space. For the proposal distribution we use fully-factorized gaussian DISPLAYFORM0. For variational inference, we train the model using different initialization and pick the model according to the best ELBO. For our procedure, we do the same and choose the model by the maximum value of the acceptance rate lower bound. In Algorithm 1 we propose to sample from the posterior distribution using the independent MH and the current proposal. It turns out in practice that it is better to use the currently learned proposal q φ (θ) = N (θ | µ, σ) as the initial state for random-walk MH algorithm. That is, we start with the mean µ as an initial point, and then use random-walk proposal q(θ | θ) = N (θ | θ, σ) with the variances σ of current independent proposal. This should be considered as a heuristic that improves the approximation of the loss function. The optimization of the acceptance rate lower bound in the better estimation of predictive distribution than the variational inference (see FIG1). Optimization of acceptance rate for the same number of epochs in nearly 30% accuracy on the test set. That is why we do not report for this procedure in FIG1 In both procedures we apply the independent MH algorithm to estimate the predictive distribution. To answer the second question we estimate predictive distribution in two ways. The first way is to perform 100 accept/reject steps of the independent MH algorithm with the learned proposal q φ (θ) after each epoch, i.e. perform MH correction of the samples from the proposal. The second way is to take the same number of samples from q φ (θ) without MH correction. For both estimations of predictive distribution, we evaluate negative log-likelihood on the test set and compare them. The MH correction of the learned proposal improves the estimation of predictive distribution for the variational inference (right plot of FIG2) but does not do so for the optimization of the acceptance rate lower bound (left plot of FIG2). This fact may be considered as an implicit evidence that our procedure learns the proposal distribution with higher acceptance rate. In the sample-based setting, we estimate density ratio using a discriminator. Hence we do not use the minibatching property (see subsection 3.1) of the obtained lower bound, and optimization problems for the acceptance rate and for the lower bound have the same efficiency in terms of using data. That is why our main goal in this setting is to compare the optimization of the acceptance rate and the optimization of the lower bound. Also, in this setting, we have Markov chain proposal that is interesting to compare with the independent proposal. Summing up, we formulate the following questions:1. Does the optimization of the lower bound has any benefits compared to the direct optimization of the acceptance rate? 2. Do we have mixing issue while learning Markov chain proposal in practice? 3. Could we improve the visual quality of samples by applying the MH correction to the learned proposal?We use DCGAN architecture for the proposal and discriminator (see Appendix C.8) and apply our algorithm to MNIST dataset. We consider two optimization problems: direct optimization of the acceptance rate and its lower bound. We also consider two ways to obtain samples from the approximation of the target distribution -use raw samples from the learned proposal, or perform the MH algorithm, where we use the learned discriminator for density ratio estimation. In case of the independent proposal, we show that the MH correction at evaluation step allows to improve visual quality of samples -figures 4(a) and 4(b) for the direct optimization of acceptance rate, figures 4(c) and 4(d) for the optimization of its lower bound. Note that in Algorithm 2 we do not apply the independent MH algorithm during training. Potentially, one can use the MH algorithm considering any generative model as a proposal distribution and learning a discriminator for density ratio estimation. Also, for this proposal, we demonstrate the negligible difference in visual quality of samples obtained by the direct optimization of acceptance rate (see Fig. 4(a) ) and by the optimization of the lower bound (see Fig. 4(c) ). Figure 4: Samples from the learned independent proposal obtained via optimization: of acceptance rate (4(a), 4(b)) and its lower bound (4(c), 4(d)). In Fig. 4(b), 4(d) we show raw samples from the learned proposal. In Fig. 4(a), 4(c) we show the samples after applying the independent MH correction to the samples, using the learned discriminator for density ratio estimation. In the case of the Markov chain proposal, we show that the direct optimization of acceptance rate in slow mixing (see Fig. 5(a) ) -most of the time the proposal generates samples from one of the modes (digits) and rarely switches to another mode. When we perform the optimization of the lower bound the proposal switches between modes frequently (see Fig. 5(b) ). Note that we obtain different distributions of the samples because of conditioning of our proposal. To show that the learned proposal distribution has the Markov property rather than being totally independent, we show samples from the proposal conditioned on two different points in the dataset (see Fig. 6). The difference in samples from two these distributions (Fig. 6(a), 6(a)) reflects the dependence on the conditioning. Additionally, in Appendix C.9 we present samples from the chain after 10000 accepted images and also samples from the chain that was initialized with noise. This paper proposes to use the acceptance rate of the MH algorithm as the universal objective for learning to sample from some target distribution. We also propose the lower bound on the acceptance rate that should be preferred over the direct maximization of the acceptance rate in many cases. The proposed approach provides many ways of improvement by the combination with techniques from the recent developments in the field of MCMC, GANs, variational inference. For example• The proposed loss function can be combined with the loss function from BID16, thus allowing to learn the Markov chain proposal in the density-based setting.• We can use stochastic Hamiltonian Monte Carlo for the loss estimation in Algorithm 1. • In sample-based setting one can use more advanced techniques of density ratio estimation. Application of the MH algorithm to improve the quality of generative models also requires exhaustive further exploration and rigorous treatment. Remind that we have random variables ξ = DISPLAYFORM0, and want to prove the following equalities. DISPLAYFORM1 Equality E ξ min{1, ξ} = P{ξ > u} is obvious. DISPLAYFORM2 Equality P{ξ > u} = 1 − 1 2 E ξ |ξ − 1| can be proofed as follows. DISPLAYFORM3 where F ξ (u) is CDF of random variable ξ. Note that F ξ = 0 since ξ ∈ (0, +∞]. Eq. 21 can be rewritten in two ways. DISPLAYFORM4 To rewrite Eq. 21 in the second way we note that Eξ = 1. DISPLAYFORM5 Summing equations 22 and 23 in the following formula DISPLAYFORM6 Using the form of ξ we can rewrite the acceptance rate as DISPLAYFORM7 In independent case we have ξ = DISPLAYFORM0 and we want to prove that E ξ |ξ − 1| is semimetric (or pseudo-metric) in space of distributions. For this appendix, we denote D(p, q) = E ξ |ξ − 1|. The first two axioms for metric obviously holds DISPLAYFORM1 There is an example when triangle inequality does not hold. DISPLAYFORM2 But weaker inequality can be proved. DISPLAYFORM3 Summing up equations 28, 30 and 32 we obtain DISPLAYFORM4 B OPTIMIZATION OF PROPOSAL DISTRIBUTION Firstly, let's consider the case of gaussian random-walk proposal q(x | x) = N (x | x, σ). The optimization problem for the acceptance rate takes the form DISPLAYFORM0 It is easy to see that we can obtain acceptance rate arbitrarly close to 1, taking σ small enough. In the case of the independent proposal, we don't have the collapsing to the delta-function problem. In our work, it is important to show non-collapsing during optimization of the lower bound, but the same hold for the direct optimization of the acceptance rate. To provide such intuition we consider one-dimensional case where we have some target distribution p(x) and independent proposal q(x) = N (x | µ, σ). Choosing σ small enough, we approximate sampling with the independent MH as sampling on some finite support x ∈ [µ − a, µ + a]. For this support, we approximate the target distribution with the uniform distribution (see FIG5).For such approximation, optimization of lower bound takes the form Here N (x | 0, σ, −a, a) is truncated normal distribution. The first KL-divergence can be written as follows. DISPLAYFORM1 DISPLAYFORM2 Here Z is normalization constant of truncated log normal distribution and DISPLAYFORM3 Summing up two KL-divergencies and taking derivative w.r.t. σ we obtain ∂ ∂σ DISPLAYFORM4 To show that the derivative of the lower bound w.r.t. σ is negative, we need to prove that the following inequality holds for positive x. DISPLAYFORM5 2 /2 dt and noting that 2φ(x) = √ 2π(Φ(x) − Φ(−x)) we can rewrite inequality 47 as DISPLAYFORM6 By the fundamental theorem of calculus, we have DISPLAYFORM7 Hence, DISPLAYFORM8 Or equivalently, DISPLAYFORM9 Using this inequality twice, we obtain DISPLAYFORM10 and DISPLAYFORM11 Thus, the target inequality can be verified by the verification of DISPLAYFORM12 Thus, we show that partial derivative of our lower bound w.r.t. σ is negative. Using that knowledge we can improve our loss by taking a bigger value of σ. Hence, such proposal does not collapse to delta-function. In this section, we provide an intuition for sample-based setting that the loss function for lower bound has better gradients than the loss function for acceptance rate. Firstly, we remind that in the sample-based setting we use a discriminator for density ratio estimation. DISPLAYFORM0 For this purpose we use the discriminator of special structure DISPLAYFORM1 We denote d(x, x) = D(x, x) − D(x, x) and consider the case when the discriminator can easily distinguish fake pairs from valid pairs. So D(x, x) is close to 1 and d(x, x) 0 for x ∼ p(x) and x ∼ q(x | x). To evaluate gradients we consider Monte Carlo estimations of each loss and take gradients w.r.t. x in order to obtain gradients for parameters of proposal distribution. We do not introduce the reparameterization trick to simplify the notation but assume it to be performed. For the optimization of the acceptance rate we have DISPLAYFORM2 DISPLAYFORM3 While for the optimization of the lower bound we have DISPLAYFORM4 DISPLAYFORM5 DISPLAYFORM6 Now we compare Eq. 59 and Eq. 62. We see that in case of strong discriminator we have vanishing gradients in Eq. 59 due to exp(−d(x, x)), while it is not the case for Eq. 62. This experiment shows that it is possible to optimize the acceptance rate, optimizing its lower bound. For the target distribution we consider bimodal Gaussian p(x) = 0.5 · N (x | − 2, 0.5) + 0.5 · N (x | 2, 0.7), for the independent proposal we consider unimodal gaussian q(x) = N (x | µ, σ). We perform stochastic gradient optimization using Algorithm 1 from the same initialization for both objectives FIG6 and obtain approximately the same local maximums. For the proposal distribution we use similar architecture to the NICE proposal. The RealNVP model BID3 use the same strategy for evaluating the Jacobian as the NICE model does. Each coupling layer define the following function. Given a D dimensional input x and d < D, the output y is evaluated by the formula DISPLAYFORM0 where the functions s, t can be arbitrary complex, since the structure of the functions doesn't influence the computation of the Jacobian. For our proposal we use 4 coupling layers with s and t consist of two fully-connected layers with hidden dimension of 256. For synthetic distributions we consider the same distributions as in BID26.The analytic form of p(x) for ring is: DISPLAYFORM0 The analytic form of p(x) for mog2 is: DISPLAYFORM1 where DISPLAYFORM2 The analytic form of p(x) for mog6 is: DISPLAYFORM3 where DISPLAYFORM4 DISPLAYFORM5 where DISPLAYFORM6 For the effective sample size formulation we follow BID26.Assume a target distribution p(x), and a Markov chain Monte Carlo (MCMC) sampler that produces a set of N correlated samples DISPLAYFORM0. Suppose we are estimating the mean of p(x) through sampling; we assume that increasing the number of samples will reduce the variance of that estimate. DISPLAYFORM1 DISPLAYFORM2 where ρ s denotes the autocorrelation under q of x at lag s. We compute the following empirical estimateρ s for ρ s:ρ DISPLAYFORM3 whereμ andσ are the empirical mean and variance obtained by an independent sampler. Due to the noise in large lags s, we adopt the approach of Hoffman & Gelman FORMULA1 where we truncate the sum over the autocorrelations when the autocorrelation goes below 0.05. In this section we provide the empirical evidence that maximization of the proposed lower bound on the acceptance rate (ARLB) in maximization of the acceptance rate (AR). For that purpose we evaluate ARLB and AR at each iteration during the optimization of ARLB. After training we evaluate correlation coefficient between ARLB and logarithm of AR. The curves are shown in FIG9: plots for the acceptance rate and the acceptance rate lower bound evaluated at every iteration during the optimization of the acceptance rate lower bound. Correlation coefficient is evaluated between the logarithm of the acceptance rate and the acceptance rate lower bound. In this section we provide levelplots of learned proposals densities (see FIG0). We also provide 2d histrograms of samples from the MH algorithm using the corresponding proposals (see FIG0). In this section, we show additional figures for Markov chain proposals. In FIG0 we show samples from the chain that was initialized by the noise. In FIG0 we show samples from the chain after 10000 accepted samples. FIG0: Samples from the chain initialized with noise. To obtain samples we use the MH algorithm with the learned proposal and the learned discriminator for density ratio estimation. In Fig. 5 (a) we use proposal and discriminator that are learned during optimization of acceptance rate. In Fig. 5(b) we use proposal and discriminator that are learned during the optimization of the acceptance rate lower bound. Samples in the chain are obtained one by one from left to right from top to bottom starting with noise (first image in the figure). FIG0: Samples from the chain after 10000 accepted samples. To obtain samples we use the MH algorithm with the learned proposal and the learned discriminator for density ratio estimation. In Fig. 5(a) we use proposal and discriminator that are learned during optimization of acceptance rate. In Fig. 5(b) we use proposal and discriminator that are learned during the optimization of the acceptance rate lower bound. Samples in chain are obtained one by one from left to right from top to bottom.

Learning to sample via lower bounding the acceptance rate of the Metropolis-Hastings algorithm

Represent the short summary for finding the corresponding scientific article (long)

We show that posterior collapse in linear VAEs is caused entirely by marginal log-likelihood (not ELBO). Experiments on deep VAEs suggest a similar phenomenon is at play.


Question: Represent the article for finding a single-sentence TLDR

Question: This paper proposes a self-supervised learning approach for video features that in significantly improved performance on downstream tasks (such as video classification, captioning and segmentation) compared to existing methods. Our method extends the BERT model for text sequences to the case of sequences of real-valued feature vectors, by replacing the softmax loss with noise contrastive estimation (NCE). We also show how to learn representations from sequences of visual features and sequences of words derived from ASR (automatic speech recognition), and show that such cross-modal training (when possible) helps even more. Recently there has been a lot of progress in self-supervised representation learning for textual sequences, followed by supervised fine-tuning (using small labeled datasets) of shallow (often linear) decoders on various downstream NLP tasks, such as sentiment classification. In this paper, we build on this work and propose a new method for self-supervised representation learning for videos, optionally accompanied by speech transcripts generated by automatic speech recognition (ASR). We show that fine-tuning linear decoders together with our self-supervised video representations, can achieve state of the art on various supervised tasks, including video classification, segmentation and captioning. Our approach builds on the popular BERT (Bidirectional Encoder Representations from Transformers) model for text. This uses the Transformer architecture to encode long sentences, and trains the model using the "masked language modeling" (MLM) training objective, in which the model must predict the missing words given their bidirectional context. The MLM loss requires that each token in the sequence be discrete. The VideoBERT model of (a) therefore applied vector quantization (VQ) to video frames before passing them (along with optional ASR tokens) to the BERT model. Unfortunately, VQ loses fine-grained information that is often critical for downstream tasks. More recently, several papers (e.g., VilBERT and LXMERT ) proposed to address this limitation by directly measuring the visual similarity between frames using pre-trained visual encoders. In this paper, we propose a way to train bidirectional transformer models on sequences of realvalued vectors (e.g., video frames), x 1:T, using noise contrastive estimation (NCE), without needing pre-trained visual encoders. We call our method "Contastive Bidirectional Transformer" or CBT. We also develop a method that combines x 1:T with an optional sequence of discrete tokens, y 1:T (e.g., derived from ASR). In contrast to the VideoBERT paper (a), we provide a "lightweight" way of combining these signals after training each modality separately. In particular, we propose a cross-modal transformer to maximize the mutual information between x 1:T and y 1:T at the sequence level (rather than at the frame level). This method is robust to small misalignments between the sequences (e.g., if the words at time t do not exactly correspond to what is visually present in frame t). We demonstrate the effectiveness of the proposed approach for learning short-term visual representations, as well as longer term temporal representations. For visual representations, we encode each window of K frames using a 3D convolutional neural network S3D , and then pass this sequence of features to the CBT model for self-supervised pretraining with the NCE loss on the Pre-training Figure 1: Summary of our method for training and evaluation. The blocks above the line are pretrained in an self-supervised way. The solid blocks represent the BERT language model, which is pre-trained on web text and frozen (see section 3.1). The black CBT visual block is trained using NCE loss on unlabeled HowTo or Kinetics videos (see section 3.2). The red cross-modal transformer is trained using cross-modal loss on HowTo with ASR (see section 3.3). The components below the line are trained in a supervised way on various tasks. The purple block is trained for next action prediction on ActivityNet, Breakfast, and 50Salads (see section 4.2). The blue block is trained for video classification on UCF101 and HMDB501 (see section 4.1). The green blocks are trained on captioning and video segmentation tasks, which are described in the supplementary material (section 6.1 and section 6.2). Lseq refers to cross-entropy sequence loss. unlabeled Kinetics dataset . We then fine-tune a linear classifier for video classification on UCF101 and HMDB51. We show that our method outperforms previous state-of-the-art self-supervised methods by large margins (UCF101 from 75.7% to 79.5% and HMDB51 from 35.7% to 44.6%). For temporal representations, we encode each window of K frames using a S3D network that is pretrained on Kinetics, and then "frozen". We then pass this sequence of features to the CBT model for self-supervised pretraining with the NCE loss on the unlabeled HowTo100M dataset (b). We also evaluate the effects of running ASR on HowTo100M, and passing this to our cross-modal transformer as an additional signal. We then fine-tune various shallow decoders for a variety of tasks, including video classification, segmentation and captioning. We show large gains compared to previous methods, especially when we use cross-modal pretraining. See fig. 1 for a summary of our training method and evaluation protocol. Video representations. Most existing work on learning video representations, such as (; ; ; ;), only captures a few seconds of video. Long-term context can be encoded by recurrent neural networks (; b), graph convolutional networks, or long-term feature banks, but these are all supervised methods. Some recent work have been done on learning self-supervised video representation;;;;; by defining pretext tasks such as ordering , rotation , temporal cycle consistency (b;) or colorization ) but similar to their supervised counterparts they capture only few seconds. Self-supervised context modeling. Recently, there has between a lot of work on self-supervised context modeling for language representations (; ;). In particular, the BERT model, which stands for Bidirectional Encoder Representations from Transformers , pre-trains deep bidirectional representations by jointly conditioning on both left and right context in all layers. The pre-trained BERT representations can be fine-tuned with just one additional output layer to create state-of-the-art models for a wide range of NLP tasks, such as question answering and linguistic entailment. Our representation builds on this approach and adapts it to continuous video data by using a contrastive loss. Mutual information estimation and maximization. For representation learning, a signal encoder can be trained to maximize the mutual information (MI) between the input signal and its encoded outputs, or the encoded outputs of the signal and its context (see e.g., (; ; ;). In particular, contrastive predictive coding (CPC) uses noise contrastive estimation (Gutmann & Hyvärinen, 2010) to maximize the MI lower bound. Unlike CPC, which relies on auto regressive models to encode context, we use BERT to encode bidirectional context within each sequence, and across different modalities. Cross-modal learning. The multi-modality of video is a rich source of information for selfsupervised learning of video representations. Since videos contain both visual and audio signals that are roughly synchronized, the two modalities can supervised each other, as explored in prior work such as (; b; a;). Another common form of weak supervision is based on video and language, where language is either obtained by automatic speech recognition (ASR) or from additional textual description. Language can be leveraged by finding a joint embedding space for both visual and textual modalities or by learning an alignment between the two modalities (; ; a). Recently, several concurrent approaches (a; a; ; ; b) generalize the BERT architecture and MLM objective to learn visual-linguistic representations. They assume the visual representations to be fixed and given by supervised pre-trained visual encoders, and define the visual MLM objective in terms of visual similarities (e.g. via vector quantization or measuring L2 distance) between the original and predicted visual representations. To the best of our knowledge, our proposed CBT is the first to demonstrate the effectiveness of BERT-style pre-training in a fully self-supervised way for video. We first give an overview of the BERT model for learning from sequences of words, y 1:T. We then discuss an extension to the case of sequences of video frames, x 1:T. Finally, we discuss how to learn from both kinds of data, even when not perfectly aligned. The BERT model takes in a sequence of discrete tokens, y 1:T, where y t ∈ {1, . . ., K}, embeds each one into a dense vector, e y t ∈ D, and then emits a sequence of dense output vectors, h y t ∈ D y, which are computed using a transformer . The output sequence captures local and global semantic information about the input sequence. The main training objective for BERT is to minimize the pseudo negative log likelihood, defined by where y −t is the sequence of all words except the t'th, and Here f enc (k) is an embedding lookup table for token k, e t = f enc (y t) is the embedding for the token at t, is the embedding sequence for the context, and is a multi-layer multi-headed transformer network that takes a T × D feature matrix as input (masked at location t) and returns a matrix of the same size. The BERT model requires a fixed discrete vocabulary. However, for images and videos, the inputs are real-valued vectors. We propose to use the softmax version of the noise contrastive estimation (NCE) loss , which has the form where where e t = f enc (x t) is the output of a 3D CNN applied to a small window around frame t (we use the S3D model from ),ê t = g context (e −t) is the output of a visual transformer, and neg(t) is a set of (indices of) "negative examples" (in practice we use all the other frames from the same minibatch as frame t). Intuitively the NCE loss encourages the model to learn to identify the correct frame (given the context) compared to a set of negative distractors. More formally, it can be shown that the NCE loss maximizes (a lower bound on) the mutual information (MI) between x t and x −t (see e.g., ). This loss has been used in other work on self-supervised visual representation learning, e.g., in the deep infomax (DIM) and contrastive predictive coding (CPC) papers. In DIM, the context predictor uses a CNN applied to neighboring patches in the same image, and in CPC, the context predictor uses a causal autoregressive model applied to "earlier" patches in the same image. In our CBT method, the context predictor is a bidirectional transformer applied to video frames. In this section we show how to learn useful representations from sequences of continuous visual features (from video) and sequences of discrete words (from ASR). More precisely, assume we have two sequences, x = x 1:T representing video, and y = y 1:T, representing ASR. Note that the sequences may not be perfectly aligned, since a person may speak about things at time t that are not visible in the frame at time t. Therefore it does not make sense to try to maximize the MI between x t and y t at the frame level. Instead, we try to maximize MI between x and y at the sequence level. To do this, we first encode each sequence using CBT and BERT to get h x 1:T = CBT(x 1:T) and h y 1:T = BERT(y 1:T), as shown in fig. 1. We then concatenate these sequences and pass them to a shallow cross-modal transformer to produce h xy 1:T +T. Finally, we pass this to a shallow MLP to compute an MI-like score MI(x, y) = f (h xy 1:T +T). (Here f extracts the features from h xy 0, but it could also use average pooling.) This model is trained using L cross = −E (x,y)∼D log NCE(y|x), where where Neg(y) is a set of ASR sequences not associated with video x. Note that our cross-modal training assumes there is something in common between the video and text streams. In practice this means we have to focus on certain kinds of videos, such as instructional videos, in which the spoken words and the visual content are "about" the same thing. By contrast, arbitrary videos may contain speech content that is unrelated to the visual frames (e.g., imagine a conversation between two characters in a drama or soap opera). Our overall model has three components: one transformer (BERT) that takes discrete ASR tokens, one transformer that takes continuous video features, and a third transformer to estimate mutual information between the two modalities. We jointly train the model by optimizing: We fix w bert = 0, since we use a pre-trained (frozen) BERT model for ASR. We set w visual = 1, and either set w cross = 1 or w cross = 0, depending on whether we use cross-modal training or not. In this section we conduct experiments to study the usefulness of the representations learned by our CBT model for various downstream tasks, including action anticipation, video captioning and action segmentation. We also consider ablations to our model, such as turning cross-modal training on or off, varying the size of the visual transformer, and varying the amount of unlabeled pre-training data. In this section we evaluate self-supervised visual representation learning on the downstream task of action recognition. Existing methods use various proxy tasks to pre-train feature extractors in a self-supervised way, and then use supervised learning to train linear classifiers on top of these frozen representations, or fine-tune the classifier plus feature extractor end-to-end. We follow the same protocol. Experimental setup. We follow the standard practice from recent works (; ; by pre-training our model (S3D feature extractor followed by CBT) on unlabeled RGB-only Kinetics videos. Kinetics is the largest action recognition dataset containing 500k short clip videos (about 10 seconds long) for 600 human actions classes. We take 1 minute sliding windows from the original YouTube videos they are selected from. We then use the (average pooled) S3D features as input to a linear classifier, and train the classifier on various datasets. For evaluation, we use UCF101 , which contains 13,320 videos from 101 human actions, and HMDB51 , which contains 7,000 videos from 51 classes. For both dataset we report the action recognition test accuracy averaged over the 3 standard train/test splits. To pre-train our CBT model, we use a curriculum learning strategy, by first pre-training the S3D feature extractor on unsupervised clips using the loss proposed in the 3DRotNet paper on 16 consecutive frames. We then jointly fine-tune the last blocks of S3D (Mixed5b and Mixed5c) with the visual transformer using the CBT visual loss. We observed that this strategy gave us better on downstream tasks compared to pre-training from scratch using CBT; it also saves memory and computation, which allows us to use longer sequences. During pre-training, we set the number of visual transformer layers to be 2, number of attention heads to be 4, and hidden unit size to be 768. We randomly take 60-second sliding windows from the Kinetics videos, and break them into sequences of 1.5-second clips. We randomly mask out 6 out of the 40 possible locations. We resize the video frames to 112 by 112 before encoding them with S3D to save memory. The model is trained for 500K iterations with batch size of 128 and learning rate of 1e-5. Comparison of pre-training strategies. In Table 1 (Left) we compare our way of pre-training the S3D model (i.e., using CBT visual loss) to existing approaches. In particular, we consider the Shuffle&Learn and 3DRotNet proxy tasks. We reimplement the two methods using S3D CNN, and pre-train them on the same Kinetics data. We also consider random initialization. We report classification on UCF101 and HMDB51 using frozen features and fine-tuned features passed to a linear classifier. We see that our method outperforms existing training methods by a large margin. Comparison to existing methods. Table 1(Right) compares the of our method to various state-of-the art self-supervised methods. (We only report the of fine-tuning, which are better for all methods than using frozen features.) Note that the methods differ both in architecture and training objective. First we compare against 2DCNN approaches Shuffle&Learn and OPN . Our method outperforms both by a very large margin. This can be explained by the fact that our backbone is a 3DCNN architecture, which is much more powerful than 2D CNNs for video action recognition. Next we compare against approaches using 3DCNN architectures similar to our S3D. We also outperform all of these methods by a very large margin, and even beat the most recent approach, DPC , by 3.8 points on UCF101 and 8.9 points on HMDB51. We believe this is due to the better contextual features that we are able to learn by using the transformer model and NCE loss. In this section, we consider self-supervised training of representations from long videos, followed by supervised fine-tuning on various downstream tasks. To avoid running out of memory, we pre-train the S3D model on the task of classifying (short) Kinetics videos. We then freeze this feature extractor, and focus on learning longer-term temporal representations using the self-supervised CBT model. That is, we precompute short term representations e x t = f enc (x t) for all videos using S3D, and focus on learning global representations h For the self-supervised pre-training, we use unlabeled videos from the HowTo100M dataset (b). This contains ∼ 1M instructional videos (details below), so the speech is informative about the vision. Therefore, we also run ASR on this dataset and use cross-modal training to compute h Details on self-supervised pre-training. We pre-train our model on HowTo100M (b). This is a new large-scale dataset of 1.22M narrated instructional videos available on YouTube and covers among 23k different visual tasks. The average duration of a video is 6.5 minutes and there are on average 110 clips per video. To extract visual features, we resize the videos to be 224 by 224, and compute visual features over sliding windows of 1.5 seconds (30 frames at 20 FPS) using an S3D network pre-trained on the Kinetics dataset . We take the feature embeddings from the final Mixed5c block of the S3D network before the classification layer, and average pool the features spatio-temporally to get vectors of size 1024. We follow the same strategy for extracting visual features on the downstream tasks. The visual features are not fine-tuned during pre-training or when applied to downstream tasks. To extract text features, we convert the audio track into sentences by calling the YouTube ASR API, followed by an off-the-shelf LSTM-based language model to add punctuation, thus converting the stream of words into a stream of sentences. We then follow the standard preprocessing steps from BERT , and use WordPieces tokenization with the same vocabulary of 30,000 tokens. To encode ASR tokens, we take the pre-trained BERT-base architecture, which has 12 layers of Transformers, each of which has 768 hidden units and 12 attention heads. To construct paired inputs to pre-train CBT with the cross-modal objective, we iterate over all the sentence segments in the HowTo100M dataset, and concatenate short segments until they reach the maximal length of 48 tokens. We then retrieve up to 48 visual features (72 seconds) starting at the same locations in videos. We mask out 6 out of 48 features randomly. For both the video and cross-modal transformers, we set the total hidden units per layer to 768. We fix the number of layers to 1 for the cross-modal transformer and explore the optimal number of layers and attention heads for the video transformer. Their weights are randomly initialized. For pre-training the CBT model on HowTo100M, we use 32 Cloud TPUs and a total batch size of 128. We use the Adam optimizer with an initial learning rate of 1e-5 and a linear decay learning rate schedule. The model is trained for 2 million iterations, which takes around 2 days. Details on supervised evaluation. We evaluate the pre-trained temporal representations by transfer learning to downstream tasks. We first focus on action anticipation, whose goal is to predict the future actions by observing video sequences preceding them. In the supplementary, we also present on video captioning and action segmentation. For the action anticipation task, we follow the standard setup described from recent work (; a). We consider three datasets: the Breakfast dataset is composed of 1712 cooking videos and contains 48 fine-grained action classes; the 50Salads dataset contains 50 cooking videos with 17 fine-grained action classes; and the ActivityNet 200 dataset contains 19994 YouTube videos with 200 human action classes (beyond the cooking and instructional domains). The inputs are video segments up to T c seconds before the actual action starts, and the outputs are categorical labels. For comparison with previous approaches, we set T c = 1 for Breakfast and 50Salads, and T c = 5 for ActivityNet. For all experiments except for ablation on sequence lengths, we fix the input sequence length to be 72 seconds (corresponding to 48 sliding windows), features for videos shorter than 72 seconds are zero-padded. The outputs of CBT are transformed features with the same length, we take the output feature at the last non-padded position to represent the whole sequence, and put a linear classifier on top to predict the future actions. We jointly fine-tune the weights of visual transformers with the linear classifier. The text transformers and cross-modal transformers are not used during fine-tuning since only visual inputs are available. We train our model for 5 epochs using a batch size of 32 with the Adam optimizer and an initial learning rate of 1e-3. We report the top-1 accuracy on the test sets for Breakfast and 50Salads, and on the validation set for ActivityNet. Comparison to existing methods. Table 2 (Left) compares to existing methods. First we compare to two existing self-supervised approaches, namely and Sun et al. (2019a). Our approach outperforms both by a very large margin. The difference with VideoBERT (Sun et al. (2019a) ), which also relies on a BERT model, can be explained by the fact that it quantizes the visual features into tokens and, hence loses discriminative power. Next we compare to some recent methods that train deep classifiers end-to-end, namely and (a). We outperform both by a large margin. Saldads, ActivityNet. Self-super = Y means the model was pre-trained in a self-supervised way, and then fine-tuned using a linear classifier. Self-super = N means the model is trained end-to-end on the specific task. (Right) Comparison with the average pooling and LSTM baselines on 50Salads Breakfast, 50Salads and ActivityNet. We vary the observation window lengths (sec.) Effect of video length. In Table 2 (Right) we show the impact of the length of the training videos on the performance. We compare with two baselines, average pooling (AvgPool) and LSTM . The AvgPool baseline simply computes the average of all input visual features over time. The LSTM baseline takes the same sequence of S3D features but recurrently updates its hidden states over time. The final hidden state is used for classification. We adjust the hidden unit size of LSTM to make its number of parameters comparable to CBT. We can see that CBT significantly outperforms the two baselines on all three datasets. Moreover, we can see that as the observed video length increases, the performance of CBT monotonically increases, while LSTM and AvgPool either plateaus or decreases. These indicate that CBT is better at modeling long-term temporal context. Table 3: Ablation study on the action anticipation task. We show accuracy on Breakfast, 50Salads and ActivityNet. (Left) Impact of the percentage of HowTo100M videos used, and the cross-modal objective during pre-training. 0% corresponds to no pretraining, ie. using random weights. (Middle, Right) Impact of the number of layers (L) and attention heads (A) for the visual transformers. Effect of dataset size and cross-modal training. In Table 3 (Left), we study the impact of pretraining data set. As expected, pre-training with more examples leads to higher performance on all three benchmarks. We also study the impact of cross-modal training. We see this helps signficantly, especially on the smaller datasets (Breakfast and Salads). Effect of model size. In Table 3 (Middle) and (Right), we study the impact of the number of layers (L) and the number of attention heads (A) for the visual transformer. Not surprisingly, model performance initially increases, but surprisingly, it then starts to decrease, in contrast to the case of NLP-BERT. We conjecture that this is because our unlabeled pre-training set is much smaller than used by the NLP-BERT model. Fortunately, our technique is quite scalable, since we can train the video representations on top of S3D features using relatively shallow transformers -our visual transformer only has 15M parameters, whereas the BERT NLP transformer has 110M parameters. Applications to other tasks. In Table 4 we show the of using of our learned temporal representation for video captioning and action segmentation. See section 6.1 and section 6.2 in the supplementary for details. Table 4: (Left) Video captioning on the YouCook2 dataset (b). We compare with previous state-of-the-art methods by Zhou et al. (2018c) and Sun et al. (2019a), the caption decoder of all methods share the same architecture, the main difference comes from the visual encoder. (Right) Action segmentation on the COIN dataset . A linear classifier is applied on the sequence of CBT output features for dense frame labeling. We compare with previous state-of-the-art methods using the standard frame accuracy metric. We have shown how to extend the BERT model to learn representations from video in a self-supervised way, without needing vector quantization or pre-trained visual features. We have also shown how to extend this to the cross-modal setting, when ASR is available. Finally, we demonstrated that our method learns features that are far more useful than existing self-supervised methods for a variety of downstream video tasks, such as classification, captioning and segmentation. We believe that the simplicity and modularity of our method will let us scale to much larger unlabeled video datasets, which we hope will let us finally surpass supervised video pretraining (e.g., on Kinetics), just as other methods (e.g., CPC++ (Hénaff et al., 2019) ) have recently surpassed supervised image pretraining (on ImageNet). In this section, we apply our model to video captioning. Dataset. We pretrain our model on HowTo100M, and then use its features as input to a captioning model (details below) which is trained on the YouCook2 dataset (b). This contains 2000 Youtube videos of an average length of 5.26 minutes for a total of 176 hours. The annotations consist of segmentation boundaries and captions with on average 7.7 segments per video and 8.8 words per caption. We made sure that there is no overlap between the videos from our pre-training datasets and YouCook2. Model. We follow the experimental setup from (c), where the ground truth video segmentations from YouCook2 are used to train a supervised model mapping video segments to captions. Our captioning model is a transformer with 2 layers and a hidden layer of size 128. During training we set the dropout probability to 0.4. We train our model for 10K iterations using batch size of 128 with the Adam optimizer and an initial learning rate of 1e-4. We report BLEU, METEOR and ROUGE metrics on the validation set. Comparison to other methods. Table 4 shows our . We outperform a simple baseline computed using average-pooled S3D features. We also outperform the approach of Zhou et al. (2018c) and VideoBERT Sun et al. (2019a) on all reported metrics. The comparison to VideoBERT is particularly interesting. The gains suggest that removing the quantization of video features is important for obtaining a fine-grained video representation. We also observe that the difference between CBT and VideoBERT is smaller for YouCook2 than for Breakfast and 50Salads action anticipation task, possibly because the YouCook2 dataset set is more similar to the cooking videos used for pre-training by VideoBERT. In this section, we apply our model to the task of temporal action segmentation. Dataset. We pretrain our model on HowTo100M and then use its features as input to a linear classifier (details below) which is trained on the COIN dataset. This contains 11827 instructional Youtube videos of an average length of 2.36 minutes. The annotations consist of segment boundaries and class label. On average there are 3.91 segments per video each of which lasts 14.9 seconds. There are in total 779 classes. Model. We extract video features using S3D and feed the sequence to the visual transformer. We use a fixed size of 72 seconds and use zero-padding for shorter sequences. The overall clip is represented by its associated output embedding of size 768. This preprocessing step is frozen. We feed the features to a linear classifier, which we train or model for 100K iterations using batch size of 32 with the Adam optmizer and initial learning rate of 1e-3. At test time we operate on a long video with a sliding window of 72 seconds. Comparison to existing approaches. In Table 4 we compare CBT against various state of the art approaches using the frame acuracy as metric, including , and . We outperform them by a large margin state (+19.6 points).

Represent the TLDR for finding the corresponding scientific article

Generalized BERT for continuous and cross-modal inputs; state-of-the-art self-supervised video representations.

Conditional VAE on top of latent spaces of pre-trained generative models that enables  transfer between drastically different domains while preserving locality and semantic alignment.


Question: Represent the article for finding a single-sentence TLDR

Question: We present a generic dynamic architecture that employs a problem specific differentiable forking mechanism to leverage discrete logical information about the problem data structure. We adapt and apply our model to CLEVR Visual Question Answering, giving rise to the DDRprog architecture; compared to previous approaches, our model achieves higher accuracy in half as many epochs with five times fewer learnable parameters. Our model directly models underlying question logic using a recurrent controller that jointly predicts and executes functional neural modules; it explicitly forks subprocesses to handle logical branching. While FiLM and other competitive models are static architectures with less supervision, we argue that inclusion of program labels enables learning of higher level logical operations -- our architecture achieves particularly high performance on questions requiring counting and integer comparison. We further demonstrate the generality of our approach though DDRstack -- an application of our method to reverse Polish notation expression evaluation in which the inclusion of a stack assumption allows our approach to generalize to long expressions, significantly outperforming an LSTM with ten times as many learnable parameters. Deep learning is inherently data driven -visual question answering, scene recognition, language modeling, speech recognition, translation, and other supervised tasks can be expressed as: given input x, predict output y. The field has attempted to model different underlying data structures with neural architectures, but core convolutional and recurrent building blocks were designed with only general notions of spatial and temporal locality. In some cases, additional information about the problem can be expressed simply as an additional loss, but when hard logical assumptions are present, it is nonobvious how to do so in a manner compatible with backpropagation. Discrete logic is a fundamental component of human visual reasoning, but there is no dominant approach to incorporating such structural information into deep learning models. For particular data structures and settings, there has been some success. However, in prior work additional structure must either be learned implicitly and without additional annotations or is available at both train and test time. For example, StackRNN BID7 ) allows recurrent architectures to push and pop from a stack. While this approach works well without explicit stack trace supervision, implicit learning only goes so far: the hardest task it was tested on is binary addition. Approaches such as recursive NN BID13 ) and TreeRNN BID15 ) allow inclusion of explicit tree structures available during both training and testing, but neither can be used when additional supervision is available only at training time. We consider this the most general problem because it is not feasible to obtain good without any additional supervision if the problem is sufficiently difficult. Our objective is to develop a general framework for differentiable, discrete reasoning over data structures, including as stacks and trees. Our approach is flexible to differing degrees of supervision and demonstrates improved when structural assumptions are available at test time. We are less concerned with the no-supervision case because of limitations in scalability, as demonstrated by the scope of StackRNN.We present our framework in the context of two broader architectures: Neural Module Networks (NMN, BID0) and Neural Programmer-Interpreters (NPI, BID11).The original NMN allows per-example dynamic architectures assembled from a set of smaller models; it was concurrently adapted in N2NMN BID4 ) and IEP as the basis of the first visual question answering (VQA) architectures successful on CLEVR BID5 ). The NPI work allows networks to execute programs by directly maximizing the probability of a successful execution trace. In the present work, we present two applications of our framework, which is a superset of both approaches. The first is our CLEVR architecture, which introduces two novel behaviors. It interleaves program prediction and program execution by using the output of each module to predict the next module; this is an important addition because it improves the differentiability of the model. For IEP/N2NMN, the discrete program in the middle of the model breaks the gradient flow. For our model, although the selection of modules is still a discrete non-differentiable choice, it is influenced by the loss gradient: the visual state gives a gradient pathway learnable through the question answer loss. The second contribution of this architecture is a novel differentiable forking mechanism that enables our network to process logical tree structures through interaction with a stack of saved states. This allows our model to perform a broad range of logical operations; DDRstack is the first architecture to obtain consistently strong performance across all CLEVR subtasks. We briefly discuss our rationale for evaluation on CLEVR as well as prior work on the task. Though CLEVR is too easy with or without program supervision, it is the best-available proxy task for high-level reasoning. Its scale, diverse logical subtask categories, and program annotations make the dataset the best current option for designing discrete visual reasoning systems. By effectively leveraging the additional program annotations, we improve over the previous state-of-the-art with a much smaller model -on the important Count and Compare Integer subtasks, we improve from 94.5 to 96.5 percent and 93.8 to 98.4 percent, respectively. However, our objective is neither the last couple percentage points of accuracy on this task nor to decrease supervision, but to motivate more complex tasks over knowledge graphs. We expect that it is possible to improve accuracy on CLEVR with a static architecture using less supervision. This is largely unrelated to the objective of our work -we view CLEVR as a good first step towards increased supervision for the learning of complex logic. Human-level general visual reasoning from scratch is less reasonable than from expressively annotated data: we consider improving and generalizing the ability of architectures to better leverage additional supervision to be the most likely means to this end. Prior work on CLEVR is largely categorized by dynamic and static approaches. IEP BID6 ) and N2NMN both generalized the original neural module networks architecture and used the functional annotations in CLEVR to predict a static program which is then assembled into a tree of discrete modules and executed. IEP further demonstrated success when program annotations are available for only a few percent of questions. These are most similar to our approach; we focus largely upon comparison to IEP, which performs significantly better. RN BID12 ) and FiLM BID10 ), the latter being the direct successor of CBN BID9 ) are both static architectures which incorporate some form of implicit reasoning module in order to achieve high performance without program annotations. In contrast, our architecture uses program annotations to explicitly model the underlying question structure and jointly executes the corresponding functional representation. As a , our architecture performs comparably on questions requiring only a sequence of filtering operations, but it performs significantly better on questions requiring higher level operations such as counting and numerical comparison. We present DDRstack as a second application of our framework and introduce a reverse Polish notation (RPN) expression evaluation task. The task is solvable by leveraging the stack structure of expression evaluation, but extremely difficult without additional supervision: a much larger LSTM baseline fails to attain any generalization on the task. We therefore use RPN as additional motivation for our framework, which introduces a simple mechanism for differentiably incorporating the relevant stack structure. Despite major quantitative differences from CLEVR VQA, the RPN task is structurally similar. In the former, questions seen at training time contain direct programmatic representations well modeled by a set of discrete logical operations and a stack requiring at most one recursive call. The latter is an extreme case with deep recursion requiring a full stack representation, but this stack structure is also available at test time. In summary: the DDR framework combines the discrete modular behavior of NMN and IEP with an NPI inspired forking behavior to leverage structural information about the input data. Our approach resolves common differentibility issues and is easily adapted to the specific of each problem: we achieve a moderate improvement over previous state-of-the-art on CLEVR and succeed on RPN where a much larger baseline LSTM fails to attain generalization. 2.1 CLEVR CLEVR is a synthetic VQA dataset that encourages approaches capable of discrete reasoning through its inclusion of functional program annotations that model the logic of each question. The dataset consists of 100k images and 1 million question/answer pairs. Over 850k of these questions are unique. Images are high quality 3D Blender BID1 ) renders of scenes containing geometric objects of various shapes, sizes, colors, textures, and materials. Thus the dataset is quite realistic despite being synthetic. Furthermore, the authors ran comprehensive tests to avoid exploitable biases in the data. As the objects are geometric figures, no external knowledge of natural images is required, as in earlier VQA datasets. Most importantly, CLEVR provides an expressive program representation of each question. For example, "How many red spheres are there?" is represented as [filter red, filter sphere, count]. Some questions require nonlinear program structure, such as "How many objects are red or spheres", which is represented by a tree with two branches [filter red] and [filter sphere] followed by a binary [union] operation and a final [count]. We include additional examples in the appendix. We are unaware of any dataset with comparable properties of this size and complexity. By raw accuracy alone, CLEVR is effectively solved. Neural architectures have already far surpassed human accuracy on the task, both with and without using program annotations at train time. One perspective is that this should motivate a return to the natural image setting without programs or with transfer learning from CLEVR. In contrast, we believe the rapid recent progress on CLEVR motivates more complex synthetic tasks -perhaps involving harder logical inference over general knowledge graphs. It is not currently obvious what form a visual Turing test should take, nor is it clear what should comprise the train set for such a task: this will likely require much iteration and experimentation. On this front, the synthetic setting is unmatched: making even slight changes to a natural image dataset often involves a length additional data collection task compared to a programmatic change in the synthetic case. We introduce the reverse Polish notation (RPN) expression evaluation dataset as a motivation for additional supervision in higher level learning tasks. The specific problem form we consider is [NUM]*(n+1)-[OP]*n, that is, n + 1 numbers followed by n operations. For example, "2 3 4 + *" evaluates to 14. This simplifies the problem by eliminating consideration for order of operations. Thus the task is: given a sequence of tokens corresponding to a valid expression in reverse Polish notation, evaluate the expression and produce a single real valued answer. This may seem like a simple task; it is not. For large n, expressions behave somewhat like a hash function. Small changes in the input can cause wild variations in the output -we found the problem intractable in general. Our objective is to make stronger structural assumptions about the problem and create an architecture to leverage them. For this reason, our framework is incomparable to StackRNN, which attempts to learn a stack structure implicitly but is unable to incorporate additional supervision when the problem is likely too difficult to solve otherwise. We therefore modify the problem as such: instead of producing only the final expression evaluation, produce the sequence of answers to all n intermediate expressions in the answer labels. For the example "2 3 4 + *", the expected output would be because 3+4=7 and 2*7=14. We further assume the stack structure of the problem is available to the architecture should it be capable of taking advantage of such information. The problem is still sufficiently complex -note that to the model, {1, 3, 4, +, *} would all be meaningless tokens: it must learn both the NUM and the OP tokens. The dataset consists of 100k train, 5k validation, and 20k test expression with n = 10 -that is, 11 numbers followed by 10 operations. We also provide a 20k expression generalization set with n = 30. The label for each question contains the n solutions to each intermediate operation. During data generation, we sample NUM and OP tokens uniformly, reject expressions including division by zero, and also omit expressions that evaluate to over 100 in magnitude. The NUM tokens correspond to 0, 0.1,..., 0.9 and the OP tokens correspond to +, -, *, /; however, architectures are not privy to this information. The purpose of the DDR framework is to naturally incorporate structured information into a neural reasoning architecture. This is important when the given problem is hard without additional supervision and allows the model to perform discrete, complex reasoning. Our framework addresses the difficulty of combining discrete logic with clean, differentiable training and is capable of interfacing with a broad range of data structures. Our framework is a clean fusion of two broad design patterns. Like IEP, we maintain a set of problem-specific neural modules to allow our model to learn relevant program primitives. Like NPI, we interleave program prediction with program execution, differentiably learning modules when module arrangement is not known at test time. This is much more general compared to either IEP/NMN or NPI independently, and the particular mechanism for combining them is non-trivial differentiable forking operation. IEP alone lacks the ability to examine the output of intermediate operations. The relevance of this is particularly clear in the CLEVR setting. The NPI architecture can learn sequences of functions, but lacks the ability to learn the functions themselves. Our approach responds flexibly to the problem supervision: in VQA, modules are known only at train time. At each timestep, the controller therefore produces an index corresponding to a neural module, which is then executed. On the RPN task, the problem structure is also known at test time; the controller is therefore deterministic and directly executes the correct module. We refer to our VQA and RPN architecture adaptations as DDRprog and DDRstack, respectively; details are provided below. DDRprog is a direct adaptation of our framework to CLEVR, requiring only some standard encoders to handle the mixed language and visual data. We provide pseudocode in Algorithm 1, a visual representation in FIG0, and subnetwork details in Table 2 (see Appendix).The input data x for each sample is a (image, question, program) triple; the label y is a (answer, program) pair. The program only available at train time, thus our model must learn to predict it. The network first applies standard LSTM and ResNet ) encoders to the question/image, producing language and visual states. The ResNet encoder is unchanged from FiLM/IEP.Both the language and visual states are passed to the controller. We use a recurrent highway network (RHN) BID17 ) as recommended by BID14 instead of an LSTM BID3 ) -both accept flat inputs. As the visual state contains convolutional maps, we flatten it with a standard classifier. At each time step, the controller outputs a standard softmax classification prediction, which is interpreted as an index over the set of learnable neural modules. These are smaller, slightly modified variants of the modules used in IEP. The selected module is executed on the visual state; Algorithm 1 DDRprog. Note that CNN produces a flattened output and Controller also performs a projection and argmax over program scores to produce programP rediction img, question DISPLAYFORM0 the visual state is set to the output. The module prediction at the final timestep is followed by a small classifier network, which uses the IEP classifier. This architecture introduces a significant advantage over IEP: as modules are predicted and executed one at a time instead of being compiled into a static program, our model can observe the of intermediate function operations -these have meaning as filtering and counting operations on CLEVR.We now motivate our differentiable forking mechanism. As presented thus far, our approach is sufficient on the subset of CLEVR programs that do not contain comparison operations and are effectively linear -indeed, we observe a large performance increase over IEP on this subset of CLEVR. However, some CLEVR questions contain a logical branching operation (e.g. are there more of ... than ... ?) and cannot be answered by structurally linear programs. In general, programs can take the form of expressive trees, but CLEVR programs contain at most two branches. Adding a differentiable forking mechanism handles the general case without modification from the CLEVR architecture. Upon encountering a program branch, our architecture pushes the current language and visual states to a stack and forks a subprocess. This subprocess is effectively a copy of the main network that maintains its own states. It takes as input the language state and the initial and current visual states. Effectively, a different copy of the network with its own state processes each branch of the program. Upon processing the last operation in the branch, a binary cell is applied to the subprocess state outputs and the main process states (popped from the stack), merging them as shown in FIG0. Our architecture is likely to generalize even past the setting of tree processing, as we could replace the stack with a priority queue or any other data structure pertinent to the problem. Finally, a technical note for reproducibility: the fork module must differ from a standard unary module, as it is necessary to pass the original ResNet features (e.g. the initial visual state) to the subprocess in addition to the current visual state. Consider the question: "Is the red thing larger than the blue thing?" In this case, the main network filters by red; it is impossible to recover the blue objects in the subprocess given a red filtered image. We found that it is insufficient to pass only the original images to the subprocess, as the controller is small and has difficulty tracking the current branch. We therefore use a variant of the binary module architecture that merges the original ResNet features with the current visual state (see Algorithm 1). As the fork module is shared across all branch patterns, it is larger than the other binary modules and also one layer deeper -refer to the appendix for full architecture details on each layer. The DDRstack architecture applies our general framework to the increased supervision setting of the RPN task -module arrangement is a fixed expression parse tree. One natural view of the task is: given a parse tree structure, simultaneously socket and refine the learnable NUM and OP nodes. Our model consists of an LSTM controller and a set of 4 learnable binary modules -one per OP -as well as an explicit stack. DDRstack processes one token at a time; similar to unary/binary modules in IEP, NUM and OP tokens are processed differently: NUM: Our model embeds the token and passes it to the LSTM. It then pushes the to the stack. OP: Our model pops twice, calls the OP specific binary cell, and then passes the to the LSTM. It then pushes the to the stack. The binary cell is a simple concatenation of the arguments followed by a single fully connected layer with no nonlinearity. DDRstack can be viewed as a neural analog to standard analytical RPN expression evaluation algorithm where the values of the NUM and OP tokens are unknown. We provide high level pseudocode for the model in Algorithm 2 and a visual representation in FIG1. We train a baseline vanilla LSTM supervised with the intermediate solutions in the last n timesteps. DDRstack uses the same baseline LSTM as its core controller, but includes the aforementioned stack behavior. For both models, predictions are made in the last n timesteps (Algorithm 2 shows only the final return). Table 1: Accuracy on all CLEVR question types for baselines and competitive models. The Human baseline is from the original CLEVR work. * denotes additional program supervision. SA refers to stacked spatial attention BID16. rate and rough network size. The network overall has 9M parameters. We exclude the ResNet feature extractor from all calculations because it is also present in the best FiLM model. Their work further demonstrated it is fairly straightforward to replace it with a from-scratch feature extractor with minimal loss in accuracy. We pass the ground truth program labels to the model during training. Critically, the program labels are only used on the validation set for the purpose of model selection, and our final accuracy is obtained by rerunning the validation set without the ground truth programs. We train on a single GTX 1080 TI and, after 35 epochs, our model matches the previous state-of-the-art accuracy of 97.7 percent. We continue training until the 52nd epoch, dropping the learning rate to 1e-5 for the last few epochs to ensure convergence, and obtain 98.3 percent accuracy. The model predicts program cells with 99.98 percent accuracy. Several models have far exceeded human accuracy on CLEVR -the task remains important for two reasons. First, though CLEVR is large and yields consistent performance across runs, different models exhibit significantly different performance across question types. Where every competitive previous work exhibits curiously poor performance on at least one important subtask, our architecture dramatically increases consistency across all tasks. Second, CLEVR remains the best proxy task for high-level visual reasoning because of its discrete program annotations -this is far more relevant than raw accuracy to our work, which is largely concerned with the creation of a general reasoning framework. However, we do achieve a modest improvement in raw accuracy over the previous state-of-the-art with a >5X smaller architecture. We presently consider RN, FiLM, IEP, and our architecture as competitive models. From Table 1, no architecture has particular difficulty with Exist, Query, or Compare questions; the main differentiating factors are Count and Compare Integer. Though Compare Integer is the smallest class of questions and is therefore assigned less importance by the cross entropy loss, the IEP suggests that this does not cause models to ignore this question type. We therefore consider Count and Compare Integer to be the hardest unary and binary tasks, respectively, and we assign most important to these question subsets in our analysis. We achieve strong performance on both subtasks and a significant increment over previous state-of-the-art on the Count subtask. We first compare to IEP. Our model is 4x smaller than IEP (see Table 1) and resolves IEP's poor performance on the challenging Count subtask. Overall, DDRprog performs at least 2x better across all unary tasks (+1.7 percent on Exist, +3.8 percent on Count, + 1.0 percent on Query) it closely matches binary performance (+0.2 percent on Compare, -0.3 percent on Compare Integer). We believe that our model's lack of similar gains on binary task performance can be attributed to the use of a singular fork module, which is responsible for cross-communication during prediction of both branches of a binary program tree, shared across all binary modules. We have observed that this module is essential to obtaining competitive performance on binary tasks; it is likely suboptimal to use a large shared fork module as opposed to a separate smaller cell for each binary cell. Our model surpasses RN in all categories of reasoning, achieving a 2.6x reduction in overall error. RN achieves impressive for its size and lack of program labels. However, it is questionable whether the all-to-all comparison model will generalize to more logically complex questions. In particular, Count operations do not have a natural formulation as a comparison between pairs of objects, in which case our model achieves a significant 6.4 percent improvement. RN also struggles on the challenging Compare Integer subtask, where we achieve a 4.8 percent improvement. Furthermore, it is unclear how essential high epoch counts are to the model's performance. As detailed in Table 1, RN was trained in a distributed setting for 1000 epochs. Both our and FiLM were obtained on single graphics cards and were only limited in number of epochs for practicality -FiLM had not fully converged, and our model was unregularized. Both IEP and our model achieve a roughly 4x improvement over FiLM on Compare Integer questions (4.9 and 4.6 percent, respectively), the difference being that our model eliminates the Count deficiency and is also 4X smaller than IEP. The contrast between FiLM's Compare Integer and Exist/Query/Compare performance suggests a logical deficiency in the model -we believe it is difficult to model the more complex binary question structures using only implicit branching through batch normalization parameters. FiLM does achieve strong Compare Attribute performance, but many such questions can be more easily resolved through a sequence of purely visual manipulations. FiLM achieves 1.5x relative improvement over our architecture on Exist questions, but this is offset by our 1.5x relative improvement on Count questions. Given proximity in overall performance, FiLM could be seen as the main competitor to our model. However, they achieve entirely different aims: DDRprog is an application of a general framework, >5X smaller, and achieves stable performance over all subtasks. FiLM is larger and suffers from a significant deficiency on the Compare Integer subtask, but it uses less supervision. As mentioned in the introduction, our model is part of a general framework that expands the ability of neural architectures to leverage discrete logical and structural information about the given problem. In contrast, FiLM is a single architecture that is likely more directly applicable to low-supervision natural image tasks. For our architecture, we use hidden dimension 32 throughout the model, ing in only 17k parameters overall. We train with Adam using learning rate 1e-3 and obtain a test L1 error of 0.17 after 63 epochs. Using the same hidden dimension in the pure LSTM baseline (9k parameters) in test error 0.28. We overcompensate for the difference in model size by increasing the hidden dimension of the LSTM to 128 (255k parameters), ing in an only slightly lower test error of 0.24 after nearly 3000 epochs. FIG2 shows training curves for the LSTM baseline and DDRstack. After training both models on problems of length n = 10, we test both models on sequences of length n = 10 and n = 30. For a sequence of length n both models predict values not only for the entire expression but also for all n subproblems where index n corresponds to evaluating the entire sequence. For sequences of both lengths, we evaluate accuracy for the predicted answers on all subproblems. Results are shown in FIG2. We argue that the LSTM fails on the RPN task. This is not immediately obvious: from FIG2, both the small and large LSTM baselines approximately match our model's performance on the first 5 subproblems of the n = 10 dataset. From n = 6 to n = 10, the performance gap grows between our models -the small LSTM is unable to learn deep stack behavior, and performance decays sharply. The n = 30 dataset reveals the failure. The LSTM's performance is far worse on the first few subproblems of this dataset than on the test set of the original task. This is not an error: recall the question formatting [NUM]*(n + 1)-[OP]*n. The leading subproblems do not correspond to the leading tokens of the question, but rather to a central crop. For example, the first two subproblems of "12345+-*/" are given by "345+-", not "12345" -the latter is not a valid expression. The rapid increase in error on the LSTM implies that it did not learn this property, let alone the stack structure. Instead, it memorized all possible subproblems of length n ∈ {1, 2, 3} expressions preceding the first few OP tokens. Performance quickly decays to L1 error greater than 2.0, which corresponds to mostly noise (the standard deviation of answers minus the first few subproblems is approximately 6.0). In contrast, our model's explicit incorporation of the stack assumption in a smooth generalization curve with a gradual decay in performance as problem length increases. We briefly address a few likely concerns with our reasoning. First, one might argue that DDRstack cannot be compared to an LSTM, as the latter does not incorporate explicit knowledge of the problem structure. While this evaluation is correct, it is antithetical to the purpose of our architecture. The LSTM baseline does not incorporate this additional information because there is no obvious way to include it -the prevailing approach would be to ignore it and then argue the model's superiority on the basis that it performs well with less supervision. This logic might suggest implicit reasoning approaches such as StackRNN, which attempt to model the underlying data structure without direct supervision. However, we do not expect such approaches to scale to RPN: the hardest task on which StackRNN was evaluated is binary addition. While StackRNN exhibited significantly better generalization compared to the LSTM baseline, the latter did not completely fail the task. In contrast, RPN is a more complex task that completely breaks the baseline LSTM. While we did not evaluate Stack-RNN on RPN (the original implementation is not compatible modern frameworks), we consider it highly improbably that StackRNN would generalize to RPN, which was intentionally designed to be difficult without additional supervision. In contrast, our dynamic approach achieves a dramatic increase in performance and generalization precisely by efficiently incorporating additional supervision. StackRNN is to DDRstack as FiLM is to DDRprog: one motive is to maximize performance with minimal supervision whereas our motive is to leverage structural data to solve harder tasks. The DDR framework facilitates high level reasoning in neural architectures by enabling networks to leverage additional structural information. Our approach resolves differentiability issues common in interfering with discrete logical data and is easily adapted to the specific of each problem. Our work represents a clean synthesis of the modeling capabilities of IEP/NMN and NPI through a differentiable forking mechanism. We have demonstrated efficacy through two applications of our framework. DDRprog achieves a moderate improvement over previous state-of-the-art on CLEVR with greatly increased consistency and reduced model size. DDRstack succeeds on RPN where a much larger baseline LSTM fails to attain generalization. It is our intent to continue refining the versatility of our architecture, including more accurate modeling of the fork module, as mentioned in our CLEVR VQA discussion. Our architecture and its design principles enable modeling of complex data structure assumptions across a wide class of problems where standard monolithic approaches would ignore such useful properties. We hope that this increase in interoperability between discrete data structures and deep learning architectures aids in motivating higher level tasks for the continued progression and development of neural reasoning. Table 2: Architectural details of subnetworks in DDRprog as referenced in FIG0 and Algorithm 1. Finegrained layer details are provided in tables 4-8. Source will be released pending publication. ResNetFeaturizer DISPLAYFORM0 DISPLAYFORM1 Residual: Add and h × 14 × 14 ReLU h × 14 × 14 InstanceNorm h × 14 × 14 Concatenate and 2h DISPLAYFORM2 Add and h × 14 × 14 ReLU h × 14 × 14 DISPLAYFORM3 Add and 6h DISPLAYFORM4 DISPLAYFORM5 Residual: Add and h × 14 × 14 ReLU h × 14 × 14 1 filter size large 1 filter color yellow 1 filter material rubber 1 filter shape cube 1 unique 1 relate left 0 fork 1 filter size large 1 filter color blue 1 filter material metal 1 filter shape cube 1 unique 1 relate behind 2 intersect 1 filter shape cube 1 unique 1 query color • Answer (predicted, label): green, blue DISPLAYFORM6 • Image Index: 8543 • Question: are there fewer small purple rubber things that are behind the green metallic cylinder than small things that are in front of the tiny matte block? • Program (label): 1 filter size small 1 filter material rubber 1 filter shape cube 1 unique 1 relate front 1 filter size small 1 count 0 fork 1 filter color green 1 filter material metal 1 filter shape cylinder 1 unique 1 relate behind 1 filter size small 1 filter color purple 1 filter material rubber 1 count 2 less than • Answer (predicted, label): no, yes

Represent this text to find a much much longer article about it with say hundreds-thousands of words

A generic dynamic architecture that employs a problem specific differentiable forking mechanism to encode hard data structure assumptions. Applied to CLEVR VQA and expression evaluation.

We present a new approach, SNIP, that is simple, versatile and interpretable; it prunes irrelevant connections for a given task at single-shot prior to training and is applicable to a variety of neural network models without modifications.