Question: represent the scientific article to find a short tldr

Question: Stochastic neural networks with discrete random variables are an important class of models for their expressivity and interpretability. Since direct differentiation and backpropagation is not possible, Monte Carlo gradient estimation techniques have been widely employed for training such models. Efficient stochastic gradient estimators, such Straight-Through and Gumbel-Softmax, work well for shallow models with one or two stochastic layers. Their performance, however, suffers with increasing model complexity. In this work we focus on stochastic networks with multiple layers of Boolean latent variables. To analyze such such networks, we employ the framework of harmonic analysis for Boolean functions. We use it to derive an analytic formulation for the source of bias in the biased Straight-Through estimator. Based on the analysis we propose \emph{FouST}, a simple gradient estimation algorithm that relies on three simple bias reduction steps. Extensive experiments show that FouST performs favorably compared to state-of-the-art biased estimators, while being much faster than unbiased ones. To the best of our knowledge FouST is the first gradient estimator to train up very deep stochastic neural networks, with up to 80 deterministic and 11 stochastic layers. Stochastic neural networks with discrete latent variables have been an alluring class of models for their expressivity and interpretability, dating back to foundational work on Helmholtz machines and sigmoid belief nets . Since they are not directly differentiable, discrete random variables do not mesh well with the workhorse of modern Deep Learning, that is the backpropagation algorithm. Monte Carlo gradient estimation is an effective solution where, instead of computing the true gradients, one can sample gradients from some distribution. The sample estimates can be either biased or unbiased. Unbiased gradient estimates like score function estimators come typically at the cost of high variance leading to slow learning. In contrast, biased gradient estimates such Straight-Through , while efficient, run the risk of convergence to poor minima and unstable training. To this end several solutions have recently been proposed that either reduce variance in unbiased estimators (; ; ; ;) or control bias in biased estimators . These methods, however, have difficulty scaling up to complex neural networks with multiple stochastic layers: low-variance unbiased estimators are too expensive 1, while the compounded bias from the continuous relaxations on multiple stochastic layers leads to poor minima. In this work we focus on biased estimators. Our goal in this paper is a gradient estimator for Boolean random variables that works for any complex -deep or wide-neural network architecture. We resort to the term Boolean instead of binary to emphasize that we work directly on the Boolean space {−1, +1}, without any continuous relaxations or quantizations. With this in mind we re-purpose the framework of harmonic analysis of Boolean functions, widely used in computational learning and computational complexity theory (; ; ;). We cast stochastic neural networks as Boolean functions f (z) over Boolean latent variables z sampled from probability 1. We introduce the framework of harmonic analysis of Boolean functions to analyze discrete stochastic neural networks and their REINFORCE and Straight-Through gradients. We show that stochastic gradients compute Fourier coefficients. 2. Based on the above harmonic analysis we present FouST -a low-bias gradient estimator for Boolean latent variables based on three bias reduction steps. As a side contribution, we show that the gradient estimator employed with DARN , originally proposed for autoregressive models, is a strong baseline for gradient estimation in large and complex models with many stochastic layers. 3. We show that FouST is amenable to complex stochastic neural networks with Boolean random variables. To the best of our knowledge, FouST is the first gradient estimate algorithm that can train very deep stochastic neural networks with Boolean latent variables. The practical outcome is a simple gradient estimate algorithm that can be plugged in complex stochastic neural networks with multiple layers of Boolean random variables. We consider Boolean functions on the n-dimensional Boolean cube, f: {−1, +1} n → R. The setting of Harmonic Analysis for Boolean functions is the space of Boolean functions f with a product probability distribution on the Boolean input, that is p(z) = n i=1 p i (z). We denote p i as the probability of the i-th dimension being one, i.e., p i:= p i (z i = +1). We denote the mean and variance of z i by µ i and σ i, respectively. An example of a Boolean function in this setting is a generative neural network f: z → y with a factorized latent distribution, as commonly done in representation learning . In this example, z is the stochastic input -also known as the latent code in stochastic neural networks -taking only two possible values and y is the output, like cross entropy loss. Often, the goal of a generative neural network is to learn or approximate the latent distribution given input data x, i.e., p(z|x), as we will also explore in the experiments. We first introduce a few basic operations in the context of Harmonic Analysis of Boolean functions, which we shall use further on. Further necessary details are in the Appendix A. For a more comprehensive introduction, however, we refer the reader to. Inner product. The inner product of two Boolean functions f and g is: Fourier expansion. Let S be any subset of dimensions of the n-dimensional Boolean cube, S ⊆ [n] = {1, ..., n}. Per S we define a basis function, φ S (z):= i∈S φ i (z i), where for the empty set φ ∅ (z) = 1 and φ i is the z-score normalized dimension, i.e., φ i:= zi−µi σi. For example, under the uniform Bernoulli distribution for the i-th dimension, The 2 n functions φ S form an orthonormal basis for the space of Boolean functions f,, for i = j, where the expectations compute the inner product between two Boolean functions. The last identity derives from the independence of any dimensions i = j. We can then expand the Boolean function f on the set of 2 n orthonormal basis functions, also known as the p-biased Fourier expansion of f. Thef (p) (S) are the Fourier coefficients computed by the inverse Fourier expansion, That is, the inverse expansion is computed by the inner product for Boolean functions defined above. The cardinality of S is the degree of the coefficientf (p) (S). For instance, we have only one degree-0 coefficient,f (p) (∅), which equals to the expected value of f under the distribution p(z), Further, we have n degree-1 coefficientsf (p) (i) = f, φ i and so on. We examine Straight-Through gradient estimates using the framework of Harmonic Analysis of Boolean functions. For training the model parameters with a loss function L:= E p(z) [f (z)] we want to compute the gradient ∂ pi L = ∂ pi E p(z) [f (z)] for the i-th dimension. As the random sampling operation in the expectation is not differentiable, propose the Straight-Through estimator that approximates the gradient with. Clearly, the Straight-Through computes a biased approximation to the true gradient. Next, we quantify the bias in the Straight-Through estimator using the harmonic analysis of f (x). For the quantification of bias we first need the following lemma that connects the REINFORCE gradients with the degree-1 Fourier coefficients. The lemma is an extension of Margulis-Russo (; ;) formula. Lemma 1. Let f be a Boolean function. Then, the REINFORCE gradient estimates the degree 1 Fourier coefficientsf Proof. For compactness and clarity, we provide the proof in the Appendix B.1. We introduce new notation. In the following lemma bias ) denotes the bias of the i-th gradient estimate under the distribution p. Also, given the distribution p(z), p i→1/2 (z) is the distribution for which we set p(z i = +1) = p(z i = −1) = 1/2 for a given dimension i. Lemma 2. Let f be a Boolean function, where c k are the Taylor coefficients for the i-th dimension on f (z), that is z i, around 0 and bias Proof. For compactness and clarity, we provide only a proof sketch here showing the basic steps. These steps are also needed later in the description of the proposed gradient estimate algorithm. For the detailed proof please refer to the Appendix B.2. The proof sketch goes as follows. First, we derive a relation between the Fourier coefficients under the unknown distribution p(z) and under the uniform Bernoulli distribution p i→1/2 (z). Then, using this relation we derive the Taylor expansions for the true gradient as well as the Straight-Through gradient estimator. Last, to prove the lemma we compare the two Taylor expansions. Relation between Fourier coefficients under p(z) and p i→1/2 (z). If we expand the function f in terms of its φ S basis as in equation 2 and focus on the i-th dimension, by Lemma 1 we can show that the REINFORCE gradient is given by Taylor expansions of the true and the Straight-Through gradients. The Taylor expansion of Let's first focus on the true gradient. Since we work with Boolean ±1 values, we have that z k i = 1 for even k and z k i = z i for odd k. This will influence the even and the odd terms of the Taylor expansions. Specifically, for the Taylor expansion of the true gradient we can show that The expression in equation 7 implies that the true gradient with respect to the p i is the expected sum of the odd Taylor coefficients. Here we note that although the final expression in equation 7 can also be derived by a finite difference method, it does not make explicit, as in equation 31, the dependence on z i and µ i of the term inside the expectation. Now, let's focus on the Straight-Through gradient. Taking the derivative of the Taylor expansion w.r.t. to z i, we have The Straight-Through gradient is the expectation of equation 8 in the i-th dimension, that is where Comparing the Taylor expansions. By comparing the expansion of the Straight-Through gradient in equation 10 and the expansion of the true gradient in equation 7 and given that bias Taking the expectation in equation 9 under p i→1/2 causes the final term in equation 11 to vanish leaving bias Combining this expression with equation 9 gives the final expression (equation 4) from the lemma. Inspired by the Harmonic Analysis of the Straight-Through gradient estimates, we present a gradient estimate algorithm for deep Boolean latent models, FouST, for Fourier Straight-Through estimator. The algorithm relies on three bias reduction steps on the Straight-Through, lines 2, 3, 5 in Algorithm 1. As detailed earlier, the bias in the Straight-Through estimator is the sum of the bias under the uniform Bernoulli distribution plus extra bias due to non-zero expectation terms in higher-order harmonics when sampling from p(z). Sampling from p i→1/2 instead of p(z) would decrease the total bias from the form in equation 11 by setting the final term to 0. As a first bias reduction step, therefore, we do importance sampling. Specifically, after getting samples from p(z) and computing the gradients ∂ zi f (z) with the Straight-Through, we estimate the expectation under p i→1/2 as arrive at equation 12 in the context of unbiased control variates for quadratic functions. Lemma 2 shows that part of the bias in the Straight-Through estimator is due to the presence of extra factors in the Taylor coefficients. We can reduce the effect of these factors by taking advantage of the moments of the uniform distribution. Recalling that, we can attempt to correct the coefficients in equation 9, which for z k have the form (k + 1)c k, with the same extra factor of k + 1 that appears in the denominator of the kth moment. This suggests that we can sample from an auxiliary variable u and then use the auxiliary variable u with f instead of z and exploit the higher moments of the uniform distribution to reduce bias. For brevity, we illustrate the method with a case study of a two-dimensional z, and a bivariate f (z 1, z 2). As in Lemma 2, the partial true gradient of f (z 1, z 2) w.r.t. the first distribution parameter p 1 equals to "Bernoulli splitting uniform" trick. Assume an auxiliary variable u = (u 1, u 2), which we choose as follows. First, we sample z = (z 1, z 2) from the uniform Bernoulli distribution p 1→1/2 with (i set to 1). Then we take a uniform sample (u 1, u 2) with u i sampled from either The expectation of the gradient under such random sampling is Further detail is in Appendix C.1. We compare equation 14 with equation 13. In equation 14 we observe that the pure terms in z 1, namely terms with j = 0, always match those of the true gradient in equation 13. For j > 0 we obtain mixed terms with coefficients that do not match those of the true gradient in equation 13. Due to the 1 j+1 factor, for small j the mixed-degree terms are closer to the original ones in equation 13. For functions with small mixed degree terms, this can lead to bias reduction, at the cost of an increased variance because of sampling an auxiliary variable. In practice, to manage this bias-variance trade-off and to deal with functions that have greater dependence on mixed degree terms, we use smaller intervals for the random samples as in Algorithm 1. To summarize, for appropriate functions, the "Bernoulli splitting uniform" relies on the continuous variable u conditioned on the binary sample to reduce the bias. However, it is important to emphasize that u is only an auxiliary variable; the actual latent variable z is always binary. Thus, the "Bernoulli splitting uniform" trick does not lead to a relaxation of the sort used by Gumbel-Softmax , where there are no hard samples. Lastly we note that for a univariate f the "Bernoulli splitting uniform" trick leads to an unbiased estimator with an increased variance. The Fourier basis does not depend on the particular input representation and any two-valued set, say {−t, t} can be used as the Boolean representation. The choice of a representation, however, does affect the bias as we show next. As a concrete example, we let our input representation be n, where p i = p(z i = +1/2). While we can change the input representation like that, in general the Fourier coefficients in equation 3 will be different than for the {−1, +1} representation. We give the final forms of the gradients here. Details are given in Appendix C.2. Under the p i→1/2 distribution the degree-1 Fourier coefficients are: Note that compared to equation 7, in equation 15 we still get the odd terms c 1, c 3 albeit decayed by inverse powers of 2. Following the same process for the Straight-Through gradient as in equation 10, we have that While this is still biased, compared to equation 7 the bias is reduced by damping higher order terms by inverse powers of 2. The algorithm, described in algorithm 1, is a Straight-Through gradient estimator with the bias reduction steps described above, where a single sample is used to estimate the gradient. We emphasize that the algorithm uses a single sample and a single evaluation of the decoder per example and latent vector sample. Thus, the algorithm has the same complexity as that of the original Straight-Through estimator. Monte Carlo gradient estimators for training models with stochastic variables can be biased or unbiased. Perhaps the best known example of an unbiased gradient estimator is the REINFORCE algorithm . Unfortunately, REINFORCE gives gradients of high variance. For continuous stochastic variables propose the reparameterization trick, which transforms the random variable into a function of deterministic ones perturbed by a fixed noise source, yielding much lower variance gradient estimates. For discrete stochastic variables, REINFORCE is augmented with control variates for variance reduction. A number of control variate schemes have been proposed: NVIL subtracts two baselines (one constant and one input-dependent) from the objective to reduce variance. MuProp uses the first-order Taylor approximation of the function as a baseline. REBAR uses the Gumbel-Softmax trick to form a control variate for unbiased gradient estimates. RELAX generalizes REBAR to include an auxiliary network in the gradient expression and uses continuous relaxations and the reparameterization trick to give unbiased gradients. Regarding biased estimators, a simple choice is the Straight-Through estimator which uses the gradient relative to the sample as that relative to the probability parameter. Another recent approach is to use continuous relaxations of discrete random variables so that the reparameterization trick becomes applicable. The most common example of this being the GumbelSoftmax estimator . Although this is a continuous relaxation, it has been used to define the Gumbel Straight-Through estimator with hard samples. This uses arg max in the forward pass and the Gumbel-Softmax gradient is used as an approximation during in the backward pass. DARN , like MuProp, also uses the first-order Taylor expansion as a baseline but does not add the analytical expectation, making the estimator biased for non-quadratic functions. In this work we focus on biased Straight-Through gradient estimators. Specifically, we analyse how to reduce bias via Fourier expansions of Boolean functions. The Fourier expansion itself is widely used in computational learning theory with applications to learning low-degree functions , decision trees , constant-depth circuits and juntas . To the best of our knowledge we are the first to explore Fourier expansions for bias reduction of biased stochastic gradient estimators. Experimental Setup. We first validate FouST on a toy setup, where we already know the analytic expression of f (z). Next we validate FouST by training generative models using the variational autoencoder framework of. We optimize the single sample variational lower bound (ELBO) of the log-likelihood. We train variational autoencoders exclusively with Boolean latent variables on OMNIGLOT, CIFAR10, mini-ImageNet and MNIST (Appendix D.1). We train all models using a regular GPU with stochastic gradient descent with a momentum of 0.9 and a batch size of 128. We compare against Straight-Through, GumbelSoftmax, and DARN, although on more complex models some estimators diverge. The were consistent over multiple runs. Details regarding the architectures and hyperparameters used are in Appendix E. Upon acceptance we will open source all code, models, data and experiments., where t ∈ is a continuous target value and z is a sample from the Bernoulli distribution p(z). The optimum is obtained for p(z = +1) ∈ {0, 1}. Figure 1, shows a case with t = 0.45, where the minimizing solution is p(z = +1) = 0. We observe that unlike the Straight-Through estimator, FouST converges to the minimizing deterministic solution (lower is better). Training Stochastic MLPs. We train MLPs with one and two stochastic layers on OMNIGLOT, following the non-linear architecture of. Each stochastic Boolean layer is preceded by two deterministic layers of 200 tanh units. All hyperparameters remain fixed throughout the training. All estimators use one sample per example and a single decoder evaluation. We present in Fig. 2. FouST outperforms other biased gradient estimators in both datasets and architectures. FouST is clearly better than the StraightThrough estimator. Despite the complicated nature of the optimized neural network function f (z) the bias reduction appears fruitful. With one or two stochastic layers we can also use the unbiased REBAR. REBAR is not directly comparable to the estimators we study, since it uses multiple decoder evaluations and for models with multiple stochastic layers, multiple passes through later layers. Nevertheless, as shown in appendix D.1 for MNIST, with two stochastic layers REBAR reaches a worse test ELBO of -94.43 v. -91.94 for FouST. A possibility of worse test than training ELBOs for REBAR was also suggested in the original work . Training Stochastic ResNets. We further validate FouST in a setting where the encoder and decoder are stochastic ResNets, S-ResNets, which are standard ResNets with stochastic layers inserted between ResNet blocks. Similar to MLPs, FouST outperforms other biased estimators in this setting on CIFAR-10 (left in Figure 3). Note that despite the hyperparameter sweep, we were unable to train S-ResNet's with Gumbel-Softmax. So we compare against DARN and Straight-Through only. With an S-ResNet with 12 ResNet blocks and 4 stochatic layers FouST yields a score of 5.08 bits per dimension (bpd). This is comparable to the 5.14 bpd with the categorical VIMCO-trained model In the plots, we observe sharp spikes or slower curving. We hypothesize these are due, respectively, to stochasticity and bias, and are corrected to some degree along the way. Efficiency. We compare the efficiency of different estimators in Tab. 1. Like other biased estimators, FouST requires a single sample for estimating the gradients and has similar wallclock times. On MNIST, the unbiased REBAR is 15x and 40x slower than the biased estimators for two and five stochastic layer MLP's respectively. From the above experiments we conclude that FouST allows for efficient and effective training of fully connected and convolutional neural networks with Boolean stochastic variables. Last, we evaluate FouST on more complex neural networks with deeper and wider stochastic layers. We perform experiments with convolutional architectures on the larger scale and more realistic mini-ImageNet . As the scope of this work is not architecture search, we present two architectures inspired from residual networks of varying stochastic depth and width. The first one is a wide S-ResNet, S-ResNet-40-2-800, and has 40 deterministic (with encoder and decoder combined), 2 stochastic layers, and 800 channels for the last stochastic layer. The second, S-ResNet-80-11-256, is very deep with 80 deterministic and 11 stochastic layers, and a last stochastic layer with 256 channels. Architecture details are given in Appendix E.2. In this setup, training with existing unbiased estimators is intractable. We present in Fig. 3. We compare against DARN, since we were unable to train the models with Gumbel-Softmax. Incomplete lines indicate failure. We observe that FouST is able to achieve better training ELBO's in both cases. We conclude that FouST allows for scaling up the complexity of stochastic neural networks in terms of stochastic depth and width. For a Boolean function f the discrete derivative on the i-th latent dimension with a basis function φi is defined as The Fourier expansion of the discrete derivative equals The Fourier expansion of the discrete derivative is derived by equation 2: (i) all bases that do not contain the i-th dimension are constant to φi and thus set to zero, while (ii) for the rest of the terms ∂φ S dφ i = φ S\i from the definition of basis functions φS(z). In the following we differentiate partial derivatives on continuous functions noted with ∂· from discrete derivatives on Boolean functions noted with D·. The i-th discrete derivative is independent of zi both in the above definitions. Proof. We follow O'Donnell (2014, §8.4). In this proof we work with two representations of the Boolean function f. The first is the Fourier expansion of f under the uniform Bernoulli distribution. This is also the representation obtained by expressing f as a polynomial in zi. Since the domain of the function f is the Boolean cube, the polynomial representation is multilinear. That is f (z) = S⊆[n]f (S) j∈S zj. To avoid confusion and to differentiate the representation from the Boolean function, we use f (u) (z) to denote this representation in the following. Note that since this representation is a polynomial it is defined over any input in R n. In particular, The second representation we use is the Fourier expansion of the Boolean function f under p(x). We denote this by f (p). The following relation follows from the fact that when working with the Fourier representation, f (z) is multilinear, E p(z) [zi] = µi and the linearity of expectation. As the partial derivative of f (u) w.r.t. µi is equivalent to discrete derivative of, and keeping in mind that φi = (zi − µi)/σi, we have that We then note that the discrete derivative of f w.r.t. zi, Dz i f (u) (µ), from the left hand side of equation 19, is equivalent to the partial derivative of f w.r.t. µi, ∂µ i f (u) (µ). We complete the proof by noting that the right hand side in equation 24 is 1 2 times the REINFORCE gradient. The detailed proof of Lemma 2 is as follows. Proof. We first derive the Taylor expansions for the true gradient as well as the Straight-Through gradient estimator. Then, we prove the lemma by comparing the two Taylor expansions. By expanding the function f in terms of its φS basis as in equation 2 and focusing on the i-th dimension, we have thatf The first term,f (p) (i), is the term corresponding to {i} in the Fourier expansion of f under p i→1/2 (z). That isf This follows from the fact that when moving from p(z) to p i→1/2 (z), (i) we have that φi = zi, and (ii) no other term under the p(z) expansion contributes to the zi term under the p i→1/2 (z) expansion. As a consequence of Lemma 1 the REINFORCE gradient for the i-th dimension is given by Next, we will derive the Taylor expansions of the true and the Straight-Through gradients. The Taylor expansion of f (z) for zi around 0 is where are the Taylor coefficients. All c k are a function of zj, j = i. Let's first focus on the true gradient. Since we work with binary ±1 values, we have that z k i = 1 for even k and z k i = zi for odd k. This will influence the even and the odd terms of the Taylor expansions. Specifically, for the Taylor expansion of the true gradient we have from equation 27 and equation 3 that The expression in equation 32 implies that the true gradient with respect to the pi is the expected sum of the odd Taylor coefficients. Here we note that the although final expression in equation 32 can also be derived by a finite difference method, it does not make explicit, as in equation 31, the dependence on zi and µi of the term inside the expectation. By comparing the expansion of the Straight-Through gradient in equation 35 and the expansion of the true gradient in equation 32, Taking the expectation in equation 34 under p i→1/2 causes the final term in equation 37 to vanish leaving C LOW-BIAS GRADIENT ESTIMATES We describe the case of a bivariate function in detail. For brevity, we focus on a case study of a two-dimensional z, and a bivariate f (z1, z2) with the bivariate Taylor expansion f (z1, z2) = i,j ci,jz As in Lemma 2, the partial true gradient of f (z1, z2) w.r.t. the first distribution parameter p1 equals to Further, the Taylor expansion of "Bernoulli splitting uniform" trick. Assume an auxiliary variable u = (u1, u2), which we choose as follows. First, we sample z = (z1, z2) from the uniform Bernoulli distribution p 1→1/2. Then we take a uniform sample (u1, u2) with ui sampled from either for zi = +1 or from [−1, 0] if zi = −1. At this point it is important to note that the moments of the uniform distribution in [a, b], which simplifies to b/2, b 2 /3, b 3 /4,... for a = 0, and where we think of b as a binary sample i.e., b ∈ {−1, 1}. The expectation of the gradient under such random sampling is We then compare equation 40 with equation 39. In equation 40 we observe that the pure terms in z1, namely terms with j = 0, always match those of the true gradient in equation 39. For j > 0 we obtain mixed terms with coefficients that do not match those of the true gradient in equation 39. However, the partial gradient obtained with the auxiliary variables in equation 40 has coefficients following a decaying trend due to the 1 j+1. For small j, that is, the mixed-degree terms are closer to the original ones in equation 39. For functions with smaller mixed degree terms this leads to bias reduction, at the cost of an increased variance due to additional sampling. In practice many functions would have greater dependence on mixed degree terms. For such functions and to manage the bias-variance trade-off we choose smaller intervals for the uniform samples, that is a → b. The Fourier basis does not depend on the particular input representation and any two-valued set, say {−t, t} can be used as the Boolean representation. The choice of a representation, however, does affect the bias as we show next. As a concrete example, we let our input representation be zi ∈ {−1/2, 1/2} n, where pi = p(zi = +1/2). While we can change the input representation like that, in general the Fourier coefficients in equation 3 will be different than for the {−1, +1} representation. Letting h(zi) = 2zi ∈ {−1, 1}, the functions φi are now given Next, we write the Taylor series of f in terms of h(zi), f (z) = c0 + c1zi + c2z Under the p i→1/2 distribution, we still have that E p→1/2 [h(zi)] = 0 and the degree-1 Fourier coefficients are: Note that compared to equation 7, in equation 43 we still get the odd terms c1, c3 albeit decayed by inverse powers of 2. Following the same process like for equation 10, we have that To further judge the effect of our proposed modifications to Straight-Through, we performed ablation experiments where we separately applied scaling and noise to the importance-corrected Straight-Through. These experiments were performed on the single stochastic layer MNIST and OMNIGLOT models. The of the ablation experiments are shown in figure 5. From the figure it can be seen that scaling alone improves optimization in both cases and noise alone helps in the case of MNIST. Noise alone in a worse ELBO in the case of OMNIGLOT, but gives an improvement when combined with scaling. From these we conclude that the proposed modifications are effective. The encoder and decoder networks in this case are MLP's with one or more stochastic layers. Each stochastic layer is preceded by 2 deterministic layers with a tanh activation function. We chose learning rates from {1 × 10 −4, 2 × 10 −4, 4 × 10 −4, 6 × 10 −4}, Gumbel-Softmax temperatures from {0.1, 0.5}, and noise interval length for FouST from {0.1, 0.2}. For these dataset we use a stochastic variant or ResNets . Each network is composed of stacks of layers. Each layer has (i) one regular residual block as in , (ii) followed by at most one stochastic layer, except for the CIFAR architecture B in figure 3 where we used two stochastic layers in the last layer. The stacks are followed by a final stochastic layer in the encoder. We do downsampling at most once per stack. We used two layers per stack. For CIFAR we downsample twice so that the last stochastic layer has feature maps of size 8x8. We chose learning rate from {9 × 10 −7, 1 × 10 −6, 2 × 10 −6, 4 × 10 −6}, the FouST scaling parameter from {0.5, 0.8, 0.9}, and the uniform interval was scaled by a factor from {0.01, 0.05, 0.1} For mini-ImageNet we downsample thrice. We chose the learning rate from {2 × 10 −7, 3 × 10 −7, 4 × 10 −7, 5 × 10 −7}.

represent the tldr for finding the corresponding scientific article

E.g.:
Why and how to constrain planning domain model verification with planning goals to avoid unreachable counterexamples (false positives verification outcomes). == The verification of planning domain models is crucial to ensure the safety, integrity and correctness of planning-based automated systems. This task is usually performed using model checking techniques. However, directly applying model checkers to verify planning domain models can in false positives, i.e. counterexamples that are unreachable by a sound planner when using the domain under verification during a planning task. In this paper, we discuss the downside of unconstrained planning domain model verification. We then propose a fail-safe practice for designing planning domain models that can inherently guarantee the safety of the produced plans in case of undetected errors in domain models. In addition, we demonstrate how model checkers, as well as state trajectory constraints planning techniques, should be used to verify planning domain models so that unreachable counterexamples are not returned. Planning and task scheduling techniques are increasingly applied to real-world problems such as activity sequencing, constraint solving and resource management. These processes are implemented in planning-based automated systems which are already used in space missions BID14 BID3 BID0, search and rescue BID12, logistics BID19 and many other domains. Since the failure of such systems could have catastrophic consequences, these applications are regarded as safety-critical. Therefore, verification methods that are robust, trustworthy and systematic are crucial to gain confidence in the safety, integrity and correctness of these systems. The literature is rich with studies on verification of planning systems. For instance, BID17 carried out scenario-based testing and model-based validation of the remote agent that controlled the Deep Space 1 mission. Another example is the verification of the safety of the autonomous science agent design that was deployed on the Earth Orbiter 1 spacecraft.A typical planning system consists of a planning domain model, planning problem, planner, plan, executive, and mon-itor. Planners take as an input a domain model which describes application-specific states and actions, and a problem that specifies the goal and the initial state. From these inputs, a sequence of actions that can achieve the goal starting from the initial state is returned as plan. The plan is then executed by an executive to change the world state to match the desired goal. Our research focuses on the verification of planning domain models wrt. safety properties. Domain models provide the foundations for planning. They describe real-world actions by capturing their pre-conditions and effects. Due to modelling errors, a domain model might be inconsistent, incomplete, or inaccurate. This could cause the planner to fail in finding a plan or to generate unrealistic plans that will fail to execute in the real world. Moreover, erroneous domain models could lead planners to produce unsafe plans that, when executed, could cause catastrophic consequences in the real world. This paper addresses the fact that the state-of-the-art verification methods for planning domain models are vulnerable to false positive counterexamples. In particular, unconstrained verification tasks might return counterexamples that are unreachable by planners. Such counterexamples can mislead designers to unnecessarily restrict domain models, thereby potentially blocking valid and possibly necessary behaviours. In addition, false positive counterexamples can lead verification engineers to overlook counterexamples that are reachable by planners. To overcome these deficiencies, we propose to employ planning goals as constraints during verification. Thus, we introduce goal-constrained planning domain model verification, a novel concept that eliminates unreachable counterexamples per se. We formally prove that goal-constrained planning domain model verification of safety properties is guaranteed to return reachable counterexamples if and only if any exist. We also demonstrate two different ways to perform goal-constrained planning domain model verification, one using model checkers and the other using state trajectory constraints planning techniques. To the best of our knowledge, this work is the first to recommend fail-safe planning domain model design practice; introduce the concept of goal-constrained planning domain model verification. and demonstrate how model checkers, as well as state trajectory constraints planning techniques, can be used to perform goal-constrained planning domain model verification The rest of this paper is organised as follows. First, Section 2, contrasts the concepts presented here with related work. Second, Section 3 discusses the problem of unreachable counterexamples in planning domain model verification. Third, Section 4 proposes a design practice for planning domain models that can inherently guarantee domain model safety even in the case of undetected modelling errors. A verification concept of planning domain models that avoids returning unreachable counterexamples is presented in Section 5. Then, Section 6 discusses the implementation of this concept on the Cave Diving planning domain using Spin and MIPS-XXL. Finally, Section 7 concludes the paper and suggests future work. Closely related, but different, is the work by BID1. Their main objective is to treat verification as a planning task, whereas our aim is to demonstrate how model checkers and planners can be used for domain model verification. They proposed to perform system model verification using classical planners. To do this, they first translated the model of the system under verification into a planning domain model. Then, the negation of the safety property to be established, is used as the goal for the planner, which is then consulted to find a plan that acts as counterexample for the given property. In our study, because our aim is to verify domain models against a given property with respect to a specific goal and initial state, we used state trajectory constraints to restrict counterexamples to identify plans that can achieve the planning goal while falsifying the safety property. Unlike BID1, where the negation of the safety property is used as the goal, in our verification as planning method, the negation of the safety property is represented as state trajectory constraint and the goal is the given planning goal. BID16 ) also apply verification as planning to verify planning domain models, starting from LTL specifications. This work fundamentally differs from our work. BID16 focused their work on translating specification properties into trap formulas which can help in testing the impact of individual atomic propositions on the validity of the overall verified property. However, their method does not consider the interaction between property testing and the original planning goal. Note that finding a planning constraint to exercise a specific atomic proposition is not enough to ensure the constraint itself would be exercised during the planning process. For example, a planning goal might be achieved through a state trajectory that does not exercise the hard constraint used to represent the tested property. Our work is mainly based on investigating this interaction. Therefore, we used state trajectory constraints to guarantee the property is tested while achieving the planning goal. Additionally, their work, just like other similar methods, requires a complete planner to give deterministic , whereas our work, as discussed in Section 5, guarantees definite verification without this requirement.(Goldman, Kuter, and Schneider 2012) also used classical planners for planning systems verification, but they examined verifying plans rather than domain models. They proposed an approach that uses classical planners to find counterexamples for a given planning problem and plan instance. Their work and ours are related in that both suggest performing planning verification for a specific planning problem rather than attempting ungrounded verification of a planning system. However, their work is limited to the verification of single plan instances, whereas our method verifies all potential plans that can be spun from a domain model for a specific goal and initial state. Among others, BID15 BID13 BID18 BID10 BID2 ) used model checkers to verify planning domain models. They translated the respective domain models into the input language of the selected model checker. The model checker is then applied to verify the domain model wrt. a given specification property. Similarly, we also proposed a method to verify domain models using model checkers. However, our method differs from the others in two aspects. First, in the way we define the planning domain model verification problem, and, second, in the way we use model checkers to perform verification. As explained in Section 5, we consider the verification of planning domain models to be constrained by a specific goal and initial state pair. In contrast, previous studies perform ungrounded verification of domain models, i.e. leaving the goal and initial state open. As discussed in Section 3, the ungrounded goal and initial state may cause the model checker to return counterexamples that are unreachable when a planner uses the DUV. These unreachable counterexamples can mislead the designers to over-restrict the DUV during the debugging process. On the other side, when the goal and initial state are constrained for verification, then we have shown that the returned counterexamples, if any, are guaranteed to be reachable by any sound planner. The second difference is that, after the planning domain model is translated to the model checker's input language, we augment the model transitions, introducing the negation of the goal as a new constraint that forces the model checker to terminate once the goal is reached. This modification prevents the model checker from returning counterexamples that falsify the given property after satisfying the goal; these are unreachable by planners. Planning domain model verification aims to demonstrate that any produced plan satisfies a set of properties. To achieve this, formal planning domain model verification methods leave the planning goal open. This, we define as unconstrained verification of planning domain models, i.e. the verification is expected to hold for any potential goal. unconstrained verification searches the domain model for a sequence of actions that can falsify the given property, regardless of any other conditions. In particular, whether or not a planner would consider this sequence to be a plan, is not taken into account. This is a critical oversight, because, when the domain model is used to solve a specific planning problem, the sequence of actions that constitutes such a counterexample might, in fact, be "pruned away" by the planner, if it does not satisfy the planning goal. Therefore, for a specific planning problem, counterexamples that do not achieve the planning goal are deemed unreachable counterexamples from the planner's perspective. To illustrate this, we use a modified version of the microwave oven example, introduced in BID5, as presented in FIG0. A safety requirement would be that the domain model does not allow the generation of erroneous plans, in LTL p 0 = G(¬Error), where G is the LTL globally operator. Unconstrained verification will return StartOven as a counterexample that when applied to s 0 will produce s 2 which is an error state. However, when this model is used to find a plan that achieves the goal (g = Heat), this sequence will not be considered by the planner as it does not lead to a state that achieves the goal. Moreover, we observe that the valid plan CloseDoor, StartOven does satisfy the property p 0, i.e. is error-free. Thus, the sequence StartOven from s 0 to s 2 is an unreachable counterexample for the planner; it does not achieve the goal, nor is it part of a valid plan towards the goal. DISPLAYFORM0 Counterexamples that are unreachable by planners exist in the literature. For example, BID18 ) used the Spin model checker to verify whether a planning domain model would permit an automated planning system to select plans that would waste resources and therefore not meet the mission's science goals. To express this requirement, they used "five data-producing activities must be scheduled by any returned plan" as a property for model checking. The automated system has two data-producing and two data-consuming activities, and a buffer that can hold four data blocks. The goal of the planner is to schedule five data-producing activity instances. The counterexample returned by the model checker represented a plan with the two data-consuming activities scheduled before four dataproducing activities. This plan did not contain a fifth dataproducing task, because the data buffer was full after four data-producing activities and the only two data-consuming tasks that would have cleared the buffer, were scheduled at the beginning of the plan with no data in the buffer. Though the model checker found a counterexample to falsify the property, we argue that any sound planner would not generate such a plan, because it does not achieve the planning goal. As such, this counterexample would have been pruned during the planner's goal search, and consequently, it would never have been returned as a plan, i.e. it is unreachable for the planner, yet reachable by a goal-ignorant model checker. The problem with unreachable counterexamples is that they mislead the designer to unnecessarily restrict the domain model in the process of removing them. Consequently, debugging is made harder and genuine counterexamples could potentially be introduced in the process. To overcome this, we observe that planning is performed for a specific goal and initial state. To exploit this observation for domain model verification, we propose to use the goal and initial state given to the planner as constraints to ensure that the counterexamples returned by a model checker, or other tools used in this context, falsify the given property while also achieving the planning goal. Thus, instead of performing unconstrained domain model verification, we propose goal-constrained verification of planning domain models. The details of this method are further explained in Section 5. Next, we describe an inherently safe domain model design practice which can help to make domain models safer. The ultimate objective of planning domain model verification is to ensure that the plans produced by the verified domains satisfy a given specification. An alternative and guaranteed way of achieving this goal is to extract plan constraints from the specification, then include them in the domain model. A sound planner using this constrained domain model cannot produce any plan that could violate these constraints. This idea was first noticed in 2005 BID18 ) but was dismissed as it was not possible to describe overall plan constraints using PDDL 2.2. However, in BID7 BID7 proposed an extension to the PDDL 2.2 language that allows the expression of plan state trajectory constraints. The extended language, called PDDL3.0, was proposed for the fifth international planning competition (IPC-5). BID18 provided an example of a system consisting of a camera, solid-state recorder and a radio, and a requirement that for all plans, if an image is taken and stored, then it is eventually uplinked. With the hard state trajectory constraints, this property can be expressed as sometimeafter((image is taken and image is stored) image is uplinked). With this constraint, any sequence of actions that does not respect this property would not be returned as a plan. Though including specification properties in the domain model as strong constraints is enough to guarantee that sound planners using the constrained domain models will produce plans that meet the specification, this method will not be able to find any errors in the domain model. Instead, it will just ensure these errors, if any, are masked and prevented from affecting any plans that could possibly be generated using the modified domain model. As such, this method can be seen as a safety defence layer, a firewall, that prevents any potential property violation. Nevertheless, note that undetected bugs in a domain model could cause what would have been valid plans to be masked, thus unnecessarily restricting the planner. Therefore, further verification efforts are needed to reveal and rectify any underlying errors. We consider including plan constraints in the domain model to be a good practice to design inherently safe domain models. The effort of extracting formal properties from the specification and inserting them as constraints in the domain model is a small investment in return to the huge benefit of guaranteed safe plans, i.e. plans that are safe "by construction". This, together with our new concept of goalconstrained verification, as introduced in the next section, can deliver safe and error-free models.5 Goal-constrained verification of planning domain modelsPlanning domain model verification covers different objectives, including the domain's correctness, completeness, robustness, effectiveness and safety. The intent of safety verification in this context is to verify that any plan produced from the DUV will satisfy a given safety property. In other words, a domain is considered safe if the domain is guaranteed only to produce plans that satisfy the given safety property when used by a sound planner. This verification task can be performed using advanced search algorithms, such as model checkers or classical planners, to find a valid counterexample for the given safety property. We define a valid counterexample to be a sequence of actions that, firstly, can falsify the given safety property, secondly, can achieve the planning goal from the given initial state, and, thirdly, none of the sub-sequences of the counterexample can achieve the goal. Formally, this is defined as follows: Let the planning problem P be a tuple (D, s 0, g), where D is the domain model that describes the set of all available actions A, s 0 is the initial state and g is the desired goal. π is a solution to P, a plan, defined as a sequence of actions, where these actions are chosen from A. π = a 0, a 1,..., a n such that π |= g i.e. when π is applied to the initial state s 0 it yields a sequence of states S, S = s 0, s 1,..., s n where the last state s n satisfies the planning goal g, s n |= g. We say a plan π satisfies a property p, π |= p, if the sequence of states S, generated by the plan π, satisfies the property p, S |= p. Furthermore, as defined in (Ghallab, Nau, and Traverso 2004), we call a plan π a redundant plan, if π contains a subsequence, π, π | π, that achieves the goal g. Definition 1: A valid counterexample for a safety property, p, of a planning problem is a non-redundant plan, π, that falsifies the safety property, π |= p. Plans are required to be non-redundant in the definition of valid counterexamples to exclude any plans that are enriched with action sequences which are unnecessary to achieve the planning goal but required to falsify the given safety property. Such plans represent counterexamples that are unreachable by any sound planner when searching for a plan to achieve a given planning problem in a planning task. Such plans represent counterexamples that are unreachable by any sound planner. To ensure the returned counterexamples are valid, we constrain the verification problem with a goal and initial state, and we exclude any counterexample that is a redundant plan. More formally, the verification problem associated with planning task P is defined as the tuple V = (D, (s 0, g), p). Where p is a formal safety property extracted from a given specification and required to hold over all valid paths that achieve the goal g from the initial state s 0.In this section, we introduced and formally defined the concept of goal-constrained verification of planning domain models. In the following subsections, we demonstrate how this concept can be realized using model checkers and state trajectory constraints planning techniques. Model checkers verify safety properties by searching for counterexamples that falsify those properties. In the case of planning applications, any sequence of actions that does not achieve the given goal, will be pruned by any sound planner. Therefore, in the verification of planning problems, any counterexample that does not achieve the goal of the planning problem should be eliminated on the bases that this counterexample is unreachable by the planner. To force model checkers to only return valid counterexamples, the safety property is first negated and then joined with the planning goal in a conjunction. This conjunction is then negated and supplied to the model checker as an input property. The final property requires the model checker to find a counterexample that both, falsifies the safety property and satisfies the planning goal. Note that, unlike Def. 1, this permits sequences that falsify the property after satisfying the goal. However, once the goal is achieved, planners terminate the search, thus rendering such sequences unreachable. To eliminate these sequences, model transitions should be augmented with an additional guard, representing the negation of the goal, to restrict all transitions once the goal is achieved. With this modification, the model checker is forced to return counterexamples that falsify the safety property before achieving the goal, because once the goal is satisfied no further transitions will be permitted. For a verification problem V = (D, (s 0, g), p) we first translate the domain model D into the model checker's input language, obtaining the model M that incorporates the initial state s 0. Then, a model checker is applied to the verification problem V = (M, ¬F (g)) to establish that DISPLAYFORM0 where F is the LTL eventually operator. The model M is modified to M by augmenting the guards of all transitions with the negation of the goal condition. The model checker is then applied to the verification problem V = (M, p) where p is defined as follows: DISPLAYFORM1 There are two possible outcomes of the verification task V. If the model checker returns a counterexample, π, then: DISPLAYFORM2 From the definition of the LTL eventually operator F: DISPLAYFORM3 It follows that there is at least one sequence S that falsifies the property p, and there is a state s j in that sequence which satisfies the goal g, according to and. In addition to that, in the sequence S, p is guaranteed to be falsified before g is satisfied, due to the modification we introduced in the model M. Thus, the plan π is a valid counterexample for the original safety property p as per Def. 1. This proves that the DUV does not satisfy the safety property p with respect to the goal and initial state. The other potential outcome is that the model checker fails to find a counterexample, then for all plans, π: DISPLAYFORM4 It follows that π |= F (¬p) ∨ π |= F (g). Furthermore, from we know that ∃π. π |= F (g). Therefore for all plans, π: DISPLAYFORM5 That means p is always true for all possible plans. Which proves that the DUV satisfies the original property with respect to the goal and initial state. Domain models can be verified to only produce valid plans, in terms of satisfying given a property, for a specific goal and initial state pair using sound planners. This is achieved by consulting the planner over the DUV to produce a plan that can satisfy the goal and the negation of the property. If the domain model permits producing plans that, along with achieving the goal, contradict the safety property, then an unsafe plan can be found. Thus, the returned plan is a counterexample that demonstrates that the safety property does not hold. On the other hand, if the domain model does not permit the generation of plans that can satisfy the negation of the safety property while achieving the goal, then the planner will fail. Thus, the property holds in any plan produced for the given goal. A benefit of goal-constrained planning domain verification is, where a planner is used to perform the verification task, there is no need to for this planner to be complete, as long as the planner used for the verification is also the planner that will be used during the planning task. This is because any counterexample not found by that planner during verification, will then also not be reached by the same planner during the planning task. The following subsection provides a description of how state trajectory constraints can be used to verify planning domain models for a specific goal and initial state. Goal-constrained planning domain verification using planning techniques with state trajectory constraints The PDDL3.0 state trajectory constraints, first mentioned in Section 4, can be used to perform planning domain model verification. First, the negation of the given property is expressed using PDDL3.0 modal operators and embedded in the original domain model as state trajectory constraint. The modified model is then used by a planner, as described earlier, to perform the verification. For a verification problem V = (D, (s 0, g), p), we first apply a planner to the planning problem P = (D, s 0, g) to establish that there is a plan that solves P ∃π. π |= g. Then, the safety property p is negated and expressed in terms of PDDL3.0 modal operators as shown in BID8 ). The is added as state trajectory constraint to the original domain model. Using the algorithm proposed in (Edelkamp, Jabbar, and Nazih 2006), the new model is transformed into a PDDL2 compatible version. This is performed by first translating the state trajectory constraint into a non-deterministic finite state automaton (NFA) which can monitor property violations by inserting additional predicates and actions conditional effects into the model to simulate and observe the behaviour of the automaton that represents the constraint. This yields a new planning problem P = (D, s 0, g), where D, s 0, g are modified instances of D, s 0, g that are supplemented with the additional predicates and actions conditional effects of the automaton that represents the introduced constraint. Then, a planner is applied to P with two possible outcomes. If the planner finds a plan then: DISPLAYFORM0 Since the satisfaction of g implies both, the satisfaction of the original goal g at the last state of the sequence S, and the satisfaction of the state trajectory constraint by the sequence S, can be rewritten as DISPLAYFORM1 Furthermore, from it follows that π |= p, confirming that there is at least one plan that achieves the goal while not respecting the safety property. Therefore, this plan is a valid counterexample for that property as per Def. 1. Hence, the DUV does not satisfy the property wrt. the planning goal and initial state. Alternatively, if the planner fails to find a plan, then, as opposed to FORMULA1, we have DISPLAYFORM2 Given FORMULA1, FORMULA1 can be simplified to: DISPLAYFORM3 Hence, all plans satisfy the safety property. Therefore, the property holds for the planning domain model wrt. the given goal and initial state. In this section, we discuss how goal-constrained planning domain verification can verify safety properties using both the Spin model checker BID11 ) and the MIPS-XXL planner BID6. We perform constrained and unconstrained verification tasks to show how unlike the latter task our method does not return unreachable counterexamples. As an example, we consider the classical cave diving planning domain taken from the Satisfying Track of the IPC-2014 (IPC2014 2014 . The problem consists of an underwater cave system with a finite number of partially interconnected locations. Divers can enter the cave from a specific location, entrance, and swim from one location to an adjacent connected one. They can hold up to four oxygen tanks and they consume one for every swim and take-photo action. Only one diver can be in the cave at a time. Finally, divers have to perform a decompression manoeuvre to go to the surface and this can be done only at the entrance. Additionally, divers can drop tanks in or take tanks from any location if they hold at least one tank or there is one tank available at the location respectively. The planning goals of this domain, as provided in the problem files in the IPC-2014, consist of two parts. The first part dictates the required underwater location of which the photo is to be taken (we call it mission target) and the second part which mandates the divers should return to the surface after completing the mission (we call it safety target).A critical safety property is that divers should not drown i.e. they should not be in an underwater location, other than the entrance, where neither the divers nor the location has one full oxygen tank at least. To enable the planner and the model checkers to explore the entire state space, we simplified this domain by ignoring the "precludes" condition from the original domain as it does not affect the verification of the drowning safety property. Consequently, we considered only one diver and we modified some actions to enable the diver to go back into the water after a dive. These modifications are further explained in the commented simplified planning domain model PDDL file which is provided along with the tasks problem PDDL and Promela files online [address hidden for blind review].First, we translated the planning domain model from PDDL to Promela. Thus, the verification using the translated model only hold provided that the translation is valid. The verification of the translation is outside the scope and focus of this paper and left for future work. In this example, the chosen planning goal is to have a photo of the first location, L 1, and to get the diver outside the water. The verification tasks are:1 -Unconstrained verification with only the safety property: Both Spin and MIPS-XXL found a counterexample prepare-tank, enter-water, swim(L 0, L 1). Indeed, this counterexample leads the diver to a drowning state. At the end of this sequence, the diver will be in underwater location L 1 which is not the entrance so they can not surface and with no oxygen tank to swim back to the entrance. However, this is not a plan because it does not achieve any useful goal. Therefore, it will not be produced by any sound planner when it is used in a practical scenario (taking a photo of any location).2-Verification with safety property and incomplete goal (mission target only): Both Spin and MIPS-XXL returned prepare-tank, prepare-tank, enter-water, swim(L 0, L 1), take-photo. This counterexample achieves the goal and violates the property. However, without the safety part of the goal, it would be possible to generate plans that imply divers should swim to an underwater location and take a photo of it without requiring the divers to return to the surface. These kind of plans are illegal as they do not respect the safety part of the goal. Therefore, such sequences are unreachable counterexamples i.e. will never be produced by any sound planner while planning for a legal goal.3-Verification using Spin with both safety property and proper goal but without the augmented model M: Spin found a counterexample prepare-tank, prepare-tank, prepare-tank, prepare-tank, enter-water, swim(L 0, L 1), take-photo, swim(L 1, L 0), decompress, enter water, swim(L 0, L 1). This counterexample achieves the goal and violates the safety property but only after the goal is achieved. Therefore, this is also an unreachable counterexample because a sound planner will terminate after achieving the goal and any counterexample that violates the property after achieving the goal will not be returned. Hence, it is unreachable. 4-Goal-constrained planning domain verification, as presented in this paper, the was: No plan is returned by the planner MIPS-XXL with complete exploration and no counterexample is returned by Spin with exhaustive verification mode. This means the planning domain model has no provision of producing a plan that can violate the safety property before achieving the goal. I.e. this model is safe with respect to the given property and goal pair. Though the counterexamples returned by the incomplete verification tasks number one, two and three are obviously unreachable and should not misguide the designers to overcomplicate the model, in a real world sized application such unreachable counterexamples can be critical and much more diffcult to recognise and avoid. We expect that our proposed concept can save practitioners a huge amount of personhours trying to alter planning domain models for behaviours that their planners will never experience in practice. The verification of planning domain models is essential to guarantee the safety of planning-based automated systems. Unreachable counterexamples returned by unconstrained planning domain model verification techniques undermine the verification . In this paper, we have discussed the potential deficiencies of this problem and provided an example of an unreachable counterexample form the literature. We then introduced goal-constrained verification, a new concept to address this problem, which restricts the verification task to a specific goal and initial state pair. This limits counterexamples to those practically reachable by a planner that is tasked with achieving the goal given the initial state. Consequently, our method verifies the domain model only wrt. a specific goal and initial state. This is an acceptable limitation, given that planners also operate on this basis. We have demonstrated how model checkers and planning techniques can be used to perform goal-constrained planning domain model verification. In addition, we have recommended an inherently safe practice for domain model design that guarantees the safety of domain models "by construction" in case of undetected modelling errors. Goalconstrained domain model verification ensures accurate verification and complements the inherently safe domain model design practice to generate safe and error-free planning domain models. In , the main message of this paper is that the direct application of verification algorithms to the planning domain model verification problem can return counterexamples that would never be reached by planners in real planning tasks. These unreachable counterexamples can mislead the designers to perform unnecessary remediations that can be prone to errors. The proposed solution is simple which makes it readily usable in practice. It is also effective as formally proven in the paper. Currently, we are investigating the use of Temporally Extended Goals (TEGs) translators BID20 to perform goal-constrained domain model verification. As future work, we intend to automate the proposed methods, so that they can be applied to real-world sized planning domain models. Finally, we would like to perform an empirical comparison of the proposed methods to assess their scalability and performance.

We present a low-bias estimator for Boolean stochastic variable models with many stochastic layers.

Represent this text to find a much much longer article about it with say hundreds-thousands of words:

This paper 1) characterizes functions representable by ReLU DNNs, 2) formally studies the benefit of depth in such architectures,  3) gives an algorithm to implement empirical risk minimization to global optimality for two layer ReLU nets.


Question: We propose a novel generative adversarial network for visual attributes manipulation (ManiGAN), which is able to semantically modify the visual attributes of given images using natural language descriptions. The key to our method is to design a novel co-attention module to combine text and image information rather than simply concatenating two features along the channel direction. Also, a detail correction module is proposed to rectify mismatched attributes of the synthetic image, and to reconstruct text-unrelated contents. Finally, we propose a new metric for evaluating manipulation , in terms of both the generation of text-related attributes and the reconstruction of text-unrelated contents. Extensive experiments on benchmark datasets demonstrate the advantages of our proposed method, regarding the effectiveness of image manipulation and the capability of generating high-quality . Image manipulation refers to the task of changing various aspects of given images from low-level colour or texture ) to high-level semantics , and has numerous potential applications in video games, image editing, and computer-aided design. Recently, with the development of deep learning and generative models, automatic image manipulation becomes possible, including image inpainting , image colourisation, style transfer , and domain or attribute translation (; . However, all the above works mainly focus on specific tasks, and only few studies concentrate on more general and user-friendly image manipulation by using natural language descriptions. Also, as shown in Fig.1, current state-of-the-art methods can only generate low-quality images and fail to effectively manipulate given images on more complicated datasets, such as COCO . The less effective performance is mainly because simply concatenating text and image cross-domain features along the channel direction, the model may fail to precisely correlate words and corresponding visual attributes, and thus cannot modified specific attributes required in the text, and conditioned only on a global sentence vector, current state-of-the-art methods lack important fine-grained information at the word-level, which prevents an effective manipulation using natural language descriptions. In this paper, we aim to manipulate given images using natural language descriptions. In particular, we focus on modifying visual attributes (e.g., category, texture, colour, and ) of input images by providing texts that describe desired attributes. To achieve this, we propose a novel generative adversarial network for visual attributes manipulation (ManiGAN), which allows to effectively manipulate given images using natural language descriptions and to produce high-quality . The contribution of our proposed method is fourfold: instead of simply concatenating hidden features generated from a natural language description and image features encoded from the input image along the channel direction, we propose a novel co-attention module where both features can collaborate to reconstruct the input image and also keep the synthetic semantically aligned with the given text description, a detail correction module (DCM) is introduced to rectify mismatched attributes, and to reconstruct text-unrelated contents existing in the input image, a new metric is proposed, which can appropriately reflect the generation of text-related visual attributes and the reconstruction of text-unrelated contents involved in the image manipulation, and extensive experiments on the CUB and COCO Figure 1: Examples of image manipulation using natural language descriptions. Current state-of-theart methods only generate low-quality images, and fail to do manipulation on COCO. In contrast, our method allows the input images to be manipulated accurately corresponding to the given text descriptions while preserving text-unrelated contents. to demonstrate the superiority of our model, which outperforms existing state-of-the-art methods both qualitatively and quantitatively. There are few studies focusing on image manipulation using natural language descriptions. proposed a GAN-based encoder-decoder architecture to disentangle the semantics of both input images and text descriptions. implemented a similar architecture, but introduced a text-adaptive discriminator that can provide specific word-level training feedback to the generator. However, both methods are limited in performance due to a less effective text-image concatenation method and a coarse sentence condition. Our work is also related to conditional image manipulation. introduced a VAE-GAN hybridisation model to modify natural images by exploring the latent features. and introduced paired and unpaired image-to-image translation methods based on conditional adversarial networks, respectively. However, all these methods focus mainly on image-to-image same-domain translation instead of image manipulation using cross-domain text descriptions. Recently, text-to-image generation has drawn much attention due to the success of GANs in generating photo-realistic images. first proposed to use conditional GANs to generate plausible images from given text descriptions. stacked multiple GANs to generate high-resolution images from coarse-to fine-scale. implemented a spatial attention mechanism to explore the fine-grained information at the word-level. However, all aforementioned methods mainly focus on generating new photo-realistic images from texts, and not on manipulating specific visual attributes of given images using natural language descriptions. Let I denote an input image required to be modified, and S denote a text description given by a user. We aim to semantically manipulate the input image I using the given text S, and also keep the visual attributes of the modified image I semantically aligned with S while preserving textunrelated contents existing in I. To achieve this, we first adopt the ControlGAN , as our basic framework, as it can effectively control text-to-image generation, and manipulate visual attributes of synthetic images. Then, we propose two novel components: co-attention module, and detail correction module to achieve effective image manipulation. We elaborate our model as follow, and the full architecture diagram is shown in Appendix A. As shown in Fig. 2 (a), our co-attention module takes two inputs: the hidden features h ∈ R C×H×D, where C is the number of channels, H and D are the height and width of the feature map, respectively, and the regional image features v ∈ R 256×17×17 of the input image I encoded by the GoogleNet . The activation value h ∈ R C×H×D is given by h = h W (v) + b(v), where W (v) and b(v) are the learned weights and biases dependent on the regional features v, and denotes Hadamard element-wise product. We use W and b to represent the functions that convert the regional features v to scaling and bias values. Then, the activation value h serves as the input for the next stage. We also apply the co-attention module before implementing an image generation network to produce synthetic images; please see Appendix A for more details. This linear combination form has been widely used in normalisation techniques (; ; ;), but, different from them, our co-attention module is only applied at specific positions instead of all normalisation layers, which requires less computational resources, and our co-attention module is designed to incorporate text and image cross-domain information, where W helps the model to focus on text-related visual attributes, while b provides input image information to help to reconstruct text-unrelated contents. Also, we experimentally find that implementing our co-attention module at all normalisation layers fails to produce reasonable images, which indicates that the normalisation techniques may not be suitable for the tasks requiring different domain information. , the functions W and b are implemented by a simple two-layer convolutional network, see Fig. 2 What has been learned by the co-attention module? To better understand what has been learned by our co-attention module, we conduct an ablation study shown in Fig. 3 to evaluate the effectiveness of W and b. As we can see, without W, some visual attributes cannot be perfectly generated (e.g., white belly in row 1 and the red head in row 2), and without b, the text-unrelated contents (e.g., ) are hard to preserve, which verify our assumption that W behaves as an attention function to help the model focus on text-related visual attributes, and b helps to complete missing text-unrelated details existing in the input image. Also, the visualisation of the channel feature maps of W (v) shown in the last three columns of Fig. 3 validates the attention mechanism of W. The main purpose of our model is to incorporate input images and then generate modified images aligned with given text descriptions. Then, it may inevitably produce some new visual attributes or mismatched contents that are not required in the given texts. To fix this issue, we propose a The bird has a black bill, a red crown, and a white belly. (top) This bird has wings that are black, and has a red belly and a red head. detail correction module (DCM) to rectify inappropriate attributes, and to reconstruct text-unrelated contents existing in the input images. The DCM consists of a generator and a discriminator, and is trained alternatively by minimising both objective functions. The generator, shown in Fig. 2 (b), takes three inputs: the last hidden features h last ∈ R C ×H ×D from the main module (we call our model without the DCM as main module), the word features, and visual features v ∈ R 128×128×128 that are extracted from the input image I by the VGG-16 pretrained on ImageNet . We have also applied GoogleNet and ResNet for feature extraction, but both do not perform well. Please refer to Appendix D for a detailed description of the detail correction module. We train the main module and detail correction module separately, and the generator and discriminator in both modules are trained alternatively by minimising both the generator loss L G and discriminator loss L D. Generator objective. The loss function for the generator follows those used in ControlGAN , but we introduce a regularisation term L reg = 1 − 1 C I H I W I ||I − I|| to prevent the network achieving identity mapping, which can penalise large perturbations when the generated image becomes the same as the input image. where the unconditional adversarial loss makes the synthetic image I indistinguishable from the real image I, the conditional adversarial loss aligns the generated image I with the given text description S, L DAMSM measures the text-image similarity at the word-level to provide finegrained feedback for image generation, L corre determines whether words-related visual attributes exist in the image, and L rec reduces randomness involved in the generation process. λ 1, λ 2, λ 3, and λ 4 are hyperparameters controlling the importance of additional losses. Note that we do not use L rec when we train the detail correction module. Discriminator objective. The loss function for the discriminator follows those used in Control-GAN , and the function used to train the discriminator in the detail correction module is the same as the one used in the last stage of the main module. conditional adversarial loss where S is a given text description randomly sampled from the dataset, the unconditional adversarial loss determines whether the given image is real, and the conditional adversarial loss reflects the semantic similarity between images and texts. Analysis. To prevent the model picking the input image as the solution, i.e., the model becomes an identity mapping network, we first introduce a regularisation term L reg to penalise large perturbations when the generated image becomes the same as the input image, and then we stop the training early when the model reaches a stage achieving the best trade-off between the generation of new visual attributes aligned with given text descriptions and the reconstruction of text-unrelated contents existing in the input images. As for when to stop training, it is based on our proposed measurement metric, called manipulative precision (see Fig. 4), which is discussed in Sec. 4. To evaluate our model, extensive quantitative and qualitative experiments are carried out. Two stateof-the-art approaches on image manipulation using natural language descriptions, SISGAN and TAGAN , are compared on the CUB birds and more complicated COCO datasets. Results for these two baselines are reproduced based on the code released by the authors. Please refer to Appendix A, B, and C for a detailed description of our network structures, the datasets, and training configurations. Quantitative . As mentioned above, our model can generate high-quality images compared with state-of-the-art methods. To demonstrate this, we adopt the inceptions score (IS) as the quantitative evaluation measure. In our experiments, we evaluate the IS on a large number of manipulated samples generated from mismatched pairs, i.e., randomly chosen input images manipulated by randomly selected text descriptions. However, as the IS cannot reflect the quality of the content preservation, the L 1 pixel difference (diff) is calculated between the input images and corresponding modified images. Moreover, using the pixel difference alone may falsely report a good reconstruction due to over-training that the model becomes an identity mapping network. To address this issue, we propose a new measurement metric, called manipulative precision (MP), incorporating both the text-image similarity (sim) and the pixel difference, where the text-image similarity is calculated by performing the cosine similarity on the text features and corresponding image features encoded from the modified images. This is based on the intuition that if the manipulated images are generated from an identity mapping network, then the text-image similarity should be low, as the synthetic images cannot perfectly keep a semantic consistence with given text descriptions. Thus, the measurement metric is defined as MP = (1 − diff) × sim. As shown in Table 1, our method has the highest MP values on both the CUB and COCO datasets compared with the state-of-the-art approaches, which demonstrates that our method can better generate text-related visual attributes, and also reconstruct text-unrelated contents existing in the input images. The model without main module (i.e., only having the DCM) gets the highest IS, the lowest L 1 pixel difference, and low text-image similarity. This is because the model has become a identity mapping network and loses the capability of image manipulation. Qualitative . Figs. 5 and 6 show the visual comparison between our ManiGAN, SISGAN , and TAGAN on the CUB and COCO datasets, respectively. It can be seen that both state-of-the-art methods are only able to produce low-quality and cannot effectively manipulate input images on the COCO dataset. However, our method is capable to perform an accurate manipulation and keep a highly semantic consistence between synthetic images and given text descriptions, while preserving text-unrelated contents. For example, shown in the last column of Fig. 6, SISGAN and TAGAN both fail to achieve an effective manipulation, while our model modifies the green grass to dry grass and also maps the cow into a sheep. Note that as birds can have many detailed descriptions (e.g., colour for different parts), we use a long sentence to manipulate them, while the text descriptions for COCO are more abstract and focus mainly on categories, thus we use words to do manipulation for simplicity, which has the same effect as using long detailed text descriptions. The effectiveness of the co-attention module. To verify the effectiveness of the co-attention module, we use the concatenation method to replace all co-attention modules, which concatenates hidden features h and regional features v along the channel direction, shown in Figs. 7 and 8 (d). As we can see that our full model can synthesise an object having exactly the same shape, pose, and position as the one existing in the input image, and also generate new visual attributes aligned with the given text description on the synthetic image. In contrast, as shown in the last two columns of Figs. 7 and 8 (d), with concatenation method, the model cannot reconstruct birds on the CUB bird dataset, and fails to do manipulation on the COCO dataset. This bird is yellow with a yellow belly, and has a yellow beak. A small bird with a red belly, a red crown, and black wings. This bird has wings that are brown, and has an orange belly and an orange breast. A bird that has a red beak, a grey head, and a grey belly. Also, to further validate the effectiveness of the co-attention module, we conduct an ablation study shown in Fig. 8 (c). It can be seen that our model without co-attention module that we just concatenate text and image features before feeding into the main module, which is used in and , fails to produce reasonable images on both datasets. In contrast, our full model can better generate text-required attributes and also reconstruct text-unrelated contents shown in the last column. Table 1 also verifies the effectiveness of our co-attention module, as the values of IS and MP increase significantly when we implement the co-attention module. The effectiveness of the detail correction module and main module. As shown in Fig. 8 (f), our model without detail correction module may miss some visual attributes (e.g., the bird missing the tail at row 2, the zebra missing the mouth at row 3), or generate new contents (e.g., new at row 1, different appearance of bus at row 4), which indicates that the detail correction module can correct inappropriate attributes and reconstruct the text-unrelated contents. Fig. 8 (e) shows that without the main module, our model fails to do image manipulation on both datasets, which just achieves an identity mapping. This is mainly because the model cannot precisely correlate words with corresponding visual attributes, which mostly has been done in the main module. A bird with black eye rings and a black bill, with a yellow crown and white belly. (matched) This bird has a yellow bill, a blue head, blue wings, and yellow belly. This beautiful bird is made up random patches of red, white, black, orange, and brown. (matched) A bird is brown and white in colour, with a grey belly and short orange bill. (given) Text Original Ours, Matched Ours, Given Concat., Matched Concat., Given Figure 7: Analysis of the co-attention module. "Matched" represents the texts matching original images. "Given" represents the texts provided by users. "Concat." denotes that instead of using co-attention, hidden features are concatenated with image features along the channel direction. This bird has a light grey belly, dark grey wings and head with a red beak. This bird has a yellow crown, blue wings and a yellow belly. removing the co-attention module and only concatenating image features and text features before feeding into the main module; d: using concatenation method to replace all co-attention modules; e: removing the main module and just training the DCM only; f: removing the DCM and just training the main module only; g: our full model. We have proposed a novel generative adversarial network for visual attributes manipulation, called ManiGAN, which can semantically manipulate the input images using natural language descriptions. Two novel components are proposed in our model: the co-attention module enables cooperation between hidden features and image features where both features can collaborate to reconstruct the input image and also keep the synthetic semantically aligned with the given text description, and the detail correction module can rectify mismatched visual attributes of the synthetic , and also reconstruct text-unrelated contents existing in the input image. Extensive experimental demonstrate the superiority of our proposed method, in terms of both the effectiveness of image manipulation and the capability of generating high-quality . We adopt the ControlGAN as the basic framework and replace batch normalisation with instance normalisation everywhere in the generator network except in the first stage. Basically, the co-attention module can be inserted anywhere in the generator, but we experimentally find that it is best to incorporate the module before upsampling blocks and image generation networks; see Fig. 9. Our method is evaluated on the CUB birds and the MS COCO datasets. The CUB dataset contains 8,855 training images and 2,933 test images, and each image has 10 corresponding text descriptions. As for the COCO dataset, it contains 82,783 training images and 40,504 validation images, and each image has 5 corresponding text descriptions. We preprocess this two datasets based on the methods introduced in. In our setting, we train the detail correction module (DCM) separately from the main module. Once the main module has converged, we train the DCM subsequently and set the main module as the eval mode. There are three stages in the main module, and each stage contains a generator and a discriminator. We train three stages at the same time, and three different-scale images 64×64, 128× 128, 256 × 256 are generated progressively. The main module is trained for 600 epochs on the CUB dataset and 120 epochs on the COCO dataset using the Adam optimiser with the learning rate 0.0002, and β 1 = 0.5, β 2 = 0.999. We do not use any learning rate decay, but for visualising generator output at any given point during the training, we use an exponential running average for the weights of the generator with decay 0.999. As for the DCM, there is a trade-off between generation of text-related attributes and the reconstruction of text-unrelated contents. Based on the manipulative precision (MP) values (see Fig. 4), we find that training 100 epochs for the CUB, and 12 epochs for the COCO to achieve an appropriate balance between generation and reconstruction. The other training setting are the same as in the main module. The hyperparameters λ 1, λ 2, λ 3, and λ 4 are set to 1, 5, 0.5, and 1 for the CUB dataset, and 15, 5, 0.5, and 1 for COCO, respectively. First, the visual features v are converted into the same size as the hidden features h last via a convolutional layer F, denotedṽ = F v, whereṽ ∈ R 128×H ×D. Then, we adopt the spatial attention and channel-wise attention introduced in to generate spatial attentive word-context features s ∈ R C ×H ×D and channel-wise attentive word-context features c ∈ R C ×H ×D, and concatenate these two features with the hidden features h last along the channel direction to generate new hidden features a ∈ R (3 * C)×H ×D. Next, to incorporate the visual featuresṽ, we adopt the co-attention module here, donatedã = a W (ṽ) + b (ṽ), where W and b are learned weights and bias dependent on visual featuresṽ. Then, the transformed featuresã are fed into a series of residual blocks followed by a convolutional layer to generate hidden features e. Before feeding e into a network to generate the output image, we apply the co-attention module on the e again to further strengthen the visual information; see Fig. 2 (b). We also track the trend of manipulation over epoch increases, as shown in Fig. 10. The image is smoothly modified to achieve the best balance between the generation of new visual attributes (e.g., dirt ) and the reconstruction of text-unrelated contents (e.g., the appearance of zebras). However, when the epoch goes larger, the generated visual attributes (e.g., dirt ) aligned with the given text description are erased, and the synthetic image becomes more and more similar to the input image. This verifies the existence of the trade-off between the generation of new visual attributes required in the given text description and the reconstruction of contents existing in the input image. We show additional comparison between our ManiGAN, SISGAN , and TAGAN on the CUB and COCO datasets. This bird is blue and grey with a red belly. This bird has wings that are grey and yellow with a yellow belly. This bird is black in colour, with a red crown and a red beak. This green bird has a black crown and a green belly. A bird with brown wings and a yellow body, with a yellow head. A white bird with grey wings and a red bill, with a white belly. Original SISGAN TAGAN Ours Figure 11: Additional between ManiGAN, SISGAN, and TAGAN on the CUB bird dataset. A small blue bird with an orange crown, with a grey belly. This bird has a red head, black eye rings, and a yellow belly. This bird is mostly red with a black beak, and a black tail. This tiny bird is blue and has a red bill and a red belly. This bird has a white head, a yellow bill, and a yellow belly. A white bird with red throat, black eye rings, and grey wings.

Represent the short summary for finding the corresponding scientific article (long)

We propose a novel method to manipulate given images using natural language descriptions.

Conditional VAE on top of latent spaces of pre-trained generative models that enables  transfer between drastically different domains while preserving locality and semantic alignment.