Question: Represent the article for finding a single-sentence TLDR

Question: While neural networks can be trained to map from one specific dataset to another, they usually do not learn a generalized transformation that can extrapolate accurately outside the space of training. For instance, a generative adversarial network (GAN) exclusively trained to transform images of cars from light to dark might not have the same effect on images of horses. This is because neural networks are good at generation within the manifold of the data that they are trained on. However, generating new samples outside of the manifold or extrapolating "out-of-sample" is a much harder problem that has been less well studied. To address this, we introduce a technique called neuron editing that learns how neurons encode an edit for a particular transformation in a latent space. We use an autoencoder to decompose the variation within the dataset into activations of different neurons and generate transformed data by defining an editing transformation on those neurons. By performing the transformation in a latent trained space, we encode fairly complex and non-linear transformations to the data with much simpler distribution shifts to the neuron's activations. We showcase our technique on image domain/style transfer and two biological applications: removal of batch artifacts representing unwanted noise and modeling the effect of drug treatments to predict synergy between drugs. Many experiments in biology are conducted to study the effect of a treatment or a condition on a set of samples. For example, the samples can be groups of cells and the treatment can be the administration of a drug. However, experiments and clinical trials are often performed on only a small subset of samples from the entire population. Usually, it is assumed that the effects generalize in a context-independent manner without mathematically attempting to model the effect and potential interactions with the context. However, mathematically modeling the effect and potential interactions with information would give us a powerful tool that would allow us to assess how the treatment would generalize beyond the samples measured. We propose a neural network-based method for learning a general edit function corresponding to treatment in the biological setting. While neural networks offer the power and flexibility to learn complicated ways of transforming data from one distribution to another, they are often overfit to the training dataset in the sense that they only learn how to map one specific data manifold to another, and not a general edit function. Indeed, popular neural network architectures like GANs pose the problem as one of learning to generate the post-treatment data distributions from pre-treatment data distributions. Instead, we reframe the problem as that of learning an edit function between the preand post-treatment versions of the data, that could be applied to other datasets. We propose to learn such an edit, which we term neuron editing, in the latent space of an autoencoder neural network with non-linear activations. First we train an autoencoder on the entire population of data which we are interested in transforming. This includes all of the pre-treatment samples and the post-treatment samples from the subset of the data on which we have post-treatment measurements. The internal layers of this autoencoder represent the data with all existing variation decomposed into abstract features (neurons) that allow the network to reconstruct the data accurately BID28 BID4 BID17 BID24. Neuron editing involves extracting differences between the observed pre-and post-treatment activation distributions for neurons in this layer and then applying them to pre-treatment data from the rest of the population to synthetically generate post-treatment data. Thus performing the edit node-by-node in this space actually encodes complex multivariate edits in the ambient space, performed on denoised and meaningful features, owing to the fact that these features themselves are complex non-linear combinations of the input features. While neuron editing is a general technique that could be applied to the latent space of any neural network, even GANs themselves, we instead focus exclusively on the autoencoder in this work to leverage three of its key advantages. First, we seek to model complex distribution-to-distribution transformations between large samples in high-dimensional space. While this can be generally intractable due to difficulty in estimating joint probability distributions, research has provided evidence that working in a lower-dimensional manifold facilitates learning transformations that would otherwise be infeasible in the original ambient space BID32 BID21 BID29. The non-linear dimensionality reduction performed by autoencoders finds intrinsic data dimensions that esentially straighten the curvature of data in the ambient space. Thus complex effects can become simpler shifts in distribution that can be computationally efficient to apply. Second, by performing the edit to the neural network internal layer, we allow for the modeling of some context dependence. Some neurons of the internal layer have a drastic change between preand post-treatment versions of the experimental subpopulation, while other neurons such as those that encode context information not directly associated with treatment have less change in the embedding layer. The latter neurons are less heavily edited but still influence the output jointly with edited neurons due to their integration in the decoding layers. These edited neurons interact with the data-context-encoding neurons in complex ways that may be more predictive of treatment than the experimental norm of simply assuming widespread generalization of context-free. Third, editing in a low-dimensional internal layer allows us to edit on a denoised version of the data. Because of the reconstruction penalty, more significant dimensions are retained through the bottleneck dimensions of an autoencoder while noise dimensions are discarded. Thus, by editing in the hidden layer, we avoid editing noise and instead edit significant dimensions of the data. We note that neuron editing makes the assumption that the internal neurons have semantic consistency across the data, i.e., the same neurons encode the same types of features for every data manifold. We demonstrate that this holds in our setting because the autoencoder learns a joint manifold of all of the given data including pre-and post-treatment samples of the experimental subpopulation and pre-treatment samples from the rest of the population. Recent show that neural networks prefer to learn patterns over memorizing inputs even when they have the capacity to do so BID31.We demonstrate that neuron editing extrapolates better than generative models on two important criteria. First, as to the original goal, the predicted change on extrapolated data more closely resembles the predicted change on interpolated data. Second, the editing process produces more complex variation, since it simply preserves the existing variation in the data rather than needing a generator to learn to create it. We compare the predictions from neuron editing to those of several generationbased approaches: a traditional GAN, a GAN implemented with residual blocks (ResnetGAN) to show generating residuals is not the same as editing BID26, and a CycleGAN BID33. While in other applications, like natural images, GANs have shown an impressive ability to generate plausible individual points, we illustrate that they struggle with these two criteria. We also motivate why neuron editing is performed on inference by comparing against a regularized autoencoder that performs the internal layer transformations during training, but the decoder learns to undo the transformation and reconstruct the input unchanged BID0.In the following section, we detail the neuron editing method. Then, we motivate the extrapolation problem by trying to perform natural image domain transfer on the canonical CelebA dataset. We then move to two biological applications where extrapolation is essential: correcting the artificial variability introduced by measuring instruments (batch effects), and predicting the combined effects of multiple drug treatments (combinatorial drug effects) BID1. Let S, T, X ⊆ R d be sampled sets representing d-dimensional source, target, and extrapolation distributions, respectively. We seek a transformation that has two properties: when applied to S it produces a distribution equivalent to the one represented by T, and when applied to T it is the identity function. GANs learn a transformation with these properties, and when parameterized with ReLU or leaky ReLU activations, as they often are, this transformation also has the property that it is piecewise linear BID11 BID14. However, the GAN optimization paradigm produces transformations that do not behave comparably on both S and X. Therefore, instead of learning such a transformation, we define a transformation with these properties on a learned space (summarized in FIG0).We first train an encoder/decoder pair E/D to map the data into an abstract neuron space decomposed into high-level features such that it can also decode from that space, i.e., an objective L: DISPLAYFORM0 where MSE is the mean-squared error. Then, without further training, we separately extract the activations of an n-dimensional internal layer of the network for inputs from S and from T, denoted by a S: S → R n, a T: T → R n. We define a piecewise linear transformation, called N euronEdit, which we apply to these distributions of activations: DISPLAYFORM1 where a ∈ R n consists of n activations for a single network input, p S j, p T j ∈ R n consist of the j th percentiles of activations (i.e., for each of the n neurons) over the distributions of a S, a T correspondingly, and all operations are taken pointwise, i.e., independently on each of the n neurons in the layer. Then, we define N euronEdit(a S): S → R n given by x → N euronEdit(a S (x)), and equivalently for a T and any other distribution (or collection) of activations over a set of network inputs. Therefore, the N euronEdit function operates on distributions, represented via activations over network input samples, and transforms the input activation distribution based on the difference between the source and target distributions (considered via their percentile disctretization).We note that the N euronEdit function has the three properties of a GAN generator: 1. N euronEdit(a S) ≈ a T (in terms of the represented n-dimensional distributions) 2. N euronEdit(a T) = a T 3. piecewise linearity. However, we are also able to guarantee that the neuron editing performed on the source distribution S will be the same as that performed on the extrapolation distribution X, which would not be the case with the generator in a GAN.To apply the learned transformation to X, we first extract the activations of the internal layer computed by the encoder, a X. Then, we cascade the transformations applied to the neuron activations through the decoder without any further training. Thus, the transformed outputX is obtained by: DISPLAYFORM2 Crucially, the nomenclature of an autoencoder no longer strictly applies. If we allowed the encoder or decoder to train with the transformed neuron activations, the network could learn to undo these transformations and still produce the identity function. However, since we freeze training and apply these transformations exclusively on inference, we turn an autoencoder into a generative model that need not be close to the identity. Training a GAN in this setting could exclusively utilize the data in S and T, since we have no real examples of the output for X to feed to the discriminator. Neuron editing, on the other hand, is able to model the variation intrinsic to X in an unsupervised manner despite not having real posttransformation data for X. Since we know a priori that X will differ substantially from S, this provides significantly more information. Furthermore, GANs are notoriously tricky to train BID22 BID12 BID30. Adversarial discriminators suffer from oscillating optimization dynamics, uninterpretable losses BID6, and most debilitatingly, mode collapse BID25 BID15 BID20. Mode collapse refers to the discriminator being unable to detect differences in variabillity between real and fake examples. In other words, the generator learns to generate a point that is very realistic, but produces that same point for most (or even all) input, no matter how different the input is. In practice, we see the discriminator struggles to detect differences in real and fake distributions even without mode collapse, as evidenced by the generator favoring ellipsoid output instead of the more complex and natural variability of the real data. Since our goal is not to generate convincing individual examples of the post-transformation output, but the more difficult task of generating convincing entire distributions of the post-transformation output, this is a worrisome defect of GANs. Neuron editing avoids all of these traps by learning an unsupervised model of the data space with the easier-to-train autoencoder. The essential step that facilitates generation is the isolation of the variation in the neuron activations that characterizes the difference between source and target distributions. There is a relationship between neuron editing and the well-known word2vec embeddings in natural language processing BID10. There, words are embedded in a latent space where a meaningful transformation such as changing the gender of a word is a constant vector in this space. This vector can be learned on one example, like transforming man to woman, and then extrapolated to another example, like king, to predict the location in the space of queen. Neuron editing is an extension in complexity of word2vec's vector arithmetic, because instead of transforming a single point into another single point, it transforms an entire distribution into another distribution. In this section we compare neuron interference as an editing method to various generating methods: a regularized autoencoder, a standard GAN, a ResnetGAN, and a CycleGAN. For the regularized autoencoder, the regularization penalized differences in the distributions of the source and target in a latent layer using maximal mean discrepancy BID0 BID8. The image experiment used convolutional layers with stride-two filters of size four, with 64-128-256-128-64 filters in the layers. All other models used fully connected layers of size 500-250-50-250-500. In all cases, leaky ReLU activation was used with 0.2 leak. Training was done with minibatches of size 100, with the adam optimizer BID16, and a learning rate of 0.001. We first consider a motivational experiment on the canonical image dataset of CelebA. If we want to learn a transformation that turns a given image of a person with black hair to that same person except with blond hair, a natural approach would be to collect two sets of images, one with all black haired people and another with all blond haired people, and teach a generative model to map between them. The problem with this approach is that the learned model will only be able to apply the hair color change to images similar to those in the collection, unable to extrapolate. This is illustrated in FIG1, where we collect images that have the attribute male and the attribute black hair and try to map to the set of images with the attribute male and the attribute blond hair. Then, after training on this data, we extrapolate and apply the transformation to females with black hair, which had not been seen during training. The GAN models are unable to successfully model this transformation on out-of-sample data. In the parts of the image that should stay the same (everything but the hair color), they do not always generate a recreation of the input. In the hair color, only sometimes is the color changed. The regular GAN model especially has copious artifacts that are a of the difficulty in training these models. This provides further evidence of the benefits of avoiding these complications when possible, for example by using the stable training of an autoencoder and editing it as we do in neuron editing. In FIG1, we motivate why we need to perform the N euronEdit transformation on the internal layer of a neural network, as opposed to applying it on some other latent space like PCA. Only in the neuron space has this complex and abstract transformation of changing the hair color (and only the hair color) been decomposed into a relatively simple and piecewise linear shift. We next demonstrate another application of neuron editing's ability to learn to transform a distribution based on a separate source/target pair: batch correction. Batch effects are differences in the observed data that are caused by technical artifacts of the measurement process. In other words, we can measure the same sample twice and get two rather different datasets back. When we measure different samples, batch effects get confounded with the true difference between the samples. Batch effects are a ubiquitous problem in biological experimental data that (in the best case) prevent combining measurements or (in the worst case) lead to wrong . Addressing batch effects is a goal of many new models BID9 BID27 BID7 BID13, including some deep learning methods BID23 BID0.One method for grappling with this issue is to repeatedly measure an identical control (spike-in) set of cells with each sample, and correct that sample based on the variation in its version of the control BID3. In our terminology of generation, we choose our source/target pair to be Control1/Control2, and then extrapolate to Sample1. Our transformed Sample1 can then be compared to raw Sample2 cells, rid of any variation induced by the measurement process. We would expect this to be a natural application of neuron editing as the data distributions are complex and the control population is unlikely to be similar to any (much less all) of the samples. The dataset we investigate in this section comes from a mass cytometry BID5 ) experiment which measures the amount of particular proteins in each cell in two different individuals infected with dengue virus BID0. The data consists of 35 dimensions, where Control1, Control2, Sample1, and Sample2 have 18919, 22802, 94556, and 55594 observations, respectively. The two samples were measured in separate runs, so in addition to the true difference in biology creating variation between them, there are also technical artifacts creating variation between them. From the controls, we can see one such batch effect characterized by artificially low readings in the amount of the protein InfG in Control1 (the x-axis in Figure 3a).We would like our model to identify this source of variation and compensate for the lower values of InfG without losing other true biological variation in Sample1. For example, Sample1 also has higher values of the protein CCR6, and as the controls show, this is a true biological difference, not a batch effect (the y-axis in Figure 3a). Because the GANs never trained on cells with high CCR6 and were never trained to generate cells with high CCR6, it is unsurprising that all of them remove that variation. Worryingly, the GAN maps almost all cells to the same values of CCR6 and InfG (Figure 3d), and the CycleGAN maps them to CCR6 values near zero (Figure 3f). This means later comparisons would not only lose the information that these cells were exceptionally high in CCR6, but now they would even look exceptionally low in it. The ResnetGAN does not fix this problem, as it is intrinsic to the specification of the generation objective, which only encourages the production of output like the target distribution, and in this case we want output different from the target distribution. As in the previous examples, the ResnetGAN also learns residuals that produces more ellipsoid data, rather than preserving the variation in the original source distribution or even matching the variation in the target distribution. The regularized autoencoder has learned to undo the transformations to its latent space and produced unchanged data. Neuron editing, on the other hand, decomposes the variability into just the separation between controls (increases in InfG), and edits the sample to include this variation. This removes the batch effect that caused higher readings of InfG, while preserving all other real variation, including both the intra-sample variation and the variation separating the populations in CCR6.Unlike the other generative models, neuron editing produced the intended transformations for the proteins InfG and CCR6, and here we go on to confirm that its are accurate globally across all dimensions. In FIG2, a PCA embedding of the whole data space is visualized for Control1 (light blue), Control2 (light red), Sample1 (dark blue), and post-transformation Sample1 (dark red). The transformation from Control1 to Control2 mirrors the transformation applied to Sample1. Notably, the other variation (intra-sample variation) is preserved. In FIG2, we see that for every dimension, the variation between the controls corresponds accurately to the variation introduced by neuron editing into the sample. These global assessments across the full data space offer additional corroboration that the transformations produced by neuron editing reasonably reflect the transformation as evidenced by the controls. Finally, we consider biological data from a combinatorial drug experiment on cells from patients with acute lymphoblastic leukemia BID1. The dataset we analyze consists of cells Figure 3: The of learning to batch correct a sample based on the difference in a repeatedlymeasured control population measured. The true transformation is depicted with a solid arrow and the learned transformation with a dashed arrow. There is a batch effect that should be corrected in IFNg (horizontal) with true variation that should be preserved in CCR6 (vertical). All of the GANs attempt to get rid of all sources of variation (but do so only partially because the input is out-ofsample). The autoencoder does not move the data at all. Neuron editing corrects the batch effect in IFNg while preserving the true biological variation in CCR6. under four treatments: no treatment (basal), BEZ-235 (Bez), Dasatinib (Das), and both Bez and Das (Bez+Das). These measurements also come from mass cytometry, this time on 41 dimensions, with the four datasets consisting of 19925, 20078, 19843, and 19764 observations, respectively. In this setting, we define the source to be the basal cells, the target to be the Das cells, and then extrapolate to the Bez cells. We hold out the true Bez+Das data and attempt to predict the effects of applying Das to cells that have already been treated with Bez. A characteristic effect of applying Das is a decrease in p4EBP1 (seen on the x-axis of FIG3). No change in another dimension, pSTATS, is associated with the treatment (the y-axis of FIG3). Neuron editing accurately models this horizontal change, without introducing any vertical change or losing variation within the extrapolation dataset FIG3 ). The regularized autoencoder, as before, does not change the output at all, despite the manipulations within its internal layer FIG3. None of the three GAN models accurately predict the real combination: the characteristic horizontal shift is identified, but additional vertical shifts are introduced and much of the original within-Bez variability is lost FIG3 -e).We note that since much of the variation in the target distribution already exists in the source distribution and the shift is a relatively small one, we might expect the ResnetGAN to be able to easily mimic the target. However, despite the residual connections, it still suffers from the same problems as the other model using the generating approach: namely, the GAN objective encourages all output to be like the target it trained on. That the GANs are not able to replicate even two-dimensional slices of the target data proves that they have not learned the appropriate transformation. But to further evaluate that neuron editing produces a meaningful transformation globally, we here make a comparison of every dimension. FIG4 compares the real and predicted means (in 6a) and variances (in 6b) of each dimension. Neuron editing more accurately predicts the principle direction and magnitude of transformation across all dimensions. Furthermore, neuron editing better preserves the variation in the real data. In almost all dimensions, the GAN generates data with less variance than really exists. In this paper, we tackled a data-transformation problem inspired by biological experimental settings: that of generating transformed versions of data based on observed pre-and post-transformation versions of a small subset of the available data. This problem arises during clinical trials or in settings where effects of drug treatment (or other experimental conditions) are only measured in a subset of the population, but expected to generalize beyond that subset. Here we introduce a novel approach that we call neuron editing, for applying the treatment effect to the remainder of the dataset. Neuron editing makes use of the encoding learned by the latent layers of an autoencoder and extracts the changes in activation distribution between the observed pre-and post treatment measurements. Then, it applies these same edits to the internal layer encodings of other data to mimic the transformation. We show that performing the edit on neurons of an internal layer in more realistic transformations of image data, and successfully predicts synergistic effects of drug treatments in biological data. Moreover, we note that it is feasible to learn complex data transformations in the non-linear dimensionality reduced space of a hidden layer rather than in ambient space where joint probability distributions are difficult to extract. Finally, learning edits in a hidden layer allows for interactions between the edit and other context information from the dataset during decoding. Future work along these lines could include training parallel encoders with the same decoder, or training to generate conditionally.

represent this text to find a much much longer article about it with say hundreds-thousands of words

We reframe the generation problem as one of editing existing points, and as a result extrapolate better than traditional GANs.

We introduce a type of neural network that is structurally resistant to adversarial attacks, even when trained on unaugmented training sets.  The resistance is due to the stability of network units wrt input perturbations.


Question: Represent the text to find a very short summary (1-2 sentences max).

Question: We present a representation for describing transition models in complex uncertain domains using relational rules. For any action, a rule selects a set of relevant objects and computes a distribution over properties of just those objects in the ing state given their properties in the previous state. An iterative greedy algorithm is used to construct a set of deictic references that determine which objects are relevant in any given state. Feed-forward neural networks are used to learn the transition distribution on the relevant objects' properties. This strategy is demonstrated to be both more versatile and more sample efficient than learning a monolithic transition model in a simulated domain in which a robot pushes stacks of objects on a cluttered table. Many complex domains are appropriately described in terms of sets of objects, properties of those objects, and relations among them. We are interested in the problem of taking actions to change the state of such complex systems, in order to achieve some objective. To do this, we require a transition model, which describes the system state that from taking a particular action, given the previous system state. In many important domains, ranging from interacting with physical objects to managing the operations of an airline, actions have localized effects: they may change the state of the object(s) being directly operated on, as well as some objects that are related to those objects in important ways, but will generally not affect the vast majority of other objects. In this paper, we present a strategy for learning state-transition models that embodies these assumptions. We structure our model in terms of rules, each of which only depends on and affects the properties and relations among a small number of objects in the domain, and only very few of which may apply for characterizing the effects of any given action. Our primary focus is on learning the kernel of a rule: that is, the set of objects that it depends on and affects. At a moderate level of abstraction, most actions taken by an intentional system are inherently directly parametrized by at least one object that is being operated on: a robot pushes a block, an airport management system reschedules a flight, an automated assistant commits to a venue for a meeting. It is clear that properties of these "direct" objects are likely to be relevant to predicting the action's effects and that some properties of these objects will be changed. But how can we characterize which other objects, out of all the objects in a household or airline network, are relevant for prediction or likely to be affected?To do so, we make use of the notion of a deictic reference. In linguistics, a deictic (literally meaning "pointing") reference, is a way of naming an object in terms of its relationship to the current situation rather than in global terms. So, "the object I am pushing," "all the objects on the table nearest me," and "the object on top of the object I am pushing" are all deictic references. This style of reference was introduced as a representation strategy for AI systems by BID0, under the name indexical-functional representations, for the purpose of compactly describing policies for a video-game agent, and has been in occasional use since then. We will learn a set of deictic references, for each rule, that characterize, relative to the object(s) being operated on, which other objects are relevant. Given this set of relevant objects, the problem of describing the transition model on a large, variable-size domain, reduces to describing a transition model on fixed-length vectors characterizing the relevant objects and their properties and relations, which we represent and learn using standard feed-forward neural networks. Next, we briefly survey related work, describe the problem more formally, and then provide an algorithm for learning both the structure, in terms of deictic references, and parameters, in terms of neural networks, of a sparse relational transition model. We go on to demonstrate this algorithm in a simulated robot-manipulation domain in which the robot pushes objects on a cluttered table. Rule learning has a long history in artificial intelligence. The novelty in our approach is the combination of learning discrete structures with flexible parametrized models in the form of neural networks. Rule learning We are inspired by very early work on rule learning by BID4, which sought to find predictive rules in simple noisy domains, using Boolean combinations of binary input features to predict the effects of actions. This approach has a modern re-interpretation in the form of schema networks BID5. The rules we learn are lifted, in the sense that they can be applied to objects, generally, and are not tied to specific bits or objects in the input representation and probabilistic, in the sense that they make a distributional prediction about the outcome. In these senses, this work is similar to that of and methods that build on it (BID9, BID8, BID6 .) In addition, the approach of learning to use deictic expressions was inspired by Pasula et al. and used also by BID7 in the form of object-oriented reinforcement learning and by BID2. BID2, however, relies on a full description of the states in ground first-order logic and does not have a mechanism to introduce new deictic references to the action model. Our representation and learning algorithm improves on the Pasula et al. strategy by using the power of feed-forward neural networks as a local transition model, which allows us to address domains with real-valued properties and much more complex dependencies. In addition, our EM-based learning algorithm presents a much smoother space in which to optimize, making the overall learning faster and more robust. We do not, however, construct new functional terms during learning; that would be an avenue for future work for us. Graph network models There has recently been a great deal of work on learning graph-structured (neural) network models BID1. There is a way in which our rule-based structure could be interpreted as a kind of graph network, although it is fairly non-standard. We can understand each object as being a node in the network, and the deictic functions as being labeled directed hyper-edges (between sets of objects). Unlike the typical graph network models, we do not condition on a fixed set of neighboring nodes and edges to compute the next value of a node; in fact, a focus of our learning method is to determine which neighbors (and neighbors of neighbors, etc.) to condition on, depending on the current state of the edge labels. This means that the relevant neighborhood structure of any node changes dynamically over time, as the state of the system changes. This style of graph network is not inherently better or worse than others: it makes a different set of assumptions (including a strong default that most objects do not change state on any given step and the dynamic nature of the neighborhoods) which are particularly appropriate for modeling an agent's interactions with a complex environment using actions that have relatively local effects. We assume we are working on a class of problems in which the domain is appropriately described in terms of objects. This method might not be appropriate for a single high-dimensional system in which the transition model is not sparse or factorable, or can be factored along different lines (such as a spatial decomposition) rather than along the lines of objects and properties. We also assume a set of primitive actions defined in terms of control programs that can be executed to make actual changes in the world state and then return. These might be robot motor skills (grasping or pushing an object) or virtual actions (placing an order or announcing a meeting). In this section, we formalize this class of problems, define a new rule structure for specifying probabilistic transition models for these problems, and articulate an objective function for estimating these models from data. A problem domain is given by tuple D = (Υ, P, F, A) where Υ is a countably infinite universe of possible objects, P is a finite set of properties P i: Υ → R, i ∈ [N P] = {1, · · ·, N P}, and F is a finite set of deictic reference functions DISPLAYFORM0 where ℘ (Υ) denotes the powerset of Υ. Each function F i ∈ Υ maps from an ordered list of objects to a set of objects, and we define it as DISPLAYFORM1 where the relation f i: Υ mi+1 → {True, False} is defined in terms of the object properties in P. For example, if we have a location property P loc and m i = 1, we can define f i (o, o 1) = 1 Ploc(o)−Ploc(o1) <0.5 so that the function F i associated with f i maps from one object to the set of objects that are within 0.5 distance of its center; here 1 is an indicator function. Finally, A is a set of action templates DISPLAYFORM2 where Ψ is the space of executable control programs. Each action template is a function parameterized by continuous parameters α i ∈ R di and a tuple of n i objects that the action operates on. In this work, we assume that P, F and A are given. A problem instance is characterized by I = (D, U), where D is a domain defined above and U ⊂ Υ is a finite universe of objects with |U| = N U. For simplicity, we assume that, for a particular instance, the universe of objects remains constant over time. In the problem instance I, we characterize a state s in terms of the concrete values of all properties in P on all objects in U; that is, DISPLAYFORM0 A problem instance induces the definition of its action space A, constructed by applying every action template A i ∈ A to all tuples of n i elements in U and all assignments α i to the continuous parameters; namely, DISPLAYFORM1 In many domains, there is substantial uncertainty, and the key to robust behavior is the ability to model this uncertainty and make plans that respect it. A sparse relational transition model (SPARE) for a domain D, when applied to a problem instance I for that domain, defines a probability density function on the ing state s ing from taking action a in state s. Our objective is to specify this function in terms of domain elements P, R, and F in such a way that it will apply to any problem instance, independent of the number and properties of the objects in its universe. We achieve this by defining the transition model in terms of a set of transition rules, T = {T k} K k=1 and a score function C: T × S → N. The score function takes in as input a state s and a rule T ∈ T, and outputs a non-negative integer. If the output is 0, the rule does not apply; otherwise, the rule can predict the distribution of the next state to be p(s | s, a; T). The final prediction of SPARE is DISPLAYFORM0 whereT = arg max T ∈T C(T, s) and the matrix DISPLAYFORM1 ) is the default predicted covariance for any state that is not predicted to change, so that our problem is well-formed in the presence of noise in the input. Here I N U is an identity matrix of size N U, and diag ([σ DISPLAYFORM2 represents a square diagonal matrix with σ i on the main diagonal, denoting the default variance for property P i if no rule applies. Note that the transition rules will be learned from past experience with a loss function specified in Section 3.3. In the rest of this section, we formalize the definition of transition rules and the score function. Transition rule T = (A, Γ, ∆, φ θ, v default) is characterized by an action template A, two ordered lists of deictic references Γ and ∆ of size N Γ and N ∆, a predictor φ θ and the default variances DISPLAYFORM3 for each property P i under this rule. The action template is defined as operating on a tuple of n object variables, which we will refer to as DISPLAYFORM4 A reference list uses functions to designate a list of additional objects or sets of objects, by making deictic references DISPLAYFORM5 Figure 2: Instead of directly mapping from current state s to next state s, our prediction model uses deictic references to find subsets of objects for prediction. In the left most graph, we illustrate what relations are used to construct the input objects with two rules for the same action template, DISPLAYFORM6 default ), where the reference list DISPLAYFORM7 1 to the target object o 2 and added input features computed by an aggregator g on o 3, o 6 to the inputs of the predictor of rule T 1. Similarly for DISPLAYFORM8, the first deictic reference selected o 3 and then γ2 is applied on o 3 to get o 1. The predictors φ θ are neural networks that map the fixed-length input to a fixed-length output, which is applied to a set of objects computed from a relational graph on all the objects, derived from the reference list DISPLAYFORM9, to compute the whole next state s. Because δ DISPLAYFORM10 θ is only predicting a single property, we use a "de-aggregator" function h to assign its prediction to both objects o 4, o 6. based on previously designated objects. In particular, Γ generates a list of objects whose properties affect the prediction made by the transition rule, while ∆ generates a list of objects whose properties are affected after taking an action specified by the action template A.We begin with the simple case in which every function returns a single object, then extend our definition to the case of sets. Concretely, for the t-th element DISPLAYFORM11 where F ∈ F is a deictic reference function in the domain, m is the arity of that function, and integer k j ∈ [n+t−1] specifies that object O n+t in the object list can be determined by applying function F to objects (O kj) m j=1. Thus, we get a new list of objects, DISPLAYFORM12. So, reference γ 1 can only refer to the objects (O i) n i=1 that are named in the action, and determines an object O n+1. Then, reference γ 2 can refer to objects named in the action or those that were determined by reference γ 1, and so on. When the function DISPLAYFORM13 ) ∈ Γ returns a set of objects rather than a single object, this process of adding more objects remains almost the same, except that the O t may denote sets of objects, and the functions that are applied to them must be able to operate on sets. In the case that a function F returns a set, it must also specify an aggregator, g, that can return a single value for each property P i ∈ P, aggregated over the set. Examples of aggregators include the mean or maximum values or possibly simply the cardinality of the set. For example, consider the case of pushing the bottom (block A) of a stack of 4 blocks, depicted in FIG1. Suppose the deictic reference is F =above, which takes one object and returns a set of objects immediately on top of the input object. Then, by applying F =above starting from the initial set O 0 = {A}, we get an ordered list of sets of objects DISPLAYFORM14 Returning to the definition of a transition rule, we now can see informally that if the parameters of action template A are instantiated to actual objects in a problem instance, then Γ and ∆ can be used to determine lists of input and output objects (or sets of objects). We can use these lists, finally, to construct input and output vectors. The input vector x consists of the continuous action parameters α of action A and property P i (O t) for all properties P i ∈ P and objects O t ∈ O NΓ that are selected by Γ in arbitrary but fixed order. In the case that O t is a set of size greater than one, the aggregator associated with the function F that computed the reference is used to compute P i (O t). Similar for the desired output construction, we use the references in the list ∆, initializeÔ = O, and gradually add more objects to construct the output set of objectsÔ =Ô (N∆). The output vector is y = [P (ô)]ô ∈Ô,P ∈P where ifô is a set of objects, we apply a mean aggregator on the properties of all the objects inô. The predictor φ θ is some functional form φ (such as a feed-forward neural network) with parameters (weights) θ that will take values x as input and predict a distribution for the output vector y. It is difficult to represent arbitrarily complex distributions over output values. In this work, we restrict ourselves to representing a Gaussian distributions on all property values in y, encoded with a mean and independent variance for each dimension. Now, we describe how a transition rule can be used to map a state and action into a distribution over the new state. A transition rule T = (A, Γ, ∆, φ θ, v default) applies to a particular state-action (s, a) pair if a is an instance of A and if none of the elements of the input or output object lists is empty. To construct the input (and output) list, we begin by assigning the actual objects o 1,..., o n to the object variables O 1,..., O n in action instance a, and then successively computing references γ i ∈ Γ based on the previously selected objects, applying the definition of the deictic reference F in each γ i to the actual values of the properties as specified in the state s. If, at any point, a γ i ∈ Γ or δ i ∈ ∆ returns an empty set, then the transition rule does not apply. If the rule does apply, and successfully selects input and output object lists, then the values of the input vector x can be extracted from s, and predictions are made on the mean and variance values Pr(DISPLAYFORM15 θ2 (x) be the vector entry corresponding to the predicted Gaussian parameters of property P i of j-th output object setô j and denote s[o, P i] as the property P i of object o in state s, for all o ∈ U. The predicted distribution of the ing state p(s | s, a; T) is computed as follows: DISPLAYFORM16 where v i ∈ v default is the default variance of property P i in rule T. There are two important points to note. First, it is possible for the same object to appear in the object-list more than once, and therefore for more than one predicted distribution to appear for its properties in the output vector. In this case, we use the mixture of all the predicted distributions with uniform weights. Second, when an element of the output object list is a set, then we treat this as predicting the same single property distribution for all elements of that set. This strategy has sufficed for our current set of examples, but an alternative choice would be to make the predicted values be changes to the current property value, rather than new absolute values. Then, for example, moving all of the objects on top of a tray could easily specify a change to each of their poses. We illustrate how we can use transition rules to build a SPARE in Fig. 2.For each transition rule T k ∈ T and state s ∈ S, we assign the score function value to be 0 if T k does not apply to state s. Otherwise, we assign the total number of deictic references plus one, N Γ + N ∆ + 1, as the score. The more references there are in a rule that is applicable to the state, the more detailed the match is between the rules conditions and the state, and the more specific the predictions we expect it to be able to make. We frame the problem of learning a transition model from data in terms of conditional likelihood. The learning problem is, given a problem domain description D and a set of experience E tuples, DISPLAYFORM0, find a SPARE T that minimizes the loss function: DISPLAYFORM1 Note that we require all of the tuples in E to belong to the same domain D, and require for any (DISPLAYFORM2 and s (i) belong to the same problem instance, but individual tuples may be drawn from different problem instances (with, for example, different numbers and types of objects). In fact, to get good generalization performance, it will be important to vary these aspects across training instances. We describe our learning algorithm in three parts. First, we introduce our strategy for learning φ θ, which predicts a Gaussian distribution on y, given x. Then, we describe our algorithm for learning reference lists Γ and ∆ for a single transition rule, which enable the extraction of x and y from E. Finally, we present an EM method for learning multiple rules. For a particular transition rule T with associated action template A, once Γ and ∆ have been specified, we can extract input and output features x and y from a given set of experience samples E. We would like to learn the transition rule's predictor φ θ to minimize Eq.. Our predictor takes the form φ θ (x) = N (µ θ (x), Σ θ (x)) and a neural network is used to predict both the mean µ θ (x) and the diagonal variance Σ θ (x). We directly optimize the negative data-likelihood loss function DISPLAYFORM0 Let E T ∈ E be the set of experience tuples to which rule T applies. Then once we have θ, we can optimize the default variance of the rule DISPLAYFORM1 It can be shown that these loss-minimizing values for the default predicted variances v default are the empirical averages of the squared deviations for all unpredicted objects (i.e., those for which φ θ does not explicitly make predictions), where averages are computed separately for each object property. We use θ, v default ← LEARNDIST(D, E, Γ, ∆) to refer to this learning and optimization procedure for the predictor parameters and default variance. Algorithm 1 Greedy procedure for constructing Γ. DISPLAYFORM0 train model using Γ 0 = ∅, save loss L 0 i ← 1 4: DISPLAYFORM0 for all γ ∈ R i do 7: DISPLAYFORM1 else breakIn the simple setting where only one transition rule T exists in our domain D, we show how to construct the input and output reference lists Γ and ∆ that will determine the vectors x and y. Suppose for now that ∆ and v default are fixed, and we wish to learn Γ. Our approach is to incrementally build up Γ by adding DISPLAYFORM2 ) tuples one at a time via a greedy selection procedure. Specifically, let R i be the universe of possible γ i, split the experience samples E into a training set E train and a validation set E val, and initialize the list Γ to be Γ 0 = ∅. For each i, compute γ i = arg min γ∈Ri L(T γ ; D, E val), where L in Eq. evaluates a SPARE T γ with a single transition rule T = (A, Γ i−1 ∪ {γ}, ∆, φ θ, v default ), where θ and v default are computed using the LEARNDIST described in Section 4.1 2. If the value of the loss function L(T γi ; D, E val) is less than the value of L(T γi−1 ; D, E val), then we let Γ i = Γ i−1 ∪{γ i} and continue. Else, we terminate the greedy selection process with Γ = Γ i−1, since further growing the list of deictic references hurts the loss. We also terminate the process when i exceeds some predetermined maximum allowed number of input deictic references, N Γ. Pseudocode for this algorithm is provided in Algorithm 1.In our experiments we set ∆ = Γ and construct the lists of deictic references using a single pass of the greedy algorithm described above. This simplification is reasonable, as the set of objects that are relevant to predicting the transition outcome often overlap substantially with the objects that are affected by the action. Alternatively, we could learn ∆ via an analogous greedy procedure nested around or, as a more efficient approach, interleaved with, the one for learning Γ. Our training data in robotic manipulation tasks are likely to be best described by many rules instead of a single one, since different combinations of relations among objects could be present in different states. For example, we may have one rule for pushing a single object and another rule for pushing a stack of objects. We now address the case where we wish to learn K rules from a single experience set E, for K > 1. We do so via initial clustering to separate experience samples into K clusters, one for each rule to be learned, followed by an EM-like approach to further separate samples and simultaneously learn rule parameters. To facilitate the learning of our model, we will additionally learn membership probabilities Z = ((z i,j) DISPLAYFORM0, where z i,j represents the probability that the i-th experience sample is assigned to transition rule T j, and DISPLAYFORM1. We initialize membership probabilities via clustering, then refine them through EM.Because the experience samples E may come from different problem instances and involve different numbers of objects, we cannot directly run a clustering algorithm such as k-means on the (s, a, s) samples themselves. Instead we first learn a single transition rule T = (A, Γ, ∆, φ θ, v default) from E using the algorithm in Section 4.2, use the ing Γ and ∆ to transform E into x and y, and then run k-means clustering on the concatenation of x, y, and values of the loss function when T is used to predict each of the samples. For each experience sample, the squared distance from the sample to each of the K cluster centers is computed, and membership probabilities for the sample to each of the K transition rules to be learned are initialized to be proportional to the (multiplicative) inverses of these squared distances. Before introducing the EM-like algorithm that simultaneously improves the assignment of experience samples to transition rules and learns details of the rules themselves, we make a minor modification to transition rules to obtain mixture rules. Whereas a probabilistic transition rule has been defined as T = (A, Γ, ∆, φ θ, v default), a mixture rule is T = (A, π Γ, π ∆, Φ), where π Γ represents a distribution over all possible lists of input references Γ (and similarly for π ∆ and ∆), of which there are a finite number, since the set of available reference functions F is finite, and there is an upper bound N Γ on the maximum number of references Γ may contain. For simplicity of terminology, we refer to each possible list of references Γ as a shell, so π Γ is a distribution over possible shells. DISPLAYFORM2 is a collection of κ transition rules (i.e., predictors φ θ (k), each with an associated Γ (k), ∆ (k), and v (k) default ). To make predictions for a sample (s, a) using a mixture rule, predictions from each of the mixture rule's κ transition rules are combined according to the probabilities that π Γ and π ∆ assign to each transition rule's Γ (k) and ∆ (k). Rather than having our EM approach learn K transition rules, we instead learn K mixture rules, as the distributions π Γ and π ∆ allow for smoother sorting of experience samples into clusters corresponding to the different rules, in contrast to the discrete Γ and ∆ of regular transition rules. As before, we focus on the case where for each mixture rule, DISPLAYFORM3, and π Γ = π ∆ as well. Our EM-like algorithm is then as follows:1. For each j ∈ [K], initialize distributions π Γ = π ∆ for mixture rule T j as follows. First, use the algorithm in Section 4.2 to learn a transition rule on the weighted experience samples E Zj with weights equal to the membership probabilities DISPLAYFORM4. In the process of greedily assembling reference lists Γ = ∆, data likelihood loss function values are computed for multiple explored shells, in addition to the shell Γ = ∆ that was ultimately selected. Initialize π Γ = π ∆ to distribute weight proportionally, ac- DISPLAYFORM5, with the summation taken over all explored shells Γ, is a normalization factor so that the total weight assigned by π Γ to explored shells is 1−. The remaining probability weight is distributed uniformly across unexplored shells., where we have dropped subscripting according to j for notational simplicity: DISPLAYFORM0 default ) using the procedure in Section 4.2 on the weighted experience samples E Zj, where we choose DISPLAYFORM1 to be the list of references with k-th highest weight according to π Γ = π ∆. (b) Update π Γ = π ∆ by redistributing weight among the top κ shells according to a voting procedure where each training sample "votes" for the shell whose predictor minimizes the validation loss for that sample. In other words, the i-th experience sample DISPLAYFORM2 ). Then, shell weights are assigned to be proportional to the sum of the sample weights (i.e., membership probability of belonging to this rule) of samples that voted for each particular shell: the number of votes received by the k-th shell is V (k) = |E| i=1 1 v(i)=k · z i,j, for indicator function 1 and k ∈ [κ]. Then, π Γ (k), the current k-th highest value of π Γ, is updated to become V (k)/ξ, where ξ is a normalization factor to ensure that π Γ remains a valid probability distribution. (Specifically, ξ = ( DISPLAYFORM3 Step 2a, in case the κ shells with highest π Γ values have changed, in preparation for using the mixture rule to make predictions in the next step.3. Update membership probabilities by scaling by data likelihoods from using each of the K rules to make predictions: DISPLAYFORM4 is the data likelihood from using mixture rule T j to make predictions for the i-th experience sample E (i), and DISPLAYFORM5 ) is a normalization factor to maintain K j=1 z i,j = 1. 4. Repeat Steps 2 and 3 some fixed number of times, or until convergence. We apply our approach, SPARE, to a challenging problem of predicting pushing stacks of blocks on a cluttered table top. We describe our domain, the baseline that we compare to and report our . In our domain D = (Υ, P, F, A), the object universe Υ is composed of blocks of different sizes and weight, the property set P includes shapes of the blocks (width, length, height) and the position of the block ((x, y, z) location relative to the table). We have one action template, push(α, o), which pushes toward a target object o with parameters α = (x g, y g, z g, d), where (x g, y g, z g) is the 3D position of the gripper before the push starts and d is the distance of the push. The orientation of the gripper and the direction of the push are computed from the gripper location and the target object location. We simulate this 3D domain using the physically realistic PyBullet (simulator. In real-world scenarios, an action cannot be executed with the exact action parameters due to the inaccuracy in the motor and hence in our simulation, we add Gaussian noise on the action parameters during execution to imitate this effect. We consider the following deictic references in the reference collection F: identity (O), which takes in an object O and returns O; above (O), which takes in an object O and returns the object immediately above O; below (O), which takes in an object O and returns the object immediately below O; nearest (O), which takes in an object O and returns the object that is closest to O. Neural network (NN) We compare to a neural network function approximator that takes in as input the current state s ∈ R N P ×N U and action parameter α ∈ R N A, and outputs the next state s ∈ R N P ×N U. The list of objects that appear in each state is ordered: the target objects appear first and the remaining objects are sorted by their poses (first sort by x coordinate, then y, then z).Graph NN We compare to a fully connected graph NN. Each node of the graph corresponds to an object in the scene, and the action α is concatenated to the state of each object. Bidirectional edges connect every node in the graph. The graph NN consists of encoders for the nodes and edges, propagation networks for message passing, and a node decoder to convert back to predict the mean and variance of the next state of each object. As a sanity check, we start from a simple problem where a gripper is pushing a stack of three blocks with two extra blocks on the table. We randomly sampled 1250 problem instances by drawing random block shapes and locations from a uniform distribution within a range while satisfying the condition that the stack of blocks is stable and the extra blocks do not affect the push. In each problem instance, we uniformly randomly sample the action parameters and obtain the training data, a collection of tuples of state, action and next state, where the target object of the push action is always the one at the bottom of the stack. We held out 20% of the training data as the validation set. We found that our approach is able to reliably select the correct combinations of the references that select all the blocks in the problem instance to construct inputs and outputs. In FIG3, we show how the performance varies as deictic references are added during a typical run of this experiment. The solid purple curves show training performance, as measured by data likelihood on the validation set, while the dashed purple curve shows performance on a held-out test set with 250 unseen problem instances. As expected, performance improves noticeably from the addition of the first two deictic references selected by the greedy selection procedure, but not from the 4th. The brown curve shows the learned default standard deviations, used to compute data likelihoods for features of objects not explicitly predicted by the rule. As expected, the learned default standard deviation drops as deictic references are added, until it levels off after the third reference is added since at that point the set of references captures all moving objects in the scene. Sensitivity analysis on the number of objects We compare our approach to the baselines in terms of how sensitive the performance is to the number of objects that exist in the problem instance. We continue the setting where a stack of three blocks lie on a table, with extra blocks that may affect the prediction of the next state. FIG3 shows the performance, as measured by the log data likelihood, as a function of the number of extra blocks. For each number of extra blocks, we used 1250 training problem instances with 20% as the validation set and 250 testing problem instances. When there are no extra blocks, SPARE learns a single rule whose x and y contain the same information as the inputs and outputs for the baselines. As more objects are added to the table, NN's performance drops as the presence of these additional objects appear to complicate the scene and NN is forced to consider more objects when making its predictions. SPARE outperforms graph NN, as the good predictions for the extra blocks contribute to the log data likelihood. Note that, performance aside, NN is limited to problems for which the number of objects in the scenes is fixed, as the it requires a fixed-size input vector containing information about all objects. Our SPARE approach does not have this limitation, and could have been trained on a single, large dataset that is the combination of the datasets with varying numbers of extra objects. However, we did not do this in our experiments for the sake of providing a more fair comparison against NN.Sample efficiency We evaluate our approach on more challenging problem instances where the robot gripper is pushing blocks on a cluttered table top and there are two additional blocks on the table that do not interfere or get affected by the pushing action. FIG3 (c) plots the data likelihood as a function of the number of training samples. We evaluate with training samples varying from 100 to 1250 and in each setting, the test dataset has 250 samples. Both our approach and the baselines benefit from having more training samples, but our approach is much more sample efficient and achieves good performance within only 500 training samples. Learning multiple transition rules Now we put our approach in a more general setting where multiple transition rules need to be learned for prediction of the next state. Our approach adopts an EM-like procedure to assign each training sample its distribution on the transition rules and learn each transition rule with re-weighted training samples. First, we construct a training dataset and 70% of it is on pushing 4-block stack. Our EM approach is able to concentrate to the 4-block case as shown in FIG4. The three curves correspond to the three stack heights in the original dataset, and each shows the average weight assigned to the "target" rule among samples of that stack height, where the target rule is the one that starts with a high concentration of samples of that particular height. At iteration 0, we see that the rules were initialized such that samples were assigned 70% probability of belonging to specific rules, based on stack height. As the algorithm progresses, the samples separate further, suggesting that the algorithm is able to separate samples into the correct groups. Conclusion These demonstrate the power of combining relational abstraction with neural networks, to learn probabilistic state transition models for an important class of domains from very little training data. In addition, the structural nature of the learned models will allow them to be used in factored search-based planning methods that can take advantage of sparsity of effects to plan efficiently. Finally, the end purpose of obtaining a transition model for robotic actions is to enable planning to achieve high-level goals. Due to time constraints, this work did not assess the effectiveness of learned template-based models in planning, but this is a promising area of future work as the assumption that features of objects for which a template makes no explicit prediction do not change meshes well with STRIPS-style planners, as they make similar assumptions. We here provide details on our experiments and more in experiments. Experimental details Our experiments on SPARE in this paper used nueral network predictors for making mean predictions and variance predictions, described in Section 4.1. Each network was implemented as a fully-connected network with two hidden layers of 64 nodes each in Keras, used ReLU activations between layers, and the Adam optimizer with default parameters. Predictors for the templates approach were trained for 1000 epochs each with a decaying learning rate starting at 1e-2 and decreasing by a factor of 0.6 every 100 epochs. The baseline NN predictor was implemented in exactly the same way. For the GNN, we used a node encoder and edge encoder to map to latent spaces of 16 dimensions. The propagation networks consisted of 2 fully connected layers of 16 units each, and the decoder mapped back to 6 dimensions: 3 for the mean, and 3 for the variance. The GNN was trained using a decaying learning rate starting at 1e-2, and decreasing by a factor of 0.5 every 100 epochs. A total of 900 epochs were used. States were parameterized by the (x, y, z) pose of each object in the scene, ordered such that the target object of the action always appeared first, and other objects appeared in random order (except for the baseline). Action parameters included the (x, y, z) starting pose of the robotic gripper, as well as a "push distance" parameter that controls how long the push action lasts. Actions were implemented to be stochastic by adding some amount of noise to the target location of each push, potentially reflecting inaccuries in robotic control. We use the clustering-based approaches for initializing membership probabilities presented in Section 4.3. In this section, we how well our clustering approach performs. TAB0 shows the sample separation achieved by the discrete clustering approach, where samples are assigned solely to their associated clusters found by k-means, on the push dataset for stacks of varying height. Each column corresponds to the one-third of samples which involve stacks of a particular height. Entries in the table show the proportion of samples of that stack height that have been assigned to each of the three clusters, where in generating these data the clusters were ordered so that the predominantly 2-block sample cluster came first, followed by the predominantly 3-block cluster, then the 4-block cluster. Values in parentheses are standard deviations across three runs of the experiment. As seen in the table, separation of samples into clusters is quite good, though 22.7% of 3-block samples were assigned to the predominantly 4-block cluster, and 11.5% of 2-block samples were assigned to the predominantly 3-block cluser. The sample separation evidenced in TAB0 is enough such that templates trained on the three clusters of samples reliably select deictic references that consider the correct number of blocks, i.e., the 2-block cluster reliably learns a template which considers the target object and the object above the target, and similiarly for the other clusters and their respective stack heights. However, initializing samples to belong solely to a single cluster, rather than initializing membership probabilities, is unlikely to be robust in general, so we turn to the proposed clustering-based approaches for initializing membership probabilities instead. TAB1 is analogous to TAB0 in structure, but shows sample separation for sample membership probabilities initialized to be proportional to the inverse distance from the sample to each of the cluster centers found by k-means. TAB2 is the same, except with membership probabilities initialized to be proportional to the square of the inverse distance to cluster centers. Sample separation is better in the case of squared distances than non-squared distances, but it's unclear whether this generalizes to other datasets. For our specific problem instance, the log data likelihood feature turns out to be very important for the success of these clustering-based initialization approaches. For example, Table 4 shows analogous to those in TAB2, where the only difference is that all log data likelihoods were multiplied by five before being passed as input to the k-means clustering algorithm. Comparing the two tables, this scaling of data likelihood to become relatively more important as a feature in better data separation. This suggests that the relative importance between log likelihood and the other input features is a parameter of these clustering approaches that should be tuned. Effect of object ordering on baseline performance The single-predictor baseline used in our experiments receives all objects in the scene as input, but this leaves open the question of in what order these objects should be presented. Because the templates approach has the target object of the action specified, in the interest of fairness this information is also provided to the baseline by having the target object always appear first in the ordering. As there is in general no clear ordering for the remainder of the objects, we could present them in a random order, but perhaps sorting the objects according to position (first by x-coordinate, then y, then z) could in better predictions than if objects are completely randomly ordered. Table 4: Sample separation from clustering-based initialization of membership probabilities, where probabiliites are assigned to be proportional to squared inverse distance to cluster centers, and log data likelihood feature used as part of k-means clustering has been multiplied by a factor of five. Standard deviations are reported in parentheses. To analyze the effect of object ordering on baseline performance, we run the same experiment where a push action is applied to the bottom-most of a stack of three blocks, and there exist some number of additional objects on the table that do not interfere with the action in any way. Figure 6 shows our . We test three object orderings: random ("none"), sorted according to object position ("xtheny"), and an ideal ordering where the first three objects in the ordering are exactly the three objects in the stack ordered from bottom up ("stack"). As expected, in all cases, predicted log likelihood drops as more extra blocks are added to the scene, and the random ordering performs worst while the ideal ordering performs best. Figure 6: Effect of object ordering on baseline performance, on task of pushing a stack of three blocks on a table top, where there are extra blocks on the table that do not interfere with the push.

A new approach that learns a representation for describing transition models in complex uncertaindomains using relational rules.

Represent the short summary for finding the corresponding scientific article (long)

This work aims to provide quantitative answers to the relative importance of concepts of interest via concept activation vectors (CAV). In particular, this framework enables non-machine learning experts to express concepts of interest and test hypotheses using examples (e.g., a set of pictures that illustrate  the concept). We show that CAV can be learned given a relatively small set of examples. Hypothesis testing with CAV can answer whether a particular concept (e.g., gender) is more important in predicting a given class (e.g., doctor) than other sets of concepts. Interpreting networks with CAV does not require any retraining or modification of the network.