Question: Represent the scientific article to find a short TLDR

Question: The Hamiltonian formalism plays a central role in classical and quantum physics. Hamiltonians are the main tool for modelling the continuous time evolution of systems with conserved quantities, and they come equipped with many useful properties, like time reversibility and smooth interpolation in time. These properties are important for many machine learning problems - from sequence prediction to reinforcement learning and density modelling - but are not typically provided out of the box by standard tools such as recurrent neural networks. In this paper, we introduce the Hamiltonian Generative Network (HGN), the first approach capable of consistently learning Hamiltonian dynamics from high-dimensional observations (such as images) without restrictive domain assumptions. Once trained, we can use HGN to sample new trajectories, perform rollouts both forward and backward in time, and even speed up or slow down the learned dynamics. We demonstrate how a simple modification of the network architecture turns HGN into a powerful normalising flow model, called Neural Hamiltonian Flow (NHF), that uses Hamiltonian dynamics to model expressive densities. Hence, we hope that our work serves as a first practical demonstration of the value that the Hamiltonian formalism can bring to machine learning. More and video evaluations are available at: http://tiny.cc/hgn Figure 1: The Hamiltonian manifold hypothesis: natural images lie on a low-dimensional manifold in pixel space, and natural image sequences (such as one produced by watching a two-body system, as shown in red) correspond to movement on the manifold according to Hamiltonian dynamics. Any system capable of a wide range of intelligent behaviours within a dynamic environment requires a good predictive model of the environment's dynamics. This is true for intelligence in both biological (; 2010;) and artificial (; ; ;) systems. Predicting environmental dynamics is also of fundamental importance in physics, where Hamiltonian dynamics and the structurepreserving transformations it provides have been used to unify, categorise and discover new physical entities . Hamilton's fundamental was a system of two first-order differential equations that, in a stroke, unified the predictions made by prior Newtonian and Lagrangian mechanics (Hamilton, 1834). After well over a century of development, it has proven to be essential for parsimonious descriptions of nearly all of physics. Hamilton's equations provide a way to predict a system's future behavior from its current state in phase space (that is, its position and momentum for classical Newtonian systems, and its generalized position and momentum more broadly). Hamiltonian mechanics induce dynamics with several nice properties: they are smooth, they include paths along which certain physical quantities are conserved (symmetries) and their time evolution is fully reversible. These properties are also useful for machine learning systems. For example, capturing the time-reversible dynamics of the world state might be useful for agents attempting to account for how their actions led to effects in the world; recovering an Figure 2: Hamiltonian Generative Network schematic. The encoder takes a stacked sequence of images and infers the posterior over the initial state. The state is rolled out using the learnt Hamiltonian. Note that we depict Euler updates of the state for schematic simplicity, while in practice this is done using a leapfrog integrator. For each unroll step we reconstruct the image from the position q state variables only and calculate the reconstruction error. abstract low-dimensional manifold with paths that conserve various properties is tightly connected to outstanding problems in representation learning (see e.g. for more discussion); and the ability to conserve energy is related to expressive density modelling in generative approaches . Hence, we propose a reformulation of the well-known image manifold hypothesis by extending it with a Hamiltonian assumption (illustrated in Fig. 1): natural images lie on a low-dimensional manifold embedded within a high-dimensional pixel space and natural sequences of images trace out paths on this manifold that follow the equations of an abstract Hamiltonian. Given the rich set of established tools provided by Hamiltonian descriptions of system dynamics, can we adapt these to solve outstanding machine learning problems? When it comes to adapting the Hamiltonian formalism to contemporary machine learning, two questions need to be addressed: 1) how should a system's Hamiltonian be learned from data; and 2) how should a system's abstract phase space be inferred from the high-dimensional observations typically available to machine learning systems? Note that the inferred state may need to include information about properties that play no physical role in classical mechanics but which can still affect their behavior or function, like the colour or shape of an object. The first question was recently addressed by the Hamiltonian Neural Network (HNN) approach, which was able to learn the Hamiltonian of three simple physical systems from noisy phase space observations. However, to address the second question, HNN makes assumptions that restrict it to Newtonian systems and appear to limit its ability to scale to more challenging video datasets. In this paper we introduce the first model that answers both of these questions without relying on restrictive domain assumptions. Our model, the Hamiltonian Generative Network (HGN), is a generative model that infers the abstract state from pixels and then unrolls the learned Hamiltonian following the Hamiltonian equations . We demonstrate that HGN is able to reliably learn the Hamiltonian dynamics from noisy pixel observations on four simulated physical systems: a pendulum, a mass-spring and two-and three-body systems. Our approach outperforms HNN by a significant margin. After training, we demonstrate that HGN produces meaningful samples with reversible dynamics and that the speed of rollouts can be controlled by changing the time derivative of the integrator at test time. Finally, we show that a small modification of our architecture yields a flexible, normalising flow-based generative model that respects Hamiltonian dynamics. We show that this model, which we call Neural Hamiltonian Flow (NHF), inherits the beneficial properties of the Hamiltonian formalism (including volume preservation) and is capable of expressive density modelling, while offering computational benefits over standard flow-based models. Most machine learning approaches to modeling dynamics use discrete time steps, which often in an accumulation of the approximation errors when producing rollouts and, therefore, to a fast drop in accuracy. Our approach, on the other hand, does not discretise continuous dynamics and models them directly using the Hamiltonian differential equations, which leads to slower divergence for longer rollouts. The density model version of HGN (NHF) uses the Hamiltonian dynamics as normalising flows along with a numerical integrator, making our approach somewhat related to the recently published neural ODE work ). What makes our approach different is that Hamiltonian dynamics are both invertible and volume-preserving (as discussed in Sec. 3.3), which makes our approach computationally cheaper than the alternatives and more suitable as a model of physical systems and other processes that have these properties. Also related is recent work attempting to learn a model of physical system dynamics end-to-end from image sequences using an autoencoder (de). Unlike our work, this model does not exploit Hamiltonian dynamics and is trained in a supervised or semi-supervised regime. One of the most comparable approaches to ours is the Hamiltonian Neural Network (HNN) . This work, done concurrently to ours, proposes a way to learn Hamiltonian dynamics from data by training the gradients of a neural network (obtained by backpropagation) to match the time derivative of a target system in a supervised fashion. In particular, HNN learns a differentiable function H(q,p) that maps a system's state (its position, q, and momentum, p) to a scalar quantity interpreted as the system's Hamiltonian. This model is trained so that H(p,q) satisfies the Hamiltonian equation by minimizing where the derivatives ∂H ∂q and ∂H ∂p are computed by backpropagation. Hence, this learning procedure is most directly applicable when the true state space (in canonical coordinates) and its time derivatives are known. Accordingly, in the majority of the experiments presented by the authors, the Hamiltonian was learned from the ground truth state space directly, rather than from pixel observations. The single experiment with pixel observations required a modification of the model. First, the input to the model became a concatenated, flattened pair of images o t = [x t,x t+1], which was then mapped to a low-dimensional embedding space z t = [q t,p t] using an encoder neural network. Note that the dimensionality of this embedding (z ∈ R 2 in the case of the pendulum system presented in the paper) was chosen to perfectly match the ground truth dimensionality of the phase space, which was assumed to be known a priori. This, however, is not always possible. The latent embedding was then treated as an estimate of the position and the momentum of the system depicted in the images, where the momentum was assumed to be equal to the velocity of the system -an assumption enforced by the additional constraint found necessary to encourage learning, which encouraged the time derivative of the position latent to equal the momentum latent using finite differences on the split latents: This assumption is appropriate in the case of the simple pendulum system presented in the paper, however it does not hold more generally. Note that our approach does not make any assumptions on the dimensionality of the learned phase space, or the form of the momenta coordinates, which makes our approach more general and allows it to perform well on a wider range of image domains as presented in Sec. 4. Figure 3: A: standard normalising flow, where the invertible function f i is implemented by a neural network. B: Hamiltonian flows, where the initial density is transformed using the learned Hamiltonian dynamics. Note that we depict Euler updates of the state for schematic simplicity, while in practice this is done using a leapfrog integrator. The Hamiltonian formalism describes the continuous time evolution of a system in an abstract phase space s = (q, p) ∈ R 2n, where q ∈ R n is a vector of position coordinates, and p ∈ R n is the corresponding vector of momenta. The time evolution of the system in phase space is given by the Hamiltonian equations: where the Hamiltonian H: R 2n → R maps the state s = (q,p) to a scalar representing the energy of the system. The Hamiltonian specifies a vector field over the phase space that describes all possible dynamics of the system. For example, the Hamiltonian for an undamped mass-spring system is, where m is the mass, q ∈ R 1 is its position, p ∈ R 1 is its momentum and k is the spring stiffness coefficient. The Hamiltonian can often be expressed as the sum of the kinetic T and potential V energies H = T (p)+V (q), as is the case for the mass-spring example. Identifying a system's Hamiltonian is in general a very difficult problem, requiring carefully instrumented experiments and researcher insight produced by years of training. In what follows, we describe a method for modeling a system's Hamiltonian from raw observations (such as pixels) by inferring a system's state with a generative model and rolling it out with the Hamiltonian equations. Our goal is to build a model that can learn a Hamiltonian from observations. We assume that the data } comes in the form of high-dimensional noisy observations, where each x i = G(s i) = G(q i) is a non-deterministic function of the generalised position in the phase space, and the full state is a non-deterministic function of a sequence of images since the momentum (and hence the full state) cannot in general be recovered from a single observation. Our goal is to infer the abstract state and learn the Hamiltonian dynamics in phase space by observing K motion sequences, discretised into T +1 time steps each. In the process, we also want to learn an approximation to the generative process G(s) in order to be able to move in both directions between the high dimensional observations and the low-dimensional abstract phase space. Although the Hamiltonian formalism is general and does not depend on the form of the observations, we present our model in terms of visual observations, since many known physical Hamiltonian systems, like a mass-spring system, can be easily observed visually. In this section we introduce the Hamiltonian Generative Network (HGN), a generative model that is trained to behave according to the Hamiltonian dynamics in an abstract phase space learned from raw observations of image sequences. HGN consists of three parts (see Fig. 2): an inference network, a Hamiltonian network and a decoder network, which are discussed next. The inference network takes in a sequence of images (x i 0,...x i T), concatenated along the channel dimension, and outputs a posterior over the initial state z ∼ q φ (·|x 0,...x T), corresponding to the system's coordinates in phase space at the first frame of the sequence. We parametrise q φ (z) as a diagonal Gaussian with a unit Gaussian prior p(z) = N (0, I) and optimise it using the usual reparametrisation trick . To increase the expressivity of the abstract phase Figure 4: A schematic representation of NHF which can perform expressive density modelling by using the learned Hamiltonians as normalising flows. Note that we depict Euler updates of the state for schematic simplicity, while in practice this is done using a leapfrog integrator. space s 0, we map samples from the posterior with another function s 0 = f ψ (z) to obtain the system's initial state. As mentioned in Sec. 3.1, the Hamiltonian function expects the state to be of the form s = (q,p), hence we initialise s 0 ∈ R 2n and arbitrarily assign the first half of the units to represent abstract position q and the other half to represent abstract momentum p. The Hamiltonian network is parametrised as a neural network with parameters γ that takes in the inferred abstract state and maps it to a scalar H γ (s t) ∈ R. We can use this function to do rollouts in the abstract state space using the Hamiltonian equations (Eq. 3), for example by Euler integration: In this work we assume a separable Hamiltonian, so in practice we use a more sophisticated leapfrog integrator to roll out the system, since it has better theoretical properties and in better performance in practice (see Sec. A.6 in Supplementary Materials for more details). The decoder network is a standard deconvolutional network (we use the architecture from) that takes in a low-dimensional representation vector and produces a high-dimensional pixel reconstruction. Given that each instantaneous image does not depend on the momentum information, we restrict the decoder to take only the position coordinates of the abstract state as input: The objective function. Given a sequence of T +1 images, HGN is trained to optimise the following objective: which can be seen as a temporally extended variational autoencoder (VAE) objective, consisting of a reconstruction term for each frame, and an additional term that encourages the inferred posterior to match a prior. The key difference with a standard VAE lies in how we generate rollouts -these are produced using the Hamiltonian equations of motion in learned Hamiltonian phase space. In this section, we describe how the architecture described above can be modified to produce a model for flexible density estimation. Learning computationally feasible and accurate estimates of complex densities is an open problem in generative modelling. A common idea to address this problem is to start with a simple prior distribution π(u) and then transform it into a more expressive form p(x) through a series of composable invertible transformations f i (u) called normalising flows (see Fig. 3A). Sampling can then be done according to x = f T •... •f 1 (u), where u ∼ π(·). Density evaluation, however, requires more expensive computations of both inverting the flows and calculating the determinants of their Jacobians. For a single flow step, this equates to the Pendulum 2-& 3-Body Problem Mass-spring Figure 5: Ground truth Hamiltonians and samples from generated datasets for the ideal pendulum, mass-spring, and two-and three-body systems used to train HGN.. While a lot of work has been done recently into proposing better alternatives for flow-based generative models in machine learning (; ; ; ; ; ; ; ;), none of the approaches manage to produce both sampling and density evaluation steps that are computationally scalable. The two requirements for normalising flows are that they are invertible and volume preserving, which are exactly the two properties that Hamiltonian dynamics possess. This can be seen by computing the determinant of the Jacobian of the infinitesimal transformation induced by the Hamiltonian H. By Jacobi's formula, the derivative of the determinant at the identity is the trace and: where i = j are the off-diagonal entries of the determinant of the Jacobian. Hence, in this section we describe a simple modification of HGN that allows it to act as a normalising flow. We will refer to this modification as the Neural Hamiltonian Flow (NHF) model. First, we assume that the initial state s 0 is a sample from a simple prior s 0 ∼ π 0 (·). We then chain several Hamiltonians H i to transform the sample to a new state s T = H T •...•H 1 (s 0) which corresponds to a sample from the more expressive final density s T ∼ p(x) (see Fig. 3B for an illustration of a single Hamiltonian flow). Note that unlike HGN, where the Hamiltonian dynamics are shared across time steps (a single Hamiltonian is learned and its parameters are shared across time steps of a rollout), in NHF each step of the flow (corresponding to a single time step of a rollout) can be parametrised by a different Hamiltonian. The inverse of such a Hamiltonian flow can be easily obtained by replacing dt by −dt in the Hamiltonian equations and reversing the order of the transformations, T (s T) (we will use the appropriate dt or −dt superscript from now on to make the direction of integration of the Hamiltonian dynamics more explicit). The ing density p(s T) is given by the following equation: Our proposed NHF is more computationally efficient that many other flow-based approaches, because it does not require the expensive step of calculating the trace of the Jacobian. Hence, the NHF model constitutes a more structured form of a Neural ODE flow, but with a few notable differences: (i) The Hamiltonian ODE is volume-preserving, which makes the computation of loglikelihood cheaper than for a general ODE flow. (ii) General ODE flows are only invertible in the limit dt → 0, whereas for some Hamiltonians we can use more complex integrators (like the symplectic leapfrog integrator described in Sec. A.6) that are both invertible and volume-preserving for any dt > 0. The structure s = (q,p) on the state-space imposed by the Hamiltonian dynamics can be constraining from the point of view of density estimation. We choose to use the trick proposed in the Hamiltonian Monte Carlo (HMC) literature (; ;), which treats the momentum p as a latent variable (see Fig. 4). This is an elegant solution which avoids having to artificially split the density into two disjoint sets. As a , the data density that our Hamiltonian flows are modelling becomes exclusively parametrised by p(q T), which takes the following form: T (q T,p T))dp T. This integral is intractable, but the Note that, in contrast to VAEs , the ELBO in Eq. 6 is not explicitly in the form of a reconstruction error term plus a KL term. In order to directly compare the performance of HGN to that of its closest baseline, HNN, we generated four datasets analogous to the data used in. The datasets contained observations of the time evolution of four physical systems: mass-spring, pendulum, two-and three-body (see Fig. 5). In order to generate each trajectory, we first randomly sampled an initial state, then produced a 30 step rollout following the ground truth Hamiltonian dynamics, before adding Gaussian noise with standard deviation σ 2 = 0.1 to each phase-space coordinate, and rendering a corresponding 64x64 pixel observation. We generated 50 000 train and 10 000 test trajectories for each dataset. When sampling initial states, we start by first sampling the total energy of the system denoted as a radius r in the phase space, before sampling the initial state (q,p) uniformly on the circle of radius r. Note that our pendulum dataset is more challenging than the one described in , where the pendulum had a fixed radius and was initialized at a maximum angle of 30 • from the central axis. Mass-spring. The dynamics of a frictionless mass-spring system are modeled by the Hamiltonian, where k is the spring constant and m is the mass. We fix k = 2 and m = 0.5, then sample a radius from a uniform distribution r ∼ U(0.1,1.0). Pendulum. The dynamics of a frictionless pendulum are modeled by the Hamiltonian H = 2mgl(1− cos (q))+ p 2 2ml 2, where g is the gravitational constant and l is the length of the pendulum. We fix g = 3, m = 0.5, l = 1, then sample a radius from a uniform distribution r ∼ U(1.3,2.3). Two-and three-body problems. In an n-body problem, particles interact with each other through an attractive force, like gravity. The dynamics are represented by the following Hamiltonian H = n i ||pi|| 2 2mi − 1≤i<j≤n gmimj ||qj −qi||. We set m = 1 and g = 1 for both systems. For the two-body problem, we set r ∼ U(0.5,1.5), and we also change the observation noise to σ 2 = 0.05. For the three-body problem, we set r ∼ U (0.9,1.2), and set the observation noise to σ 2 = 0.2. Learning the Hamiltonian We tested whether HGN and the HNN baseline could learn the dynamics of the four systems described above. To ensure that our re-implementation of HNN was correct, we replicated all the presented in the original paper by verifying that it could learn the dynamics of the mass-spring, pendulum and two-body systems well from the ground truth state, and the dynamics of a restricted pendulum from pixels. We also compared different modifications of HGN: a version trained and tested with an Euler rather than a leapfrog integrator (HGN Euler), a version trained with no additional function between the posterior and the prior (HGN no f ψ) and a deterministic version (HGN determ), which did not include the sampling step from the posterior q φ (z|x 0 ...x T). : the original architecture and a convolutional version closely matched to the architecture of HGN. We also compare four versions of our proposed Hamiltonian Generative Network (HGN): the full version, a version trained and tested with an Euler rather than a leapfrog integrator, a deterministic rather than a generative version, and a version of HGN with no extra network between the posterior and the initial state. Tbl. 1 and Fig. 6 demonstrate that HGN and its modifications learned well on all four datasets. However, when we attempted to train HNN on the four datasets described above, its Hamiltonian often collapsed to 0 and the model failed to reproduce any dynamics, defaulting to a static single image. We were unable to improve on this performance despite our best efforts, including a modification of the architecture to closely match ours (referred to as HNN Conv) (see Sec. A.3 of the appendix for details). Tbl. 1 shows that the average mean squared error (MSE) of the pixel reconstructions on both the train and test data is an order of magnitude better for HGN compared to both versions of HNN. The same holds when visualising the average per-frame MSE of a single train and test rollout for each dataset shown in Fig. 6. Note that the different versions of HGN have different trade-offs. The deterministic version produces more accurate reconstructions but it does not allow sampling. This effect is equivalent to a similar distinction between autoencoders and VAEs. Using the simpler Euler integrator rather than the more involved leapfrog one might be conceptually more appealing, however it does not provide the same energy conservation and reversibility properties as the leapfrog integrator, as evidenced by the increase by an order of magnitude of the variance of the learned Hamiltonian throughout a sequence rollout as shown in Tbl. 2. The full version of HGN, on the other hand, is capable of reproducing the dynamics well, is capable of producing diverse yet plausible rollout samples (Fig. 8) and its rollouts can be reversed in time, sped up or slowed down by either changing the value or the sign of dt used in the integrator (Fig. 7). Expressive density modelling using learned Hamiltonian flows We evaluate whether NHF is capable of expressive density modelling by stacking learned Hamiltonians into a series of normalising flows. Fig. 9 demonstrates that NHF can transform a simple soft-uniform prior distribution π(s 0 ;σ,β) into significantly more complex densities with arbitrary numbers of modes. The soft-uniform density, 2 )), where f is the sigmoid function and β is a constant, was chosen to make it easier to visualise the learned attractors. The model also performed well with other priors, including a Normal distribution. It is interesting to note that the trained model is very interpretable. When decomposed into the equivalents of the kinetic and potential energies, it can be MODEL MASS-SPRING PENDULUM TWO-BODY THREE-BODY TRAIN TEST TRAIN TEST TRAIN TEST TRAIN Figure 7: Example of a train and a test sequence from the dataset of a three-body system, its inferred forward, backward, double speed and half speed rollouts in time from HGN, and a forward rollout from HNN. HNN did not learn the dynamics of the system and instead learned to reconstruct an average image. Mass-spring seen that the learned potential energy V (q) learned to have several local minima, one for each mode of the data. As a consequence, the trajectory of the initial samples through the flow has attractors at the modes of the data. We have also compared the performance of NHF to that of the RNVP baseline . Fig. 10 shows that the two approaches are comparable in their performance, but NHF is more computationally efficient as discussed at the end of Sec. 3.3. The RNVPs use alternating masks with two layers (red) or 3 layers (purple). NHF uses 1, 2 or 3 leapfrog steps (blue, yellow and green respectively). We have presented HGN, the first deep learning approach capable of reliably learning Hamiltonian dynamics from pixel observations. We have evaluated our approach on four classical physical systems and demonstrated that it outperformed the only relevant baseline by a large margin. Hamiltonian dynamics have a number of useful properties that can be exploited more widely by the machine learning community. For example, the Hamiltonian induces a smooth manifold and a vector field in the abstract phase space along which certain physical quantities are conserved. The time evolution along these paths is also completely reversible. These properties can have wide implications in such areas of machine learning as reinforcement learning, representation learning and generative modelling. We have demonstrated the first step towards applying the learnt Hamiltonian dynamics as normalising flows for expressive yet computationally efficient density modelling. We hope that this work serves as the first step towards a wider adoption of the rich body of physics literature around the Hamiltonian principles in the machine learning community. A SUPPLEMENTARY MATERIALS The Hamiltonian Generative Network (HGN) consists of three major parts, an encoder, the Hamiltonian transition network and a decoder. During training the encoder starts with a sequence of raw training images and encodes it into a probabilistic prior representation transformed with an additional network on top into a start state consisting of a downsized spatial representation in latent space (4×4), where each abstract pixel is the the concatenation of abstract position (q) and momentum (p) (each of dimension 16). The encoder network is a convolutional neural network with 8 layers, with 32 filters on the first layer, then 64 filters on each subsequent layer, while in the last layer we have 48 filters. The final encoder transformer network is a convolutional neural network with 3 layers and 64 filters on each layer. Starting from this initial embedded state, the Hamiltonian transition network generates subsequent states using a symplectic integrator approximating the Hamiltonian equations. The Hamiltonian transition network represents the Hamiltonian function as a function from the abstract position and momentum space to the real numbers at any time step t. The Hamiltonian transition network is a convolutional neural network of 6 layers, each consisting of 64 filters. The discrete timestep we use for the symplectic integrator update step is dt = 0.125. At each time step t the decoder network d θ takes only the abstract position part q t of the state s t and decodes it back to an output image x t ∈ R 32×32×3 of the same shape as the input images. The decoder network is a progressive network consisting of 3 residual blocks, where each residual block resizes the current input image by a factor of 2 using the nearest neighbor method (at the end we have to upscale our latent spatial dimension of 4 to the desired output image dimension of 32 in these steps), followed by 2 blocks of a one layer convolutional neural network with 64 filters and a leaky ReLU activation function, closing by a sigmoid activation in each block. After the 3 blocks a final one layer convolutional neural network outputs the output image with the right number of channels. We use Adam optimisier with learning rate 1.5e-4. When optimising the loss, in practice we do not learn the variance of the decoder p θ (x|s) and fix it to 1, which makes the reconstruction objective equivalent to a scaled L2 loss. Furthermore, we introduce a Lagrange multiplier in front of the KL term and optimise it using the same method as in. For all NHF experiments the Hamiltonian was of the form H(q, p) = K(p) + V (q). The kinetic energy term K and the potential energy term V are soft-plus MLPs with layer-sizes [d,128,128,1] where d is the dimension of the data. Soft-plus non-linearities were chosen because the optimisation of Hamiltonian flows involves second-order derivatives of the MLPs used for parametrising the Hamiltonians. This makes ReLU non-linearities unsuitable. The encoder network was parametrized as f ψ (p|q) = N (p;µ(q),σ(q)), where µ and σ are ReLU MLPs with size [d,128,128,d]. The Hamiltonian flow H dt, was approximated using a leapfrog integrator since it preserves volume and is invertible for any dt (see also section A.6). We found that only two leapfrog steps where sufficient for all our examples. Parameters were optimised using Adam (learning rate 3e-4) and Lagrange multipliers were optimised using the same method as in. All shown kernel density estimate (KDE) plots used 1000 samples and isotropic Gaussian kernel bandwidth of 0.3. For the RNVP baseline we used alternating masks with two or three layers. Each RNVP layer used an affine coupling parametrized by a two-layer relu-MLP that matched those used in the leapfrog. The Hamiltonian Neural Network (HNN) learns a differentiable function H(q,p) that maps a system's state in phase space (its position q and momentum p) to a scalar quantity interpreted as the system's Hamiltonian. This model is trained so that H(q,p) satisfies the Hamiltonian equation by minimizing where the derivatives ∂H ∂q and ∂H ∂p are computed by backpropagation. In the original paper, these targets are either assumed to be known or are estimated by finite differences using the state at times t and t+1. Accordingly, in the majority of the experiments presented by the authors, the Hamiltonian was learned from the ground truth state space directly, rather than from pixel observations. The original HNN model is trained in a supervised fashion on the ground truth state of a physical system and its time derivatives. As such, it is not directly comparable to our method, which learns a Hamiltonian directly from pixels. Instead, we compare to the PixelHNN variant of the HNN, which is introduced in the same paper, and which is able to learn a Hamiltonian from images and in the absence of true state or time derivative in some settings. This required a modification of the model. First, the input to the model became a concatenated, flattened pair of images X t = (x t, x t+1), which was then mapped to a low-dimensional embedding space z t = (q t,p t) using an encoder neural network. Note that the dimensionality of this embedding (z ∈ R 2 in the case of the pendulum system presented in the paper) is chosen to perfectly match the ground truth dimensionality of the phase space, which was assumed to be known a priori. This, however, is not always possible, as when a system has not yet been identified. The latent embedding was then treated as an estimate of the position and the momentum of the system depicted in the images, where the momentum was assumed to be equal to the velocity of the system -an assumption enforced by the additional constraint found necessary to encourage learning, which encouraged the time derivative of the position latent to equal the momentum latent using finite differences on the split latents: This loss is motivated by the observation that in simple Newtonian systems with unit mass, the system's state is fully described by the position and its time derivative (the system's velocity). An image embedding that corresponds to the position and velocity of the system will minimize this loss. This assumption is appropriate in the case of the simple pendulum system presented in the paper, however it does not hold more generally. As mentioned earlier, PixelHNN takes as input a concatenated, flattened pair of images and maps them to an embedding space z t = (q t,p t), which is treated as an estimate of the position and momentum of the system depicted in the images. Note that X t always consists of two images in order to make the momentum observable. This embedding space is used as the input to an HNN, which is trained to learn the Hamiltonian of the system as before, but using the embedding instead of the true system state. To enable stable learning in this configuration, the PixelHNN uses a standard mean-squared error autoencoding loss: whereX t is the autoencoder output and X i t is the value of pixel i of N total pixels in X t. 2 This loss encourages the network embedding to reflect the content of the input images and to avoid the trivial solution to. The full PixelHNN loss is: where L HNN is computed using the finite difference estimate of the time derivative of the embedding. λ HNN is a Lagrange multiplier, which is set to 0.1, as in the original paper. L WD is a standard L2 weight decay and its Lagrange multiplier λ WD is set to 1e-5, as in the original paper. In the experiments presented here, we reimplemented the PixelHNN architecture as described in and trained it using the full loss. As in the original paper, we used a PixelHNN with HNN, encoder, and decoder subnetworks each parameterized by a multi-layer perceptron (MLP). The encoder and decoder MLPs use ReLU nonlinearities. Each consists of 4 layers, with 200 units in each hidden layer and an embedding of the same size as the true position and momentum of the system depicted (2 for mass-spring and pendulum, 8 for two-body, and 12 for 3-body). The HNN MLP uses tanh nonlinearities and consists of two hidden layers with 200 units and a one-dimensional output. To ensure the difference in performance between the PixelHNN and HGN are not due primarily to archiectural choices, we also compare to a variant of the PixelHNN architecture using the same convolutional encoder and decoder as used in HGN. We used identical hyperparameters to those described in section A.1. We map between the convolutional latent space used by the encoder and decoder and the vector-valued latent required by the HNN using one additional linear layer for the encoder and decoder. In the original paper, the PixelHNN model is trained using full-batch gradient descent. To make it more comparable to our approach, we train it here using stochastic gradient descent using minibatches of size 64 and around 15000 training steps. As in the original paper, we train the model using the Adam optimizer and a learning rate of 1e-3. As in the original paper, we produce rollouts of the model using a Runge-Kutta integrator (RK4). See Section A.6 for a description of RK4. Note that, as in the original paper, we use the more sophisticated algorithm implemented in scipy (scipy.integrate.solve_ivp) . The datasets for the experiments described in 4 were generated in a similar manner to for comparative purposes. All of the datasets simulate the exact Hamiltonian dynamics of the underlying differential equation using the default scipy initial value problem solver. After creating a dataset of trajectories for each system, we render those into a sequence of images. The system depicted in each dataset can be visualized by rendering circular objects: • For the mass-spring the mass object is rendered as a circle and the spring and pivot are invisible. • For the pendulum only the weight (the bob) is rendered as a circle, while the massless rod and pivot are invisible. • For the two and three body problem we render each point mass as a circle in a different color. Additionally, we smooth out the circles such that they do not have hard edges, as can be seen in Fig. 7. In order to obtain the presented in Tbl. 1 we trained both HGN and HNN for 15000 iterations, with batch size of 16 for HGN, and 64 for the HNN. Given the dataset sizes, this means that HGN was trained for around 5 epochs and HNN was trained for around 19 epochs, which took around 16 hours. Figs. 11-12 plot the convergence rates for HGN (leapfrog) and HNN (conv) on the four datasets. Mass-spring Two-Body Three-Body Throughout this paper, we estimate the future state of systems from inferred values of the system position and momentum by numerically integrated the Hamiltonian. We explore three methods of numerical integration: (i) Euler integration, (ii) Runge-Kutta integration and (iii) leapfrog integration. Euler integration estimates the value of a function at time t+dt by incrementing the function's value with the value accumulated by the function's derivative, assuming it stays constant in the interval [t,t+dt]. In the Hamiltonian framework, Euler integration takes the form: Because Euler integration estimates a function's future value by extrapolating along its first derivative, the method ignores the contribution of higher-order derivatives to the function's change in time. Accordingly, while Euler integration can reasonably estimate a function's value over short periods, its errors accumulate rapidly as it is integrated over longer periods or when it is applied multiple times. This limitation motivates the use of methods that are stable over more steps and longer integration times. One such method is four-step Runge-Kutta integration (RK4), the most widely used member of the Runge-Kutta family of integrators. Whereas Euler integration estimates the value of a function at time t+dt using only the function's derivative at time t, RK4 accumulates multiple estimates of the function's value in the interval [t,t + dt]. This integral more correctly reflects the behavior of the function in the interval, ing in a more stable estimate of the function's value. RK4 estimates the state at time t+dt as: Non-symplectic Symplectic q p dt A B Figure 13: A: example of using a symplectic (leapfrog) and a non-symplectic (Euler) integrators on the Hamiltonian of a harmonic oscillator. The blue quadrilaterals depict a volume in phase space over the course of integration. While the symplectic integrator conserves the volume of this region, but the non-symplectic integrator causes it to increase in volume with each integration step. The symplectic integrator clearly introduces less divergence in the phase space than the non-symplectic alternative over the same integration window. B: an illustration of the leapfrog updates in the phase space, where q is position and p is momentum. If we assume that the Hamiltonian equations take this form, we can integrate them using the leapfrog integrator, which in essence updates the position and momentum variables at interleaved time points in a way that resembles the updates "leapfrogging" over each other (see Fig. 13B for an illustration). In particular, the following updates can be applied to a Hamiltonian of the form H = V (q) + T (p), where V is the potential energy and T is the kinetic energy of the system: q t+dt = q t +dtp t+ dt 2. As discussed above, leapfrog integration is more stable and accurate over long rollouts than integrators like Euler or RK4. This is because the leapfrog integrator is an example of a symplectic integrator, which means it is guaranteed to preserve the special form of the Hamiltonian even after repeated application. An example visual comparison between a symplectic (leapfrog) and non-symplectic (Euler) integrator applied over the Hamiltonian for a harmonic oscilator is shown in Fig. 13A. For a more thorough discussion of the properties of leapfrog integration, see . The Hamiltonian Flow consists of two components. Firstly, it defines a density model over the joint space s T = (q T,p T) using the Hamiltonian Flow as described in section 3.3. However, we assume that the observable variable represents only q T and treat p T as a latent variable which we have to marginalize over. Since the integral is intractable using the introduced variational distribution we can derive the lower bound on the marginal likelihood:

Represent the TLDR for finding the corresponding scientific article

We introduce a class of generative models that reliably learn Hamiltonian dynamics from high-dimensional observations. The learnt Hamiltonian can be applied to sequence modeling or as a normalising flow.

We find movement in function space is not proportional to movement in parameter space during optimization. We propose a new natural-gradient style optimizer to address this.


Question: Represent the scientific article to find a short TLDR.

Question: Cloud Migration transforms customer’s data, application and services from original IT platform to one or more cloud en- vironment, with the goal of improving the performance of the IT system while reducing the IT management cost. The enterprise level Cloud Migration projects are generally com- plex, involves dynamically planning and replanning various types of transformations for up to 10k endpoints. Currently the planning and replanning in Cloud Migration are generally done manually or semi-manually with heavy dependency on the migration expert’s domain knowledge, which takes days to even weeks for each round of planning or replanning. As a , automated planning engine that is capable of gener- ating high quality migration plan in a short time is particu- larly desirable for the migration industry. In this short paper, we briefly introduce the advantages of using AI planning in Cloud Migration, a preliminary prototype, as well as the challenges the requires attention from the planning and scheduling society. Automated planning and AI planning have been investigated extensively by researchers and successfully applied in many areas for decades, for example, health care BID0, semiconductor manufacturing BID1, and aviation BID2, to name a few. Meanwhile, attracted by the promise of the scalability, flexibility and potentially lower cost of the resources, more and more enterprises are considering moving their IT infrastructure and applications to Cloud or Hybrid Cloud service platforms, which is called Cloud Migration in general (Armbrust et. al. 2010, Khajeh-Hosseini, Greenwood, and Sommerville 2010). Noticing that the discussions of using AI planning in the Cloud Migration are limited both in academia and in industry, in this short paper we identify the advantages and challenges of applying AI planning to Cloud Migration by (i) introducing Cloud Migration and its planning problem; (ii) demonstrate problem feasibility by showing a prototype AI planning model; and (iii) discuss the limits of current model and future research. Cloud Migration transforms customer's data, application and services from original IT platform, hosted on servers hosted in-house or cloud environment, to one or more cloud environment, with the goal of improving the performance of the IT system while reducing the IT management cost. Generally speaking, enterprise-level Cloud Migration is a complex and usually long running process that requires careful planning. Cloud Migration includes four major steps: Discovery, Planning, Execute and Validation BID6. In the Discovery stage, migration experts investigate the current IT system, collect data and identify customer's migration goals. Then migration experts allocate resources and schedule the executions activities, refer to as the Planning stage. Next the migration are executed as planned, which is called Execute stage. In the last Valication stage, all the applications are tested that they are running as expected on the new cloud environment. Due to the complexity of the IT system and IT infrastructure, there may not be clear boundaries between the major steps. It could happen that in the Planning stage some data inconsistency were observed and additional discoveries are performed and the migration execution needed to be re-scheduled or re-planned. Due to the complexity of migration projects, low tolerance on errors and its heavy dependence on the migration expert's domain knowledge, current practitioners mostly perform migration planning either manually. or using tools manually created runbooks (Transition Manager, Velostrata). For example, in Transition Manager, the user has to upload manually scripts, e.g. groovy scripts (The Apache Groovy Programming Language), and ask the tool to generate runbook for existing wave bundles. Meanwhile, Velostrata Manager would create a.csv format template for the user to manually put tasks in and create the runbook (Creating and modifying runbooks). These planning and replanning approaches rely heavily on the practitioner's previous migration experience and domain knowledge, hence is not scalable. With the fast evolution of computing speed and machine learning technologies, domain-independent AI planners are becoming more and more powerful (Ghallab, Nau, In a Cloud Migration planning problem, there are N assets to be migrated. Assets may communicate with each other, for example, an application reads/writes a database, hence causes dependencies between assets and enforces precedence constraints for migration tasks. For instance, if asset A depends on asset B, the migration of asset A has to be done before asset B's migration. The goal of migration planning is to allocate resources and create sequence of tasks to be executed. In the case of enterprise level migration, the execution should be performed in a limited time window to minimize potential business disruption. From application point of view, the main step in developing an AI planner based on domain-independent planner is to formulate the planning problem in Cloud Migration as a planning problem for the planner. In one of our prototype AI planner, a simplest scenario, in which only migration of physical servers and virtual machines are considered, is modeled as Domain file and Problem file using Planning Domain Definition Language (PDDL). The objects in the domain file are server and wave. Each server has is assigned a numeric value called'effort hours', which represents the cost of migrating this server. Each wave is assigned a numeric value called'effort hour limit', which enforces a capacity constraint for the number of servers to be migrated in each wave. The goal is to migrate all the servers without violating the capacity constraint. A planner supported by Metric-FF planner is developed to test the performance and a graphical UI is created for users to upload spreadsheet containing server's information BID7. In particular, translation engines are developed to generate the Domain.pddl file and Problem.pddl file automatically. When there are limited number of servers, the planner finds solution in a few seconds. However, when tested with 500 servers, the planner did not find any solution in 2 hours. Figure 1 shows an overview of the prototype planner. In , in this short paper, the Cloud Migration process is investigated and a prospective research direction is identified around using domain-independent AI planner in Cloud Migration Planning. Automate migration planning is desirable for practitioners from both the cost perspective and the quality consideration. It also brings in new research topics. Some of them are listed as following.• Optimize the modeling of migration planning problem so that the domain and problem file can be generated faster, shorten the auto-planning time.• Noticing that many top-performed planners in ICP does not support metric feature, efficient algorithms that removes the metric requirements in the resource planning part of a migration planning problem needs to be developed.• Improve planner's computational speed or develop algorithms so it can generate migration plan for thousands assets and more complicated migration scenarios.

Represent the short summary for finding the corresponding scientific article (long).

In this short paper, we briefly introduce the advantages of using AI planning in Cloud Migration, a preliminary prototype, as well as the chal- lenges the requires attention from the planning and schedul- ing society.

Introduce an approach to allow agents to learn  PPDDL action models incrementally over multiple planning problems under the framework of reinforcement learning.


Question: Represent the scientific article to find a short TLDR

Question: Unsupervised domain adaptation aims to generalize the hypothesis trained in a source domain to an unlabeled target domain. One popular approach to this problem is to learn a domain-invariant representation for both domains. In this work, we study, theoretically and empirically, the explicit effect of the embedding on generalization to the target domain. In particular, the complexity of the class of embeddings affects an upper bound on the target domain's risk. This is reflected in our experiments, too. Domain adaptation is critical in many applications where collecting large-scale supervised data is prohibitively expensive or intractable, or conditions at prediction time can change. For instance, self-driving cars must be robust to various conditions such as different weather, change of landscape and traffic. In such cases, the model learned from limited source data should ideally generalize to different target domains. Specifically, unsupervised domain adaptation aims to transfer knowledge learned from a labeled source domain to similar but completely unlabeled target domains. One popular approach to unsupervised domain adaptation is to learn domain-invariant representations BID7 BID5, by minimizing a divergence between the representations of source and target domains. The prediction function is learned on the latent space, with the aim of making it domain-independent. A series of theoretical works justifies this idea BID9 BID1 BID3.Despite the empirical success of domain-invariant representations, exactly matching the representations of source and target distribution can sometimes fail to achieve domain adaptation. For example, BID13 show that exact matching may increase target error if label distributions are different between source and target domain, and propose a new divergence metric to overcome this limitation. BID14 establish lower and upper bounds on the risk when label distributions between source and target domains differ. BID6 point out the information lost in non-invertible embeddings, and propose different generalization bounds based on the overlap of the supports of source and target distribution. In contrast to previous analyses that focus on changes in the label distributions or on joint support, we here study the effect of the complexity of the joint representation. In particular, we show a general bound on the target risk that reflects a tradeoff between the embedding complexity and the divergence of source and target in the latent representation space. In particular, a too powerful class of embedding functions can in overfitting the source data and the distribution matching, leading to arbitrarily high target risk. Hence, a restriction (taking into account assumptions about correspondences and invariances) is needed. Our experiments reflect these trends empirically, too. For simplicity, we consider binary classification with input space X ✓ R n and output space Y = {0, 1}. Define H to be the hypothesis class from X to Y. The learning algorithm obtains two datasets: labeled source data X S with distribution p S, and unlabeled target data X T with distribution p T. We will use p S and p T to denote the joint distribution on data and labels X, Y and the marginals, i.e., p S (X) and p S (Y). Unsupervised domain adaptation seeks a hypothesis h 2 H that minimizes the risk in the target domain measured by a loss function`(here, zero-one loss): DISPLAYFORM0 We will not assume common support in source and target domain, in line with standard benchmarks for domain adaptation such as adapting from MNIST to M-MNIST. A common approach to domain adaptation is to learn a joint embedding of source and target data BID5 BID12. The idea is that aligning source and target distributions in this latent space Z in a domaininvariant representation, and hence a subsequent classifier f from the embedding to Y will generalize from source to target. Formally, this in the following objective function on the hypothesis h = fg:= f g, where G is the class of embedding functions to Z, and we minimize a divergence d between the distributions p S (Z g) = p S (g(X)), p T (Z g) of source and target after mapping to Z: DISPLAYFORM0 The divergence d could be, e.g., the Jensen-Shannon BID5 or Wasserstein distance BID11. introduced the H H-divergence to bound the worst-case loss from extrapolating between ] be the expected disagreement between two hypotheses, then the H H-divergence is defined as follows. Definition 1. (H H-divergence) Given two domain distributions p S and p T over X, and a hypothesis class H, the DISPLAYFORM0 This divergence allows to bound the risk on the target domain: Theorem 1. For all hypotheses h 2 H, the target risk is bounded as DISPLAYFORM1 where H is the best joint risk DISPLAYFORM2 Similar have been obtained for continuous labels BID3 BID9 ).Theorem 1 is an influential theoretical in unsupervised domain adaptation, and motivated work on domain invariant representations. For example, recent work BID5; BID6 ) applied Theorem 1 to the hypothesis space F that maps the representation space Z induced by an encoder g to the output space: DISPLAYFORM3 where F (g) is the best hypothesis risk with fixed g, i.e., DISPLAYFORM4 The F F divergence implicitly depends on the fixed g and can be small if g provides a suitable representation. However, if g induces a wrong alignment, then the best hypothesis risk F (g) is large with any function class F. The following example will illustrate such a situation, motivating to explicitly take the class of embeddings into account when bounding the target risk. We begin with an illustrative toy example. FIG1 shows a binary classification problem in 2D with disjoint support and a slight shift in the label distributions from source to target: p S (y = 1) = p T (y = 1) + 2✏. Assume the representation space is one dimensional, so the embedding g is a function from 2D to 1D. If we allow arbitrary, nonlinear embeddings, then, for instance, the embedding shown in FIG1 (b), together with an optimal predictor, achieves zero source loss and a zero divergence, and is hence optimal according to the objective. However, the target risk of this combination of embedding and predictor is maximal: R T (fg) = 1.If we restrict the class G of embeddings to linear maps g(x) = Wx where W 2 IR 1⇥2, then the embeddings that are optimal with respect to the objective are of the form W = ⇥ a, 0 ⇤. Together with an optimal source classifier f, they achieve a non-zero value of 2✏ for objective due to the shift in class distributions. However, these embeddings retain label correspondences, and can lead to a zero target risk. This example illustrates that a too rich class of embeddings can "overfit" the alignment, and hence lead to arbitrarily bad solutions. Hence, the complexity of the encoder class plays an important role in learning domain invariant representation too. Motivated by the above example, we next expose how the bound on the target risk depends on the complexity of the embedding class. To do so, we apply Theorem 1 to the hypothesis h = fg: DISPLAYFORM0 Comparing the bound to the previous bound, we notice two differences: the best in-class joint risk now minimizes over both F and G, i.e., DISPLAYFORM1 which is smaller than Fg and reflects the fact that we are learning both f and g. In return, the divergence term d FG FG (p S, p T) becomes larger than the one in bound.To better understand these tradeoffs, we derive a more interpetable form of the bound on the target risk. Before presenting the bound, we define an extended version of DISPLAYFORM2 For two domain distributions p S and p T over X, an encoder class G, and predictor class F, the F G G -divergence between p S and p T is DISPLAYFORM3 Note that the F G G -divergence is strictly smaller than the FG FG-divergence, since the two hypotheses in the supremum, fg and fg 0, share the same predictor f. We are ready to state the following . Theorem 2. For all f 2 F and g 2 G, DISPLAYFORM4 where FG (g) is the best in-class joint risk defined as DISPLAYFORM5 A detailed proof of the theorem may be found in the Appendix. The first term of the bound is the source risk. The second term i is the F F-divergence between the distributions p S (Z g) and p T (Z g) in the representation space; this also appears in the previous bound. The first term in ii measure the F G G -divergence between source and target distribution, which may decrease as the complexity of the encoder decreases. However, a less complex encoder class G can lead to increasing the best hypothesis risk FG (g). Therefore, ii makes a trade-off explicit between the divergence and the model complexity. Note that, as opposed to different FG, FG (g) also measures the correctness of the encoder in the source domain. If the encoder fails to provide informative representations in the source domain, then first term in FG (g) can be large. The last two terms in Theorem 1 express a similar complexity trade-off as ii, but this time with repect to the hypothesis class H, which here combines the encoder and predictor. Hence, directly applying Theorem 1 to the composition (Equation) treats both jointly and does not make the role of the embedding as explicit as Theorem 2. For example, Theorem 2 shows that we can also make the bound tighter by minimizing the divergence between the corresponding distributions in the embedding space, as long as the encoder provides useful representations in the source domain. If i is sufficiently small, the FG FG-divergence reduces to the F G G -divergence, which is strictly smaller than the FG FG-divergence. Comparing to the previous bound in Equation FORMULA6, which assumes a fixed g, we do not assume a known encoder and instead quantify the effect of the encoder family. Moreover, the term F (g) in bound involves the source and target risk, whereas in FG (g) the encoder g only affects the source risk, which can be estimated empirically. Importantly, without restricting the complexity of the encoder or embedding, the F G G -divergence can be large, indicating that the target risk may be large too. This suggests that restricting the model complexity of the embedding is crucial for domain invariant representation learning. To reduce the worst case divergence i, we need to restrict the encoder family to those that can approximately minimize i, in coordination with the predictor class F. Practically, we can optimize the original objective of domain invariant representations in Equation 2 to align the latent distributions. Term ii implies that we should choose the minimal complexity encoder class G that is is still expressive enough to encode the data from both domains. Practically, this can be done by regularizing the encoder, e.g., restricting Lipschitz constants or norms of weight matrices. More explicitly, one may limit the number of layers of a neural network, or apply inductive biases via selecting network architectures. For instance, comparing to fully connected networks (FCs), convolutional neural networks (CNNs) restrict the output to be spatially consistent with respect to the input. Next, we empirically test Theorem 2 via one example of domain-invariant representations: Domain-Adversarial Neural Networks (DANN) BID5, which measure the latent divergence via a domain discriminator (JensenShannon divergence). We use the standard benchmark MNIST! MNIST-M (Ganin & Lempitsky FORMULA0, where the task is to classify unlabeled handwritten digits overlayed with random photographs (MNIST-M) based on labeled images of digits alone (MNIST). We consider two categories of complexity: number of layers and inductive bias (CNN). To analyze the effect of the encoder's complexity, we augment the original two-layer CNN encoders with 1 to 5 additional CNN layers, leaving other settings unchanged. We retrain each model for 5 times and plot the mean and standard deviation of target error with respect to the number of layers in FIG2 (a): Initially, the target error decreases, and then increases when more layers are added. This corroborates our theory: the CNN encoder without additional layers does not have enough expressive power. As a consequence, the best hypothesis risk term FG is larger. However, when more layers are added, the complexity increases and subsequently makes the disagreements larger. To investigate the importance of inductive bias in domain invariant representations, we replace the CNN encoder with an MLP encoder. The experimental are shown in 2(b). Comparing the target error between (a) and (b) in FIG2, we can see that the target error with an MLP encoder is significantly higher than with a CNN encoder. Comparing to CNNs, which encode invariance via pooling and learned filters, MLPs do not have any inductive bias and lead to worse performance. In fact, the target error with MLP-based domain adaptation is higher than just training on the source, suggesting that, without an appropriate inductive bias, learning domain invariant representations can even worsen the performance. To gain deeper insight, we use t-SNE BID8 to visualize source and target embedding distributions in FIG2 (c),(d). With the inductive bias of CNNs, the representations of the target domain aligns well with those of source domain. In contrast, the MLP encoder in a strong label mismatch. The experiments show that the complexity of the encoder can have a direct effect on the target error. A more complex encoder class leads to larger theoretical bound on the target error, and, indeed, aligned with the theory, we see a significant performance drop in target domain. Moreover, the experiments suggest that inductive bias is important too. With a suitable inductive bias such as CNNs, DANN achieves higher performance than the with the MLP encoder, even if the CNN encoder has twice the number of layers. CNNs are standard for many vision tasks, such as digit recognition. However, explicit supervision may be required to identify the encoder class when we have less prior knowledge about the task BID10 BID2. In this work, we study the role of embedding complexity for domain-invariant representations. We theoretically and empirically show that restricting the encoder is necessary for successful adaptation, a fact that has mostly been overlooked by previous work. In fact, without carefully selecting the encoder class, learning domain invariant representations might even harm the performance. Our observations motivate future research on identifying eappropriate encoder classes for various tasks. Theorem 2. For all f 2 F and g 2 G, DISPLAYFORM0 where FG (g) is the best in-class joint risk defined as DISPLAYFORM1 Proof. We first define the optimal composition hypothesis f? g? with respect to an encoder g to be the hypothesis which minimizes the following error DISPLAYFORM2 By the triangle inequality for classification error ), DISPLAYFORM3 The second term in the R.H.S of Eq. 4 can be bounded as DISPLAYFORM4 The third term in the R.H.S of Eq. 4 can be bounded as DISPLAYFORM5 Combine the above bounds, we have DISPLAYFORM6 where DISPLAYFORM7

A general upper bound on the target domain's risk that reflects the role of embedding-complexity.

Represent this text to find a much much longer article about it with say hundreds-thousands of words.

We show that posterior collapse in linear VAEs is caused entirely by marginal log-likelihood (not ELBO). Experiments on deep VAEs suggest a similar phenomenon is at play.


Question: Represent the scientific article to find a short TLDR

Question: The domain of time-series forecasting has been extensively studied because it is of fundamental importance in many real-life applications. Weather prediction, traffic flow forecasting or sales are compelling examples of sequential phenomena. Predictive models generally make use of the relations between past and future values. However, in the case of stationary time-series, observed values also drastically depend on a number of exogenous features that can be used to improve forecasting quality. In this work, we propose a change of paradigm which consists in learning such features in embeddings vectors within recurrent neural networks. We apply our framework to forecast smart cards tap-in logs in the Parisian subway network. Results show that context-embedded models perform quantitatively better in one-step ahead and multi-step ahead forecasting. Classical statistical forecasting methods rely on the existence of temporal correlation between past and future values. In particular, the auto-regressive component of ARIMA estimators BID0 ) models the relation between past and future as a linear regression. In the deep learning paradigm, Recurrent Neural Networks have long been used to tackle sequential problems. Increasingly complex models such as ConvLSTM BID13 ) or Graph Neural Networks ) are developed to model multivariate phenomena and allow a precise modeling of the temporal dynamics. However, exogenous factors can greatly influence the observed values and are not taken into account by the mentioned models. For example, the type of road can drastically change traffic flow predictions, the period of the year will determine the values of sales time-series, and so on. In this work, we refer to these features as contextual information, or context. Such context is naturally used when dealing with stationary time-series to construct baselines based on the average of past values given a context. NARX models and their neural networks variations also make use of context by inputting it jointly with previous values of the forecast variable BID17 ).Similar to how Graph NN learn relations between nodes, we propose for multivariate stationary timeseries to learn context within a recurrent architecture and we introduce context-embedded RNN. For each contextual feature, we concatenate to the observed value an embedding that is to be learned jointly with the weights of the network. We do not deal with the case of continuous features but these could be transformed into categories. We tested our framework on public transportation tap-in logs one-step ahead and multi-step ahead forecasting, where we consider spatial context in the form of subway stations and temporal context through day of the week and time of the day. To the best of our knowledge, there exists no good-quality public dataset containing subway logs at a satisfying granularity. We realized experiments on data provided by Ile-de-France Mobilités 1 (Parisian region public transportation agency) but we expect that the fast development of data collection in this domain will entail the availability of public datasets in a near future. On the other hand, all of the source code used to realize the experiments is available on https://github.com/XXXX.Results of the experiments show that contextual models consistently outperform other recurrent models as well as the historical average baseline which is especially strong in the case of stationary 1 https://www.iledefrance-mobilites.fr/ time-series. Contextual models perform particularly well for long-term forecasting. In summary, in this paper we propose a new paradigm for learning contextual information within RNNs, which quantitatively improves forecasting performance by allowing a fine-grained modeling of local dynamics. The remainder of this paper is organized as follows: in time-series forecasting and use of context is presented in Section 2; proposed models are introduced in Section 3 and are tested in prediction experiments in Section 4. Time-series forecasting When it comes to time-series forecasting, the classical methods rely on ARMA models BID0 ) and their variants ARIMA for non-stationary series or SARIMA in the case of seasonality. However, RNNs have long been used for this task BID3 ) and perform well on a variety of applications BID20 ). They are now employed to model more complex data. For instance, spatio-temporal data, which are similar to the application studied in this work, can be dealt with using a combination of CNN and RNN as in BID13. More generally, it is viewed as a graph problem in many works;; BID4; BID21 ). In particular, applied to traffic forecasting, BID4 learn weighted convolutional features representing the relations between each node of the graph. These features are then processed by a LSTM. While we could deal with the use case of transportation logs forecasting with such Graph NN, we choose to develop a more general framework where we learn peculiarities of each location instead of the relations between them. Contextual information Jointly with complex architectures, contextual features can be used to improve forecasting performance. In an early work, BID15 develop the KARIMA algorithm. It uses a Kohonen neural network to cluster data based on present and past observations, but also time-step and day of the week. Then an ARIMA is used to predict the next value. More recently BID18 and BID5 use additional temporal features in LSTM and gradient boosting decision trees respectively. In general, predictive models with exogenous features belong to the class of NARX models such as BID6 which forecast groundwater level based on precipitation or BID8 where building heat load depends on many features. A different method is adopted by BID10 in the prediction of the next location problem. They replace the weight matrix multiplied by the input of a RNN by transition matrices representing spatial and temporal information. In this work, we choose to let the neural network learn its representation of the contextual features. Public transportation data We apply our models on public transportation data, a domain which has not been as extensively studied as traffic forecasting because of the late apparition of data. BID19 combine SVM and Kalman filters to predict bus arrival times while BID1 only consider historical average for tap-in and tap-out forecasting. Closer to our work, BID14 use LSTM networks for tap-in data in the Parisian area. However, they do not use context in the proposed models, whether spatial context because they study a small business zone, or temporal context. We describe notations for the considered transportation problem but the developed ideas can be extended to other context-dependent forecasting problems. A particularity of the data is the discontinuity caused by the closure of the subway network every night (as mentioned in BID1 ; BID12). Therefore the observations for each day form a multivariate time-series containing the number of passengers entering the transportation network. Data is processed in the form of a 3D tensor X ∈ R N ×S×T, with N the number of days, S the number subway stations and T the number of time-steps. In particular, for a station s, X s = X:,s,: ∈ R N ×T contains all the values for a specific location. We also denote x s = X d,s,: ∈ R T the vector of values for a day d and station s and x t = X d,:,t ∈ R S the values for a day d at time t. In the recurrent models, the hidden state at time t of size h will be noted h t ∈ R h, or h s t when it represents a single location s. We will also introduce embeddings vectors for spatial location z s, day of the week z d and time-step z t whose sizes are respectively λ s, λ d and λ t. Recurrent neural networks are a natural choice for forecasting time-series as they encode a latent state for each time step which is used to predict the next value. These architectures can model the dynamics of time-series and have a memory allowing the use of several observations in the past. In particular, they may be able to adapt themselves to anomalous behaviors, making them more robust. We propose three recurrent architectures, introducing three different ways of dealing with spatial context. Two of them model it implicitly while the third one explicitly learns it. Each architecture is composed of a recurrent encoder E transforming the observations into hidden latent states. These states are then decoded into predictions using a linear layer D. Each of the models can then be completed with temporal context. Univariate RNN First of all, we consider each station separately. That is, we explicitly train S distinct RNNs over as many matrices of samples X s ∈ R N ×T. In this case the input dimension of each RNN is 1, i.e. we compute p(x for each t which is decoded into the prediction by D s . DISPLAYFORM0 Multivariate RNN In this model we consider that each sample represents a single day over the whole network and is a multi-dimensional series X d ∈ R S×T . This representation assumes a correlation between the values of the different stations at each time t. In this setting we compute p(x t+1 |x t, ..., x 0). This is similar to spatio-temporal models, but here the relations are not specified and the network must discover them during training. At time t the vector sample x t ∈ R S represents the observed state of the subway network, which is combined with the current hidden state h t ∈ R h by the recurrent encoder to compute the next hidden state. During this stage, the recurrent encoder E has used several layers to combine past and current information into a synthetic state which is decoded back to S predictions by D (see Equation 2 and Figure 2). At the end of the day, this architecture captures the dynamics and the correlation of the entire network. Spatial context is not explicitly specified but included in the weights of the network. This second model offers a natural way to deal with multivariate series. However, because of the large number of relations to learn between stations compared to the scarcity of samples, it may face overfitting and perform poorly. DISPLAYFORM1 Spatial RNN Finally, we propose a hybrid architecture which mixes the univariate and multivariate ones. As with the Univariate RNN we consider N * S samples x s ∈ R T that are encoded into a singular hidden state. However, there is a single couple (E, D) shared across all the stations -as in the Multivariate RNN -that allows to take into account the correlations between the stations and greatly reduces the number of weights. This time, spatial context is explicitly learned in the form of a matrix of spatial embeddings Z S ∈ R S×λs, hence the name Spatial RNN. For each station s, the corresponding embedding z s is concatenated to the observation as in FIG3 where c is the concatenation operation. At time step t, for a station s, the observation x s t ∈ R is concatenated to the embedding z s ∈ R λs. The ing vector and hidden state h s t ∈ R h are encoded via the common encoder E into a hidden state representing only this station. This state is then decoded into a single-valued predictionx DISPLAYFORM2 Observation PredictionLinear decoder D Recurrent encoder E T are concatenated with a vector of embeddings z s ∈ R λs and then processed by a recurrent encoder which computes a hidden state h s t+1 ∈ R h for each t. This state is then decoded into a single prediction. In addition to directly learning spatial context, this architecture offers the possibility to scale to a network of thousands or tens of thousands of stations because the number of recurrent weights is fixed. More generally, learning embeddings greatly helps to reduce dimensionality when dealing with a large number of contextual cases, compared to NARX models. We proposed three different ways to deal with spatial context, one of them being to learn it. A promising way to improve performance is to introduce temporal context in the models. We consider two distinct time scales for temporal context, that are the days of the week and the time of the day. Indeed, the number of logs during one day at a specific time step is expected to be the same from one week to another. We wish to see if the model is able to learn and discover meaningful representations of such temporal entities. Therefore, for each recurrent architectures we add the possibility to concatenate temporal embeddings to the observations. It is noteworthy that the temporal embeddings are shared across every networks i.e. there is one set of embeddings for the entire Univariate architecture, and not one different set per station. Similarly to the way we dealt with spatial context, we could design multivariate and univariate architectures for days and time-steps. However we lack data to learn such models and the overfitting risk would be especially high for the day of the week scale. Day embeddings We first introduce embeddings corresponding to the day of the week, via a matrix (z d) d={1,..,7} ∈ R 7×λ d containing 7 different embeddings. Because we focus on fully-contextual models we only present in Equation 4 the prediction in the Spatial case, but temporal embeddings can be used for the other architectures as well. DISPLAYFORM0 Time-step embeddings Similarly, the number of logs is very dependent on the time of the day, with notable morning and evening peak hours separated by off-peak time. Therefore we learn a matrix of embeddings (z t) t={1,..,T −1} ∈ R T −1×λt. Prediction in the Spatial case is presented in Equation 5. DISPLAYFORM1 These embeddings can be learned using each of the architectures presented before and the two types of temporal embeddings can obviously be combined. An illustration for the Spatial model with day and time embeddings is presented in FIG4.......: Computing predictions for a particular station s using the spatial architecture with temporal context. Given a day d, at each step t, the observed value x s t is concatenated with three embeddings representing the station, the day and the time, respectively z s ∈ R λs, z d ∈ R λ d and z t ∈ R λt. The obtained vector is processed by a recurrent encoder E (common to all stations) to compute a hidden state h s t+1. Finally this vector is decoded into a single predictionx s t+1 We train our models on a data set provided by Ile-de-France Mobilites (the transport agency of the Parisian region). It contains 256,028,548 logs (user, station, time) between October and Decem-ber 2015 across 300 subway stations. We aggregate the logs by windows of 15 minutes, in order to reduce inherent noise in the data and allow tractable computation. From the data set we remove 3 stations which were undergoing planned works during the period. We also pull out 15 days with disturbed traffic pattern and that can be considered as anomalies. Finally we have S = 297 stations and N = 77 days. Those days are split into 70% train and 30% test samples. Splits are stratified with regards to the day of week, meaning that we try to have as many Sundays in train and test splits. In addition, 15% of the train split is kept for validation. In the end, there are 45 days in train split, 8 in validation and 24 in test. Scaling The transportation network is comprised of few hubs and many less crowded stations. The existence of utterly different scales complicates the prediction problem, especially in the multivariate setting. To tackle this problem we rescale the data set, considering each station separately. Note that this also simplifies gradient computation. In more details we apply a two-step procedure:• First, for a station s we calculate the 99.9th percentile and replace all superior or equal values by this bound. This removes the local outliers we missed when skipping some days.• Then the values of s are scaled between -1 and 1 by min-max scaling. Treating each station one bye one prevents more important stations to squeeze the values of minor ones. For these two steps, the percentile and the scaling values are computed on the train set and then applied on the other sets. In this work we use vanilla RNN as well as Gated Recurrent Units networks BID2 for the encoder. Models are trained with pytorch BID11 ) on GPU using the well-known optimizer Adam (Kingma & Ba FORMULA0) with a learning rate of 0.0001 and Mean Squared Error (MSE).To select the best hyperparameters and epoch during training we monitor root MSE applied on descaled predictions of the validation set. Hyperparameters are presented in TAB1 and we use λ s = 80, λ d = 4 and λ t = 30 for embeddings' sizes. Experiments are run with 5 different random seeds to compute standard deviation of the error. A strong baseline is constructed by averaging previous values given their context. Dealing with a similar application of tap-in logs forecasting, BID12 propose a Bayesian network but performs slightly worse than the average baseline. Indeed, the considered series are very stationary and heavily depend on the context. The baseline model is a tensor of predictions of size 7×S ×T, where the first dimension corresponds to each day of the week. For a specific day d, station s and time-step t, the average baseline is equal DISPLAYFORM0, D being a look-up table from date stamp to day. This model is only based on domain expert knowledge and contextual information. Unlike machine learning models, it cannot adapt to anomalous behaviors but it is context aware. RMSE of the different architectures before the addition of temporal context are presented in TAB2. All recurrent models, using RNN or GRU, significantly outperform the baseline. In particular, we check in FIG5 that the models learn more than the average behavior by plotting predictions during November 4th. An anomaly seems to occur during the day, disturbing the baseline while recurrent models precisely fit to the unusually low traffic. This means that the proposed models learned the dynamics of the time-series and are robust to unseen values. is not a particular day by itself but an anomaly seems to have happened. The baseline mispredicts while our recurrent models correctly fit with the ground truth. We assumed that it would be beneficial to combine the dynamics of recurrent models with the temporal contextual information used in the baseline. To that end we learned day and time embeddings within the previous models and present the in TAB3. Since RNN and GRU performed similarly we chose to display only GRU . The first column corresponds to the previous . With the exception of day embeddings for Multivariate and Univariate GRU, the addition of temporal context benefits all models. Interestingly, the combination of time and day embeddings for these two architectures is better than time embeddings alone. On the opposite, the Spatial model benefits more from day embeddings. In the previous experiments we focused on predicting one value from each observation. However, we would like our model to deliver predictions in a window wider than 15 minutes. Therefore, for each step t, after the model has generated prediction t + 1, we feed it with this prediction in order to get value at t + 2, etc. Obviously the errors made at a previous step will propagate and the prediction will degrade, ing in a increase in loss. In FIG6 we plot this loss evolution against the number of time-steps predicted. We find that the addition of temporal embeddings noticeably improves the quality of predictions until a further horizon. While vanilla models perform similarly or worse than the baseline after one hour of predictions, augmented models adopt a concave curve deteriorate much slower. In particular, the addition of temporal embedding to the Spatial model allows to double the horizon during which it beats the baseline. As a second evidence that temporal context is especially useful when predicting farther in the future, we input p observed values to the model to compute a starting hidden state h p and then feed it only with its own predictions. Results of this experiment are presented in Figure 7 for p = 16, i.e. we input values until 8AM. Figure 7a shows, for each time-step starting from 8AM, the difference between RMSE for the baseline and three recurrent models, averaged on the test set. We observe that the vanilla Multivariate model performs significantly worse than the baseline as the day progresses, especially during peak hours. On the other hand, temporal models tend to converge to the average model. Indeed, when predicting long term sequences, the historical mean is the best estimator in the least square sense. Therefore, spatial and temporal context allow the Day & Time Spatial GRU to predict as well as the baseline with very partial information. Besides, as seen in FIG6, it is even better for around one hour after the last observed value was inputted. In Figure 7b, the new protocol is applied to the same disrupted sample as in FIG5 and in this particular case, the baseline is not a good estimator. On the opposite, contextual models are able to detect from the first 4 hours of information that the traffic is disrupted and that they should diverge from the baseline. Even in this unusual case, temporal context entails competitive long-term predictions. In this paper we presented a novel idea for time-series forecasting with contextual features. It consists in learning contextual information that strongly conditions the observed phenomenom within a recurrent neural network. We applied this general idea to the concrete case of transportation logs forecasting in the subway and observed significant improvement of the prediction error when using spatial and temporal context. In particular, the proposed framework performs significantly better in one-step ahead prediction and remains competitive in long-term forecasting. In a very applicated perspective, robust recurrent models could be used in the case of anomalies to accurately predict traffic recovery and help users adapt their behavior. Figure 7: Predictions for the test set are computed using only he 16th first values of each day, i.e. until 8AM and we plot: (a) the average RMSE difference between the baseline and some proposed models for every time-step. 0 corresponds to the baseline performance & (b) the predicted logs for the same day and place than in FIG5.

In order to forecast multivariate stationary time-series we learn embeddings containing contextual features within a RNN; we apply the framework on public transportation data

This work aims to provide quantitative answers to the relative importance of concepts of interest via concept activation vectors (CAV). In particular, this framework enables non-machine learning experts to express concepts of interest and test hypotheses using examples (e.g., a set of pictures that illustrate  the concept). We show that CAV can be learned given a relatively small set of examples. Hypothesis testing with CAV can answer whether a particular concept (e.g., gender) is more important in predicting a given class (e.g., doctor) than other sets of concepts. Interpreting networks with CAV does not require any retraining or modification of the network.