Question: Represent the scientific article to find a short TLDR.

Question: In many applications, it is desirable to extract only the relevant information from complex input data, which involves making a decision about which input features are relevant. The information bottleneck method formalizes this as an information-theoretic optimization problem by maintaining an optimal tradeoff between compression (throwing away irrelevant input information), and predicting the target. In many problem settings, including the reinforcement learning problems we consider in this work, we might prefer to compress only part of the input. This is typically the case when we have a standard conditioning input, such as a state observation, and a ``privileged'' input, which might correspond to the goal of a task, the output of a costly planning algorithm, or communication with another agent. In such cases, we might prefer to compress the privileged input, either to achieve better generalization (e.g., with respect to goals) or to minimize access to costly information (e.g., in the case of communication). Practical implementations of the information bottleneck based on variational inference require access to the privileged input in order to compute the bottleneck variable, so although they perform compression, this compression operation itself needs unrestricted, lossless access. In this work, we propose the variational bandwidth bottleneck, which decides for each example on the estimated value of the privileged information before seeing it, i.e., only based on the standard input, and then accordingly chooses stochastically, whether to access the privileged input or not. We formulate a tractable approximation to this framework and demonstrate in a series of reinforcement learning experiments that it can improve generalization and reduce access to computationally costly information. A model that generalizes effectively should be able to pick up on relevant cues in the input while ignoring irrelevant distractors. For example, if one want to cross the street, one should only pay attention to the positions and velocities of the cars, disregarding their color. The information bottleneck formalizes this in terms of minimizing the mutual information between the bottleneck representation layer with the input, while maximizing its mutual information with the correct output. This type of input compression can improve generalization , and has recently been extended to deep parametric models, such as neural networks where it has been shown to improve generalization . The information bottleneck is generally intractable, but can be approximated using variational inference . This variational approach parameterizes the information bottleneck model using a neural network (i.e., an encoder). While the variational bound makes it feasible to train (approximate) information bottleneck layers with deep neural networks, the encoder in these networks -the layer that predicts the bottleneck variable distribution conditioned on the input -must still process the full input, before it is compressed and irrelevant information is removed. The encoder itself can therefore fail to generalize, and although the information bottleneck minimizes mutual information with the input on the training data, it might not compress successfully on new inputs. To The information bottleneck (IB) objective is formulated as the maximization of I(Z; Y) − βI(Z; X), where X refers to the input signal, Y refers to the target signal, Z refers to the compressed representation of X, and β controls the trade-off between compression and prediction. The IB has its roots in channel coding, where a compression metric I(Z; X) represents the capacity of the communication channel between Z and X. Assuming a prior distribution r(Z) over the random variable Z, constraining the channel capacity corresponds to limiting the information by which the posterior p(Z|X) is permitted to differ from the prior r(Z). This difference can be measured using the Kullback-Leibler (KL) divergence, such that D KL (p(Z|X) r(Z)) refers to the channel capacity. Now, we write the equations for the variational information bottleneck, where the bottleneck is learnt on both the standard input S as well as a privileged input G. The Data Processing Inequality (DPI) for a Markov chain x → z → y ensures that I(x; z) ≥ I(x; y). Hence for a bottleneck where the input is comprised of both the standard input as well as privileged input, we have I(Z; G|S) ≥ I(Y ; G|S). To obtain an upper bound on I(Z; G|S), we must first obtain an upper bound on I(Z; G|S = s), and then average over p(s). We get the following : We ask the reader to refer to the section on the conditional bottleneck in the supplementary material for the full derivation. The variational bandwidth bottleneck: Based on the standard input S, the channel capacity network determines the capacity of the bottleneck Z. The channel capacity then determines the probability of accessing the privileged input. In the event that the privileged input is not accessed, no part of the model actually reads its value. We now introduce our proposed method, the variational bandwidth bottleneck (VBB). The goal of the variational bandwidth bottleneck is to avoid accessing the privileged input G if it is not required to make an informed decision about the output Y. This means that the decision about whether or not to access G must be made only on the basis of the standard input S. The standard input is used to determine a channel capacity, d cap, which controls how much information about G is available to compute Z. If d cap denotes the channel capacity, one way to satisfy this channel capacity is to access the input losslessly with probability d cap, and otherwise send no information about the input at all. In this communication strategy, we have p(Z|S, G) = δ(f enc (S, G)) if we choose to access the privileged input (with probability d cap), where f enc (S, G) is a deterministic encoder, and δ denotes the Dirac delta function. The full posterior distribution p(Z|S, G) over the compressed representation can be written as a weighted mixture of (a) (deterministically) accessing the privileged input and standard input and (b) sampling from the prior (when channel capacity is low), such that z is sampled using This modified distribution p(Z|S, G) allows us to dynamically adjusts how much information about G is transmitted through Z. As shown in the Figure 1, if d cap is set to zero, Z is simply sampled from the prior and contains no information about G. If it is set to one, the privileged information in G is deterministically transmitted. The amount of information about G that is transmitted is therefore determined by d cap, which will depend only on the standard input S. This means that the model must decide how much information about the privileged input is required before accessing it. Optimizing the information bottleneck objective with this type of bottleneck requires computing gradients through the term D KL (p(Z|S, G) r(Z)) (as in Eq. 1), where z ∼ p(Z|S, G) is sampled as in Eq. 2. The non-differentiable binary event, whose probability is represented by d cap, precludes us from differentiating through the channel capacity directly. In the next sections, we will first show that this mixture can be used within a variational approximation to the information bottleneck, and then describe a practical approximation that allows us to train the model with standard backpropagation. In this section, we show how we can evaluate the channel capacity in a tractable way. We learn a deterministic function B(S) of the standard input S which determines channel capacity. This function outputs a scalar value for d cap ∈, which is treated as the probability of accessing the information about the privileged input. This deterministic function B(S) is parameterized as a neural network. We then access the privileged input with probability d cap = B(S). Hence, the ing distribution over Z is a weighted mixture of accessing the privileged input f enc (S, G) with probability d cap and sampling from the prior with probability 1 − d cap. At inference time, using d cap, we sample from the Bernoulli distribution b ∼ Bernoulli(d prob) to decide whether to access the privileged input or not. Here, we show the KL objective which allows for tractable optimization of D KL (p(Z|S, G) r(Z)) (as in Eq. 2, 1). Proposition 1 Given the standard input s, privileged input g, bottleneck variable z, and a deterministic encoder f enc (s, g), we can express the D KL between the weighed mixture and the prior as The proof is given in the section "Tractable Optimization of the KL Objective" in the supplementary appendix. This equation is fully differentiable with respect to the parameters of f (g, s) and B(s) = d cap, making it feasible to use standard gradient-based optimizers. Summary: As in Eq. 2, we approximate p(Z|S, G) as a weighted mixture of f enc (S, G) and the normal prior, such that can be seen as a bound on the information bottleneck objective. When we access the privileged input G, we pay a cost equal to I(Z, G|S), which is bounded by D KL (p(Z|S, G) r(Z)) as in Eq. 1. Hence, optimizing this objective causes the model to avoid accessing the privileged input when it is not necessary. In order to show how the proposed model can be implemented, we consider a sequential decision making setting, though our variational bandwidth bottleneck could also be applied to other learning problems. In reinforcement learning, the problem of sequential decision making is cast within the framework of MDPs . Our proposed method depends on two sources of input, standard input and "privileged" input. In reinforcement learning, privileged inputs could be the of performing any upstream computation, such as running model based planning. It can also be the information from the environment, such as the goal or the of active perception. In all these settings, the agent must decide whether to access the privileged input or not. If the agent decides to access the privileged input, then the the agent pays an "information cost." The objective is to maximize the expected reward and reduce the cost associated with accessing privileged input, such that across all states on average, the information cost of using the privileged information is minimal. We parameterize the agent's policy π θ (A|S, G) using an encoder p enc (Z|S, G) and a decoder p dec (A|S, Z), parameterized as neural networks. Here, the channel capacity network B(S) would take in the standard input that would be used to determine channel capacity, depending on which we decide to access the privileged input as in Section 4.1, such that we would output the distribution over the actions. That is, Y is A, and, z). This would correspond to minimizing I(A; G|S), ing in the objective 6 RELATED WORK A number of prior works have studied information-theoretic regularization in RL. For instance, van use information theoretic measures to define relevant goal-information, which then could be used to find subgoals. Our work is related in that our proposed method could be used to find relevant goal information, but without accessing the goal first. Information theoretic measures have also been used for exploration (; ; ;). More recently proposed InfoBot, where "decision" states are identified by training a goal conditioned policy with an information bottleneck. In InfoBot, the goal conditioned policy always accesses the goal information, while the proposed method conditionally access the goal information. The VBB is also related to work on conditional computation. Conditional computation aims to reduce computation costs by activating only a part of the entire network for each example . Our work is related in the sense that we activate the entire network, but only conditionally access the privileged input. Another point of comparison for our work is the research on attention models ((; ;) ). These models typically learn a policy, that allows them to selectively attend to parts of their input. However, these models still need to access the entire input in order to decide where to attend. Our method dynamically decides whether to access privileged information or not. As shown in our experiments, our method performs better than the attention method of. Recently, many models have been shown to be effective at learning communication in multi-agent reinforcement learning . learns a deep neural network that maps inputs of all the agents to their respective actions. In this particular architecture, each agent sends its state as the communication message to other agents. Thus, when each agent takes a decision, it takes information from all the other agents. In our proposed method, each agent communicates with other agents only when its necessary. In this section, we evaluate our proposed method and study the following questions: (a) Better generalization? Does the proposed method learn an effective bottleneck that generalizes better on test distributions, as compared to the standard conditional variational information bottleneck? (b) Learn when to access privileged input?: Does the proposed method learn when to access the privileged input dynamically, minimizing unnecessary access?. We compare the proposed method to the following methods and baselines: Conditional variational information bottleneck (VIB): The agent always access the privileged input, with a VIB using both the standard and the privileged input (InfoBot ). Deterministically accessing privileged input: The agent can deterministically access both the state as well as the privileged input. This has been shown to improve generalization in RL problems UVFA . We compare the proposed method to simpler reinforcement-learning baselines, where accessing privileged information can be formalized as one of the available actions that lead to the same state but with more information, at the cost of a small negative reward. This baseline evaluates whether the explicit VBB formulation provides a benefit over a more conventional approach, where the MDP itself is reformulated to account for the cost of information. Randomly accessing goal (RAG) -Here, we compared the proposed method to the scenario where we randomly access the privileged input (e.g., 50% of the time). This baseline evaluates whether the VBB is selecting when to access the goal in an intentional and intelligent way. Model-based planning can be computationally expensive, but beneficial in temporally extended decision making domains. In this setting, we evaluate whether the VBB can dynamically choose to invoke the planner as infrequently as possible, while still attaining good performance. While it is easy to plan using a planner (like a model based planner, which learns the dynamics model of the environment), it is not very cheap, as it involves running a planner at every step (which is expensive). So, here we try to answer whether the agent can decide based on the standard input when to access privileged input (the output of model based planner by running the planner). Experimental Setup: We consider a maze world as shown in Figure 2 (a). The agent is represented by a blue dot, and the agent has to reach the goal (represented by a green dot). The agent has access to a dynamics model of the environment (which is pretrained and represented using a parameterized neural network). In this task, the agent only gets a partial view of the surrounding i.e. the agent observes a small number of squares in front of it. The agent has to reach the goal position from the start position, and agent can use the pretrained dynamics model to sample multiple plausible trajectories, and the output of the dynamics model is fed as a conditional input to the agent's policy (similar to (Racanière et al., 2017) ), thus the agent can use this dynamics model to predict possible futures, and then make an informed decision based on its current state as well as the of the prediction from the dynamic model. Near the junction 72% ± 5% In the Hallway 28% ± 4% In this setup, the current state of the agent (i.e. the egocentric visual observation) acts as the standard input S, and the of running the planner acts as the privileged input G. In order to avoid running the model based planner, the agent needs to decide when to access the more costly planner. Results: -Here, we analyze when the agent access the output of the planner. We find that most of the times agent access the privileged information (output of model based planner) near the junctions as shown in Table 1. Figure 3: Partially Observable FindObjSX environmentsThe agent is placed in the central room. An object is placed in one of the rooms and the agent must navigate to the object in a randomly chosen outer room to complete the mission. The agent again receives an egocentric observation (7 x 7 pixels), and the difficulty of the task increases with X. For more details refer to supplementary material. The goal of this experiment is to show that, by selectively choosing when to access the privileged input, the agent can generalize better with respect to this input. We consider an agent navigating through a maze comprising sequences of rooms separated by doors, as shown in Figure 7. We use a partially observed formulation of the task, where the agent only observes a small number of squares ahead of it. These tasks are difficult to solve with standard RL algorithms, not only due to the partial observability of the environment but also the sparsity of the reward, since the agent receives a reward only upon reaching the goal. The low probability of reaching the goal randomly further exacerbates these issues. The privileged input in this case corresponds to the agent's relative distance to the goal G. At junctions, the agent needs to know where the goal is so that it can make the right turn. While in a particular room, the agent doesn't need much information about the goal. Hence, the agent needs to learn to access goal information when it is near a door, where it is most valuable. The current visual inputs act as a standard input S, which is used to compute channel capacity d cap. RoomN12S10 ) ) (a conditional variant of VIB). We use different mazes for training, validation, and testing. We evaluate generalization to an unseen distribution of tasks (i.e., more rooms than were seen during training). We experiment on both RoomNXSY (X number of rooms with atmost size Y, for more details, refer to the Appendix G) as well as the FindObjSY environment. For RoomNXSY, we trained on RoomN2S4 (2 rooms of at most size 6), and evaluate on RoomN6S6 (6 rooms of at most size 6) and RoomN12S10 (12 rooms, of at most size 10). We also evaluate on the FindObjSY environment, which consists of 9 connected rooms of size Y − 2 × Y − 2 arranged in a grid. For FindObjSY, we train on FindObjS5, and evaluate on FindObjS7 and FindObjS10. Percentage of times VBB 76% ± 6% InfoBot 60% ± 3% AIC 62% ± 6% Table 3: Goal Driven Navigation -Percentage of time steps on which each method acsess the goal information when the agent is near the junction point (or branching points in the maze. We show that the proposed method learns to access the privileged input (in this case, the goal) only when necessary. Results: Tables 3a, 3b compares an agent trained with the proposed method to a goal conditioned baseline (UVFA) , a conditional variant of the VIB , as well as to the baseline where accessing goal information is formulated as one of the actions (AIC). We also investigate how many times the agent accesses the goal information. We first train the agent on MultiRoomN2S4, and then evaluate this policy on MultiRoomN12S10. We sample 500 trajectories in MultiRoomN12S10env. Ideally, if the agent has learned when to access goal information (i.e., near the doorways), the agent should only access the goal information when it is near a door. We take sample rollouts from the pretrained policy in this new environment and check if the agent is near the junction point (or doorway) when the agent access the goal information. Table 3 quantitatively compares the proposed method with different baselines, showing that the proposed method indeed learns to generalize with respect to the privileged input (i.e., the goal). Next, we investigate the case where the privileged input is expensive to obtain, and we therefore would like to minimize how often the agent must access it. We specifically consider multiagent communication, where in order to solve a task, agents must communicate with other agents. Here we show that selectively deciding when to communicate with another agent can in better learning. Experimental setup: We use the setup proposed by. The environment consists of N agents and M landmarks. Both the agents and landmarks exhibit different characteristics such as different color and shape type. Different agents can act to move in the environment. They can also be affected by the interactions with other agents. Asides from taking physical actions, agents communicate with other agents using verbal communication symbols. Each agent has a private goal that is not observed by another agent, and the goal of the agent is grounded in the real physical environment, which might include moving to a particular location. It could also involve other agents (like requiring a particular agent to move somewhere) and hence communication between agents is required. We consider the cooperative setting, in which the problem is to find a policy that maximizes expected return for all the agents. In this scenario, the current state of the agent is the standard input S, and the information which might be obtained as a of communication with other agents is the privileged input G. For more details refer to the Appendix (D). Emergent Communication 4.85 (100%) ± 0.1% 5.44 (100%) ± 0.2% Randomly Accessing (RAG) 4.95 (50%) ± 0.2% 5.65 (50%) ± 0.1% InfoBot 4.81 (100%) ± 0.2% 5.32 (100%) ± 0.1% VBB (ours) 4.72 (23%) ± 0.1% 5.22 (34%) ± 0.05% Table 4: Multiagent communication: The VBB performs better, as compared to the baselines. In the baseline scenario, all of the agents communicate with all the other agents all the time. Averaged over 5 random seeds. Tasks: Here we consider two tasks: (a) 6 agents and 6 landmarks, (b) 10 agents and 10 landmarks. The goal is for the agents to coordinate with each other and reach their respective landmarks. We measure two metrics: (a) the distance of the agent from its destination landmark, and (b) the percentage of times the agent accesses the privileged input (i.e., information from the other agents). Table 4 shows the relative distance as well as the percentage of times agents access information from other agents (in brackets). Results: Table 4 compares an agent trained with proposed method to and Infobot . We also study how many times an agent access the privileged input. As shown in Table 4 (within brackets) the VBB can achieve better , as compared to other methods, even when accessing the privileged input only less than 40% of the times. The VBB performs better, as compared to the baselines. The VBB transmits a similar number of bits, while accessing privileged information a fraction of the time (in brackets % of times access to privileged information). Using REINFORCE to learn the parameter of the Bernoulli, does not perform as well as the proposed method. Channel Capacity: We can quantify the average information transmission through both the VBB and the VIB in bits. The average information is similar to the conventional VIB, while the input is accessed only a fraction of the time (the VIB accesses it 100% of the time). In order to show empirically that the VBB is minimizing information transmission (Eq. 1 in main paper), we measure average channel capacity D KL (p(z|s, g) r(z)) numerically and compare the proposed method with the VIB, which must access the privileged input every time (See Table 5). We demonstrated how the proposed variational bandwidth bottleneck (VBB) helps in generalization over the standard variational information bottleneck, in the case where the input is divided into a standard and privileged component. Unlike the VIB, the VBB does not actually access the privileged input before deciding how much information about it is needed. Our experiments show that the VBB improves generalization and can achieve similar or better performance while accessing the privileged input less often. Hence, the VBB provides a framework for adaptive computation in deep network models, and further study applying it to domains where reasoning about access to data and computation is an exciting direction for future work. Current limitation of the proposed method is that it assumes independence between standard input and the privileged input but we observe in practice assuming independence does not seem to hurt the . Future work would be to investigate how we can remove this assumption. In this section, we construct our objective function, such that minimizing this objective function minimizes I(Y, G|S). Recall that the IB objective is formulated as the minimization of I(Z, X) − βI(Z, Y), where X refers to the input, Y refers to the model output, Z refers to compressed representation or the bottleneck. For the proposed method, we construct our objective as follows: we minimize the mutual information between privileged input and output given the standard input, I(Y, G|S), to encode the idea that the we should avoid unnecessary access to privileged input G, and maximize the I(Z, Y). Hence, for the VBB, using the data processing inequality , this implies that To obtain an upper bound on I(G; Z|S), we must first obtain an upper bound on I(G; Z|S = s), and then we average over p(s). We get the following : We assume that the privileged input G and the standard input S are independent of each other, and hence p(g|s) = p(g). we get the following upper bound: where the inequality in the last line is because we replace p(z|s) with p prior (z). We also drop the dependence of the prior z on the standard input s. While this loses some generality, recall that the predictive distribution p(y|s, z) is already conditioned on s, so information about s itself does not need to be transmitted through z. Therefore, we have that Marginalizing over the standard input therefore gives us We approximate p(z|s, g) as a weighted mixture of p enc (z enc |s, g) and the normal prior such that z ∼ d cap * (p enc (z enc |s, g)) + (1 − d cap) * N. Hence, the weighted mixture p(z|s, g) can be seen as a bound on the information bottleneck objective. Whenever we access the privileged input G, we pay an information cost (equal to I(Z, G|S) which is bounded by D KL (p(z|s, g) p prior (z)). Hence, the objective is to avoid accessing the privileged input, such that on average, the information cost of using the privileged input is minimal. Here, we first show how the weighted mixture can be a bound on the information bottleneck objective. Recall, Hence, D KL (p(z|s, g) p prior (z)) where p(z|s, g) is expressed as a mixture of direc delta and prior, and hence it can be written as Further expanding the RHS using eq. 9, we get Here, we can assume the δ(f (s, g)) to be zero under the prior (as it is a Direc delta function). This can further be simplified to: And hence, reducing the above term reduces t0 D KL (p(z|s, g) p prior (z)), our original objective. In the main paper we show how can we evaluate channel capacity in a tractable way. The way we do is to learn a function B(S) which determines channel capacity. Here's another way, which we (empirically) found that parameterizing the channel capacity network helps. In order to represent this function B(S) which satisfies these constraints, we use an encoder of the form, where S refers to the standard input, and f µ, f σ are learned functions (e.g., as a multi-layer perceptron) that outputs µ and σ respectively for the distribution over z cap. Here, D KL (B(S)|N) refers to the channel capacity of the bottleneck. In order to get a probability prob out of B(S), we convert B(S) into a scalar prob ∈ such that the prob can be treated as a probability of accessing the privileged input. We perform this transformation by normalizing B(S) such that B(S) ∈ [−k, k], (in practice we perform this by clamping B(S) ∈ [−2, 2]) and then we pass the normalized B(S) through a sigmoid activation function, and treating the output as a probability, prob, we access the privileged input with probability prob. Hence, the ing distribution over z is a weighted mixture of accessing the privileged input f enc (s, g) with probability prob and sampling from the prior with probability 1 − prob. Here we assume prior to be N, but it can also be learned. At test time, using prob, we can sample from the Bernouilli distribution b ∼ Bernoulli(prob) to decide whether to access the privileged input or not. Experimental Setup: We use the setup proposed by. The environment consists of N agents and M landmarks. Both the agents and landmarks exhibit different characteristics such as different color and shape type. Different agents can act to move in the environment. They can also be affected by the interactions with other agents. Asides from taking physical actions, agents communicate with other agents using verbal communication symbols. Each agent has a private goal which is not observed by another agent, and the goal of the agent is grounded in the real physical environment, which might include moving to a particular location, and could also involve other agents (like requiring a particular agent to move somewhere) and hence communication between agents is required. Each agent performs actions and communicates utterances according to a policy, which is identically instantiated for all of the agents in the environment, and also receive the same reward signal. This policy determines both the actions and communication protocols. We assume all agents have identical action and observation spaces and receive the same reward signal. We consider the cooperative setting, in which the problem is to find a policy that maximizes expected return for all the agents. In order to study generalization across a wide variety of environmental conditions and linguistic inputs, develop an extension of the puddle world reinforcement learning benchmark. States in a 10 X 10 grid are first filled with either grass or water cells, such that the grass forms one connected component. We then populate the grass region with six unique objects which appear only once per map (triangle, star, diamond, circle, heart, and spade) and four non-unique objects (rock, tree, horse, and house) which can appear any number of times on a given map. We followed the same experimental setup and hyperparameters as in . Here, an agent is rewarded for reaching the location specified by the language instruction. Agent is allowed to take actions in the world. Here the goal is to be able to generalize the learned representation for a given instruction such that even if the environment observations are rearranged, this representation is still useful. Hence, we want to learn such representations that ties observations from the environment and the language expressions. Here we consider the Puddle World Navigation map as introduced in . We followed the same experiment setup as . Here, the current state of the agent acts as a standard input. Based on this, agent decides to access the privileged input. We start by converting the instruction text into a real valued vector using an LSTM. It first convolves the map layout to a low-dimensional repesentation (as opposed to the MLP of the UVFA) and concatenates this to the LSTM's instruction embedding (as opposed to a dot product). These concatenated representations are then input to a two layered MLP. Generalization over both environment configurations and text instructions requires a model that meets two desiderata. First, it must have a flexible representation of goals, one which can encode both the local structure and global spatial attributes inherent to natural language instructions. Second, it must be compositional, in order to learn a generalizable representation of the language even though each unique instruction will only be observed with a single map during training. Namely, the learned representation for a given instruction should still be useful even if the objects on a map are rearranged or the layout is changed entirely. The goal of this experiment is to study if using the proposed method enables learning a dynamic representation of an image which can be then used to accurately classify an image. In order to show this, we follow the setup of the Recurrent Attention Model (RAM) . Here, the attention process is modeled as a sequential decision process of a goal-directed agent interacting with the visual image. A recurrent neural network is trained to process the input sequentially, attending to different parts within the image one at a time and hence combining information from these different parts to build up a dynamic representation of the image. The agent incrementally combines information because of attending to different parts and then chooses this integrated information to choose where next to attend to. In this case, the information due to attending at a particular part of the image acts as a standard input, and the information which is being integrated over time acts as a privileged input, which is then used to select where the model should attend next. The entire process repeats for N steps (for our experiment N = 6). FC denotes a fully connected network with two layers of rectifier units, each containing 256 hidden units. MNIST 60 * 60 Cluttered MNIST FC (2 layers) 1.69% 11.63% RAM Model (6 locs) 1.55% 4.3% VIB (6 locs) 1.58% 4.2% VBB (6 locs) (Ours) 1.42% 3.8% Table 6: Classification error . Averaged over 3 random seeds. Quantitative Results: Table 6 shows the classification error for the proposed model, as well as the baseline model, which is the standard RAM model. For both the proposed model, as well as the RAM model, we fix the number of locations to attend to equal to 6. The proposed method outperforms the standard RAM model. We evaluate the proposed framework using Advantage Actor-Critic (A2C) to learn a policy π θ (a|s, g) conditioned on the goal. To evaluate the performance of proposed method, we use a range of maze multi-room tasks from the gym-minigrid framework and the A2C implementation from. For the maze tasks, we used agent's relative distance to the absolute goal position as "goal". For the maze environments, we use A2C with 48 parallel workers. Our actor network and critic networks consist of two and three fully connected layers respectively, each of which have 128 hidden units. The encoder network is also parameterized as a neural network, which consists of 1 fully connected layer. We use RMSProp with an initial learning rate of 0.0007 to train the models, for both InfoBot and the baseline for a fair comparison. Due to the partially observable nature of the environment, we further use a LSTM to encode the state and summarize the past observations. The MultiRoom environments used for this research are part of MiniGrid, which is an open source gridworld package. This package includes a family of reinforcement learning environments compatible with the OpenAI Gym framework. Many of these environments are parameterizable so that the difficulty of tasks can be adjusted (e.g., the size of rooms is often adjustable). In MiniGrid, the world is a grid of size NxN. Each tile in the grid contains exactly zero or one object. The possible object types are wall, door, key, ball, box and goal. Each object has an associated discrete color, which can be one of red, green, blue, purple, yellow and grey. By default, walls are always grey and goal squares are always green. Rewards are sparse for all MiniGrid environments. In the MultiRoom environment, episodes are terminated with a positive reward when the agent reaches the green goal square. Otherwise, episodes are terminated with zero reward when a time step limit is reached. In the FindObj environment, the agent receives a positive reward if it reaches the object to be found, otherwise zero reward if the time step limit is reached. The formula for calculating positive sparse rewards is 1 − 0.9 * (step_count/max_steps). That is, rewards are always between zero and one, and the quicker the agent can successfully complete an episode, the closer to 1 the reward will be. The max_steps parameter is different for each environment, and varies depending on the size of each environment, with larger environments having a higher time step limit. There are seven actions in MiniGrid: turn left, turn right, move forward, pick up an object, drop an object, toggle and done. For the purpose of this paper, the pick up, drop and done actions are irrelevant. The agent can use the turn left and turn right action to rotate and face one of 4 possible directions (north, south, east, west). The move forward action makes the agent move from its current tile onto the tile in the direction it is currently facing, provided there is nothing on that tile, or that the tile contains an open door. The agent can open doors if they are right in front of it by using the toggle action. Observations in MiniGrid are partial and egocentric. By default, the agent sees a square of 7x7 tiles in the direction it is facing. These include the tile the agent is standing on. The agent cannot see through walls or closed doors. The observations are provided as a tensor of shape 7x7x3. However, note that these are not RGB images. Each tile is encoded using 3 integer values: one describing the type of object contained in the cell, one describing its color, and a flag indicating whether doors are open or closed. This compact encoding was chosen for space efficiency and to enable faster training. The fully observable RGB image view of the environments shown in this paper is provided for human viewing. The level generation in this task works as follows: Generate the layout of the map (X number of rooms with different sizes (at most size Y) and green goal) Add the agent to the map at a random location in the first room. Add the goal at a random location in the last room. MultiRoomNXSY -In this task, the agent gets an egocentric view of its surroundings, consisting of 3×3 pixels. A neural network parameterized as MLP is used to process the visual observation. Here, the privileged input involves accessing information from the external memory like neural turing machines (NTM) (; . Reading from external memory is usually an expensive operation, and hence we would like to minimize access to the external memory. For our experiments, we consider external memory in the form of neural turning machines. NTM processes inputs in sequences, much like a normal LSTM but NTM can allow the network to learn by accessing information from the external memory. In this context, the state of controller (the NTM's controller which processes the input) becomes the standard input, and based on this (the standard input), we decide the channel capacity, and based on channel capacity we decide whether to read from external memory or not. In order to test this, we evaluate our approach on copying task. This task tests whether NTMs can store and recall information from the past. We use the same problem setup as. As shown in fig 8, we found that we can perform slightly better as compared to NTMs while accessing external memory only 32% of the times. The only hyperparameter we introduce with the variational information bottleneck is β. For both the VIB baseline and the proposed method, we evaluated with 5 values of β: 0.01, 0.09, 0.001, 0.005, 0.009. We use the following parameters for lower level policies throughout the experiments. Each training iteration consists of 5 environments time steps, and all the networks (value functions, policy, and observation embedding network) are trained at every time step. Every training batch has a size of 64. The value function networks and the embedding network are all neural networks comprised of two hidden layers, with 128 ReLU units at each hidden layer. All the network parameters are updated using Adam optimizer with learning rate 3 · 10 −4.

Represent this text to find a much much longer article about it with say hundreds-thousands of words

Training agents with adaptive computation based on information bottleneck can promote generalization.

represent the short summary for finding the corresponding scientific article (long)
E.g. We present a novel unified architecture that restores video frames from a single motion-blurred image in an end-to-end manner. == We propose a novel framework to generate clean video frames from a single motion-blurred image. While a broad range of literature focuses on recovering a single image from a blurred image, in this work, we tackle a more challenging task i.e. video restoration from a blurred image. We formulate video restoration from a single blurred image as an inverse problem by setting clean image sequence and their respective motion as latent factors, and the blurred image as an observation. Our framework is based on an encoder-decoder structure with spatial transformer network modules to restore a video sequence and its underlying motion in an end-to-end manner. We design a loss function and regularizers with complementary properties to stabilize the training and analyze variant models of the proposed network. The effectiveness and transferability of our network are highlighted through a large set of experiments on two different types of datasets: camera rotation blurs generated from panorama scenes and dynamic motion blurs in high speed videos. Our code and models will be publicly available. Capturing an image is not an instant process; to capture enough photons, the photosensitive elements of a camera have to be exposed to light for a certain interval of time, called exposure time. Therefore, during this interval if an object is moving in the observed scene or the camera is undergoing an arbitrary motion, the ing image will contain a blurring artifact known as motion blur. In general, motion blur is an unwanted behaviour in vision applications e.g.image editing , visual SLAM (and 3D reconstruction , as it degrades the visual quality of images. To cope with this type of artifact, image deblurring aims to restore a sharp image from a blurred image. This problem is known to be ill-posed since the blur kernel used for deconvolution is generally assumed to be unknown. Earlier studies assume a uniform-blur over the image to simplify the estimation of the single deconvolution blur kernel used to remove the blur (; ;). Even though the methods deploy deblurring tasks with uniform-blur assumption, the assumption is often violated in practice. For instance, when the blur is caused by out-of-plane camera rotation, the blur pattern becomes spatially variant. Moreover, the problem is more complex when objects in a scene are moving i.e.dynamic blur. While previous literature focuses on recovering a sharp image from a blurred image, we tackle a more challenging task i.e.video restoration from a blurred image. Restoring the underlying image sequence of a blurred image requires both contents and motion prediction. We formulate video restoration from a blurred image as an inverse problem where a clean sequence of images and their motion as latent factors, and a blurred image as an observation. Some of previous deblurring approaches (; ; ; ;) also estimate the underlying motion in a blurred image, however, their goal remains in single frame restoration. proposed to extract video frames from a single motion-blurred image. Their approach is close to image translation model without inferring underlying motions between the latent frames. addressed this issue by estimating pixel level motion from a given blurred input. However, their model is still prone to sequential error propagation as frames are predicted in a sequential manner using a deblurred middle frame. In this paper, we propose a novel framework to generate a clean sequence of images from a single motion-blurred image. Our framework is based on a single encoder-decoder structure with Spatial Transformer Network modules (STN) and Local Warping layers (LW) to restore an image sequence and its underlying motion. Specifically, a single encoder is used to extract intermediate features which are passed to multiple decoders with predicted motion from STN and LW modules to generate a sequence of deblurred images. We evaluate our model on two types of motion blur. For rotation blur, which is caused by abrupt camera motion, we generated a synthetic dataset from panoramic images (J. Xiao & Torralba., 2012). For dynamic blur caused by fast moving objects in a scene, we used a high speed video dataset . The proposed model is evaluated on the panorama and the high speed video datasets under various motion patterns. Both the quantitative metrics and qualitative highlight that our method is more robust and performs favorably against the competing approach 1. For further investigation, we demonstrate the transferability of our model by cross-dataset evaluation. We also propose a simpler and lighter variation of our model guiding that our approach is flexible and can be easily extended to arbitrary number of frame prediction model with negligible performance trade-off. In short, our contributions are as follows. 1) We propose a novel unified architecture to restore clean video frames from a single motion-blurred image in an end-to-end manner. 2) Loss terms are designed to stably train the proposed network. 3) We perform thorough experiments to analyze the transferability and flexibility of the proposed architecture. 4) The performance of our model quantitatively and qualitatively performs favorably against the competing approach. Moreover due to flexibility of our model, we show that our approach is robust to heavy blurs where the previous approach fails. Image deblurring in general refers to the restoration of an image affected by blur. In this paper, we focus exclusively on the motion blur. Image deblurring is an ill-posed inverse problem when a blur kernel is unknown i.e.blind deconvolution problem, as different latent images can be transformed to a blurred image depending on its blur kernel. Early stage of deblurring studies (; ; ; ; ; ; ;) assume a single blur kernel that is applied to an image globally. The restoration of blur images is often modeled as a maximization problem of probabilistic models . To narrow down the ambiguity of the blur kernel estimation, natural image priors (; ; 2016;) are exploited. For instance, formulate the blur kernel estimation as a process to recover internal recurrence of image patches. 0 regularization , dark channel prior and extreme channel prior are also used to improve image deblurring. While single blur kernel estimation approaches are effective when blur kernels are shift-invariant, they fail when the blur is not spatially uniform. A non-uniform blur can be caused by camera rotation, depth variation or moving objects in a scene. To restore images affected by motion blur from pure rotations, use the geometric information of the camera motion as a prior to recover the non-uniform blur model. Recently, deep network based methods are proposed to handle general blur patterns without the uniform blur assumption. propose multi-scale deep networks with multi-scale loss that mimics coarse-to-fine approaches to restore sharp images under non-uniform blurred images. proposed a spatially variant neural networks to learn spatially variant kernels. proposed to extract a video sequence from a single motion-blurred image using multiple deep networks. They showed that deep networks can successfully generate an image sequence from a blurred image, however there remains a few limitations. Their proposed framework consists of multiple networks of which each network is specialized to predict a specific frame in a sequence. Each network is trained separately and sequentially starting from the middle frame and then adjacent frames taking previously predicted frames as inputs. As a , the non-middle frame prediction heavily relies on previously predicted frames including the middle frame itself, therefore when the middle frame is erroneous the error propagates across frames. proposed a two-step strategy to generate a video from a motion-blurred image using three complementary networks. They used video autoencoder to learn motion and frame generation from clean frames as a pretraining phase. Latter, they introduced a motion disentangle network to extract motion from blurred image. They also used independent deblurring network as their approach requires a clean middle frame generated from a blurred image in advance. Although their approach takes motion information into account, the approach generates frames sequentially starting from the middle frame to adjacent frames which in error propagation just as in. Unlike the previous works, our approach runs in an end-to-end manner within a single training stage without error propagation across frames. Collecting a large number of natural motion-blurred images is a daunting task. Hence, a common practice in computer vision research is to generate blurry images by combining a sequence of sharp images using various approaches ranging from simple averaging to learnable methods . The source of motion blur in an image can be generalized into two main categories: rapid camera motion (camera shake) and dynamic motion of objects in the scene. In this section, we briefly explain how we generate a blurry image dataset by considering each case individually. In order to generate a rotation blurred image dataset, we use the SUN360 panorama dataset (J. Xiao & Torralba., 2012). This dataset provides various panoramas with 360 • field of view. Hence, a virtual camera can be modeled to point at different orientations to represent the camera rotation in SO. Given a panorama P of size H × W, we developed a simple yet effective framework to generate blurred images. First, the panorama is projected onto a unit sphere by linearly mapping each pixel coordinate (x, y) ∈ P into spherical coordinates (θ, φ) with θ ∈ (0, 2π) and φ ∈ (−π/2, π/2). Then, a synthetic image can be captured via a virtual camera by re-projecting the 3D points on the sphere into an image plane as briefly discussed in and. Using this procedure we first capture an image by positioning the virtual camera at an arbitrary orientation. We call the image generated at this orientation initial image. Then, we rotate the camera by a random rotation matrix (with β = (β x, β y, β z) its Euler angle representation) and capture a second image at the new camera position called final image. We finally use a quaternion spherical linear interpolation technique (Slerp) The camera rotation angle is uniformly sampled from [−10 •, 10 •]. In order to generate a realistic blurred image, the number of intermediate images have to be adjusted automatically depending upon the rotation magnitude between the initial and final frames. Therefore, we use a simple linear relationship between the number of frames to be generated (n) and the rotation magnitude as follows: n = c + 1 3 β, where c is a constant and β is the magnitude of β. In this manner, we use 1000 panoramic images from which we generate 26, 000 training and 3, 200 test images of size 128 × 128px. In order to generate more realistic and generic (arbitrary camera motions and dynamic scene) blurred images, we take advantage of a GoPro high speed video dataset . This dataset provides 22 training and 11 test scenes, each scene containing frames of size 1280 × 720px. A blurry image is generated by averaging n consecutive frames . In our experiments, we fixed n = 7 and generated 20, 000 training and 2000 test images by randomly cropping images of size 256 × 256px. In this section, we describe our network structure and loss functions. To explicitly take the camera motion into consideration, our network is designed with spatial transformer networks (STNs) in an encoder-decoder structure (Fig. 2). Given a blurry image I b as an input, our model outputs {I 1, ..., I m−1, I m, I m+1, ..., I n}, where I m is the deblurred middle frame and {I j} n j=1 where j = m are the recovered non-middle frames. To ensure sharp image generation and stable training, the network is trained using three loss terms: multi-scale photometric loss, transformation consistency loss and penalty term. The middle frame I m is reconstructed using a U-net like network. The encoder contains five convolutional blocks, each block containing two layers of convolutions with spatial kernel size of 3 × 3 and stride size of 2 and 1, respectively. The feature maps are downsampled to half size and the number of channels is doubled after each convolutional block. The decoder network also contains five convolutional blocks to upsample features and to predict images at different scales. In each block, a feature is first upscaled using a transposed convolution (deconvolution) layer of kernel size 4×4 and a stride size of 2. The image predicted in the previous block is also upscaled in the same manner. The upsampled feature and its respective image are then concatenated channel-wise with the corresponding feature from the encoder (skip connection as shown in the Fig. 2a), then passed through five layers of convolutions with dense connections to output a feature, which will be used to predict an image at current block. In this manner, features and images are successively upsampled to predict a full scale middle frame. Along with the last feature map from the decoder, the predicted image is finally passed through a refining convolutional block. It contains seven dilated convolutional layers each with kernel size of 3×3 and different dilation constants. The purpose of this network is to further refine the predicted frame with contextual information by effectively enlarging the receptive field size of the network. The non-middle frames are reconstructed based on the encoded features of the middle frame via learned transformation by STN modules and local warping (LW) networks. First, the encoded middle frame feature U e ∈ R H×W ×C with width W, height H and C channels from the encoder is transformed into a non-middle frame feature using a feature transformer network (FTN). Second, the decoded middle frame image I d ∈ R H×W ×3 predicted from the corresponding middle frame decoder is transformed using an image transformer network (ITN). Third, the transformed feature and image are concatenated channel-wise and are passed through a decoder to predict a non-middle frame (Fig. 2b). We also input the middle frame feature into the non-middle frame decoder in order to guide the decoder to learn the spatial relation between the middle and the non-middle frame. The decoder network here is similar to the one used for predicting a middle frame in the previous section. These mechanisms are summarized in the following equations, where l = {1, ..., k} is an index for k feature levels (scales) and t is a subscript for transformed feature/image. I p is the predicted frame from a non-middle frame decoder D. Each non-middle frame is reconstructed by applying multi-scale transformer networks to the middle frame encoder. Given ground truth non-middle frames during training, our model learns the transformation parameters to be applied to the middle frame at different scales in order to output the desired non-middle frames. The fact that unique transformer networks are applied at each feature and image scale gives the model a capacity to learn various types of transformations, hence, making it robust to different blur patterns including large blurs. STNs in our model learn non-local transformations in a given motion-blurred image. In order to compensate for locally variant transformations, we designed a local warping network. This network is conditioned on the input feature like STN, however, instead of predicting global transformation parameters, it predicts pixel-wise displacement i.e.motion flow. Given an input feature U ∈ R H×W ×C, the local warping network outputs a motion flow of size H × W × 2. We used two convolutional layers of kernel size 3 × 3 for the network. By warping the input feature with the predicted motion flow, we obtain a locally transformed feature which is used as an input to the decoder (Fig. 2b). Given a blurry input, in order to generate video frames that are sharp both locally and globally, we trained our network with a multi-scale photometric loss between the images predicted by the decoder network and the ground truth image. A bilinear downsampling is used to resize the ground truth image to the corresponding predicted frame size at different scales. Let l denotes a scale level, k denotes total number of scales, {ŷ} k l=1 denotes a set of predicted images from the smallest size (ŷ 1) to the full scale (ŷ k), and {y} k l=1 represent a set of downsampled ground truth images where y k is a full scale ground truth image. For training a model predicting a sequence with n frames from a single blurry image, we compute multi-scale photometric loss as the term L mp in Eq., We use individual transformer networks for each feature level when predicting non-middle frames. This augments our model with the capacity to learn transformations at different levels making it robust to various blur patterns. However, we expect the transformations at different scales to be aligned for successfully reconstructing temporally consistent non-middle frames. Especially at the initial stages of the training where the transformer parameters are random, it is beneficial that our model understands the relationship between the transformations across different frame levels. In order to impose this notion into our model and facilitate a smooth training, we propose the transformation consistency loss. Let {θ} k l=1 be the set of predicted transformation parameters at different scales. The transformation consistency loss for predicting n − 1 non-middle frames can be defined as the term L tc in Eq., where |.| 2 is an 2 loss between the transformation parameters. Predicting multiple frames from a single blurry image can be problematic at times when the model fails to learn any type of transformation and simply replicates the middle frame predicition as nonmiddle frames. In order to remedy this issue, we design a penalty term to enforce diversity among generated images. This is accomplished by explicitly maximizing the sum of absolute difference (SAD) i.e. minimizing the negative SAD between a predicted frame and its time-symmetric (about the middle frame) ground truth frame. For example, when predicting seven frames {I 1, ..., I 4, ...., I 7}, we enforce the predicted image I 1 to be different content-wise from the ground truth image I 7 and vice versa. The penalty is imposed in a symmetric manner such that the model learns to be sensitive to smaller transformations close to the middle frame as well as larger transformations at the end frames. Given a predicted frameŷ i and the corresponding time-symmetric ground truth y n+1−i, the penalty term is computed as the term L p in Eq., where m is the middle frame index, n is the total number of frames. The final training loss function is defined as follows, where λ tc and λ p are weight coefficients for transformation consistency loss and penalty term, respectively. We used λ tc = 1e + 3 and λ p = 0.5. The task at hand has two main ambiguities. i. temporal shuffling and ii. reverse ordering. As explained in section 3, motion-blur is the of an averaging process and, restoring temporally consistent (no shuffling) sharp frame sequence from a given motion-blurred input is a non-trivial task as the averaging destroys the temporal order. mentions that photometric loss is not a sufficient constraint to make their network converge. Hence, they propose a pair-wise order invariant loss to train their network. also uses the same loss function to fine-tune the recurrent video decoder in their network. We find experimentally that a multi-scale photometric loss is a sufficient constraint to train our network. We further impose more constraints using other loss terms to improve performance (see ablation studies in Sec. 5.4.2). By design nature, our model allows motions to be learned in a symmetric manner (about the middle frame) with transformer networks close to the middle frame decoding smaller motions and those further from the middle frame decoding larger motions. This notion is enforced by transformation consistency loss and symmetric constraint term during training. The fact that our model is optimized in a joint manner allows frames to be reconstructed in a motion-guided sequence. Other than temporal shuffling, another issue is reverse ordering. Given a single motion-blurred input, recovering ground truth order is a highly ill-posed problem which is intractable since reversely ordered frames in the same motion-blurred image. Neither our work nor previous works (, ) are capable of predicting the right order. Hence, we evaluate frame reconstructions using both ground truth order and its reverse order, then report the higher metric in the experiment section. Our model is implemented and trained using pyTorch . We chose Adam as an optimizer with β 1 and β 2 fixed to 0.9 and 0.999, respectively. On our synthetic blur dataset, we train the model using images of size 128 × 128px and a mini-batch size of 8 to predict initial, middle and final frames. A mini-batch size of 4 and input size of 256 × 256px is used to predict sequences of frames when training on the high speed video dataset. In all experiments, we train our model for 80 epochs. We set the learning rate λ = 1e − 4 at the start of the training and decay it by half at epochs 40 and 60. All the training and test images are cropped from the original resolution images without resizing. In this section, we analyze the performance of our model qualitatively and quantitatively on both camera shake blurs generated from panoramic scenes and dynamic blurs obtained from averaging frames in high speed videos. We report test using Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity (SSIM) metrics. To purely evaluate the quality of generated images without ordering estimation issue due to reverse ordering, we report the higher PSNR/SSIM metric of either ground truth order or reverse order of frames i.e.max{PSNR/SSIM( For dynamic blur, we compare our with 2 as they also use the same high speed video dataset . The averaged metrics obtained on the testing set are summarized in Table 1. The show that our model performs favorably against their model on both PSNR and SSIM on the high speed video dataset. While the middle frame prediction shows moderate performance increase, for the initial and final frame predictions, our model outperforms by a large margin. The performance gap between the middle frame and non-middle frames is relatively larger in than our method. This is due to sequential prediction in which makes non-middle frame prediction heavily dependent on the generated middle frame, ing in error propagation. As stated in , this limitation is particularly problematic when a heavy blur affects the input image since the middle frame prediction becomes less reliable. Our approach is Figure 3: Rotation blurred images generated from panorama scenes. The top row is ground truth frames and the bottom row is restored frames from the blurs. Input Figure 5: Partially blurred dynamic motion example. The man is blurred with dynamic motion while the is close to static. relatively robust to heavy blur as the proposed model generates frames independently from multiple decoders, therefore the error is not propagated (Fig. 7). We also evaluate our model on camera rotational blurs (Table 1 Panorama blur). In terms of blurriness, the panorama scenario is more challenging as we simulate broader range of blurs than the blurs in the high speed video dataset i.e.from sharp images to heavy blurred images. We observed visually better for the panorama scenario but lower quantitative number. One main reason is that panorama ground truths are relatively sharper while the high speed video contains blurry ground truth images due to dynamic motion and short exposure time. Therefore, the visual of panorama scenes from our model are sharp even though the quantitative performance is relatively lower. Please refer to Sec. 7.1 in the appendix for the quantitative on seven frame predictions. The qualitative for panoramic scenes and high speed videos show that our model can successfully restore multiple frames from a blurred input under various blur patterns. The restored frames from a blurred image in the panorama scenario (Fig. 3) are relatively sharper than dynamic blur case. As mentioned earlier, one of reasons is due to high quality ground truth image. In the high speed video scenario (Fig. 4), ground truth images have locally blurred contents due to fast dynamic motions. We compare our approach and previous method on relatively heavily blurred images from the high speed video dataset. As can be seen from Fig. 4, our method reconstructs contents consistently across frames and restores visually sharper videos compared to. In case of dynamic blurs where the motion of multiple objects and static objects are Figure 6: STN transformation visualization. The ground truth middle frame is transformed to non-middle frame using pure STN transformation. mixed as shown in Fig. 5, our model can generate video frames unravelling the underlying spatially varying motion. Failure cases happen for undersampled and heavily blurred inputs as can be seen from Fig. 7 and Fig. 9d. Please refer to Sec. 8 in the appendix for more qualitative . In order to visually analyze if the STN module in our model reasonably infers the blur motion, we apply the STN transformation to the middle frame and compare it with the ground truth non-middle frames. Let i be the index of the initial frame, m the middle frame, and f the final frame. The input image is obtained by averaging a set of frames {F j} f j=i. We can obtain the STN transformation parameters {θ} f j=1,j =m from the feature transformer network which transforms middle frame features to non-middle frame features. We apply the STN transformation T with the parameter θ j to middle frame F m to visualize whether the transformation implies valid motion information. Fig. 6 shows that the transformation spatially aligns contents of the middle frame to that of the non-middle frames. We report a cross-dataset panorama→high speed video evaluation to assess the generalization capability of our model. A model trained on the panoramic scenes is evaluated on high speed video test set (Table 2). Despite a performance degradation, our model trained on the panorama dataset performs on par with the competing approach trained on the high speed video dataset. The absence of dynamic motion on the panorama dataset, which is apparent in high speed videos, can be one contributing factor explaining the performance loss in addition to the domain gap e.g.image contents, sharpness, blurriness. Model size can be a bottleneck when the number of frames in a video to be predicted increases since our model uses multiple decoders and transformer networks. Hence, we experiment with a lighter model by replacing decoders and STN modules to weight shared layers. As opposed to using individual decoders for each non-middle frame, we use a single decoder i.e.weight shared decoders. For STN, we apply inverse transformations by assuming symmetric motion about the middle frame, therefore, reducing the number of transformer networks by half. We tested this light architecture on a model that predicts three (initial, middle and final) frames and it reduces the number of model parameters by 48%, yet yielding only a 0.4dB performance decrease on average (Table 2). The feature transformer network (FTN) at different levels of abstraction is the core part of our model for network convergence. Local warping (LW) layer also significantly improves the performance of our model. The best model performance is, yet, achieved with the three network components (FTN, LW, ITN) combined (Table 3). As mentioned earlier, the multi-scale photometric loss (PML) is a sufficient constraint to make our network converge during training. We also experimentally find that a model trained with transformation consistency loss (TCL) not only converges faster with smoother behavior but also gives a better performance during testing. The penalty term (PT) gives a marginal performance improvement when predicting fewer frames as photometric loss is already a sufficient constraint. In 3 frame prediction model, the penalty term improved performance marginally around 0.25dB while in 7 frame prediction model, it improved approximately 0.6dB. Penalty term enforces the model to consider subtle differences especially when the motion is small. We present a novel unified architecture that restores video frames from a single blurred image in an end-to-end manner. During training, feature and image transformer networks indirectly learn blur motions without motion supervision. The designed a loss function with regularizers enforce the model to consider subtle differences with fast loss convergence. We evaluate our model on the two datasets with rotation blurs and dynamic blurs and demonstrate qualitatively and quantitatively favorable performance against the competing approach. The cross-dataset evaluation demonstrates that our model can generalize even when the training and test set have significantly different blur patterns and domain gap. We additionally propose a lighter version of the model by weight-sharing of the decoders and STN modules under symmetric frame motion assumption. This modification enables the model to have negligible parameter size increment even when the number of predicted frames are high. Unlike the previous approaches, our model predicts frames in a single step without middle frame dependency. It is advantageous not only because it is simple to use but also robust to heavy blurs where middle frame prediction often fails. Overall, the simplicity and flexibility of our method makes it a promising approach for future applications such as deblurring, temporal super resolution and flow estimation from a motion-blurred image. Here, we present additional details pertaining to the experiments that could not be included in the main text due to space constraints. As can be inferred from Table 4, our method consistently performs favorably against the competing method. The prediction performs best for middle frames and consistently decreases for non-middle frames on both our method and the competing method. The overall performance reported here is consistent with the in the main paper. Here, we show qualitative (frame-by-frame comparison) from the high speed video dataset that could not be included in the main text due to space constraints. We also tested our model on motion-blurred examples from and the restored videos are shown in Fig. 9. Here, we show qualitative from the SUN360 panorama dataset (J. Xiao & Torralba., 2012) that could not be included in the main text due to space constraints. The samples contain various blur patterns e.g.static, partial blur, and heavy blur.

A backward model of previous (state, action) given the next state, i.e. P(s_t, a_t | s_{t+1}), can be used to simulate additional trajectories terminating at states of interest! Improves RL learning efficiency.


Question: Represent the text to find a very short summary (1-2 sentences max)

Question: Breathing exercises are an accessible way to manage stress and many mental illness symptoms. Traditionally, learning breathing exercises involved in-person guidance or audio recordings. The shift to mobile devices has led to a new way of learning and engaging in breathing exercises as seen in the rise of multiple mobile applications with different breathing representations. However, limited work has been done to investigate the effectiveness of these visual representations in supporting breathing pace as measured by synchronization. We utilized a within-subjects study to evaluate four common breathing visuals to understand which is most effective in providing breathing exercise guidance. Through controlled lab studies and interviews, we identified two representations with clear advantages over the others. In addition, we found that auditory guidance was not preferred by all users. We identify potential usability issues with the representations and suggest design guidelines for future development of app-supported breathing training.

Represent the TLDR for finding the corresponding scientific article

We utilized a within-subjects study to evaluate four paced breathing visuals common in mobile apps to understand which is most effective in providing breathing exercise guidance.

We explored the visualization designs that can support chronic patients to present and review their health data with healthcare providers during clinical visits.