Question: Represent the article for finding a single-sentence TLDR.

Question: We explore the collaborative multi-agent setting where a team of deep reinforcement learning agents attempt to solve a shared task in partially observable environments. In this scenario, learning an effective communication protocol is key. We propose a communication protocol that allows for targeted communication, where agents learn \emph{what} messages to send and \emph{who} to send them to. Additionally, we introduce a multi-stage communication approach where the agents co-ordinate via several rounds of communication before taking an action in the environment. We evaluate our approach on several cooperative multi-agent tasks, of varying difficulties with varying number of agents, in a variety of environments ranging from 2D grid layouts of shapes and simulated traffic junctions to complex 3D indoor environments. We demonstrate the benefits of targeted as well as multi-stage communication. Moreover, we show that the targeted communication strategies learned by the agents are quite interpretable and intuitive. Effective communication is a key ability for collaborative multi-agents systems. Indeed, intelligent agents (humans or artificial) in real-world scenarios can significantly benefit from exchanging information that enables them to coordinate, strategize, and utilize their combined sensory experiences to act in the physical world. The ability to communicate has wide-ranging applications for artificial agents -from multi-player gameplay in simulated games (e.g. DoTA, Quake, StarCraft) or physical worlds (e.g. robot soccer), to networks of self-driving cars communicating with each other to achieve safe and swift transport, to teams of robots on search-and-rescue missions deployed in hostile and fast-evolving environments. A salient property of human communication is the ability to hold targeted interactions. Rather than the'one-size-fits-all' approach of broadcasting messages to all participating agents, as has been previously explored BID19 BID4, it can be useful to direct certain messages to specific recipients. This enables a more flexible collaboration strategy in complex environments. For example, within a team of search-and-rescue robots with a diverse set of roles and goals, a message for a fire-fighter ("smoke is coming from the kitchen") is largely meaningless for a bomb-defuser. In this work we develop a collaborative multi-agent deep reinforcement learning approach that supports targeted communication. Crucially, each individual agent actively selects which other agents to send messages to. This targeted communication behavior is operationalized via a simple signaturebased soft attention mechanism: along with the message, the sender broadcasts a key which encodes properties of agents the message is intended for, and is used by receivers to gauge the relevance of the message. This communication mechanism is learned implicitly, without any attention supervision, as a of end-to-end training using a downstream task-specific team reward. The inductive bias provided by soft attention in the communication architecture is sufficient to enable agents to 1) communicate agent-goal-specific messages (e.g. guide fire-fighter towards fire, bomb-defuser towards bomb, etc.), 2) be adaptive to variable team sizes (e.g. the size of the local neighborhood a self-driving car can communicate with changes as it moves), and 3) be interpretable through predicted attention probabilities that allow for inspection of which agent is communicating what message and to whom. Multi-agent systems fall at the intersection of game theory, distributed systems, and Artificial Intelligence in general BID18, and thus have a rich and diverse literature. Our work builds on and is related to prior work in deep multi-agent reinforcement learning, the centralized training and decentralized execution paradigm, and emergent communication protocols. Multi-Agent Reinforcement Learning (MARL). Within MARL (see BID1 for a survey), our work is related to recent efforts on using recurrent neural networks to approximate agent policies BID6, algorithms stabilizing multi-agent training BID13 BID5, and tasks in novel application domains such as coordination and navigation in 3D simulated environments BID17 BID16 BID8.Centralized Training & Decentralized Execution. Both BID19 and adopt a fully centralized framework at both training and test time -a central controller processes local observations from all agents and outputs a probability distribution over joint actions. In this setting, any controller (e.g. a fully-connected network) can be viewed as implicitly encoding communication. BID19 present an efficient architecture to learn a centralized controller invariant to agent permutations -by sharing weights and averaging as in BID23. proposes to replace averaging by an attentional mechanism to allow targeted interactions between agents. While closely related to our communication architecture, his work only considers fully supervised one-next-step prediction tasks, while we tackle the full reinforcement learning problem with tasks requiring planning over long time horizons. Moreover, a centralized controller quickly becomes intractable in real-world tasks with many agents and high-dimensional observation spaces (e.g. navigation in House3D BID22). To address these weaknesses, we adopt the framework of centralized learning but decentralized execution (following BID4 ; BID13 BID19 No No Yes Yes (REINFORCE) VAIN No Yes Yes No (Supervised) ATOC BID9 Yes sonable trade-off between allowing agents to globally coordinate while retaining tractability (since the communicated messages are much lower-dimensional than the observation space).Emergent Communication Protocols. Our work is also related to recent work on learning communication protocols in a completely end-to-end manner with reinforcement learning -from perceptual input (e.g. pixels) to communication symbols (discrete or continuous) to actions (e.g. navigating in an environment). While BID4 BID10 BID3 BID14 BID12 constrain agents to communicate with discrete symbols with the explicit goal to study emergence of language, our work operates in the paradigm of learning a continuous communication protocol in order to solve a downstream task BID19; BID9. While BID9 ) also operate in a decentralized execution setting and use an attentional communication mechanism, their setup is significantly different from ours as they use attention to decide when to communicate, not who to communicate with ('who' depends on a hand-tuned neighborhood parameter in their work). TAB2 summarizes the main axes of comparison between our work and previous efforts in this exciting space. Decentralized Partially Observable Markov Decision Processes (Dec-POMDPs). A Dec-POMDP is a cooperative multi-agent extension of a partially observable Markov decision process BID15 ). For N agents, it is defined by a set of states S describing possible configurations of all agents, a global reward function R, a transition probability function T, and for each agent i P 1,..., N a set of allowed actions A i, a set of possible observations Ω i and an observation function O i. Operationally, at each time step every agent picks an action a i based on its local observation ω i following its own stochastic policy π θi pa i |ω i q. The system randomly transitions to the next state s 1 given the current state and joint action T ps 1 |s, a 1,..., a N q. The agent team receives a global reward r " Rps, a 1,..., a N q while each agent receives a local observation of the new state O i pω i |s 1 q. Agents aim to maximize the total expected return J " ř T t"0 γ t r t where γ is a discount factor and T is the episode time horizon. Actor-Critic Algorithms. Policy gradient methods directly adjust the parameters θ of the policy in order to maximize the objective Jpθq " E s"pπ,a"π θ psq rRps, aqs by taking steps in the direction of ∇Jpθq. We can write the gradient with respect to the policy parameters as DISPLAYFORM0 where Q π ps, aq is called the action-value, it is the expected remaining discounted reward if we take action a in state s and follow policy π thereafter. Actor-Critic algorithms learn an approximation of the unknown true action-value functionQps, aq by e.g. temporal-difference learning BID20. ThisQps, aq is called the Critic while the policy π θ is called the Actor. Multi-Agent Actor-Critic. BID13 propose a multi-agent Actor-Critic algorithm adapted to centralized learning and decentralized execution. Each agent learns its own individual policy π θi pa i |ω i q conditioned on local observation ω i, using a centralized Critic which estimates the joint action-valueQps, a 1,..., a N q. We now describe our multi-agent communication architecture in detail. Recall that we have N agents with policies tπ 1,..., π N u, respectively parameterized by tθ 1,..., θ N u, jointly performing a cooperative task. At every timestep t, the ith agent for all i P t1,..., N u sees a local observation ω t i, and must select a discrete environment action a t i " π θi and a continuous communication message m t i, received by other agents at the next timestep, in order to maximize global reward r t " R. Since no agent has access to the underlying state of the environment s t, there is incentive in communicating with each other and being mutually helpful to do better as a team. Policies and Decentralized Execution. Each agent is essentially modeled as a Dec-POMDP augmented with communication. Each agent's policy π θi is implemented as a 1-layer Gated Recurrent Unit BID2. At every timestep, the local observation ω t i and a vector c t i aggregating messages sent by all agents at the previous timestep (described in more detail below) are used to update the hidden state h t i of the GRU, which encodes the entire message-action-observation history up to time t. From this internal state representation, the agent's policy π θi pa t i | h t i q predicts a categorical distribution over the space of actions, and another output head produces an outgoing message vector m t i. Note that for all our experiments, agents are symmetric and policies are instantiated from the same set of shared parameters; i.e. θ 1 "... " θ N. This considerably speeds up learning. Centralized Critic. Following prior work BID13 BID5, we operate under the centralized learning and decentralized execution paradigm wherein during training, a centralized critic guides the optimization of individual agent policies. The centralized Critic takes as input predicted actions ta t 1,..., a t N u and internal state representations th t 1,..., h t N u from all agents to estimate the joint action-valueQ t at every timestep. The centralized Critic is learned by temporal difference BID20 and the gradient of the expected return Jpθ i q " ErRs with respect to policy parameters is approximated by: DISPLAYFORM0 Note that compared to an individual criticQ i ph t i, a t i q for each agent, having a centralized critic leads to considerably lower variance in policy gradient estimates since it takes into account actions from all agents. At test time, the critic is not needed anymore and policy execution is fully decentralized. At the receiving end, each agent (indexed by j) predicts a query vector q DISPLAYFORM1 and uses it to compute a dot product with signatures of all N messages. This is scaled by 1{? d k followed by a softmax to obtain attention weight α ji for each message value vector: DISPLAYFORM2 Note that equation 2 also includes α ii corresponding to the ability to self-attend BID21, which we empirically found to improve performance, especially in situations when an agent has found the goal in a coordinated navigation task and all it is required to do is stay at the goal, so others benefit from attending to this agent's message but return communication is not needed. and internal state h t j are first used to predict the next internal state h 1 t j taking into account a first round of communication: DISPLAYFORM0 Next, h 1 t j is used to predict signature, query, value followed by repeating Eqns 1-4 for multiple rounds until we get a final aggregated message vector c t`1 j to be used as input at the next timestep. We evaluate our targeted multi-agent communication architecture on a variety of tasks and environments. All our models were trained with a batched synchronous version of the multi-agent ActorCritic described above, using RMSProp with a learning rate of 7ˆ10´4 and α " 0.99, batch size 16, discount factor γ " 0.99 and entropy regularization coefficient 0.01 for agent policies. All our agent policies are instantiated from the same set of shared parameters; i.e. θ 1 " ... " θ N . Each agent's GRU hidden state is 128-d, message signature/query is 16-d, and message value is 32-d (unless specified otherwise). All are averaged over 5 independent runs with different seeds. The SHAPES dataset was introduced by BID0 1, and originally created for testing compositional visual reasoning for the task of visual question answering. It consists of synthetic images of 2D colored shapes arranged in a grid (3ˆ3 cells in the original dataset) along with corresponding question-answer pairs. There are 3 shapes (circle, square, triangle), 3 colors (red, green, blue), and 2 sizes (small, big) in total (see FIG1 .We convert each image from SHAPES into an active environment where agents can now be spawned at different regions of the image, observe a 5ˆ5 local patch around them and their coordinates, and take actions to move around -tup, down, left, right, stayu. Each agent is tasked with navigating to a specified goal state in the environment -t'red', 'blue square', 'small green circle', etc. u -and the reward for each agent at every timestep is based on team performance i.e. r t " # agents on goal # agents . 1 github.com/jacobandreas/nmn2/tree/shapes (a) 4 agents have to find rred, red, green, blues respectively. t " 1: inital spawn locations; t " 2: 4 was on red at t " 1 so 1 and 2 attend to messages from 4 since they have to find red. 3 has found its goal (green) and is self-attending; t " 6: 4 attends to messages from 2 as 2 is on 4's target -blue; t " 8: 1 finds red, so 1 and 2 shift attention to 1; t " 21: all agents are at their respective goal locations and primarily self-attending.(b) 8 agents have to find red on a large 100ˆ100 environment. t " 7: Agent 2 finds red and signals all other agents; t " 7 to t " 150: All agents make their way to 2's location and eventually converge around red. Having a symmetric, team-based reward incentivizes agents to cooperate with each other in finding each agent's goal. For example, as shown in FIG1, if agent 2's goal is to find red and agent 4's goal is to find blue, it is in agent 4's interest to let agent 2 know if it passes by red (t " 2) during its exploration / quest for blue and vice versa (t " 6). SHAPES serves as a flexible testbed for carefully controlling and analyzing the effect of changing the size of the environment, no. of agents, goal configurations, etc. How does targeting work in the communication learnt by TarMAC? Recall that each agent predicts a signature and value vector as part of the message it sends, and a query vector to attend to incoming messages. The communication is targeted because the attention probabilities are a function of both the sender's signature and receiver's query vectors. So it is not just the receiver deciding how much of each message to listen to. The sender also sends out signatures that affects how much of each message is sent to each receiver. The sender's signature could encode parts of its observation most relevant to other agents' goals (for example, it would be futile to convey coordinates in the signature), and the message value could contain the agent's own location. For example, in FIG1, at t " 6, we see that when agent 2 passes by blue, agent 4 starts attending to agent 2. Here, agent 2's signature encodes the color it observes (which is blue), and agent 4's query encodes its goal (which is also blue) leading to high attention probability. Agent 2's message value encodes coordinates agent 4 has to navigate to, as can be seen at t " 21 when agent 4 reaches there. Environment and Task. The simulated traffic junction environments from BID19 consist of cars moving along pre-assigned, potentially intersecting routes on one or more road junctions. The total number of cars is fixed at N max and at every timestep, new cars get added to the environment with probability p arrive. Once a car completes its route, it becomes available to be sampled and added back to the environment with a different route assignment. Each car has a limited visibility of a 3ˆ3 region around it, but is free to communicate with all other cars. The action space for each car at every timestep is gas and brake, and the reward consists of a linear time penalty´0.01τ, where τ is the number of timesteps since car has been active, and a collision penalty r collision "´10. Quantitative Results. We compare our approach with CommNets BID19 on the easy and hard difficulties of the traffic junction environment. The easy task has one junction of two one-way roads on a 7ˆ7 grid with N max " 5 and p arrive " 0.30, while the hard task has four connected junctions of two-way roads on a 18ˆ18 grid with N max " 20 and p arrive " 0.05.See FIG5, 4b for an example of the four two-way junctions in the hard task. As shown in Model Interpretation. Interpreting the learned policies, FIG5 shows braking probabilities at different locations: cars tend to brake close to or right before entering traffic junctions, which is reasonable since junctions have the highest chances for collisions. Turning our attention to attention probabilities FIG5 ), we can see that cars are most-attended to when in the'internal grid' -right after crossing the 1st junction and before hitting the 2nd junction. These attention probabilities are intuitive: cars learn to attentively attend to specific sensitive locations with the most relevant local observations to avoid collisions. Finally, FIG5 compares total number of cars in the environment vs. number of cars being attended to with probability ą 0.1 at any time. Interestingly, these are (loosely) positively correlated, with Spearman's σ " 0.49, which shows that TarMAC is able to adapt to variable number of agents. Crucially, agents learn this dynamic targeting behavior purely from task rewards with no handcoding! Note that the right shift between the two curves is expected, as it takes a few timesteps of communication for team size changes to propagate. At a relative time shift of 3, the Spearman's rank correlation between the two curves goes up to 0.53.Message size vs. multi-stage communication. We study performance of TarMAC with varying message value size and number of rounds of communication on the'hard' variant of the traffic junction task. As can be seen in FIG3, multiple rounds of communication leads to significantly higher performance than simply increasing message size, demonstrating the advantage of multistage communication. In fact, decreasing message size to a single scalar performs almost as well as 64-d, perhaps because even a single real number can be sufficiently partitioned to cover the space of meanings/messages that need to be conveyed for this task. Finally, we benchmark TarMAC on a cooperative point-goal navigation task in House3D BID22. House3D provides a rich and diverse set of publicly-available 2 3D indoor environments, wherein agents do not have access to the top-down map and must navigate purely from first-person vision. Similar to SHAPES, the agents are tasked with finding a specified goal (such as 'fireplace'), spawned at random locations in the environment and allowed to communicate with each other and move around. Each agent gets a shaped reward based on progress towards the specified target. An episode is successful if all agents end within 0.5m of the target object in 50 navigation steps. TAB10 shows success rates on a find[fireplace] task in House3D. A no-communication navigation policy trained with the same reward structure gets a success rate of 62.1%. Mean-pooled communication (no attention) performs slightly better with a success rate of 64.3%, and TarMAC achieves the best success rate at 68.9%. FIG6 visualizes predicted navigation trajectories of 4 agents. Note that the communication vectors are significantly more compact (32-d) than the high-dimensional observation space, making our approach particularly attractive for scaling to large teams. We introduced TarMAC, an architecture for multi-agent reinforcement learning which allows targeted interactions between agents and multiple stages of collaborative reasoning at every timestep. Evaluation on three diverse environments show that our model is able to learn intuitive attention behavior and improves performance, with downstream task-specific team reward as sole supervision. While multi-agent navigation experiments in House3D show promising performance, we aim to exhaustively benchmark TarMAC on more challenging 3D navigation tasks because we believe this is where decentralized targeted communication can have the most impact -as it allows scaling to a large number of agents with large observation spaces. Given that the 3D navigation problem is hard in and of itself, it would be particularly interesting to investigate combinations with recent advances orthogonal to our approach (e.g. spatial memory, planning networks) with the TarMAC framework.

Targeted communication in multi-agent cooperative reinforcement learning

Represent the short summary for finding the corresponding scientific article (long)

A backward model of previous (state, action) given the next state, i.e. P(s_t, a_t | s_{t+1}), can be used to simulate additional trajectories terminating at states of interest! Improves RL learning efficiency.


Question: represent the scientific article to find a short tldr

Question: It is difficult for the beginners of etching latte art to make well-balanced patterns by using two fluids with different viscosities such as foamed milk and syrup. Even though making etching latte art while watching making videos which show the procedure, it is difficult to keep balance. Thus well-balanced etching latte art cannot be made easily. In this paper, we propose a system which supports the beginners to make well-balanced etching latte art by projecting a making procedure of etching latte art directly onto a cappuccino. The experiment show the progress by using our system. We also discuss about the similarity of the etching latte art and the design templates by using subtraction. Etching latte art is the practice of literally drawing on a coffee with a thin rod, such as a toothpick, in order to create images in the coffee. There are several kinds of making method of etching latte art depending on tools and toppings. A method which is often introduced as easy one for beginners is putting syrup directly onto milk foam and etching to make patterns as shown in Figure 1. The color combination automatically makes the drink look impressive by using syrup, so baristas are under less pressure to create a difficult design. However, it is difficult for beginners to imagine how they ought to put syrup and etch in order to make beautiful patterns since etching latte art offers two fluids with different viscosities. On top of this, even though they can watch videos which show a making procedure of etching latte art, etching latte art made by imitating hardly looks well-balanced. It is impossible to make well-balanced etching latte art without repeated practice. In this paper, we develop a support system which helps even beginners to make well-balanced latte art by directly projecting a making procedure of etching latte art using syrup, which has high needs, onto a cappuccino. Moreover, projecting a deformation of fluid with viscosity such as syrup which is difficult to imagine as animations in order to support beginners to understand the deformation of fluid with viscosity. We indicate the usefulness of this system through a questionnaire survey and the similarity to the design templates using subtraction. There are two kinds of latte art, etching and free pouring. The typical making method of the former is putting syrup on the milk foam and making patterns by etching whereas the latter does not use any tools and makes patterns with only flow of milk as poring. Hu and Chi proposed a simulation method which considered viscosity of milk to express the flow of milk in latte art. Their research expressed the flow of milk which is really similar to one in actual latte art. From the viewpoint of practicing latte art, however, users have to estimate paths of pouring milk and acquire the manipulation of the milk jug from the of the simulation. Moreover, it will not seem to be able to understand the flow of milk unless the users have advanced skills. Pikalo developed an automated latte art machine which used a modified inkjet cartridge to infuse tiny droplets of colorant into the upper layer of the beverage. By this machine, unlimitedly designed latte art can be easily made like a printer. This machine lets everyone have original latte art without any barista skills. However, this machine cannot make latte art with milk foam like one in free pour latte art. Therefore, baristas still have to practice latte art to make other kinds of latte art. Kawai et al. developed a free pour latte art support system by showing how to pour the milk to make latte art designed by the users as animated lines. This system targets baristas who have experience of making basic free pour latte art and know how much milk has to be poured. People who never made free pour latte art have to repeat the practice several times. Flagg et al. developed a painting support system by projecting a painting procedure on the canvas. In order to avoid users' shadow hiding a projected painting procedure, they put two projectors behind users. This system is quite large-scale and involves costs. Morioka et al. visually supported cooking by projecting how to chop ingredients on the proper place of that. This system is able to indicate how to chop ingredients, detailed notes, and cooking procedures which is hard to understand as cooking while reading a recipe. However, to use this system, users have to prepare a dedicated kitchen which has a hole on the ceiling. This system is quite large-scale since it projects instructions from a hole made on the ceiling. Xie et al. proposed an interactive system which lets common users build large-scale balloon art in an easy and enjoyable way by using spatial segmented reality solution. This system provides fabrication guidance to illustrate the differences between the depth maps of the target three-dimensional shape and the current work in progress. In addition, they design a shaking animation for each number to increase user immersion. Yoshida et al. proposed an architecture-scale, computerassisted digital fabrication method which used a depth camera and projection mapping. This system captured the current work using a depth camera, compared the scanned geometry with the target shape, and then projected the guiding information based on an evaluation. They mentioned that AR devices such as head-mounted displays (HMDs) could serve as interfaces, but would require calibration on each device. The proposed projector-camera guidance system can be prepared with a simple calibration process, and would also allow for intuitive information-sharing among workers. As indicated in this paper, projectors are able to project instructions on hand. Therefore, difference between instructions and users' manipulation would be reduced. We decided to support making etching latte art with a small projector as well. We show the system overview in Figure 2. The system configuration is written below (Figure 3). • A laptop computer connecting to a projector to show a making procedure on a cappuccino. • A small projector shows a making procedure on a cappuccino. • A tripod for a small projector. Firstly, users select a pattern from Heart Ring, Leaf, or Spider Web as shown in Figure 2 (a). Next, a making procedure of selected etching latte art is projected on the cappuccino (Figure 2(b) ). Then, the animation of syrup deformation is projected on the cappuccino (Figure 2(c) ). Finally, a making procedure is projected on the cappuccino once again. Users put syrup to trace the projected image. Then, etching latte art is completed by etching to trace the projected lines (Figure 2(d) ). Animations at each step can be played whenever the user wants. In our system, users select a pattern from three kinds of etching latte art (the first column in Table 1). After the selection, a making procedure written below is projected on the cappuccino. Firstly, proper places to put syrup corresponding on selected etching latte art are projected on the cappuccino (the second column in Table 1). Next, manipulation of the pick is projected on the cappuccino (the third column in Table 1). It is hard to confirm a making procedure of etching latte art in books since they just line up several frames of a making video as shown in Figure 1. Our system displays how to put syrup and how to manipulate the pick separately. As far as the authors know, there is no system which shows a making procedure of etching latte art like our system. It is difficult for beginners to imagine the syrup deformation by manipulating the pick. Our system helps users to understand the syrup deformation as manipulating the pick by directly projecting prepared animations (Table 2). In our system, the syrup is drawn in brown and the manipulation of the pick is drawn in blue. The animations were created by Adobe After Effects considering two fluids with different viscosities. It takes approximately 30 minutes to create each design template and two hours to create each animation of syrup deformation. We have developed a system which has functions mentioned in the previous sub-sections. As shown in Figure 3, a cappuccino is put in front of a user and a projector mounted on a tripod is placed on the left side of the cappuccino (if the user is left-handed, it is placed on the right side of the cappuccino). We project a making procedure of etching latte art and animations of syrup deformation from the top. In order to evaluate our system, we conducted an experiment. Twelve etching latte art beginners participated in the experiment. We divided them into two groups (Group 1 and Group 2). Participants make two etching latte art in different methods (making by oneself and making with our system). Group 1 makes etching latte art by themselves firstly. Then they make etching latte art with our system. Whereas Group 2 makes etching latte art with our system firstly. Then they make etching latte art by themselves. In this experiment, we selected patterns from three etching latte art in order to avoid everyone making the same pattern. After making etching latte art, participants filled out the questionnaire. We also took a questionnaire survey to inexperienced people in order to ask which etching latte art (made by oneself or made with our system) looks more well-balanced for each participant's etching latte art. Moreover, we created foreground images from subtraction for each design template and etching latte art in order to show which etching latte art is more similar to each design template. Making by Oneself Participants watch a making video of etching latte art they will make (Table 3). Then, they make etching latte art by themselves while watching the making video. Making with Our System Participants make etching latte art with our system. Firstly, they watch a making procedure projected on the cappuccino. Secondly, CG animations of syrup deformation (animation speed is almost the same as actually making etching latte art) are projected. Finally, a making procedure of etching latte art is projected on the cappuccino once again and participants make etching latte art by tracing the projected making procedure. Notes as Making Etching Latte Art Generally, baristas use well-steamed silky foamed milk which has fair quality tiny bubbles steamed by a steamer attached to an espresso machine for business use. However, it is difficult to make such good quality foamed milk with a household milk frother. Milk foamed by a milk frother has big bubbles and they break easily, so syrup put on such milk foam must spread. Therefore, in this experiment, participants made etching latte art with yoghurt. There is no difference between yoghurt and foamed milk as manipulating the pick. Thus, we consider that using yoghurt in the experiment does not affect the evaluation of this system. We do not have to care about the difference from using our system with a cappuccino. The of the making etching latte art are shown in Table 4. We compare and evaluate the etching latte art made by oneself (Table 4 "By oneself" Line) and the etching latte art made with our system (Table 4 "Our system" Line). Participants A, B, G, and H made "Heart Ring" (Table 4 A, B, G, H). Participants B and H put too much syrup as making by themselves. Therefore, the hearts are too big and their shape is not desired one. Whereas, the participants were able to adjust the amount of syrup as making with our system. As a , the shape of each heart is clearer and the etching latte art looks better quality. Participants C, D, I, and J made "Leaf" (Table 4 C, D, I, J). Participants C and J were not able to draw a line vertically. Participants J was not able to give the same distance between the syrup. Due to these problems, their etching latte art looks distorted. They were able to make well-balanced etching latte art which has the same distance between the syrup by using our system. Participants E, F, K, and L made "Spider Web" (Table 4 E, F, K, L). Participants E, F, and K were not able to draw a spiral with the certain space. The equally spaced spiral was made with our system and they were able to make better balanced etching latte art. As we mentioned, the etching latte art supported by our system is good quality. It is indicated that even beginners are able to make well-balanced etching latte art with our system. Participants compared two etching latte arts made in different methods. We conducted a questionnaire survey. The questionnaire consists of the five questions below. Question 1. Can you imagine how to make the etching latte art before watching the making video? Question 2. Is it easy to make the etching latte art while watching the making video? We also took some views, impressions, and improvement points of our system. The of the participants' questionnaires are shown in Table 5. The participants who answered 1pt or 2pt in Question 1 over 80 percent and the average is 1.92pt which is low. From this , patterns of etching latte art are complex for people who see them for the first time. It is difficult to imagine how to make it for them. The Average of Question 2 is 3.00pt, however, five participants answered 2pt and only two participants answered 5pt. There are some comments about this question. I could not understand where I should put the syrup since it was hard to get sense of the place and the size of syrup from the making video. It was difficult to manipulate the pick and I could not make desired pattern. Whereas, all participants answered 4pt or 5pt in Question 5. We consider that it is possible to make certain quality etching latte art with our system. Our system is popular with participants since it projects the making procedure directly on the cappuccino, so they do not have to watch another screen displaying making videos while making etching latte art. In Question 3 and 4, the participants who answered 4pt or 5pt over 90 percent and the average is over 4.50pt. We can say that the animations of syrup deformation in our system help users to properly understand how syrup deforms as drawing lines by the pick. Also, the animation speed of syrup deformation is ideal. The comments about our system are written below. It was easy to draw a line with a pick since I just needed to trace a line projected on the cappuccino, so it was clear where and how much syrup I should put. I was delighted that I could make etching latte art even though I had never tried it since the making procedure was easy to understand how to make it. The animation of syrup deformation indicated how the syrup deforms, so I could imagine it. From the of Table 4 and comments to, comparing to making latte art as watching making videos on another screen, even beginners are able to make etching latte art easily by using our system since it directly projects the making procedure on the cappuccino, so it is popular with users. We show improvement points from the participants. I might have been able to put proper amount of syrup if I also confirm how to put the syrup in animations. When I trace lines from left to right, the lines were hard to confirm because of my shadow at times. I got a bit confused because a lot of syrup stayed at the center and the line projected on the syrup could not be seen. Users adjust the amount of the syrup by putting it on the pattern projected on the cappuccino, however, this pattern is a static image, so some participants put too much syrup as written in comment. We consider that preparing animations which show how to put the syrup helps users to clearly understand and imagine the speed and the amount of the syrup. Also, as written in comments and, sometimes the projected making procedure was hard to see due to the position of the user's hands or the color of the . We will resolve this problem by using multiple projectors. We conducted a questionnaire survey about etching latte art had been made by participants. Sixty inexperienced people who had not participated in the experiment were asked which etching latte art (made by oneself or made with our system) looked more similar to the design template for each participant's etching latte art. The information that if the etching latte art had been made by oneself or with our system was hidden. The of inexperienced people's questionnaires are shown in Figre 4. Ten participants out of twelve got the that their etching latte art made with our system is similar to the design template than one made by themselves. About the etching latte art made by Participant G, they put the proper amount of syrup to the proper place even without our system. Their two etching latte arts are both look well-balanced. About the etching latte art made by Participant I, they could not trace the making procedure properly since they put the syrup too quickly. After making etching latte art with our system, they said that it might have been easier to make well-balanced etching latte art if our system showed how to put the syrup in animations as well. We will improve the system to resolve this issue by creating new animations which show the proper speed to put the syrup. We created foreground images from subtraction for each design template and etching latte art in order to quantitatively evaluate which etching latte art is more similar to the design template. White pixels in the foreground images indicate the difference between the design template and the etching latte art and black pixels indicate the same parts. We normalize the black pixels in order to quantify the similarity between each design template and etching latte art. The larger number indicates the higher similarity. The of the subtraction are shown in Table 6. Ten participants out of twelve got the that their etching latte art made with our system is similar to the design template than one made by themselves. About the etching latte art made by Participant A, the place of each heart was adjusted by our system, however, they put too much syrup. As a , the difference from the design template is big. About the etching latte art made by Participant D, the syrup is off to right. As a , the difference from the design template is big. However, the difference of similarities between the etching latte art made by themselves and the etching latte art made with our system is only 0.001 which is really little difference. We need to indicate the proper amount of the syrup more clearly in order to get higher similarity from subtraction. We have developed the system which supports etching latte art beginners to practice and make etching late art and also help them to understand the syrup deformation by directly projecting a making procedure and animations of syrup deformation onto the cappuccino. The participants' evaluations verified the usefulness of our system. The of the inexperienced people's questionnaire and the participants' questionnaire show that more than 80 percent of participants made better-balanced etching latte art with our system. However, each evaluation says that two participants made betterbalanced etching latte art by themselves and they are all different participants. From this , we confirm there are some instances that human beings suppose the etching latte art is similar to the design template even though the of the subtraction says it is not similar to the design template, and vice versa. In our future work, we will improve the system with considering what kind of etching latte art human beings prefer and develop a system which creates animations of syrup deformation automatically. We also handle the development factors got in the survey. Table 4: Experimental . Group 1 makes etching latte art by themselves firstly. Whereas Group 2 makes etching latte art with our system firstly. Table 5: Participants' questionnaire . Table 6: Results of subtraction. Similarities are represented by a number in the range of 0.000 to 1.000 (1.000 indicates totally the same as the design template).

represent this text to find a much much longer article about it with say hundreds-thousands of words!

We have developed an etching latte art support system which projects the making procedure directly onto a cappuccino to help the beginners to make well-balanced etching latte art.

Represent the TLDR for finding the corresponding scientific article.

We explored the visualization designs that can support chronic patients to present and review their health data with healthcare providers during clinical visits.


Question: Represent the article for finding a single-sentence TLDR

Question: We focus on temporal self-supervision for GAN-based video generation tasks. While adversarial training successfully yields generative models for a variety of areas, temporal relationship in the generated data is much less explored. This is crucial for sequential generation tasks, e.g. video super-resolution and unpaired video translation. For the former, state-of-the-art methods often favor simpler norm losses such as L2 over adversarial training. However, their averaging nature easily leads to temporally smooth with an undesirable lack of spatial detail. For unpaired video translation, existing approaches modify the generator networks to form spatio-temporal cycle consistencies. In contrast, we focus on improving the learning objectives and propose a temporally self-supervised algorithm. For both tasks, we show that temporal adversarial learning is key to achieving temporally coherent solutions without sacrificing spatial detail. We also propose a novel Ping-Pong loss to improve the long-term temporal consistency. It effectively prevents recurrent networks from accumulating artifacts temporally without depressing detailed features. We also propose a first set of metrics to quantitatively evaluate the accuracy as well as the perceptual quality of the temporal evolution. A series of user studies confirms the rankings computed with these metrics. Generative adversarial models (GANs) have been extremely successful at learning complex distributions such as natural images . However, for sequence generation, directly applying GANs without carefully engineered constraints typically in strong artifacts over time due to the significant difficulties introduced by the temporal changes. In particular, conditional video generation tasks are very challenging learning problems where generators should not only learn to represent the data distribution of the target domain, but also learn to correlate the output distribution over time with conditional inputs. Their central objective is to faithfully reproduce the temporal dynamics of the target domain and not resort to trivial solutions such as features that arbitrarily appear and disappear over time. In our work, we propose a novel adversarial learning method for a recurrent training approach that supervises both spatial content as well as temporal relationships. We apply our approach to two video-related tasks that offer substantially different challenges: video super-resolution (VSR) and unpaired video translation (UVT). With no ground truth motion available, the spatio-temporal adversarial loss and the recurrent structure enable our model to generate realistic while keeping the generated structures coherent over time. With the two learning tasks we demonstrate how spatio-temporal adversarial training can be employed in paired as well as unpaired data domains. In addition to the adversarial network which supervises the short-term temporal coherence, long-term consistency is self-supervised using a novel bi-directional loss formulation, which we refer to as "Ping-Pong" (PP) loss in the following. The PP loss effectively avoids the temporal accumulation of artifacts, which can potentially benefit a variety of recurrent architectures. The central contributions of our work are: a spatio-temporal discriminator unit together with a careful analysis of training objectives for realistic and coherent video generation tasks, a novel PP loss supervising long-term consistency, in addition to a set of metrics for quantifying temporal coherence based on motion estimation and perceptual distance. Together, our contributions lead to models that outperform previous work in terms of temporally-coherent detail, which we quantify with a wide range of metrics and user studies. features, but collapses to essentially static outputs of Obama. It manages to transfer facial expressions back to Trump using tiny differences encoded in its Obama outputs, instead of learning a meaningful mapping. Being able to establish the correct temporal cycle-consistency between domains, ours and RecycleGAN can generate correct blinking motions. Our model outperforms the latter in terms of coherent detail that is generated. Deep learning has made great progress for image generation tasks. While regular losses such as L 2 offer good performance for image super-resolution (SR) tasks in terms of PSNR metrics, GAN researchers found adversarial training to significantly improve the perceptual quality in multi-modal problems including image SR , image translations , and others. Perceptual metrics are proposed to reliably evaluate image similarity by considering semantic features instead of pixel-wise errors. Video generation tasks, on the other hand, require realistic to change naturally over time. Recent works in VSR improve the spatial detail and temporal coherence by either using multiple low-resolution (LR) frames as inputs (; ;), or recurrently using previously estimated outputs. The latter has the advantage to re-use high-frequency details over time. In general, adversarial learning is less explored for VSR and applying it in conjunction with a recurrent structure gives rise to a special form of temporal mode collapse, as we will explain below. For video translation tasks, GANs are more commonly used but discriminators typically only supervise the spatial content. E.g., does not employ temporal constrains and generators can fail to learn the temporal cycle-consistency. In order to learn temporal dynamics, RecycleGAN proposes to use a prediction network in addition to a generator, while a concurrent work chose to learn motion translation in addition to spatial content translation. Being orthogonal to these works, we propose a spatiotemporal adversarial training for both VSR and UVT and we show that temporal self-supervision is crucial for improving spatio-temporal correlations without sacrificing spatial detail. While L 2 temporal losses based on warping are used to enforce temporal smoothness in video style transfer tasks , concurrent GAN-based VSR work (Pérez-) and UVT work , it leads to an undesirable smooth over spatial detail and temporal changes in outputs. Likewise, the L 2 temporal metric represents a sub-optimal way to quantify temporal coherence and perceptual metrics that evaluate natural temporal changes are unavailable up to now. We work on this open issue, propose two improved temporal metric and demonstrate the advantages of temporal self-supervision over direct temporal losses. Previous work, e.g. tempoGAN and vid2vid (b), have proposed adversarial temporal losses to achieve time consistency. While tempoGAN employs a second temporal discriminator with multiple aligned frames to assess the realism of temporal changes, it is not suitable for videos, as it relies on ground truth motions and employs a single-frame processing that is sub-optimal for natural images. On the other hand, vid2vid focuses on paired video translations and proposes a video discriminator based on a conditional motion input that is estimated from the paired ground-truth sequences. We focus on more difficult unpaired translation tasks instead, and demonstrate the gains in quality of our approach in the evaluation section. For tracking and optical flow estimation, L2-based time-cycle losses were proposed to constrain motions and tracked correspondences using symmetric video inputs. By optimizing indirectly via motion compensation or tracking, this loss improves the accuracy of the . For video generation, we propose a PP loss that also makes use of symmetric sequences. However, we directly constrain the PP loss via the generated video content, which successfully improves the long-term temporal consistency in the video . Generative Network Before explaining the temporal self-supervision in more detail, we outline the generative model to be supervised. Our generator networks produce image sequences in a frame-recurrent manner with the help of a recurrent generator G and a flow estimator F. We follow previous work, where G produces output g t in the target domain B from conditional input frame a t from the input domain A, and recursively uses the previous generated output g t−1. F is trained to estimate the motion v t between a t−1 and a t, which is then used as a motion compensation that aligns g t−1 to the current frame. This procedure, also shown in Fig. 2a ), can be summarized as: g t = G(a t, W (g t−1, v t)), where v t = F(a t−1, a t) and W is the warping operation. While one generator is enough to map data from A to B for paired tasks such as VSR, unpaired generation requires a second generator to establish cycle consistency. . In the UVT task, we use two recurrent generators, mapping from domain A to B and back. As shown in Fig. 2b, vt)) to enforce consistency. A ResNet architecture is used for the VSR generator G and a encoder-decoder structure is applied to UVT generators and F. We intentionally keep generators simple and in line with previous work, in order to demonstrate the advantages of the temporal self-supervision that we will explain in the following paragraphs. Spatio-Temporal Adversarial Self-Supervision The central building block of our approach is a novel spatio-temporal discriminator D s,t that receives triplets of frames. This contrasts with typically used spatial discriminators which supervise only a single image. By concatenating multiple adjacent frames along the channel dimension, the frame triplets form an important building block for learning because they can provide networks with gradient information regarding the realism of spatial structures as well as short-term temporal information, such as first-and second-order time derivatives. We propose a D s,t architecture, illustrated in Fig. 3 and Fig. 4, that primarily receives two types of triplets: three adjacent frames and the corresponding warped ones. We warp later frames backward and previous ones forward. While original frames contain the full spatio-temporal infor-mation, warped frames more easily yield temporal information with their aligned content. For the input variants we use the following notation: Ig = {gt−1, gt, gt+1}, I b = {bt−1, bt, bt+1}; For VSR tasks, D s,t should guide the generator to learn the correlation between LR inputs and highresolution (HR) targets. Therefore, three LR frames I a = {a t−1, a t, a t+1} from the input domain are used as a conditional input. The input of D s,t can be summarized as I b s,t = {I b, I wb, I a} labelled as real and the generated inputs I g s,t = {I g, I wg, I a} labelled as fake. In this way, the conditional D s,t will penalize G if I g contains less spatial details or unrealistic artifacts according to I a, I b. At the same time, temporal relationships between the generated images I wg and those of the ground truth I wb should match. With our setup, the discriminator profits from the warped frames to classify realistic and unnatural temporal changes, and for situations where the motion estimation is less accurate, the discriminator can fall back to the original, i.e. not warped, images. For UVT tasks, we demonstrate that the temporal cycleconsistency between different domains can be established using the supervision of unconditional spatio-temporal discriminators. This is in contrast to previous work which focuses on the generative networks to form spatio-temporal cycle links. Our approach actually yields improved , as we will show below, and Fig. 1 shows a preview of the quality that can be achieved using spatio-temporal discriminators. In practice, we found it crucial to ensure that generators first learn reasonable spatial features, and only then improve their temporal correlation. Therefore, different to the D s,t of VST that always receives 3 concatenated triplets as an input, the unconditional D s,t of UVT only takes one triplet at a time. Focusing on the generated data, the input for a single batch can either be a static triplet of Isg = {gt, gt, gt}, the warped triplet I wg, or the original triplet I g. The same holds for the reference data of the target domain, as shown in Fig. 4. With sufficient but complex information contained in these triplets, transition techniques are applied so that the network can consider the spatio-temporal information step by step, i.e., we initially start with 100% static triplets I sg as the input. Then, over the course of training, 25% of them transition to I wg triplets with simpler temporal information, with another 25% transition to I g afterwards, leading to a (50%,25%,25%) distribution of triplets. Details of the transition calculations are given in Appendix D. Here, the warping is again performed via F. While non-adversarial training typically employs loss formulations with static goals, the GAN training yields dynamic goals due to discriminative networks discovering the learning objectives over the course of the training run. Therefore, their inputs have strong influence on the training process and the final . Modifying the inputs in a controlled manner can lead to different and substantial improvements if done correctly, as will be shown in Sec. 4. Although the proposed concatenation of several frames seems like a simple change that has been used in a variety of projects, it is an important operation that allows discriminators to understand spatio-temporal data distributions. As will be shown below, it can effectively reduce temporal problems encountered by spatial GANs. While L 2 −based temporal losses are widely used in the field of video generation, the spatiotemporal adversarial loss is crucial for preventing the inference of blurred structures in multi-modal data-sets. Compared to GANs using multiple discriminators, the single D s,t network can learn to balance the spatial and temporal aspects from the reference data and avoid inconsistent sharpness as well as overly smooth . Additionally, by extracting shared spatio-temporal features, it allows for smaller network sizes. Self-Supervision for Long-term Temporal Consistency When relying on a previous output as input, i.e., for frame-recurrent architectures, generated structures easily accumulate frame by frame. In an adversarial training, generators learn to heavily rely on previously generated frames and can easily converge towards strongly reinforcing spatial features over longer periods of time. For videos, this especially occurs along directions of motion, and these solutions can be seen as a special form of temporal mode collapse. We have noticed this issue in a variety of recurrent architectures, examples are shown in Fig. 5 a) and the Dst in Fig. 1. While this issue could be alleviated by training with longer sequences, we generally want generators to be able to work with sequences of arbitrary length for inference. To address this inherent problem of recurrent generators, we propose a new Result trained with PP loss. These artifacts are removed successfully for the latter. c) The ground-truth image. With our PP loss (shown on the right), the L 2 distance between gt and g t is minimized to remove drifting artifacts and improve temporal coherence. bi-directional "Ping-Pong" loss. For natural videos, a sequence with forward order as well as its reversed counterpart offer valid information. Thus, from any input of length n, we can construct a symmetric PP sequence in form of a 1,...a n−1, a n, a n−1,...a 1 as shown in Fig. 5. When inferring this in a frame-recurrent manner, the generated should not strengthen any invalid features from frame to frame. Rather, the should stay close to valid information and be symmetric, i.e., the forward gt = G(at, gt−1) and the one generated from the reversed part, g t = G(at, g t+1), should be identical. Based on this observation, we train our networks with extended PP sequences and constrain the generated outputs from both "legs" to be the same using the loss: Note that in contrast to the generator loss, the L 2 norm is a correct choice here: We are not faced with multi-modal data where an L 2 norm would lead to undesirable averaging, but rather aim to constrain the recurrent generator to its own, unique version over time. The PP terms provide constraints for short term consistency via gn−1 − gn−1 2, while terms such as g1 − g1 2 prevent long-term drifts of the . As shown in Fig. 5 (b), this PP loss successfully removes drifting artifacts while appropriate high-frequency details are preserved. In addition, it effectively extends the training data set, and as such represents a useful form of data augmentation. A comparison is shown in Appendix E to disentangle the effects of the augmentation of PP sequences and the temporal constrains. The show that the temporal constraint is the key to reliably suppressing the temporal accumulation of artifacts, achieving consistency, and allowing models to infer much longer sequences than seen during training. Perceptual Loss Terms As perceptual metrics, both pre-trained NNs (; a) and in-training discriminators were successfully used in previous work. Here, we use feature maps from a pre-trained VGG-19 network , as well as D s,t itself. In the VSR task, we can encourage the generator to produce features similar to the ground truth ones by increasing the cosine similarity between their feature maps. In UVT tasks without paired ground truth data, we still want the generators to match the distribution of features in the target domain. Similar to a style loss in traditional style transfer , we here compute the D s,t feature correlations measured by the Gram matrix instead. The feature maps of D s,t contain both spatial and temporal information, and hence are especially well suited for the perceptual loss. We now explain how to integrate the spatio-temporal discriminator into the paired and unpaired tasks. We use a standard discriminator loss for the D s,t of VSR and a least-square discriminator loss for the D s,t of UVT. Correspondingly, a non-saturated L adv is used for the G and F of VSR, and a least-squares one is used for the UVT generators. As summarized in Table 1, G and F are trained with the mean squared loss L content, adversarial losses L adv, perceptual losses L φ, the PP loss L PP, and a warping loss L warp, where again g, b and Φ stand for generated samples, ground truth images and feature maps of VGG-19 or D s,t. We only show losses for the mapping from A to B for UVT tasks, as the backward mapping simply mirrors the terms. We refer to our full model for both tasks as TecoGAN below. 1 Training parameters and details are given in Appendix G. In the following, we illustrate the effects of temporal supervision using two ablation studies. In the first one, models trained with ablated loss functions show how L adv and L PP change the overall learning objectives. Next, full UVT models are trained with different D s,t inputs. This highlights how differently the corresponding discriminators converge to different spatio-temporal equilibriums, and the general importance of providing suitable data distributions from the target domain. While we provide qualitative and quantitative evaluations in the following, we also refer the reader to our supplemental html document 2, with video clips that more clearly highlight the temporal differences. Loss Ablation Study Below we compare variants of our full TecoGAN model to EnhanceNet (ENet) , FRVSR , and DUF for VSR, and CycleGAN and RecycleGAN for UVT. Specifically, ENet and CycleGAN represent state-of-the-art single-image adversarial models without temporal information, FRVSR and DUF are state-of-the-art VSR methods without adversarial losses, and RecycleGAN is a spatial adversarial model with a prediction network learning the temporal evolution. For VSR, we first train a DsOnly model that uses a frame-recurrent G and F with a VGG-19 loss and only the regular spatial discriminator. Compared to ENet, which exhibits strong incoherence due to the lack of temporal information, DsOnly improves temporal coherence thanks to the framerecurrent connection, but there are noticeable high-frequency changes between frames. The temporal profiles of DsOnly in Fig. 6 and 8, correspondingly contain sharp and broken lines. When adding a temporal discriminator in addition to the spatial one (DsDt), this version generates more coherent , and its temporal profiles are sharp and coherent. However, DsDt often produces the drifting artifacts discussed in Sec. 3, as the generator learns to reinforce existing details from previous frames to fool D s with sharpness, and satisfying D t with good temporal coherence in the form of persistent detail. While this strategy works for generating short sequences during training, the strengthening effect can lead to very undesirable artifacts for long-sequence inferences. By adding the self-supervision for long-term temporal consistency L pp, we arrive at the DsDtPP model, which effectively suppresses these drifting artifacts with an improved temporal coherence. In Fig. 6 and Fig. 8, DsDtPP in continuous yet detailed temporal profiles without streaks from temporal drifting. Although DsDtPP generates good , it is difficult in practice to balance the generator and the two discriminators. The shown here were achieved only after numerous runs manually tuning the weights of the different loss terms. By using the proposed D s,t discriminator instead, we get a first complete model for our method, denoted as TecoGAN. This network is trained with a discriminator that achieves an excellent quality with an effectively halved network size, as illustrated on the right of Fig. 7. The single discriminator correspondingly leads to a significant reduction in resource usage. Using two discriminators requires ca. 70% more GPU memory, and leads to a reduced training performance by ca. 20%. The TecoGAN model yields similar perceptual and temporal quality to DsDtPP with a significantly faster and more stable training. Since the TecoGAN model requires less training resources, we also trained a larger generator with 50% more weights. In the following we will focus on this larger single-discriminator architecture with PP loss as our full TecoGAN model for VSR. Compared to the TecoGAN model, it can generate more details, and the training process is more stable, indicating that the larger generator and D s,t are more evenly balanced. Result images and temporal profiles are shown in Fig. 6 and Fig. 8. Video are shown in Sec. 4 of the supplemental material. We also carry out a similar ablation study for the UVT task. Again, we start from a single-image GAN-based model, a CycleGAN variant which already has two pairs of spatial generators and discriminators. Then, we train the DsOnly variant by adding flow estimation via F and extending the spatial generators to frame-recurrent ones. By augmenting the two discriminators to use the triplet inputs proposed in Sec. 3, we arrive at the Dst model with spatio-temporal discriminators, which does not yet use the PP loss. Although UVT tasks are substantially different from VSR tasks, the comparisons in Fig. 1 and Sec. 4.6 of our supplemental material yield similar . In these tests, we use renderings of 3D fluid simulations of rising smoke as our unpaired training data. These simulations are generated with randomized numerical simulations using a resolution of 64 3 for domain A and 256 3 for domain B, and both are visualized with images of size 256 2. Therefore, video translation from domain A to B is a tough task, as the latter contains significantly more turbulent and small-scale motions. With no temporal information available, the CycleGAN variant generates HR smoke that strongly flickers. The DsOnly model offers better temporal coherence by relying on its frame-recurrent input, but it learns a solution that largely ignores the current input and fails to keep reasonable spatio-temporal cycle-consistency links between the two domains. On the contrary, our D s,t enables the Dst model to learn the correlation between the spatial and temporal aspects, thus improving the cycle-consistency. However, without L pp, the Dst model (like the DsDt model of VSR) reinforces detail over time in an undesirable way. This manifests itself as inappropriate smoke density in empty regions. Using our full TecoGAN model which includes L pp, yields the best , with detailed smoke structures and very good spatio-temporal cycle-consistency. For comparison, a DsDtPP model involving a larger number of separate networks, i.e. four discriminators, two frame-recurrent generators and the F, is trained. By weighting the temporal adversarial losses from Dt with 0.3 and the spatial ones from Ds with 0.5, we arrived at a balanced training run. Although this model performs similarly to the TecoGAN model on the smoke dataset, the proposed spatio-temporal D s,t architecture represents a more preferable choice in practice, as it learns a natural balance of temporal and spatial components by itself, and requires fewer resources. Continuing along this direction, it will be interesting future work to evaluate variants, such as a shared D s,t for both domains, i.e. a multi-class classifier network. Besides the smoke dataset, an ablation study for the Obama and Trump dataset from Fig. 1 shows a very similar behavior, as can be seen in the supplemental material. Spatio-temporal Adversarial Equilibriums Our evaluation so far highlights that temporal adversarial learning is crucial for achieving spatial detail that is coherent over time for VSR, and for enabling the generators to learn the spatio-temporal correlation between domains in UVT. Next, we will shed light on the complex spatio-temporal adversarial learning objectives by varying the information provided to the discriminator network. The following tests D s,t networks that are identical apart from changing inputs, and we focus on the smoke dataset. In order to learn the spatial and temporal features of the target domain as well as their correlation, the simplest input for D s,t consists of only the original, unwarped triplets, i.e. {I g or I b}. Using these, we train a baseline model, which yields a sub-optimal quality: it lacks sharp spatial structures, and contains coherent but dull motions. Despite containing the full information, these input triplets prevent D s,t from providing the desired supervision. For paired video translation tasks, the vid2vid network achieves improved temporal coherence by using a video discriminator to supervise the output sequence conditioned with the ground-truth motion. With no ground-truth data available, we train a vid2vid variant by using the estimated motions and original triplets, i.e {I g + F (g t−1, g t) + F (g t+1, g t) or I b + F (b t−1, b t) + F (b t+1, b t)}, as the input for D s,t. However, the do not significantly improve. The motions are only partially reliable, and hence don't help for the difficult unpaired translation task. Therefore, the discriminator still fails to fully correlate spatial and temporal features. We then train a third model, concat, using the original triplets and the warped ones, i.e. {I g +I wg or I b +I wb}. In this case, the model learns to generate more spatial details with a more vivid motion. I.e., the improved temporal information from the warped triplets gives the discriminator important cues. However, the motion still does not fully resemble the target domain. We arrive at our final TecoGAN model for UVT by controlling the composition of the input data: as outlined above, we first provide only static triplets {I sg or I sb}, and then apply the transitions of warped triplets {I wg or I wb}, and original triplets {I g or I b} over the course of training. In this way, the network can first learn to extract spatial features, and build on them to establish temporal features. Finally, discriminators learn features about the correlation of spatial and temporal content by analyzing the original triplets, and provide gradients such that the generators learn to use the motion information from the input and establish a correlation between the motions in the two unpaired domains. Consequently, the discriminator, despite receiving only a single triplet at once, can guide the generator to produce detailed structures that move coherently. Video comparisons are shown in Sec 5. of the supplemental material. Results and Metric Evaluation While the visual discussed above provide a first indicator of the quality our approach achieves, quantitative evaluations are crucial for automated evaluations across larger numbers of samples. Below we focus on the VSR task as ground-truth data is available in this case. We conduct user studies and present evaluations of the different models w.r.t. established spatial metrics. We also motivate and propose two novel temporal metrics to quantify temporal coherence. A visual summary is shown in Fig. 7. For evaluating image demonstrated that there is an inherent trade-off between the perceptual quality of the and the distortion measured with vector norms or lowlevel structures such as PSNR and SSIM. On the other hand, metrics based on deep feature maps such as LPIPS can capture more semantic similarities. We measure the PSNR and LPIPS using the Vid4 scenes. With a PSNR decrease of less than 2dB over DUF which has twice the model size of ours, TecoGAN outperforms all methods by more than 40% on LPIPS. While traditional temporal metrics based on vector norm differences of warped frames, e.g. T-diff, can be easily deceived by very blurry , e.g. bi-cubic interpolated ones, we propose to use a tandem of two new metrics, tOF and tLP, to measure the consistence over time. tOF measures the pixel-wise difference of motions estimated from sequences, and tLP measures perceptual changes over time using deep feature map: where OF represents an optical flow estimation with and LP is the perceptual LPIPS metric. In tLP, the behavior of the reference is also considered, as natural videos exhibit a certain degree of changes over time. In conjunction, both pixel-wise differences and perceptual changes are crucial for quantifying realistic temporal coherence. While they could be combined into a single score, we list both measurements separately, as their relative importance could vary in different application settings. Our evaluation with these temporal metrics in Table 2 shows that all temporal adversarial models outperform spatial adversarial ones, and the full TecoGAN model performs very well: With a large amount of spatial detail, it still achieves good temporal coherence, on par with non-adversarial methods such as DUF and FRVSR. For VSR, we have confirmed these automated evaluations with several user studies. Across all of them, we find that the majority of the participants considered the TecoGAN to be closest to the ground truth. For the UVT tasks, where no ground-truth data is available, we can still evaluate tOF and tLP metrics by comparing the motion and the perceptual changes of the output data w.r.t. the ones from the input data, i.e., tOF = OF (at−1, at) − OF (g Table 3, although it is worth to point out that the tOF is less informative in this case, as the motion in the target domain is not necessarily pixel-wise aligned with the input. Overall, TecoGAN achieves good tLP scores thanks to its temporal coherence, on par with RecycleGAN, and its spatial detail is on par with CycleGAN. As for VSR, a perceptual evaluation by humans in the right column of Table 3 confirms our metric evaluations for the UVT task (details in Appendix C). In paired as well as unpaired data domains, we have demonstrated that it is possible to learn stable temporal functions with GANs thanks to the proposed discriminator architecture and PP loss. We have shown that this yields coherent and sharp details for VSR problems that go beyond what can be achieved with direct supervision. In UVT, we have shown that our architecture guides the training process to successfully establish the spatio-temporal cycle consistency between two domains. These are reflected in the proposed metrics and user studies. While our method generates very realistic for a wide range of natural images, our method can generate temporally coherent yet sub-optimal details in certain cases such as under-resolved faces and text in VSR, or UVT tasks with strongly different motion between two domains. For the latter case, it would be interesting to apply both our method and motion translation from concurrent work . This can make it easier for the generator to learn from our temporal self supervision. In our method, the interplay of the different loss terms in the non-linear training procedure does not provide a guarantee that all goals are fully reached every time. However, we found our method to be stable over a large number of training runs, and we anticipate that it will provide a very useful basis for wide range of generative models for temporal data sets. In the following, we first provide qualitative analysis(Appendix A) using multiple that are mentioned but omitted in our main document due to space constraints. We then explain details of the metrics and present the quantitative analysis based on them(Appendix B). The conducted user studies are in support of our TecoGAN network and proposed temporal metrics (Appendix C). Then, we give technical details of our spatio-temporal discriminator (Sec. D), details of network architectures and training parameters (Appendix F, Appendix G). In the end, we discuss the performance of our approach in Appendix H. For the VSR task, we test our model on a wide range of video data, including the generally used Vid4 dataset shown in Fig. 8 and 12, detailed scenes from the movie Tears of Steel shown in Fig. 12, and others shown in Fig. 9. As mentioned in the main document, the TecoGAN model is trained with down-sampled inputs and it can similarly work with original images that were not down-sampled or filtered, such as a data-set of real-world photos . In Fig. 10, we compared our to two other methods that have used the same dataset. With the help of adversarial learning, our model is able to generate improved and realistic details in down-sampled images as well as captured images. For UVT tasks, we train models for Obama and Trump translations, LR-and HR-smoke simulation translations, as well as translations between smoke simulations and real-smoke captures. While smoke simulations usually contain strong numerical viscosity with details limited by the simulation resolution, the real smoke, captured using the setup from , contains vivid fluid motions with many vortices and high-frequency details. As shown in Fig. 11, our method can be used to narrow the gap between simulations and real-world phenomenon. Spatial Metrics We evaluate all VSR methods with PSNR together with the human-calibrated LPIPS metric . While higher PSNR values indicate a better pixel-wise accuracy, lower LPIPS values represent better perceptual quality and closer semantic similarity. Mean values of the Vid4 scenes are shown on the top of Table 4. Trained with direct vector norms losses, FRVSR and DUF achieve high PSNR scores. However, the undesirable smoothing induced by these losses manifests themselves in larger LPIPS distances. ENet, on the other hand, with no information from neighboring frames, yields the lowest PSNR and achieves an LPIPS score that is only slightly better than DUF and FRVSR. TecoGAN model with adversarial training achieves an excellent LPIPS score, with a PSNR decrease of less than 2dB over DUF, which is very reasonable, since PSNR and perceptual quality were shown to be anti-correlated , especially in regions where PSNR is very high. Based on good perceptual quality and reasonable pixel-wise accuracy, TecoGAN outperforms all other methods by more than 40% for LPIPS. Temporal Metrics For both VSR and UVT, evaluating temporal coherence without ground-truth motion is a very challenging problem. The metric T-diff = gt − W (gt−1, vt) 1 was used by as a rough assessment of temporal differences. As shown on bottom of Table 4, T-diff, due to its local nature, is easily deceived by blurry method such as the bi-cubic interrelation and can not correlate well with visual assessments of coherence. By measuring the pixel-wise motion difference using tOF in together with the perceptual changes over time using tLP, we show the temporal evaluations for the VSR task in the middle of Table 4. Not surprisingly, the of ENet show larger errors for all metrics due to their strongly flickering content. Bi-cubic up-sampling, DUF, and FRVSR achieve very low T-diff errors due to their smooth , representing an easy, but undesirable avenue for achieving coherency. However, the overly smooth changes of the former two are identified by the tLP scores. While our DsOnly model generates sharper at the expense of temporal coherence, it still outperforms ENet there. By adding temporal information to discriminators, our DsDt, DsDt+PP, TecoGAN and TecoGAN improve in terms of temporal metrics. Especially the full TecoGAN model stands out here. For the UVT tasks, temporal motions are evaluated by comparing to the input sequence. With sharp spatial features and coherent motion, TecoGAN outperforms previous work on the Obama&Trump dataset, as shown in Table 3. Since temporal metrics can trivially be reduced for blurry image content, we found it important to evaluate with a combination of spatial and temporal metrics. Given that perceptual metrics are already widely used for image evaluations, we believe it is the right time to consider perceptual changes in temporal evaluations, as we did with our proposed temporal coherence metrics. Although not perfect, they are not easily deceived. Specifically, tOF is more robust than a direct pixel-wise metric as it compares motions instead of image content. In the supplemental material, we visualize the motion difference and it can well reflect the visual inconsistencies. On the other hand, we found that our calculation of tLP is a general concept that can work reliably with different perceptual metric: When repeating the tLP evaluation with the PieAPP metric instead of LP, i.e., tPieP = f (yt−1, yt) − f (gt−1, gt) 1, where f(·) indicates the perceptual error function of PieAPP, we get close to identical , listed in Fig. 13. The from tPieP also closely match the LPIPS-based evaluation: our network architecture can generate realistic and temporally coherent detail, and the metrics we propose allow for a stable, automated evaluation of the temporal perception of a generated video sequence. Besides the previously evaluated the Vid4 dataset, with graphs shown in Fig. 14, 15, we also get similar evaluation on the Tears of Steel data-sets (room, bridge, and face, in the following referred to as ToS scenes) and corresponding are shown in Table 5 and Fig. 16. In all tests, we follow the procedures of previous work (; to make the outputs of all methods comparable, i.e., for all images, we first exclude spatial borders with a distance of 8 pixels to the image sides, then further shrink borders such that the LR input image is divisible by 8 and for spatial metrics, we ignore the first two and the last two frames, while for temporal metrics, we ignore first three and last two frames, as an additional previous frame is required for inference. In the following, we conduct user studies for the Vid4 scenes. By comparing the user study and the metric breakdowns shown in Table 4, we found our metrics to reliably capture the human temporal perception, as shown in Appendix C. We conduct several user studies for the VSR task using five different methods, namely bi-cubic interpolation, ENet, FRVSR, DUF and our TecoGAN. The established 2AFC design (Fechner & Wundt, 1889;) is applied, i.e., participants have a pair-wise choice, with the groundtruth video shown as reference. One example can be seen in Fig. 17. The videos are synchronized and looped until user made the final decision. With no control to stop videos, users Participants cannot stop or influence the playback, and hence can focus more on the whole video, instead of specific spatial details. Videos positions (left/A or right/B) are randomized. After collecting 1000 votes from 50 users for every scene, i.e. twice for all possible pairs (5 × 4/2 = 10 pairs), we follow common procedure and compute scores for all models with the Bradley-Terry model. The outcomes for the Vid4 scenes can be seen in Fig. 18 (overall scores are listed in Table 2 of the main document). From the Bradley-Terry scores for the Vid4 scenes we can see that the TecoGAN model performs very well, and achieves the first place in three cases, as well as a second place in the walk scene. The latter is most likely caused by the overall slightly smoother images of the walk scene, in conjunction with the presence of several human faces, where our model can lead to the generation of unexpected details. However, overall the user study shows that users preferred the TecoGAN output over the other two deep-learning methods with a 63.5% probability. This also matches with our metric evaluations. In Table 4, while TecoGAN achieves spatial (LPIPS) improvements in all scenes, DUF and FRVSR are not far behind in the walk scene. In terms of temporal metrics tOF and tLP, TecoGAN achieves similar or lower scores compared to FRVSR and DUF for calendar, foliage and city scenes. The lower performance of our model for the walk scene is likewise captured by higher tOF and tLP scores. Overall, the metrics confirm the performance of our TecoGAN approach and match the of the user studies, which indicate that our proposed temporal metrics successfully capture important temporal aspects of human perception. For UVT tasks which have no ground-truth data, we carried out two sets of user studies: One uses an arbitrary sample from the target domain as the reference and the other uses the actual input from the source domain as the reference. On the Obama&Trump data-sets, we evaluate from CycleGAN, RecycleGAN, and TecoGAN following the same modality, i.e. a 2AFC design with 50 users for each run. E.g., on the left of Fig. 19, users evaluate the generated Obama in reference with the input Trump on the y-axis, while an arbitrary Obama video is shown as the reference on the x-axis. Effectively, the y-axis is more important than the x-axis as it indicates whether the translated preserves the original expression. A consistent ranking of TecoGAN > RecycleGAN > CycleGAN is shown on the y-axis with clear separations, i.e. standard errors don't overlap. The x-axis indicates whether the inferred matches the general spatio-temporal content of the target domain. Our TecoGAN model also receives the highest scores here, although the responses are slightly more spread out. On the right of Fig. 19, we summarize both studies in a single graph highlighting that the TecoGAN model is consistently preferred by the participants of our user studies. Figure 20: Near image boundaries, flow estimation is less accurate and warping often fails to align well. First two columns show original and warped frames and the third one shows differences after warping (ideally all black). The top row shows things move into the view with problems near lower boundaries, while the second row has objects moving out of the view. with the help of the flow estimation network F. However, at the boundary of images, the output of F is usually less accurate due to the lack of reliable neighborhood information. There is a higher chance that objects move into the field of view, or leave suddenly, which significantly affects the images warped with the inferred motion. An example is shown in Fig. 20 Figure 18: Tables and bar graphs to warp the previous frame in accordance with the detail that G can synthesize. However, F does not adjust the motion estimation only to reduce the adversarial loss. Curriculum Learning for UVT Discriminators As mentioned in the main part, we train the UVT D s,t with 100% spatial triplets at the very beginning. During training, 25% of them gradually transfer into warped triplets and another 25% transfer into original triplets. The transfer of the warped triplets can be represented as: (1−α)I cg +αI wg, with α growing form 0 to 1. For the original triplets, we additionally fade the "warping" operation out by using (1 − α)I cg + α{W (g t−1, v t * β), g t, W (g t+1, v t * β)}, again with α growing form 0 to 1 and β decreasing from 1 to 0. We found this smooth transition to be helpful for a stable training. Since training with sequences of arbitrary length is not possible with current hardware, problems such as the streaking artifacts discussed above generally arise for recurrent models. In the proposed PP loss, both the Ping-Pang data augmentation and the temporal consistency constraint contribute to solving these problems. In order to show their separated contributions, we trained another TecoGAN variant that only employs the data augmentation without the constraint (i.e., λ p = 0 in Table 1 In this section, we use the following notation to specify all network architectures used: conc represents the concatenation of two tensors along the channel dimension; C/CT (input, kernel size, output channel, stride size) stands for the convolution and transposed convolution operation, respectively; "+" denotes element-wise addition; BilinearUp2 up-samples input tensors by a factor of 2 using bi-linear interpolation; BicubicResize4(input) increases the resolution of the input tensor to 4 times higher via bi-cubic up-sampling; Dense(input, output size) is a densely-connected layer, which uses Xavier initialization for the kernel weights. The architecture of our VSR generator G is: conc(x t, W (g t−1, v t)) → l in; C(l in, 3, 64, 1), ReLU → l 0; ResidualBlock(l i) → l i+1 with i = 0,..., n − 1; CT (l n, 3, 64, 2), ReLU → l up2; CT (l up2, 3, 64, 2), ReLU → l up4; C(l up4, 3, 3, 1), ReLU → l res; BicubicResize4(x t) + l res → g t. In TecoGAN, there are 10 sequential residual blocks in the generator (l n = l 10), while the TecoGAN generator has 16 residual blocks (l n = l 16). Each ResidualBlock(l i) contains the following operations: C(l i, 3, 64, 1), ReLU → r i; C(r i, 3, 64, 1) + l i → l i+1. The VSR D s,t's architecture is: IN g s,t or IN y s,t → l in; C(l in, 3, 64, 1), Leaky ReLU → l 0; C(l 0, 4, 64, 2), BatchNorm, Leaky ReLU → l 1; C(l 1, 4, 64, 2), BatchNorm, Leaky ReLU → l 2; C(l 2, 4, 128, 2), BatchNorm, Leaky ReLU → l 3; C(l 3, 4, 256, 2), BatchNorm, Leaky ReLU → l 4; Dense(l 4, 1), sigmoid → l out. VSR discriminators used in our variant models, DsDt, DsDtPP and DsOnly, have a similar architecture as D s,t. They only differ in terms of their inputs. The flow estimation network F has the following architecture: conc(x t, x t−1) → l in; C(l in, 3, 32, 1), Leaky ReLU → l 0; C(l 0, 3, 32, 1), Leaky ReLU, MaxPooling → l 1; C(l 1, 3, 64, 1), Leaky ReLU → l 2; C(l 2, 3, 64, 1), Leaky ReLU, MaxPooling → l 3; C(l 3, 3, 128, 1), Leaky ReLU → l 4; C(l 4, 3, 128, 1), Leaky ReLU, MaxPooling → l 5; C(l 5, 3, 256, 1), Leaky ReLU → l 6; C(l 6, 3, 256, 1), Leaky ReLU, BilinearUp2 → l 7; C(l 7, 3, 128, 1), Leaky ReLU → l 8; For all UVT tasks, we use a learning rate of 10 −4 to train the first 90k batches and the last 10k batches are trained with the learning rate decay from 10 −4 to 0. Images of the input domain are cropped into a size of 256 × 256 when training, while the original size is 288 × 288. While the Additional training parameters are also listed in Table 6. For UVT, L content and L φ are only used to improve the convergence of the training process. We fade out the L content in the first 10k batches and the L φ is used for the first 80k and faded out in last 20k. TecoGAN is implemented in TensorFlow. While generator and discriminator are trained together, we only need the trained generator network for the inference of new outputs after training, i.e., the whole discriminator network can be discarded. We evaluate the models on a Nvidia GeForce GTX 1080Ti GPU with 11G memory, the ing VSR performance for which is given in Table 2. The VSR TecoGAN model and FRVSR have the same number of weights (843587 in the SRNet, i.e. generator network, and 1.7M in F), and thus show very similar performance characteristics with around 37 ms spent for one frame. The larger VSR TecoGAN model with 1286723 weights in the generator is slightly slower than TecoGAN, spending 42 ms per frame. In the UVT task, generators spend around 60 ms per frame with a size of 512 × 512. However, compared with the DUF model, with has more than 6 million weights in total, the TecoGAN performance significantly better thanks to its reduced size.

Represent the TLDR for finding the corresponding scientific article

We propose temporal self-supervisions for learning stable temporal functions with GANs.

Conditional VAE on top of latent spaces of pre-trained generative models that enables  transfer between drastically different domains while preserving locality and semantic alignment.