Question: Represent the text to find a very short summary (1-2 sentences max).

Question: Inferring temporally coherent data features is crucial for a large variety of learning tasks. We propose a network architecture that introduces temporal recurrent connections for the internal state of the widely used residual blocks. We demonstrate that, with these connections, convolutional neural networks can more robustly learn stable temporal states that persist between evaluations. We demonstrate their potential for inferring high-quality super-resolution images from low resolution images produced with real-time renderers. This data arises in a wide range of applications, and is particularly challenging as it contains a strongly aliased signal. Hence, the data differs substantially from the smooth inputs encountered in natural videos, and existing techniques do not succeed at producing acceptable image quality. We additionally propose a series of careful adjustments of typical generative adversarial architectures for video super-resolution to arrive at a first model that can produce detailed, yet temporally coherent images from an aliased stream of inputs from a real-time renderer. Learning expressive and stable representations is a goal that lies at the heart of a vast range of deep learning tasks (; ; . While typical recurrent architectures focus on feedback loops to form persistent latent-spaces , we show that for inference tasks where the is conditioned on a stream of inputs, these existing architectures unnecessarily complicate the learning task, and fail to reliably stabilize the inference. With our work, we propose a new type of connection for the very widely used building blocks of ResNet architectures that lets the network easily compare internal states in-place. The learned representation can then, e.g., yield a detailed image sequence with natural changes. We demonstrate this with a particularly challenging learning objective: we aim for the synthesis of detailed images from a stream of strongly aliased inputs. Specifically, we show that adversarially trained convolutional neural networks (CNNs) can be leveraged to produce detailed images from unfiltered, low-resolution images generated via point-sampling with a rasterization-based real-time renderer. Real-time graphics are the basis for a wide range of applications: Generating images with a sufficient resolution from low resolution, yet computationally light-weight renderings is a task that is, e.g., important for generating content for the high resolution screens of mobile devices, and is especially interesting for streaming services of games in order to compute the final resolution only on the client. Our work shares its goal with a variety of approaches that have been proposed for generating highquality images for raytracing algorithms and purely image-based super-resolution algorithms (; . Our architecture differs from previous works as the proposed recurrent connection allows the network to learn a temporally stable latent-space representation that does not negatively impact the residual flow of a ResNet architecture. Also, the temporal connections for deeper layers of the network are important for successful learning, as we will demonstrate below. While the basic concept of depth-recurrent connections could potentially be applied to a variety of sequence-based learning tasks, we focus on demonstrating its potential for pushing forward the limits of real-time rendering. Hence, we additionally outline a series of modifications to existing architectures which are crucial for achieving high quality of the strongly aliased input images from LR RDA modified TecoGAN re-trained DRR Figure 1 : Given a strongly aliased low-resolution input rendering with one sample per pixel, recurrent non-adversarial training ( with modifications for fair comparisons) produces blurry , and existing adversarial methods (, re-trained) introduce strong flickering artifacts. Trained on the same data, due to the proposed DRR connections our network infers more consistent spatio-temporal features (see the supplemental footage for a clear assessment of the temporal differences). typical real-time rendering pipelines. A typical input for our network is shown on the left of Fig. 1. This application scenario is especially challenging for CNNs, since it requires to work with images that need to be rendered at very high frame rates and, thus, exhibit severe aliasing due to point sampling and typically low resolutions. The aliasing not only distorts the spatial signal, but likewise affects the temporal changes. Therefore, a super-resolution (SR) network can't rely on receiving smoothly changing, filtered inputs that allow for localization of small features. Rather, it has to learn over the course of multiple frames to infer consistent output images (Fig. 1, right) from spatially and temporally aliased input content. As we will demonstrate in a number of studies below, this task is where our proposed depth-recurrent connections unfold their strength. They enable the network to match the data distribution of the targets, i.e., to synthesize images with a high visual quality in terms of detail as well as their temporal behavior. We show and comparisons in the paper, and provide many additional evaluations in the supplemental material 1, where videos more clearly show spatial and temporal differences. Deep learning has been successfully applied to a large variety of image-based super-resolution tasks (; ; . Here, convolutional architectures (CNNs) with residual blocks are a very popular generator architecture that offers training stability as well as high qualify inference. Targeting photo-realism, Generative Adversarial Networks (GAN) were introduced to prevent the undesirable smoothing of direct loss formulations (; . For GAN architectures, a second discriminator network is trained to classify real and generated samples that is used to guide the generator network. The of these GAN approaches were improved upon, e.g., by modifying the residual blocks and perceptual loss function , by employing the Earth Movers distance to stabilize the training , and by accounting for the a-priori knowledge that fake samples exist . Using the feature-space differences of image classification networks, e.g. a pre-trained VGG network or the discriminator in a GAN setting, as perceptual loss was shown to be highly effective as well. Natural temporal changes of the generated content are crucial for video SR tasks. Often multiple subsequent frames are used to generate a high-resolution (HR) frame . These multi-frame approaches benefit from aligning the frames via warping which requires an estimation of the image space motions. As this is usually not readily available for natural videos, optical flow estimation networks are a popular choice. As for spatial content, employing an L 2 loss to enforce temporal coherence is not optimal. Instead, extending the adversarial loss to the temporal domain improves the temporal coherence of small-scale details (Pérez-; . We likewise employ a spatio-temporal GAN architecture in the following. For deep learning methods, a variety of recurrent neural networks has been proposed (; ;), and were shown to be useful for image generation tasks. uses recurrent connections to propagate a latent state over time inside the network, while use the previously generated high-resolution (HR) output as input. The SR task also bears a certain similarity to other image enhancement techniques like inpainting (; a), where progressive methods provide state-of-the-art quality, or the removal of compression artifacts and text . Likewise, temporal coherence is an important aspect in style transfer. We use spatio-temporal self-supervision similar to previous work, but target a substantially more challenging data domain than natural videos in the following. Images of real-time renderers that can be generated quickly often exhibit strong aliasing. In this context, the existing work focuses on enhancing path-traced images. A common application is image denoising, e.g. for Monte-Carlo ray-tracing , where a network learns to predict a noise pattern in order to infer a smooth image from a sparse set of importance samples. Computer games and other real-time graphics applications, on the other hand, use a rasterization-based rendering pipeline. Simple geometry is shaded with simplified but increasingly complex lighting and texturing computations, and often only 1 sample per pixel is taken to maintain the required frame rates. Compared to natural and path-traced synthetic images these images are strongly under-sampled and exhibit aliasing artifacts that are different from the noisy images of path-tracing algorithms. Super-Sampling (SSAA), i.e. rendering at a higher resolution and averaging, is a straight-forward, but expensive anti-aliasing (AA) solution. While Multi-Sampling (MSAA) is a possible optimization, image-space methods (; ; 2012) are popular and try to reduce aliasing by detecting and smoothing edges in image-space after rendering, temporal methods like Temporal Anti-Aliasing (TAA) use the warped previous samples to smooth and stabilize edges. Here, the use of deep leaning models for image enhancement in real-time settings is sparse. To the best of our knowledge, only a closed-source solution exist, provided by Nvidia in the Turing architecture , and little is known about its internal realization. Instead, our goal is to provide an open solution, and at the same time improve the state-of-the-art in GAN-based SR. Our main goal is to facilitate the learning of a stable internal network representation that persists between repeated SR inference steps. In this way, the network can pass information from one frame to the next such that a persistent and stable prediction over time is achieved. This is crucial for processing strongly aliased input data streams. The internal network representation can then be used to generate the desired output distribution and, as demonstrated below, this improves temporal coherence and generates stable small-scale details in SR rendering. Fig. 2 demonstrates the quality that is achieved by the proposed architecture. To enable the network to learn this internal representation, we augment the generator network with aligned, temporal connections inside the residual blocks (ResBlocks). In order to demonstrate the advantages of our depth-recurrent architecture and the extensions for the real-time setting, we build on a GAN-based video super-resolution architecture from previous work. Here, the generator G is trained on image sequences and processes the current LR input as well as its warped previous output through a series of ResBlocks to produce a detailed and temporally coherent output. The spatio-temporal discriminator D s,t sees shorter 3-frame sequences of the output as well as the LR input as condition. We also employ a perceptual feature loss via the discriminator network. In addition to the adversarial and the feature loss, the generator is trained with a Ping-Pong term for long-term coherence and an L 2 loss in image space, the content loss, for stabilization. The residual blocks of frame t−1 and frame t are connected to pass the activations of a convolutional layer from frame t − 1 to t. As we work with video data, we align these activations by warping them with the screen-space motions before they are concatenated to the ones in frame t and passed as input to the next layer. We call this setup Depth-Recurrent Residuals, and it is visualized in Fig. 3, DRR. In contrast to frame-recurrent networks, we employ connections in latent space, and different Figure 2: Our DRR model applied to 2 different test scenes. The comparisons show input LR color images and the high-resolution outputs inferred by our method. The inputs exhibit strong spatial aliasing, and a similar amount of temporal aliasing, as is visible in the videos of the supplemental material. Despite these challenges, our model infers a stable and detailed output sequence. to commonly used "feed-back" loops of recurrent architectures, we use a "feed-forward" design that includes a warping step. Within a multi-layer CNN, a variety of possible recurrent connections are imaginable. The goal of the specific variant we propose is that of temporal stability: with our connections a receiving layer can compare an aligned set of activations from the previous frame with the current activations. Hence, operations such as temporal derivatives are trivial to perform with in-place differences. The operations of a regular ResBlock at depth b can be summarized as follows: two convolutional layers + are followed by an addition, yielding the final output F b,+ = F b,2 + F b−1,+. Here, C <ResBlock>,<layer> denotes a convolution, and + ReLU activation. For our depth-recurrent residual blocks, we replace the second convolutional layer with the following operation: b,1 )). This is followed by a summation as before to produce the output F b,+. Here, W applies warping for the current layer based on externally computed velocities (either with another CNN, or via rendering as for our inputs), and ⊕ denotes feature concatenation. Compared to regular ResBlock convolution, we use 18, 75% fewer features for DRR blocks in order to keep the overall number of weights per ResBlock constant. During our experiments, we apply the following curriculum learning steps, which typically help to stabilize the training runs and lead to improved final in our tests: The depth recurrent connections are linearly faded in during training in a staggered fashion, beginning with fading in the connection of first block, then the second block and so on. Similarly, the discriminator network is faded in after a pre-training step of the generator (details are given in App. B). Figure 3: We propose to connect the center latentspace of ResBlock n at time t − 1 to the same ResBlock n at time t, yielding our DRR connection. A variant of our approach is DRR-in, in addition to a recurrent connections similar to previous work (RDA). Discussion To analyze whether the intuition that the DRR connection support learning stable features is correct, we have evaluated alternative connectivities. One variant from previous work connects the outputs of a ResBlock as inputs into the same ResBlock for the next frame to produce a hidden state. This is inspired by the recurrent denoising autoencoders , and shown as RDA in Fig. 3. Note however, that this version in the following uses our improved architecture and warping operations, both of which are not employed in the original version. In terms of the notation above, this variant is obtained via +. It yields spatially slightly worse but strongly flickering , as the layer now effectively has to compare different feature sets: that of the previous ResBlock, and the features of its own from the previous frame. RDA DRR-in DRR Figure 4: A visual comparison of the different types of recurrent connections with otherwise identical models (all three using our discriminator supervision, generator and warping). Our DRRs yield an improved image quality and temporal stability, as shown in the supplemental material. A variant of our recurrent ResNet architecture is to use an in-place connection instead. Thus, feeding the outputs of the previous ResBlock of both the current and previous frame as input to each ResBlock as shown in Fig. 3 DRR-in. I.e., F +. This variant of our DRR connections stabilizes the , but does typically in slightly lower quality inference. Qualitative for each variant are shown in Fig. 4. We believe that the primary reason for the improved stability of our variant is that the proposed connection is de-coupled from the regular flow of information along the residual blocks. Hence, the learned temporal features change the way the latent-space within the ResBlock is shaped, but it does influence and possibly impede the residuals added at the beginning and end of a ResBlock. Therefore, the network can focus on learning suitable temporal features, instead of additionally having to learn to work with a combination of spatial and temporal information (as with previous work and the variants discussed above). Thus, our approach preserves the structure which led to the success and widespread use of the original ResBlocks . In all three variants, applying the W function, i.e. warping with the screen-space motion vectors, improves the . Otherwise, the networks easily converge towards solutions with noticeable streak-like artifacts. It is also worth noting that the depth recurrent connections are substantially different from the recurrent input to the generator network, i.e., the previously generated HR frame, as they pass on learned latent-space features at the same location within the network. Thus, for new inference tasks the network does not need to analyze again the information encoded in an image. However, it is still challenging for the network to encode and transport coherent small-scale features across frames. Removing the recurrent input, using only our depth recurrent connections leads to temporally coherent , but induces a significant loss of spatial detail. Hence, rather than replacing it, the depth recurrent connections work best in conjunction with a recurrent input. We use images rendered in real-time with a rasterization-based rendering pipeline with 1 sample/pixel as our training data. The network is trained with 15-frame sequences of matching HR-LR image pairs, both rendered using Unity's high-definition rendering pipeline (HDRP). From there, we also capture the rendered screen-space motion that is used for warping. Details are given in App. A. Typical super-resolution models for natural videos are trained with a down-sampled version of the target. This gives the network a smooth, reliable average of the reference content. A generator network can detect the changes and sharpen existing features to generate detail. As our synthetic data exhibit strong aliasing artifacts, there are larger changes between adjacent pixels as well as temporal aliasing. This makes the data much more difficult to analyze. In contrast to a spatio-temporally integrated signal, as for natural video SR, sub-pixel features only show up rarely, and most of the time do not have a signal at all in our setting. This makes it much harder to correlate image space content in subsequent frames and achieve temporal coherence. Aliasing also makes it more difficult to detect the actual, underlying edges over a larger area and correct the input accordingly. These combined challenges make it necessary to introduce more temporal context, which we enable with the DRR connections. In addition to the depth-recurrent residuals described above, our model is also subject to multiple changes which we found crucial for dealing with the challenging input data from real-time renderers. First, we use the motion vectors generated by the renderer for all warping operations. These are faster to compute and more accurate than the motion estimated by an auxiliary network. We also found resize-convolutions with bilinear interpolation instead of (strided) deconvolutions important to reduce checkerboard artifacts. The specific task the network needs to perform in our target application reveals the limits of the deconvolution approach, requiring to incorporate resize-convolutions as described below. Providing depth as LR input to the generator also slightly enhances edges and reduces aliasing artifacts such as staircasing. This effect is stronger near the camera, which suggests that it is related to the non-linear dependency of depth to view-space distance. Introducing additional data fields from the rasterization pipeline, such as surface normals, did not yield improvements. Previous work proposes to let the generator output be a residual that is added to the bicubically upscaled LR input color, instead of directly generating the final output. Interestingly, for our strongly aliased data we found it beneficial to instead use bilinear interpolation for up-scaling the inputs, as bicubic interpolation often leads to overshooting near edges. This caused difficulties for the generator and did not yield any improvements in terms of image quality. However, also the bilinearly up-scaled LR color, to which the residual is applied, still contains strong aliasing. To perform the necessary anti-aliasing, the network has to detect structures based on single samples in the LR input, as it does not have direct access to the up-scaled version in this case, and then perform the necessary correction via the residual addition to the HR version on the other end of the network. This is sufficient for the smoother data of natural videos, where detail enhancement and sharpening operations can be less accurate and still yield good . Replacing the final addition of the residual with 2 additional convolutional layers further improves image quality. For these convolutions, we concatenate the up-scaled color after the resize convolutions (details in App. B). With this modification, the network does not need to transport the LR color through the LR ResBlocks and can choose based on the up-sampled LR color what parts of it to use. This adds ca. 1% additional weights compared to the standard setup, but is computationally more expensive as high-res tensors are convolved. Nonetheless, the gain in image quality outweighed the slight increases in computations and memory. The concatenation of the DRRs increases the number of input channels and, therefore, the number of weights of the receiving layer. Thus, to keep the overall amount of weights consistent with previous versions and allow for a fair comparison, the number of base channels of the ResBlocks is reduced when using these recurrent connections. Discriminator The discriminator facilitates the generation of sharp and detailed images. While a network trained with only L 2 supervision produces a more blurred output, "L 2 only" in Fig. 5, activating the spatio-temporal discriminator yields a significant improvement, see "D s,t ". As visible in Fig. 5, we found it important to use a discriminator without conditional input in order to balance the networks. As a large part of the loss function employs the discriminator (both adversarial and feature losses), its quality is crucial. E.g., by increasing the size, i.e., adding layers or channels, and therefore complexity of the discriminator, further improvements are achieved. Increasing depth values allows D to detect more complex features, thus providing gradients towards more complex image content. This also avoids that the generator quality is limited by the lack of discriminator gradient information. The feature loss similarly profits from more complex features, and while a larger discriminator is slower to train, it does not affect performance during inference. However, a larger discriminator can make the training more unstable, and eventually become unbalanced. We found a discriminator with 14 layers and 3.7M weights (D s,t), and a generator with 26 layers and 769k weights to be a stable combination that yields high quality , as long as adaptive balancing is used as outlined in App. B. Depth-Recurrent Residuals A generator with Depth-Recurrent Residuals (DRRs) gives good , even when trained with the largest discriminator where a generator without DRRs tends to produce strong artifacts, Fig. 5 (D s,t). The generator network can process the detailed gradient feedback it receives from the large discriminator network, and use it to generate a large amount of image-space detail. Letting the DRR connections fade into the network further improves the . When the connections are activated right from the beginning of the training process, the image quality is typically reduced. Loss Functions Despite the stabilization via DRR connections, the proposed learning setup requires careful balancing. The additional details provided by a larger discriminator, combined with the strong learning objective to produce detail, yields overall better but more small-scale differences to the target. This in turn increases the L 2 content loss, which however is needed to stabilize the training. In practice, a parameter is introduced to choose between temporal stability and large amounts of image-space detail. An important source of detail seems to stem from the discriminator feature loss, i.e., by using only the regular adversarial loss, details are noticeably reduced. In practice, a strong feature loss effectively reduces the smoothing caused by the content loss and gives the best . Similar to a spatial L 2 loss, we found that a temporal L 1 loss, as employed by , leads to an undesirable smoothing in our GAN setting. Even worse, when using a temporal L 1 loss without warping, it produces strong streak-like artifacts. Modifications such as additional edge loss terms did not alleviate this behavior. While others have reported improvements by using perceptual losses with pre-trained networks (such as VGG), we likewise found that this yields different, but not necessarily better outputs. Presumably, the different nature of the images produced by the rasterization pipelines is less amenable to the VGG features. The loss variants used in previous works are shown in the bottom row of Fig. 5. We compare our model with and without DRR connections to state-of-the-art methods from previous works. Specifically, we compare to supervised approaches (DUF , Enet , FRVSR ), and TecoGAN as a GAN-based alternative. As shown by the qualitative comparison in Fig. 6 and the supplemental videos, all existing approaches have difficulties to stabilize the input data and produce strong artifacts. We also re-trained a TecoGAN model with our data set, which also leads to undesirable . Standard metrics for image quality often fail to relate to the image quality perceived by humans. PSNR values fluctuate during training and across and in practice have very limited significance. We thus use perceptual metrics, measuring LPIPS for perceptual similarity to single target images, i.e. LPIPS(G(x t), y t ) (b). As measure for temporal coherence we evaluate tLP = |LPIPS(G(x t−1), G(x t)) − LPIPS(y t−1, y t)| common problem that the temporal coherence of blurry data is generally better, as can be seen from "L 2 ". Thus, it should be viewed in conjunction with the spatial LPIPS for assessing the overall video quality. For completeness, we also compute the in-place metric T-diff = ||W (G(x t−1)) − G(x t)|| 2 ), using the captured HR motion for increased accuracy. The are given in table 1, and confirm the qualitative comparisons so far: our method yields the best temporal coherence in terms of tLP score. In addition, our generators with warped connections all yield excellent LPIPS scores around 2.7 (last three columns), but the DRR connections are crucial to obtain good temporal changes in addition to the details. The DRR tLP score is only surpassed by L 2, which, however, yields clearly sub-optimal image details. As we focus on real-time rendering as our use case scenario, ideally the performance of the inference step needs to surpass the performance of the renderer. For a desired output resolution of 1920×1080, our pre-trained model takes 113ms per frame on average. 2 Although this is not yet fast enough for real-time applications, we expect that techniques such as network compression and evaluation of the models with dedicated hardware will easily yield very significant performance improvements. We have demonstrated how depth-recurrent residual connections can be leveraged to learn stable internal latent-space representations in conditional generator architectures. The DRR connections are particularly promising for iterative models with strongly aliased data, such as low-resolution inputs from a real-time renderer. We have additionally shown how to achieve high quality synthesis in the context of real-time rendering by carefully analyzing and adjusting the network architecture. We anticipate that DRRs could be beneficial for a variety of other tasks such as object tracking and physics predictions . A DATA As source of our data we use the projects "FPS Sample" and "Book of the Dead: Environment" (b;a) for the Unity engine, both use the HDRP. We captured a total of 57 120-frame sequences, split 50-5-2 for training, validation and testing. For each frame we have lit color (the final image), unlit diffuse color, view-space surface normals, roughness, screen-space motion and depth for both HR and LR. This data is easy to acquire as it can be inferred from the scene, geometry and materials and is rendered by default in Unity's HDRP. However, the use of unlit color, normals or roughness had no tangible effects during our tests. Most post-processing effect have been turned off, but the HR color is augmented with TAA. HR is rendered and captured at a resolution of 512 × 512, LR at 128 × 128. The details of our network architecture are given in figures 8 and 9. Our generator network represents a modified TecoGAN generator, and a lager version of TecoGAN's discriminator is likewise used. A typical training takes 400,000 iterations (ca. 287 epochs). All discriminator related losses, the loss for the discriminator itself as well as the adversarial and feature loss to the generator, are linearly faded in over the first 40k iterations starting from 0. When training with DRR the temporal connections are faded in later during training in a staggered fashion, beginning at 60k iterations with the first block and 8k iterations between each. Every block takes 10k iterations to fully fade in. We also use an exponential learning rate decay during the last 150k iterations, decaying to 65% every 30k iterations. The most dominant losses are the L 2, the feature loss and the ping-pong loss. To balance the adversarial and feature loss during training, the discriminator is only trained if it is not too strong to keep it from overwhelming the generator. We train the discriminator only if the EMA of the sum of b disc = − log(D(y))− logD(G(x)), is below a threshold. For our setting, 0.4 yields stable GAN training runs. For our final DRR version we use use following weights for the terms of our loss function. L2 content loss: 1.0, ping-pong loss: 0.5, feature loss: 3.6, and adversarial loss: 0.1. In addition, we use the following hyper-parameter settings. discriminator learning rate: 3.5e-5, discriminator balancing threshold: 0.4, learning rate: 5e-5, adam beta: 0.9. Figure 8: Our modified generator: in addition to the LR color and frame-recurrent input (generator output of the previous frame encoded in 3 * 16 channel) we also add the LR depth. After an initial convolution most work is done in the 10 sequential ResBlocks (same as TecoGAN, 9 omitted for visibility). The latent image is then scaled to output resolution by 2 resize convolutions and fine-tuned by another 3 convolutions after the bilinear interpolated LR color is appended. All convolutions have 3 × 3 kernel size and ReLU or no activation. To highlight the range of content our trained model can produce, we show additional sequences in terms of their low-resolution input and inferred output in figures 10 and 11. In order to fully assess the quality, please check the supplemental material document, which contains animated sequences.

A method for persistent latent states in ResBlocks demonstrated for super-resolution of alised image sequences.

Represent the TLDR for finding the corresponding scientific article

We introduce an alternative GAN design based on random routes in generator, which can serve as a tool for generative models interpretability.


Question: Represent the article for finding a single-sentence TLDR

Question: The backpropagation algorithm is the most popular algorithm training neural networks nowadays. However, it suffers from the forward locking, backward locking and update locking problems, especially when a neural network is so large that its layers are distributed across multiple devices. Existing solutions either can only handle one locking problem or lead to severe accuracy loss or memory inefficiency. Moreover, none of them consider the straggler problem among devices. In this paper, we propose \textbf{Layer-wise Staleness} and a novel efficient training algorithm, \textbf{Diversely Stale Parameters} (DSP), which can address all these challenges without loss of accuracy nor memory issue. We also analyze the convergence of DSP with two popular gradient-based methods and prove that both of them are guaranteed to converge to critical points for non-convex problems. Finally, extensive experimental on training deep convolutional neural networks demonstrate that our proposed DSP algorithm can achieve significant training speedup with stronger robustness and better generalization than compared methods.

Represent this text to find a much much longer article about it with say hundreds-thousands of words

We propose Diversely Stale Parameters to break lockings of the backpropoagation algorithm and train a CNN in parallel.

represent the tldr for finding the corresponding scientific article

SlowMo improves the optimization and generalization performance of communication-efficient decentralized algorithms without sacrificing speed.


Question: Represent the scientific article to find a short TLDR

Question: The emergence of language in multi-agent settings is a promising research direction to ground natural language in simulated agents. If AI would be able to understand the meaning of language through its using it, it could also transfer it to other situations flexibly. That is seen as an important step towards achieving general AI. The scope of emergent communication is so far, however, still limited. It is necessary to enhance the learning possibilities for skills associated with communication to increase the emergable complexity. We took an example from human language acquisition and the importance of the empathic connection in this process. We propose an approach to introduce the notion of empathy to multi-agent deep reinforcement learning. We extend existing approaches on referential games with an auxiliary task for the speaker to predict the listener's mind change improving the learning time. Our experiments show the high potential of this architectural element by doubling the learning speed of the test setup. Natural language is not as rule-based as researchers in supervised language learning would prefer. There are limitless context-dependent notions to it, and flexible language use is considered as a necessary aspect of general AI. Originally, natural language emerged through a necessity to achieve successful coordination. Hence, a general AI would need to understand the functional aspects of language and learn communication through interaction . These considerations led to the research field of emergent communication and the attempt to ground natural language through reinforcement learning. Deep reinforcement learning has achieved some impressive over the last years . One of its principal aspects is the ability to extract features from high dimensional input data without manual preprocessing. This capability is especially useful if the necessary representation is unknown to the designer. Classical deep reinforcement learning approaches rely on a large number of training examples, mainly because the sparse reward hardly provides enough feedback to shape the deep layers. These deep layers are responsible for the embedding of input data into a meaningful representation. Therefore, it takes many training steps before a useful representation emerges; if it converges at all. According to the theory of the predictive mind , the human brain generates richer feedback through learning several unsupervised prediction tasks while training on the main task. The purpose of these predictions is to produce more and more expressive models and representations of the world. achieved a far more expressive representation of their visual inputs by learning an auxiliary prediction task. The sole purpose of the auxiliary net is to predict the change in the visual input given the last movement action. Training this net does not directly affect the original task, but it refines the visual representation to reflect the concepts of a 3D world. used predictive tasks to ground natural language, but only focused on better understanding an existent language. We transfer the auxiliary prediction to the task of active communication. This goes along with the theory of mind stating that an essential part of intelligence in interaction emerges through predicting the mental state of the interaction partner. We let the speaker train an auxiliary net that tries to predict how the speaker's utterance will change the listener's hidden state. That resembles humans empathetic way of understanding what a message will do to the listener. We assume this leads to a more communication effective representation of the sensory input; in other words, the input encoding becomes more communicatable. The effect is visible in the essential acceleration of learning successes in developing a shared language. Our main contribution is an elegant extension to multi-agent deep reinforcement learning (MADRL) algorithms aiming to emerge a communication. It resembles an empathic connection between speaker and listener, which leads to faster convergence to a shared language. We doubled the learning speed of a MADRL algorithm playing a referential game by introducing this auxiliary prediction task to the speaking agent. We attribute the improvement to the richer gradients in the lower layers of the neural network to embed the input. Reinforcement Learning (RL) An agent in a reinforcement learning setting can fully or partially observe its current state s ∈ S and is able to choose an action a ∈ A through a policy π(s) = a. The chosen action will lead to receiving a reward R. The agent's goal in its environment is to maximize the expected reward . RL with neural networks (NN) Using neural networks as a policy representation for reinforcement learning has the benefit of being able to represent any policy function and the downside of needing a huge number of data samples to learn. In our case, the policy outputs a direct probability for taking each action. Such policies can be updated by using Policy Gradient methods . The policy parameters θ, in this case, the parameters of the neural net, are updated according to their effect on the objective J with a learning rate β: Using the REINFORCE algorithm the effect on the objective can be estimated as the following: Long Short-Term Memory Network (LSTM) Recurrent neural networks (RNN) can accumulate input in an internal representation over time as well as produce a consistent output over several time steps from it. LSTMs are RNNS that are specifically created to remember information over an extended period of steps . Auxiliary tasks in RL Auxiliary unsupervised tasks were introduced into RL with NNs by. They proposed an architecture that predicts the next visual input, given the internal representation of the last visual inputs and the last taken action. The unsupervised task of correctly predicting the next visual input leads to better performance on the main task, which was playing an atari game. They assume that the auxiliary task enforces a more expressive internal representation of the visual input, which then aids the main task. transferred this auxiliary task to natural language acquisition by predicting the next word spoken by an instructor. 3 started the field of learning communication in artificial agents with the aim to research the mechanisms by which language and communication emerge. contributed by using classical genetic scenarios, where "male" agents had to find "female" agents based on signals they emitted. They extended their setting in 1993 to include predator and prey agents and showed that known prey strategies as herding emerge if the agents have the possibility to communicate . achieved the emergence of more robust signals by introducing lying agents (parasites) in this setting. The successive advances in the field of learning communication can be assigned to the progress of learning algorithms for neural networks for a big part. started using Q-learning on the pursuit problem, including learned communication, but also follow up work only reached simple information sharing about the prey position . The field was then alleviated to a new level of complexity by and. They transferred the new progress in deep learning to multi-agent coordination to emerge even more complex communication patterns. In previous work, we have shown that these algorithms can be further improved even to solve tasks that lie outside the communication range [blind]. Though many multi-agent learning setups use communication as a matter of success, only some focus on the emerging protocols and their properties . From those focusing on the emergence of communication or language, a significant number of publications used referential games as a testbed (; ; ; ; ;). Speaker Listener cat lives legs fluffy car cat bird sofa Figure 1: Illustration of the referential game used as a testbed. The speaker agent takes the concept "cat" as input and forms a message to describe it. The listener agent takes the message and several candidate concepts as input, and decides which concept is the target seen by the speaker. Especially interesting in that context is the work of , as they could vividly show, that this approach to language emergence can lead to a flexible language use which could be understood by humans even when applied to objects unknown to the algorithm, yet. We introduce the idea of auxiliary tasks into the field of language emergence. The speaking agent is equipped with an auxiliary single-layer perceptron, to predict the hidden state of the listener agent, after this ultimately encoded the message. The input for this prediction is the hidden state of the speaker right before it starts forming the message. The aim is to achieve a high relation between the hidden states of both agents. This signifies the speaker can communicate its means well. We state that in this application, the auxiliary prediction resembles empathy in humans, as the speaker tries to predict how its utterance will affect the listener's mindset. The prediction task is unsupervised and can be trained on the same samples and at the same time as the main task. Training the main RL task automatically generates the samples for the unsupervised task. The gradients can be backpropagated into the encoding layers of the speaker, where they are added to the gradients of the RL task and optimized together. With our approach, we further enhance the possibilities in language emergence by providing richer feedback to form the internal communicatable representation in the speaking agent. We provide experimental evidence that these extensions can lead to a doubled learning speed when added to an existing approach to language emergence in referential games. To test the potential of the auxiliary prediction task, we used the referential game setup proposed by shown in Fig. 1. Out of the existing implementations, we chose this one because the setup has proven to converge to an emergent communication at a relatively low computational cost. Dataset We use the Visual Attributes for Concepts Dataset (VisA) of. It contains attribute annotations for 500 concrete concepts (like cat or sofa), annotated with 636 general attributes (like is black or made of wood). The annotations are human-made and therefore carry an inherent structure that can be seen as disentangled. Agent Setup A speaker agent gets shown a target concept t that is realized as a binary vector with as many entries as possible attributes. The speaker then uses a policy π S to produce a message m out of an alphabet of discrete symbols (numbers 1 to 100 in our case). The message is then interpreted by a listener agent that observes several candidate concepts C at the same time. The listener uses a pointing policy π L to decide, which of the candidate concepts the speaker agent is describing. Both agents receive a shared reward R if the listener correctly identifies the described concept. The speaker agent consists of a single encoding layer to encode the input vector into a dense representation h S and an LSTM to produce a message out of h S. The listener agent encodes the message with an embedding layer and an LSTM into a dense representation h L. The listener contains an encoding layer as well, which it applies to every candidate concept respectively to generate a set of representations. It calculates the compliance between message and candidate concepts with the dot product between the message representation and the concept representation. The is treated as a Gibbs Distribution. Both policies π S and π L output a probability distribution over all possible actions. For the speaker, the possible actions are the elements of the alphabet, once for every symbol over the length of the message. For the listener, the actions consist of choosing each of the candidate concepts in C. For more details see. Learning As part of the reinforcement learning setting, the agents try to maximize the expected reward. They do not share any parameters but try to maximize the probability of their action that ed in a positive reward, respectively. Therefore together, they maximize the objective function in each training instance: Empathy Extension To generate richer gradients for shaping the deep encoding layers of the speaker, we assign an auxiliary unsupervised prediction task to it. We add a single layer MultiLayer-Perceptron (MLP) to the graph, which predicts the activation of the listener's hidden layer h L after hearing the full message. The input is the activation of the speaker's hidden layer h S before starting the utterance. That corresponds to predicting the effect of the to-be-made sentence on the mindset of the listener. We use the mean absolute error as the loss function for the prediction task: where α is a weighting factor that ensures that the unsupervised task does not corrupt the main reinforcement learning task. An α close to 1 would mean, that effectively manipulating the listener's mind is as important to the speaker as communicating the target concept. w θ is the linear transformation through the MLP with sigmoid activation function σ. The gradients of the unsupervised task are calculated on the same trial and added to the gradients of the reinforcement task. Hence, no additional training steps are necessary. The optimization then uses the summed up gradient. We implemented the setup using the EGG toolkit Kharitonov et al.. We found that with an α of around 0.1, i.e. weighting the prediction gradients 10% compared to the main task, we can increase the learning speed to double or triple. To be comparable, we used the same initialization and sampling seeds on both options. Good or bad initialization can make up for half the learning speed, but the relative learning speed improvement through using the prediction task stays consistent over different initializations. In Fig. 3 we compared the learning curves with and without the prediction task. For a game setup with two candidate concepts and a maximum message length of two, all marks are reached in half the time. For a more complex game setup with five candidate concepts and a maximum message length of five, some marks are even reached in a third of the time, when using the prediction task. Using an auxiliary predictive task on a communication learning task has proven auspicious. Sampleefficiency is highly desirable when acquiring language, so the fact that our auxiliary task doubles the learning speed is of high significance. Our experiments do only feature a small partition of the potential of this elegant mechanism, yet. Higher sample-efficiency at no computational cost now allows acquiring more complicated language tasks, that were previously impossible to learn in a reasonable time. We plan to apply our algorithm to much more challenging tasks in the future. We did, for example, only test disentangled input due to computational limitations. The mechanism would be even more useful when applied on entangled input because developing an expressive representation is then of higher importance. For future research, we propose the use of an auxiliary prediction task for the listener to align with the word usage of the speaker, as well. We hope that this simple but powerful mechanism brings the field of language emergence a big step forward.

Represent the TLDR for finding the corresponding scientific article

An auxiliary prediction task can speed up learning in language emergence setups.

Represent this text to find a much much longer article about it with say hundreds-thousands of words

We introduce the capacity to exploit information about the degree to which an arbitrary goal has been achieved while another goal was intended to policy gradient methods.


Question: Represent the article for finding a single-sentence TLDR

Question: Image paragraph captioning is the task of automatically generating multiple sentences for describing images in grain-fined and coherent text. Existing typical deep learning-based models for image captioning consist of an image encoder to extract visual features and a language model decoder, which has shown promising in single high-level sentence generation. However, only the word-level scalar guiding signal is available when the image encoder is optimized to extract visual features. The inconsistency between the parallel extraction of visual features and sequential text supervision limits its success when the length of the generated text is long (more than 50 words). In this paper, we propose a new module, called the Text Embedding Bank (TEB) module, to address the problem for image paragraph captioning. This module uses the paragraph vector model to learn fixed-length feature representations from a variable-length paragraph. We refer to the fixed-length feature as the TEB. This TEB module plays two roles to benefit paragraph captioning performance. First, it acts as a form of global and coherent deep supervision to regularize visual feature extraction in the image encoder. Second, it acts as a distributed memory to provide features of the whole paragraph to the language model, which alleviating the long-term dependency problem. Adding this module to two existing state-of-the-art methods achieves a new state-of-the-art by a large margin on the paragraph captioning Visual Genome dataset. Automatically generating a natural language description for visual content like image or video is an emerging interdisciplinary task. This task involves computer vision, natural language processing and artificial intelligence. Thanks to the advent of large datasets;; Krishna et al. (2017b), many recent works; have shown promising in generating a single high-level scene for images and videos. However, the coarse, scene-level descriptions that these models produce cannot meet real-world applications such as video retrieval, automatic medical report generation;; Li et al. (2018a), blind navigation and automatic video subtitling which capture fine-grained entities and have a coherent and logically detailed description. To tackle this challenge, a relatively new task called paragraph captioning is emerging. Paragraph captioning is the task of generating coherent and logically detailed descriptions by capturing the fine-grained entities of the image or video. A few works;; have pushed the performance to new heights with the main paragraph captioning dataset, the Visual Genome corpus, a dataset introduced by. Compared with the performance of single-sentence caption generating models, the performance paragraph-length caption generating models is lower by a large margin. Paragraph captioning for images and videos is challenging due to the requirement of both fine-grained image understanding and long-term language reasoning. To overcome these challenges, we propose the TEB module, a module that is easy to integrate with existing image captioning models. This module maps variedlength paragraphs to a fixed-length vector which we call TEB. Each unique vector in the TEB has distance meaning and indexed by the order of the word in the vocabulary. The TEB has a distributed memory. This is illustrated in detail in section 3. Existing deep learning based models typically consist of an image encoder to extract visual features in parallel with a RNN language model decoder to generate the sentences word by word sequentially. In the training stage, only a tiny partial scalar guiding information from the word level loss is available to optimize the image encoding training. This in an insufficient fine-grained and coherent image visual feature extraction. The TEB module, which holds the whole paragraph in a distributed memory model, can provide global supervision to better regularize the image encoder in the training stage. The RNNs are known to have a long-term dependency problem because of vanishing and exploding gradients which make it unable to meet long-term language reasoning. Since the TEB module has distributed memory and can provide ordering, it is better with long-term language reasoning. We integrated our TEB module with the state-of-the-art methods on the only available paragraph captioning dataset, the Visual Genome corpus, and achieved new state-of-the-art by a large margin. This image to text problem is a classic problem in computer vision and NLP. The first work to use deep neural networks to solve this problem was the Neural Image Caption (NIC) in , which uses a pre-trained CNN as the visual model and a RNN as the language model. The visual model extracts visual features which are fed to the first time step of the RNN. The language model takes visual features from the visual model at the first time step and predicts the first word, before feeding the predicted word into the next time step and so on. At each time step, the difference between the predicted word and the ground truth word is optimized by softmax with cross entropy loss. This work can only predict one short simple sentence for each natural image. The performance of this one sentence caption task is improved in by introducing an attention mechanism which focuses on related regions when generating a word per time step in the RNN model. In order to give a description for every object in an image, proposed a fully convolutional localization network which upgraded the region proposal network from to localize the salient regions. The RNN model then takes the corresponding visual features for each localized region to generate a sentence. However, simply joining all of the generated sentences together doesn't produce a coherent paragraph as there are semantic relationships between sentences, which is a shortcoming of DenseCap. Similarly, dense video captioning, a task which gives each event a description in a video, was first explored in Krishna et al. (2017a) by a variant of the existing proposal module and using 3D features. Later it was further improved by jointly localizing and describing eventsLi et al. (2018b). Recently, the RNN/LSTM language model was replaced by a CNN in; with comparable performance and the potential for parallel computing, which is a drawback of sequential models. In the inference process, however, this CNN model also need to be computed sequentially. Since computation cost is a big issue for video captioning, introduced a new method to find the useful frames which cut redundant information and reduce computation cost. Standard captioning generates single high-level sentence. Dense captioning generates a description for each salient object in an incoherent way. Paragraph captioning, however, overcomes the weaknesses of the previous two tasks by generating fine-grained and coherent natural language descriptions, like a story. To meet long-term language reasoning and the requirement of multiple topics in multiple sentences, a hierarchical recurrent neural network architectureLi et al.;;;; is widely used in paragraph captioning. For example, generate multiple sentences for video captioning by capturing strong temporal dependencies. uses a hierarchical recurrent network to build relationships between sentences. Regional features are passed to a sentence RNN to generate topic vectors with a halting distribution to control the ending of new topic generation. The generated topic vectors are then consumed by a word RNN to generate sentences. In this way, this hierarchical RNN and DenseCap offer two ways of generating new topics, which is essential for multiple sentence generation. The IU Chest X-ray dataset is used for automatic report generation on this unstructured reportJing et al. by using co-attention and the hierarchical LSTM. The Diversity model improves sentence diversity by introducing a repetitive penalty in the sequence-level training. However, all of these methods suffer from the fact that only a tiny partial scalar from the word level loss can be used as guiding information to optimize the image encoding in training. Our TEB module can overcome this and provides an alternative for the hierarchical recurrent neural network architecture. With our TEB module, only one level recurrent neural network is enough to generate multiple sentences with multiple find-grained topics. GANs have proved to improve real text generation in. is proposed to deal with the sequential and discrete property of text for text generation. solves the sparse signal from generator problem by leaking feature from the generator to the discriminator for long sentence generation. introduced a way to fill in the blank with GAN. use long-term feature banks for detailed video understanding. The proposed TEB module improves paragraph captioning by describing the rich content of a given image. Figure 1 shows an example of how the TEB module can be integrated with an existing typical image captioning pipeline. The paragraph vector is based on word vectors. A word vector is the concept of using a distributed vector representation of words. The basic idea is to predict a word given the other words in a context. The framework is shown in Figure 2. In this framework, each word is mapped to a unique vector which is a column of a matrix W. The column is indexed by the order of the word in the vocabulary. The features to predict the next word are the sum or concatenation of the vectors. To express this in a mathematical equation, let w 1, w 2, w 3,..., w T represent the vectors of a sequence of training words. The objective function of the framework is to maximize the average log probability Typically, a multi-class classifier such as softmax is used for the prediction task. So, we have where y i is the un-normalized log-probability for each output word i, which is computed as This framework is implemented in a neural network and trained using stochastic gradient descent through back-propagation. This type of model is the well known neural language model Bengio et al.. Compared to existing image captioning models, which only using recurrent neural networks, after training converges, this framework can map words with similar meaning to a similar position in the vector space. For example, "wind" and "beautiful" are far away from each other in the vector space, while "beautiful" and "pretty" are closer. Additionally, the distance between each unique word vector also carries meaning. This means that it can be used for analogy questions answering in a simple vector algebra manipulation: "waiter" -"man" + "women" = "waitress". This makes it easy to learn a linear matrix, such as a fully connected layer, to translate between visual features and these word vectors. Inspired by the word vector framework which can capture the semantics as a of a prediction task, The paragraph vector also contributes the prediction of the next word. In this paragraph vector framework (See Figure 3), similarly to the word vector framework, each word is still mapped to a unique vector which is a column of a matrix W, while each paragraph is mapped to a unique vector which is a column of a matrix D. Then both the word vector and paragraph vector are fused (either sum or concatenated) as features to predict the next word. We use concatenation in our implementation. The paragraph vector can be treated as a super word (or the topic of the paragraph) which acts as memory of the missing information from the current context. Hence, this framework is known as a distributed memory model. This property can compensate the recurrent neural network for its lack of generating logical connections between sentences or paragraphs. Figure 1: Integration of the paragraph vector framework as a TEB module to an existing deep learning based image captioning model. There are three interconnected components divided into three dashed rectangular boxes. In the green box on the top left, the image encoder extracts visual features through a CNN model. In the yellow box on the bottom, a RNN based language model decoder is used to generate paragraphs. Existing deep learning based models only contain these two components. The red box on the top right box is the TEB module: In the training stage, for an image, paragraph pair, the varied-length paragraph is mapped to a fixed-length vector which is called TEB through the paragraph vector framework. The visual features from the image encoder are converted to the predicted TEB (called TEB') through several fully connected layers. The TEB' is supervised by the TEB through an L1 loss, which acts as global deep supervision to regularize the visual feature extraction for the image encoder. The visual features and TEB' are concatenated and feed into the RNN as input. The generated paragraph is supervised by the ground truth paragraph through a word-level loss. In the inference stage, the TEB is not available and the TEB' acts as the TEB to provide the features of the whole paragraph to alleviate the long-term dependency problem for the language model. The integration of the paragraph vector as a TEB module for image paragraph captioning is illustrated in Figure 1.. The hyperparameters are as follows: The vector size (TEB size) is 512, the sliding window size is 50, the sampling threshold is 1e − 5, the negative size is 5. The paragraph vector model is trained for 1000 epochs before performing the inference to generate the TEB. Regardless of the dimension size of the visual features from the image encoder, the visual features are converted to the same dimension of the TEB by several fully connected layers. In the concatenation of the TEB' and visual features, a weight of 0.1 is applied to the TEB'. We integrate our TEB module with the Diversity model as its backbone. Self-critical sequence training (SCST) and repetitive training are also used. We also integrate the TEB module with a transformer model. The transformer model is adapted from the Bottom-Up and Top-Down model with the following modification: The LSTM-based language model is replaced by the transformer modelVaswani et al.. We used both cross-entropy and SCST training, without the repetition penalty, and beam search instead o greedy search. 5 Table 1 shows the quantitative . We have three models. The "Diversity + TEB" model is the Diversity model Melas-Kyriazi et al. with SCST trainingRennie et al. (2017, repetition penalty and TEB module. The "Transformer" model is Replacing the LSTM model with in the Bottom-UP and Top-Down modelAnderson et al.. The "Transformer + TEB" is the "Transformer" model with TEB module. The TEB module improve both baseline model by a large margin and our model "Diversity + TEB' achieve state of the art on the visual genome . Figure 2 shows the qualitative between A man is standing in a white basket. He is wearing a black shirt and a black hat. The man is holding a hot dog in his hand. There is a man in a black shirt standing behind the man. There is a man. The man is wearing a yellow shirt. The man is standing at a park. There are more people in the park. There are people sitting under the threes. There are people walking in the paths. The man is holding a sandwich. The sandwich is a hot dog. The sandwich has a sausage. The sandwich has onions. A large elephant is standing in the grass. The elephant is a baby elephant. The elephant has a long trunk. The baby elephant is walking. The grass is green. The elephants are standing on the grass. The water is calm. The elephants are white. The tusks are white. They are a few trees in the water. A large elephant is standing in the water. The elephant is walking in the water. There is a large body of water behind the elephant. There are a small rock behind the elephant. The elephant stands in the grass. The elephant is small. The elephant has tusks. The tusks are little. The elephant is grey. The elephant is standing by the water. The water is like a river. The grass is mostly yellow. There is a hill on the other side of the water. The water is still. there is wood in the water. The grass is short. There are trees on the other side of the river. A white toilet is in the bathroom. The toilet is white. The lid is white. There is a white toilet in the toilet. The toilet lid is up. The floor is made of white. The tiles are white. There is a white wall behind the toilet. A white toilet is sitting on the ground. There is a white toilet in the toilet. There is a toilet in front of the toilet. The toilet lid is up. The toilet bowl is cleaning. The toilet is a very light beige color. There's a white bar between the toilet lid and the toilet seat. The toilet is encased in a cubby space. The water in the toilet is low. The floor around the toilet is made of tiles. There are wires on the bottom left side of the toilet bowl. In this paper, we propose the Text Embedding Bank (TEB) module for visual paragraph captioning, a task which requires capturing fine-grained entities in the image to generate a detailed and coherent paragraph, like a story. Our TEB module provides global and parallel deep supervision and distributed memory for find-grained image understanding and long-term language reasoning. Integrating the TEB module to existing state-of-the-art methods achieves new state-of-the-art by a large margin.

Represent this text to find a much much longer article about it with say hundreds-thousands of words:

We invent a novel cluster-to-cluster framework for NMT training, which can better understand the both source and target language diversity.


Question: Represent the article for finding a single-sentence TLDR

Question: Learning Mahalanobis metric spaces is an important problem that has found numerous applications. Several algorithms have been designed for this problem, including Information Theoretic Metric Learning (ITML) [Davis et al. 2007] and Large Margin Nearest Neighbor (LMNN) classification [Weinberger and Saul 2009]. We consider a formulation of Mahalanobis metric learning as an optimization problem,where the objective is to minimize the number of violated similarity/dissimilarity constraints. We show that for any fixed ambient dimension, there exists a fully polynomial time approximation scheme (FPTAS) with nearly-linear running time. This is obtained using tools from the theory of linear programming in low dimensions. We also discuss improvements of the algorithm in practice, and present experimental on synthetic and real-world data sets. Our algorithm is fully parallelizable and performs favorably in the presence of adversarial noise. Learning metric spaces is a fundamental computational primitive that has found numerous applications and has received significant attention in the literature. We refer the reader to; for detailed exposition and discussion of previous work. At the high level, the input to a metric learning problem consists of some universe of objects X, together with some similarity information on subsets of these objects. Here, we focus on pairwise similarity and dissimilarity constraints. Specifically, we are given S, D Ă`X 2˘, which are sets of pairs of objects that are labeled as similar and dissimilar respectively. We are also given some u, ą 0, and we seek to find a mapping f: X Ñ Y, into some target metric space pY, ρq, such that for all x, y P S, ρpf pxq, f pyqq ď u, and for all x, y P D, ρpf pxq, f pyqq ě. In the case of Mahalanobis metric learning, we have X Ă R d, with |X| " n, for some d P N, and the mapping f: R d Ñ R d is linear. Specifically, we seek to find a matrix G P R dˆd, such that for all tp, qu P S, we have and for all tp, qu P D, we have 1.1 OUR CONTRIBUTION In general, there might not exist any G that satisfies all constraints of type 1 and 2. We are thus interested in finding a solution that minimizes the fraction of violated constraints, which corresponds to maximizing the accuracy of the mapping. We develop a p1`εq-approximation algorithm for optimization problem of computing a Mahalanobis metric space of maximum accuracy, that runs in near-linear time for any fixed ambient dimension d P N. This algorithm is obtained using tools from geometric approximation algorithms and the theory of linear programming in small dimension. The following summarizes our . Theorem 1.1. For any d P N, ε ą 0, there exists a randomized algorithm for learning d-dimensional Mahalanobis metric spaces, which given an instance that admits a mapping with accuracy r˚, computes a mapping with accuracy at least r˚´ε, in time d Op1q nplog n{εq Opdq, with high probability. The above algorithm can be extended to handle various forms of regularization. We also propose several modifications of our algorithm that lead to significant performance improvements in practice. The final algorithm is evaluated experimentally on both synthetic and real-world data sets, and is compared against the currently best-known algorithms for the problem. Several algorithms for learning Mahalanobis metric spaces have been proposed. Notable examples include the SDP based algorithm of , the algorithm of Globerson and Roweis for the fully supervised setting , Information Theoretic Metric Learning (ITML) by , which casts the problem as a particular optimization minimizing LogDet divergence, as well as Large Margin Nearest Neighbor (LMNN) by , which attempts to learn a metric geared towards optimizing k-NN classification. We refer the reader to the surveys; for a detailed discussion of previous work. Our algorithm differs from previous approaches in that it seeks to directly minimize the number of violated pairwise distance constraints, which is a highly non-convex objective, without resorting to a convex relaxation of the corresponding optimization problem. The rest of the paper is organized as follows. Section 2 describes the main algorithm and the proof of Theorem 1.1. Section 3 discusses practical improvements used in the implementation of the algorithm. Section 4 presents the experimental evaluation. In this Section we present an approximation scheme for Mahalanobis metric learning in d-dimensional Euclidean space, with nearly-linear running time. We begin by recalling some prior on the class of LP-type problems, which generalizes linear programming. We then show that linear metric learning can be cast as an LP-type problem. Let us recall the definition of an LP-type problem. Let H be a set of constraints, and let w: 2 H Ñ R Y t´8,`8u, such that for any G Ă H, wpGq is the value of the optimal solution of the instance defined by G. We say that pH, wq defines an LP-type problem if the following axioms hold: (A1) Monotonicity. For any F Ď G Ď H, we have wpF q ď wpGq. (A2) Locality. For any F Ď G Ď H, with´8 ă wpF q " wpGq, and any h P H, if wpGq ă wpG Y thuq, then wpF q ă wpF Y thuq. More generally, we say that pH, wq defines an LP-type problem on some H 1 Ď H, when conditions (A1) and (A2) hold for all F Ď G Ď H 1. A subset B Ď H is called a basis if wpBq ą´8 and wpB 1 q ă wpBq for any proper subset B 1 Ĺ B. A basic operation is defined to be one of the following: (B0) Initial basis computation. Given some G Ď H, compute any basis for G. (B1) Violation test. For some h P H and some basis B Ď H, test whether wpB Y thuq ą wpBq (in other words, whether B violates h). (B2) Basis computation. For some h P H and some basis B Ď H, compute a basis of B Y thu. We now show that learning Mahalanobis metric spaces can be expressed as an LP-type problem. We first note that we can rewrite and as and where A " G T G is positive semidefinite. We define H " t0, 1uˆ`R d 2˘, where for each p0, tp, quq P H, we have a constraint of type, and for every p1, tp, quq P H, we have a constraint of type. Therefore, for any set of constraints F Ď H, we may associate the set of feasible solutions for F with the set A F of all positive semidefinite matrices A P R nˆn, satisfying and for all constraints in F. Let w: 2 H Ñ R, such that for all F P H, we have where r P R d is a vector chosen uniformly at random from the unit sphere from some rotationallyinvariant probability measure. Such a vector can be chosen, for example, by first choosing some r 1 P R d, where each coordinate is sampled from the normal distribution N p0, 1q, and setting r " r 1 {}r 1 } 2. Lemma 2.1. When w is chosen as above, the pair pH, wq defines an LP-type problem of combinatorial dimension Opd 2 q, with probability 1. Moreover, for any n ą 0, if each r i is chosen using Ωplog nq bits of precision, then for each F Ď H, with n " |F |, the assertion holds with high probability. Proof. Since adding constraints to a feasible instance can only make it infeasible, it follows that w satisfies the monotonicity axiom (A1). We next argue that the locality axion (A2) also holds, with high probability. Let F Ď G Ď H, with 8 ă wpF q " wpGq, and let h P H, with wpGq ă wpG Y thuq. Let A F P A F and A G P A G be some (not necessarily unique) infimizers of wpAq, when A ranges in A F and A G respectively. The set A F, viewed as a convex subset of R d 2, is the intersection of the SDP cone with n half-spaces, and thus A F has at most n facets. There are at least two distinct infimizers for wpA G q, when A G P A G, only when the randomly chosen vector r is orthogonal to a certain direction, which occurs with probability 0. When each entry of r is chosen with c log n bits of precision, the probability that r is orthogonal to any single hyperplane is at most 2´c log n " n´c; the assertion follows by a union bound over n facets. This establishes that axiom (A2) holds with high probability. It remains to bound the combinatorial dimension, κ. Let F Ď H be a set of constraints. For each A P A F, define the ellipsoid For any A, A 1 P A F, with E A " E A 1, and Therefore in order to specify a linear transformation G, up to an isometry, it suffices to specify the ellipsoid E A. Each tp, qu P S corresponds to the constraint that the point pp´qq{u must lie in E A . Similarly each tp, qu P D corresponds to the constraint that the point pp´qq{ must lie either on the boundary or the exterior of E A . Any ellipsoid in R d is uniquely determined by specifying at most pd`3qd{2 " Opd 2 q distinct points on its boundary (see ;). Therefore, each optimal solution can be uniquely specified as the intersection of at most Opd 2 q constraints, and thus the combinatorial dimension is Opd 2 q. The basis computation step (B2) can be performed starting with the set of constraints B Y thu, and iteratively remove every constraint whose removal does not decrease the optimum cost, until we arrive at a minimal set, which is a basis. In total, we need to solve at most d SDPs, each of size Opd 2 q, which can be done in total time d Op1q. Finally, by the choice of w, any set containing a single constraint in S is a valid initial basis. Using the above formulation of Mahalanobis metric learning as an LP-type problem, we can obtain our approximation scheme. Our algorithm uses as a subroutine an exact algorithm for the problem (that is, for the special case where we seek to find a mapping that satisfies all constraints). We first present the exact algorithm and then show how it can be used to derive the approximation scheme. An exact algorithm. obtained a simple randomized linear-time algorithm for the minimum enclosing ball and minimum enclosing ellipsoid problems. This algorithm naturally extends to general LP-type problems (we refer the reader to ; for further details). With the interpretation of Mahalanobis metric learning as an LP-type problem given above, we thus obtain a linear time algorithm for in R d, for any constant d P N. The ing algorithm on a set of constraints F Ď H is implemented by the procedure Exact-LPTMLpF; Hq, which is presented in Algorithm 1. The procedure LPTMLpF; Bq takes as input sets of constraints F, B Ď H. It outputs a solution A P R dˆd to the problem induced by the set of constraints F Y B, such that all constraints in B are tight (that is, they hold with equality); if no such solution solution exists, then it returns nil. The procedure Basic-LPTMLpBq computes LPTMLpH; Bq. The analysis of implies that when Basic-LPTMLpBq is called, the cardinality of B is at most the combinatorial dimension, which by Lemma 2.1 is Opd 2 q. Thus the procedure Basic-LPTML can be implemented using one initial basis computation (B0) and Opd 2 q basis computations (B2), which by Lemma 2.2 takes total time d Op1q. Algorithm 1 An exact algorithm for Mahalanobis metric learning. An p1`εq-approximation algorithm. It is known that the above exact linear-time algorithm leads to an nearly-linear-time approximation scheme for LP-type problems. This is summarized in the following. We refer the reader to for a more detailed treatment. Lemma 2.3 (, Ch. 15). Let A be some LP-type problem of combinatorial dimension κ ą 0, defined by some pair pH, wq, and let ε ą 0. There exists a randomized algorithm which given some instance F Ď H, with |F | " n, outputs some basis B Ď F, that violates at most p1`εqk constraints in F, such that wpBq ď wpB 1 q, for any basis B 1 violating at most k constraints in F, in, log κ`2 n kε 2κ`2 )¯p t 1`t2 q¯, where t 0 is the time needed to compute an arbitrary initial basis of A, and t 1, t 2, and t 3 are upper bounds on the time needed to perform the basic operations (B0), (B1) and (B2) respectively. The algorithm succeeds with high probability. For the special case of Mahalanobis metric learning, the corresponding algorithm is given in Algorithm 2. The approximation guarantee for this algorithm is summarized in 1.1. We can now give the proof of our main . Proof of Theorem 1.1. Follows immediately by Lemmas 2.2 and 2.3. Algorithm 2 An approximation algorithm for Mahalanobis metric learning. procedure LPTML(F) for i " 0 to log 1`ε n do p Ð p1`εq´i for j " 1 to log Opd 2 q n do subsample F j Ď F, where each element is chosen independently with probability p A i,j Ð Exact-LPTMLpF j q end for end for return a solution out of tA i,j u i,j, violating the minimum number of constraints in F end procedure Regularization. We now argue that the LP-type algorithm described above can be extended to handle certain types of regularization on the matrix A. In methods based on convex optimization, introducing regularizers that are convex functions can often be done easily. In our case, we cannot directly introduce a regularizing term in the objective function that is implicit in Algorithm 2. More specifically, let costpAq denote the total number of constraints of type and that A violates. Algorithm 2 approximately minimizes the objective function costpAq. A natural regularized version of Mahalanobis metric learning is to instead minimize the objective function cost 1 pAq:" costpAq`η¨regpAq, for some η ą 0, and regularizer regpAq. One typical choice is regpAq " trpACq, for some matrix C P R dˆd; the case C " I corresponds to the trace norm (see). We can extend the Algorithm 2 to handle any regularizer that can be expressed as a linear function on the entries of A, such as trpAq. The following summarizes the . Theorem 2.4. Let regpAq be a linear function on the entries of A, with polynomially bounded coefficients. For any d P N, ε ą 0, there exists a randomized algorithm for learning d-dimensional Mahalanobis metric spaces, which given an instance that admits a solution A 0 with cost 1 pA 0 q " c˚, computes a solution A with cost 1 pAq ď p1`εqc˚, in time d Op1q nplog n{εq Opdq, with high probability. Proof. If η ă ε t, for sufficiently large constant t ą 0, since the coefficients in regpAq are polynomially bounded, it follows that the largest possible value of η¨regpAq is Opεq, and can thus be omitted without affecting the . Similarly, if η ą p1{εqn t 1, for sufficiently large constant t 1 ą 0, since there are at most`n 2˘c onstraints, it follows that the term costpAq can be omitted form the objective. Therefore, we may assume w.l.o.g. that regpA 0 q P rε Op1q, p1{εqn Op1q s. We can guess some i " Oplog n`logp1{εqq, such that regpA 0 q P pp1`εq i´1, p1`εq i s. We modify the SDP used in the proof of Lemma 2.2 by introducing the constraint regpAq ď p1`εq i . Guessing the correct value of i requires Oplog n`logp1{εqq executions of Algorithm 2, which implies the running time bound. We now discuss some modifications of the algorithm described in the previous section that significantly improve its performance in practical scenarios, and have been integrated in our implementation. Move-to-front and pivoting heuristics. We use heuristics that have been previously used in algorithms for linear programming ; , minimum enclosing ball in R 3 , minimum enclosing ball and ellipsoid is R d, for any fixed d P N , as well as in fast implementations of minimum enclosing ball algorithms Gärtner. The move-to-front heuristic keeps an ordered list of constraints which gets reorganized as the algorithm runs; when the algorithm finds a violation, it moves the violating constraint to the beginning of the list of the current sub-problem. The pivoting heuristic further improves performance by choosing to add to the basis the constraint that is "violated the most". For instance, for similarity constraints, we pick the one that is mapped to the largest distance greater than u; for dissimilarity constraints, we pick the one that is mapped to the smallest distance less than. Approximate counting. The main loop of Algorithm 2 involves counting the number of violated constraints in each iteration. In problems involving a large number of constraints, we use approximate counting by only counting the number of violations within a sample of Oplog 1{εq constraints. We denote by LPTML t for the version of the algorithm that performs a total of t iterations of the inner loop. Early termination. A bottleneck of Algorithm 2 stems from the fact that the inner loop needs to be executed for log Opd 2 q n iterations. In practice, we have observed that a significantly smaller number of iterations is needed to achieve high accuracy. Parallelization. Algorithm 2 consists of several executions of the algorithm Exact-LPTML on independently sampled sub-problems. Therefore, Algorithm 2 can trivially be parallelized by distributing a different set of sub-problems to each machine, and returning the best solution found overall. We have implemented Algorithm 2, incorporating the practical improvements described in Section 3, and performed experiments on synthetic and real-world data sets. Our LPTML implementation and documentation can be found at the supplementary material 1 . We now describe the experimental setting and discuss the main findings. Classification task. Each data set used in the experiments consists of a set of labeled points in R d . The label of each point indicates its class, and there is a constant number of classes. The set of similarity constraints S (respt. dissimilarity constraints D) is formed by uniformly sampling pairs of points in the same class (resp. from different classes). We use various algorithms to learn a Mahalanobis metric for a labeled input point set in R d, given these constraints. The values u and are chosen as the 90th and 10th percentiles of all pairwise distances. We used 2-fold cross-validation: At the training phase we learn a Mahalanobis metric, and in the testing phase we use k-NN classification, with k " 4, to evaluate the performance. Data sets. We have tested our algorithm on the following synthetic and real-world data sets: 1. Real-world: We have tested the performance of our implementation on the Iris, Wine, Ionosphere and Soybean data sets from the UCI Machine Learning Repository 2. 2. Synthetic: Next, we consider a synthetic data set that is constructed by first sampling a set of 100 points from a mixture of two Gaussians in R 2, with identity covariance matrices, and with means p´3, 0q and p3, 0q respectively; we then apply a linear transformation that stretches the y axis by a factor of 40. This linear transformation reduces the accuracy of k-NN on the underlying Euclidean metric with k " 4 from 1 to 0.68. 3. Synthetic + Adversarial Noise: We modify the above synthetic data set by introducing a small fraction of points in an adversarial manner, before applying the linear transformation. Figure 3b depicts the noise added as five points labeled as one of the classes, and sampled from a Gaussian with identity covariance matrix and mean p´100, 0q (Figure 3a). Algorithms. We compare the performance of our algorithm against ITML and LMNN. We used the implementations provided by the authors of these works, with minor modifications. Accuracy. Algorithm 2 minimizes the number of violated pairwise distance constraints. It is interesting to examine the effect of this objective function on the accuracy of k-NN classification. Comparison to ITML and LMNN. We compared the accuracy obtained by LPTML t, for t " 2000 iterations, against ITML and LMNN. Table 1 summarizes the findings on the real-world and data sets and the synthetic data set without adversarial noise. We observe that LPTML achieves accuracy that is comparable to ITML and LMNN. We observe that LPTML outperforms ITML and LMNN on the Synthetic + Adversarial Noise data set. This is due to the fact that the introduction of adversarial noise causes the relaxations used in ITML and LMNN to be biased towards contracting the x-axis. In contrast, the noise does not "fool" LPTML because it only changes the optimal accuracy by a small amount. The are summarized in Figure 2.

Represent the short summary for finding the corresponding scientific article (long)

Fully parallelizable and adversarial-noise resistant metric learning algorithm with theoretical guarantees.

We prove that for a large class of functions f there exists an interval certified robust network approximating f up to arbitrary precision.