Question: Represent the scientific article to find a short TLDR!

Question: We tackle unsupervised domain adaptation by accounting for the fact that different domains may need to be processed differently to arrive to a common feature representation effective for recognition. To this end, we introduce a deep learning framework where each domain undergoes a different sequence of operations, allowing some, possibly more complex, domains to go through more computations than others. This contrasts with state-of-the-art domain adaptation techniques that force all domains to be processed with the same series of operations, even when using multi-stream architectures whose parameters are not shared. As evidenced by our experiments, the greater flexibility of our method translates to higher accuracy. Furthermore, it allows us to handle any number of domains simultaneously. While deep learning has ushered in great advances in automated image understanding, it still suffers from the same weaknesses as all other machine learning techniques: when trained with images obtained under specific conditions, deep networks typically perform poorly on images acquired under different ones. This is known as the domain shift problem: the changing conditions cause the statistical properties of the test, or target, data, to be different from those of the training, or source, data, and the network's performance degrades accordingly. Domain adaptation aims to address this problem, especially when annotating images from the target domain is difficult, expensive, or downright infeasible. The dominant trend is to map images to features that are immune to the domain shift, so that the classifier works equally well on the source and target domains (; ; . In the context of deep learning, the standard approach is to find those features using a single architecture for both domains (; ; ;). Intuitively, however, as the domains have different properties, it is not easy to find one network that does this effectively for both. A better approach is to allow domains to undergo different transformations to arrive at domain-invariant features. This has been the focus of recent work (; Bermúdez-Chacón et al., 2018; ;, where source and target data pass through two different networks with the same architecture but different weights, nonetheless related to each other. In this paper, we introduce a novel, even more flexible paradigm for domain adaptation, that allows the different domains to undergo different computations, not only in terms of layer weights but also in terms of number of operations, while selectively sharing subsets of these computations. This enables the network to automatically adapt to situations where, for example, one domain depicts simpler images, such as synthetic ones, which may not need as much processing power as those coming from more complex domains, such as images taken in-the-wild. Our formulation reflects the intuition that source and target domain networks should be similar because they solve closely related problems, but should also perform domain-specific computations to offset the domain shift. To turn this intuition into a working algorithm, we develop a multibranch architecture that sends the data through multiple network branches in parallel. What gives it the necessary flexibility are trainable gates that are tuned to modulate and combine the outputs of these branches, as shown in, each of which processes the data in parallel branches, whose outputs are then aggregated in a weighted manner by a gate to obtain a single response. To allow for domain-adaptive computations, each domain has its own set of gates, one for each computational unit, which combine the branches in different ways. As a , some computations are shared across domains while others are domain-specific. computations should be carried out for each one. As an additional benefit, in contrast to previous strategies for untying the source and target streams (;, our formulation naturally extends to more than two domains. In other words, our contribution is a learning strategy that adaptively adjusts the specific computation to be performed for each domain. To demonstrate that it constitutes an effective approach to extracting domain-invariant features, we implement it in conjunction with the popular domain classifier-based method of . Our experiments demonstrate that our Domain Adaptive Multibranch Networks, which we will refer to as DAMNets, not only outperform the original technique of , but also the state-of-the-art strategy for untying the source and target weights of , which relies on the same domain classifier. We will make our code publicly available upon acceptance of the paper. Domain Adaptation. Domain adaptation has achieved important milestones in recent years (; ; ; ;), with deep learning-based methods largely taking the lead in performance. The dominant approach to deep domain adaptation is to learn a domain-invariant data representation. This is commonly achieved by finding a mapping to a feature space where the source and target features have the same distribution.; Long et al. (2015; ; , the distribution similarity was measured in terms of Maximum Mean Discrepancy , while other metrics based on second-and higher-order statistics were introduced in;;. , the distribution alignment process was disambiguated by exploiting the class labels, and in Häusser et al.; by leveraging anchor points associating embeddings between the domains. Another popular approach to learning domain-invariant features is to train a classifier to recognize the domain from which a sample was drawn, and use adversarial training to arrive to features that the classifier can no longer discriminate (; ; . This idea has spawned several recent adversarial domain adaptation classification , semantic segmentation ), and active learning techniques, and we will use such a classifier. Closest in spirit to our approach are those that do not share the weights of the networks that process the source and target data (; Bermúdez-Chacón et al., 2018; ; . , the weights were simply allowed to vary freely. ; Bermúdez-Chacón et al., it was shown that regularizing them to remain close to each other was beneficial. More recently, proposed to train small networks to map the source weights to the target ones. While these methods indeed untie the source and target weights, the source and target data still undergo the same computations, i.e., number of operations. In this paper, we argue that the amount of computation, that is, the network capacity, should adapt to each domain and reflect their respective complexities. We rely on a domain classifier as in; Ganin et al. (2016; . However, we do not force the source and target samples to go through the same transformations, which is counterintuitive since they display different appearance statistics. Instead, we start from the premise that they should undergo different computations and use domain-specific gates to turn this premise into our DAMNet architecture. Dynamic Network Architectures. As the performance of a neural network is tightly linked to its structure, there has been a recent push towards automatically determining the best architecture for the problem at hand. While neural architecture search techniques (; ; ; ;) aim to find one fixed architecture for a given dataset, other works have focused on dynamically adapting the network structure at inference time (; ; ; ;). In particular, in;;; , gates were introduced for this purpose. While our DAMNets also rely on gates, their role is very different: first, we work with data coming from different domains, whereas these gated methods, with the exception of , were all designed to work in the single-domain scenario. Second, and more importantly, these techniques aim to define a different computational path for every test sample. By contrast, we seek to determine the right computation for each domain. Another consideration is that we freeze our gates for inference while these methods must constantly update theirs. We believe this to be illsuited to domain adaptation, particularly because learning to adapt the gates for the target domain, for which only unlabeled data is available, is severely under-constrained. This lack of supervision may be manageable when one seeks to define operations for a whole domain, but not when these operations are sample-specific. We now describe our deep domain adaptation approach, which automatically adjusts the computations that the different domains undergo. We first introduce the multibranch networks that form the backbone of our DAMNet architecture and then discuss training in the domain adaptation scenario. is an aggregation of the outputs of parallel computations, or branches, f Let us first consider a single domain. In this context, a traditional deep neural network can be thought of as a sequence of N f operations f (i) (·) 1≤i≤N f, each transforming the output of the previous one. Given an input image x, this can be expressed as As a general convention, each operation f (i) (·) can represent either a single layer or multiple ones. Our formulation extends this definition by replacing each f (i) by multiple parallel computations, as shown in Fig. 2. More specifically, we replace each f (i) by a computational unit {f K} consisting of K parallel branches. Note that this K can be different at each stage of the network and should therefore be denoted as K (i). However, to simplify notation, we drop this index below. Given this definition, we write the output of each computational unit as whereΣ(·) is an aggregation operator that could be defined in many ways. It could be a simple summation that gives all outputs equal importance, or, at the opposite end of the spectrum, a multiplexer that selects a single branch and ignores the rest. To cover the range between these two alternatives, we introduce learnable gates that enable the network to determine what relative importance the different branches should be given. Our gates perform a weighted combination of the branch outputs. Each gate is controlled by a set of K activation weights {φ, and a unit returns If ∀j, φ (i) j = 1, the gate performs a simple summation. If φ (i) j = 1 for a single j and 0 for the others, it behaves as a multiplexer. The activation weights φ (i) j enable us to modulate the computational graph of network block f (i). To bound them and encourage the network to either select or discard each branch in a computational unit, we write them in terms of sigmoid functions with adaptive steepness. That is, where the g j s are learnable unbounded model parameters, and π (i) controls the plasticity of the activation-the rate at which φ j varies between the extreme values 0 and 1 for block i. During training, we initially set π (i) to a small value, which enables the network to explore different gate configurations. We then apply a cooling schedule on our activations, by progressively increasing π (i) over time, so as to encourage the gates to reach a firm decision. Note that our formulation does not require, that is, we do not require the aggregated output x (i) to be a convex combination of the branch outputs f ). This is deliberate because allowing the activation weights to be independent from one another provides additional flexibility for the network to learn general additive relationships. Finally, a Multibranch Network is the concatenation of multiple computational units, as shown in Fig. 1. For the aggregation within each unit f (i) to be possible, the f (i) j s' outputs must be of matching shapes. Furthermore, as in standard networks, two computational units can be attached only if the output shape of the first one matches the input shape of the second. Although it would be possible to define computational units at any point in the network architecture, in practice, we usually take them to correspond to groups of layers that are semantically related. For example, one would group a succession of convolutions, pooling and non-linear operations into the same computational unit. Our goal is to perform domain adaptation, that is, leverage a large amount of labeled images, drawn from a source domain, to train a model for a target domain, whose data distribution is different and for which we only have access to unlabeled images To this end, we extend the gated networks of Section 3.1 by defining two sets of gates, one for the source domain and one for the target one. Let {(φ s) be the corresponding source and target activation weights for computational unit f (i), respectively. Given a sample x d coming from a domain d ∈ {s, t}, we take the corresponding output of the i-th computational unit to be Note that under this formulation, the domain identity d of the sample is required in order to select the appropriate The concatenated computational units forming the DAMNet encode sample x from domain d into a feature vector z = f (x, d). Since the gates for different domains are set independently from one another, the outputs of the branches for each computational unit are combined in a domainspecific manner, dictated by the activation weights (φ d) j. Therefore, the samples are encoded to a common space, but arrive to it through potentially different computations. Fig. 3 depicts this process. Ultimately, the network can learn to share weights for computational unit f (i) by setting; Rozantsev et al. (2018;, it can learn to use more computation for one domain than for the other by setting (φ s) while having only a single non-zero (φ t) Figure 3: Computational graphs for the source (top) and target (bottom) domains, for the same network. While both domains share the same computational units, their outputs are obtained by different aggregations of their inner operations, e.g., in the first unit, the source domain does not use the middle two operations, whereas the target domain does; by contrast, both exploit the fourth operation. In essence, this scheme adapts the amount of computation that each domain is subjected to. The above formulation treats all branches for each computational unit as potentially sharable between domains. However, it is sometimes desirable not to share at all. For example, batchnormalization layers that accumulate and update statistics of the data over time, even during the forward pass, are best exposed to a single domain to learn domain-specific statistics. We allow for this by introducing computational units whose gates are fixed, yet domain specific, and that therefore act as multiplexers. After the last computational unit, a small network p y operates directly on the encodings and returns the class assignmentŷ = p y (z), thus subjecting the encodings for all samples to the same set of operations. The formulation outlined above extends naturally to more than two domains, by assigning one set of gates per domain. This enables us to exploit annotated data from different source domains, and even to potentially handle multiple target domains simultaneously. In this generalized case, we introduce governing sets of gates with activations φ d1,..., φ d D for D different domains. They act in the same way as in the two-domain case and the overall architecture remains similar. When training our models, we jointly optimize the gate parameters (g d) (i) j, from Eq. 4, along with the other network parameters using standard back-propagation. To this end, we make use of a composite loss function, designed to encourage correct classification for labeled samples from the source domain(s) and align the distributions of all domains, using labeled and unlabeled samples. This loss can be expressed as where and u are the sets of labeled and unlabeled samples, respectively, and where we assumed, without loss of generality, that the samples are ordered. The first term in this loss, L y (y,ŷ), is the standard cross-entropy, which compares the groundtruth class probabilities y with the predicted onesŷ = p y (z), where, as discussed in Section 3.2.1, ) is the feature encoding of sample x from domain d. For the second term, which encodes distribution alignment, we rely on the domain confusion strategy of , which is commonly used in existing frameworks. Specifically, for D domains, we make use of an auxiliary domain classifier network p d that predicts a D-dimensional vector of domain probabilitieŝ d given the feature vector z. Following the gradient reversal technique of Ganin & Lempitsky, where d is the D-dimensional binary vector encoding the ground-truth domain, d i indicates the i-th element of d, andd = p d (R(z)), with R the gradient reversal pseudofunction of that enables to incorporate adversarial training directly into back-propagation. That is, with this loss, standard back-propagation trains jointly the domain classifier to discriminate the domains and the feature extractor f (·) to produce features that fool this classifier. When training is complete and the gates have reached a stable state, the branches whose activations are close to zero are deactivated. This prevents the network from performing computations that are irrelevant and allows us to obtain a more compact network to process the target data. Since we rely on the domain confusion loss to train our model, we treat the Domain-Adversarial Neural Network (DANN) method of , as our first baseline. To demonstrate the benefits of our approach over simply untying the source and target stream parameters, we compare our approach against the Residual Parameter Transfer (RPT) method of , which constitutes the state of the art in doing so. Note that RPT also relies on the domain confusion loss, which makes our comparison fair. In addition, we report the of directly applying a model trained on the source domain to the target, without any domain adaptation, which we refer to as "No DA". We also provide the oracle accuracy of a model trained on the fully-labeled target domain, referred to as "On TD". We adapt different network architectures to the multibranch paradigm for different adaptation problems. For all cases, we initialize our networks' parameters by training the original versions of those architectures on the source domains, either from scratch, for simple architectures, or by fine-tuning weights learned on ImageNet, for very deep ones. We then set the parameters of all branches to the values from the corresponding layers. We perform this training on the predefined training splits, when available, or on 75% of the images, otherwise. The initial values of the gate parameters are defined so as to set the activations to 1 K, for each of the K branches. This prevents our networks from initially favoring a particular branch for any domain. To train our networks, we use Stochastic Gradient Descent with a momentum of 0.9 and a variable learning rate defined by the annealing schedule of as µ p = µ0 (1+α·p) β, where p is the training progress, relative to the total number of training epochs, µ 0 is the initial learning rate, which we take to be 10 −2, and α = 10 and β = 0.75 as in. We eliminate exploding gradients by 2 -norm clipping. Furthermore, we modulate the plasticity of the activations at every gate as π (i) = 1 − p, that is, we make π (i) decay linearly as training progresses. As data preprocessing, we apply mean subtraction, as in. We train for 200 epochs, during which the network is exposed to all the image data from the source and target domains, but only to the annotations from the source domain(s). Our "On TD" oracle is trained on either the preset training splits, when available, or our defined training data, and evaluated on the corresponding test data. For the comparison to this oracle to be meaningful, we follow the same strategy for our DAMNets. That is, we use the unlabeled target data from the training splits only and report on the testing splits. This protocol differs from that of , which relied on a transductive evaluation, where all the target images, training and test ones, were seen by the networks during training. We evaluate our method in the task of image recognition for which we use several domain adaptation benchmark problems: Digits, which comprises three domains: MNIST , MNIST-M , and SVHN ; Office (Saenko et al., Table 1 : Domain Adaptation datasets and . We compare the accuracy of our DAMNet approach with that of DANN and of RPT , for image classification tasks commonly used to evaluate domain adaptation methods. Our DAMNets yield a significant accuracy boost in the presence of large domain shifts, particularly when using more than one source domain. A more comprehensive evaluation on all datasets is provided in Appendix D. Office-Home: Art (A), Clipart (C), Product (P), Real (R) Setup. As discussed in Section 3, our method is general and can work with any feed-forward network architecture. To showcase this, for the digit recognition datasets, we apply it to the LeNet and SVHNet architectures , which are very simple convolutional networks, well suited for small images. , we employ LeNet when using the synthetic datasets MNIST and MNIST-M as source domains, and SVHNet when SVHN acts as source domain. We extend these architectures to multibranch ones by defining the computational units as the groups of consecutive convolution, pooling and non-linear operations defined in the original model. For simplicity, we use as many branches within each computational unit as we have domains, and all branches from a computational unit follow the same architecture, which we provide in Appendix A, Figures 1 and 2. As backbone network to process all the rest of the datasets, we use a ResNet-50 , with the bottleneck layer modification of. While many multibranch configurations can be designed for such a deep network, we choose to make our gated computational units coincide with the layer groupings defined in , namely conv1, conv2 x, conv3 x, conv4 x, and conv5 x. The ing multibranch network is depicted in Appendix A, Figure 4. We feed our DAMNets images resized to 224 × 224 pixels, as expected by ResNet-50. Results. The for the digit recognition and Office-Home datasets are provided in Table 1. Results for Office and VisDA17 datasets are presented in Appendix D. Our approach outperforms the baselines in all cases. For the Digits datasets, in addition to the traditional two-domain setup, we also report when using two source domains simultaneously. Note that the reference method RPT does not apply to this setting, since it was designed to transform a single set of source parameters to the target ones. Altogether, our method consistently outperforms the others. Note that the first two columns correspond to the combinations reported in the literature. We believe, however, that the SVHN MNIST one is quite artificial, since, in practice, one would typically annotate simpler, synthetic images and aim to use real ones at test time. We therefore also report synthetic SVHN cases, which are much more challenging. The multi-source version of our method achieves a significant boost over the baselines in this scenario. To further demonstrate the potential of our approach in this setting, we replaced its backbone with the much deeper ResNet-50 network and applied it on upscaled versions of the images. As shown in the column indicated by a, this allowed us to achieve an accuracy close to 80%, which is remarkable for such a difficult adaptation task. On Office-Home, the gap between DAMNet and the baselines is again consistent across the different domain pairs. Note that, here, because of the relatively large number of classes, the overall performance is low for all methods. Importantly, our show that we gain performance by training on more than one source domain, and by leveraging all synthetic domains to transfer to the real one, our approach reaches an accuracy virtually equal to that of using full supervision on the target domain. Despite our best efforts, we were unable to obtain convincing for RPT using the authors' publicly available code, as for this dataset were not originally reported for RPT. Gate dynamics. To understand the way our networks learn the domain-specific branch assignments, we track the state of the gates for all computational units over all training epochs. In Figure 4, we plot the corresponding evolution of the gate activations for the DSLR+Webcam Amazon task on Office. Note that our DAMNet leverages different branches over time for each domain before reaching a firm decision. Interestingly, we can see that, with the exception of the first unit, which performs low-level computations, DSLR and Webcam share all branches. By contrast, Amazon, which has a significantly different appearance, mostly uses its own branches, except in two computational units. This evidences that our network successfully understands when domains are similar and can thus use similar computations. No adaptation 0.377 DANN 0.715 ADDA 0.731 Two-stream 0.732 RPT 0.743 DAMNet 0.792 We evaluate our method for the detection of drones from video frames, on the UAV-200 dataset , which contains examples of drones both generated artificially and captured from real video footage. Full details and example images are provided in Appendix B.3 Setup. Our domain adaptation leverages both the synthetic examples of drones, as source domain, and the limited amount of annotated real drones, as target domain, as well as the negative examples, to predict the class of patches from the validation set of real images. We follow closely the supervised setup and network architecture of , including the use of AdaDelta as optimizer, cross-entropy as loss function, and average precision as evaluation metric. Our multibranch computational units are defined as groupings of successive convolutions, nonlinearities, and pooling operations. The details of the architecture are provided in Appendix A, Figure 3. Results. Our method considerably surpasses all the others in terms of average precision, as shown in Table 2, thus validating DAMNets as effective models for leveraging synthetic data for domain adaptation in real-world problems. We validate the effectiveness of our method as a feature extractor, by combining it with the Maximum Classifier Discrepancy (MCD) method of. As MCD operates on the extracted encodings, we replace the encoding strategy that MCD uses, which is the same as DANN, with our DAMNet. Or, in other words, we replace the domain classifier in our approach with the corresponding MCD term. Specifically, we use a single computational unit with two branches, each of which replicates the architectures proposed in. We present the of combining MCD with DAMNet in Table 3. In all tested scenarios, we obtain improvements over using MCD as originally proposed. To obtain more insights about specific branch decisions, we evaluate the effects of adding extra branches to the network, as well as using branches with different capacities. When computational units are composed of branches of different capacities, DAMNets often assign branches with more capacity to more complex domains. To exemplify this, we trained a modified multibranch SVHNet for adaptation between MNIST and SVHN. Instead of the identical branches originally used, we replace the second branch in each computational unit with a similar branch where the convolution operation is performed by 1x1 rather than 5x5 kernels. These second branches, with Figure 4: Evolution of the gates' activations for each of the computational units in a multibranch ResNet-50 network, for the Office DSLR + Webcam Amazon domain adaptation problem. In the top two rows, we show the gates for the source domains and in the bottom row for the target one. All branches are initialized to parameters obtained from a single ResNet-50 trained on ImageNet. Note how for the first computational unit, conv1, each domain chooses to process the data with different branches. In the remaining units, the two source domains, which have similar appearance, share all the computations. By contrast, the target domain still uses its own branches in conv3 x, and conv4 x to account for its significantly different appearance. When arriving at conv 5x, the data has been converted to a domain-agnostic representation, and hence the same branch can operate on all domains. 25 times fewer parameters each, are mostly used by the simpler domain-MNIST in this case. We provide the gate evolution that reflects this in Appendix C, Figures 5 and 6. We explore the effects of using more branches than domains, so as to provide the networks with alternative branches from where to choose. In particular, we explore the case where K = D + 1. We evaluate multibranch LeNet and ResNet architectures under this setting. We show the gate activation evolution in Appendix C, Figures 7 and 8. During the training process, we have observed that the networks quickly choose to ignore extra branches when K > D. This suggests that they did not contribute to the learning of our feature extraction. We did not find experimental evidence to support that K > D is beneficial. We have introduced a domain adaptation approach that allows for adaptive, separate computations for different domains. Our framework relies on computational units that aggregate the outputs of multiple parallel operations, and on a set of trainable domain-specific gates that adapt the aggregation process to each domain. Our experiments have demonstrated the benefits of this approach over the state-of-the-art weight untying strategy; the greater flexibility of our method translates into a consistently better accuracy. Although we only experimented with using the same branch architectures within a computational unit, our framework generalizes to arbitrary branch architectures, the only constraint being that their outputs are of commensurate shapes. An interesting avenue for future research would therefore be to automatically determine the best operation to perform for each domain, for example by combining our approach with neural architecture search strategies. Figure 1: Multibranch LeNet. This architecture is a multibranch extension to the LeNet used by DANN (Figure 2 : Multibranch SVHNet. This architecture is a multibranch extension to the SVHNet used by DANN ( . We preserve the groupings described in the original paper . N denotes the number of classes in the dataset. MNIST consists of black and white images of handwritten digits from 0 to 9. All images are of size 28 × 28 pixels. The standard training and testing splits contain 60,000 and 10,000 examples, respectively. MNIST-M is synthetically generated by randomly replacing the foreground and pixels of random MNIST samples with natural images. Its image size is 32 × 32, and the standard training and testing splits contain 59,001 and 9,001 images, respectively. SVHN , the Street View House Numbers dataset, consists of natural scene images of numbers acquired from Google Street View. Its images are also of size 32 × 32 pixels, and its preset training and testing splits are of 73,257 and 26,032 images, respectively. The SVHN images are centered at the desired digit, but contain clutter, visual artifacts, and distractors from its surroundings. Office is a multiclass object recognition benchmark dataset, containing images of 31 categories of objects commonly found in office environments. It contains color images from three different domains: 2,817 images of products scraped from Amazon, 498 images acquired using a DSLR digital camera, and 795 images captured with a webcam. The images are of arbitrary sizes and aspect ratios. Figure 5: Gate evolution for a multibranch SVHN network with branches of different capacities. Branch 1 is the original branch that applies 5x5 convolutions to the image, whereas branch 2 is a similar architecture but with 1x1 convolutions instead. The network quickly recognizes that SVHN requires a more complex processing and hence assigns the respective branch to it for computational units 1 and 3. Figure 6: Gate evolution for a multibranch LeNet network with branches of different capacities. We have simplified the architecture to encapsulate the feature extraction into a single computational unit in this case. Similarly to the above, we modify the second branch for a simpler computation. The original branches apply convolution operations to extract 32 channels with a 5x5 kernel, and then to extract 48 channels from those with a 5x5 kernel. We replace them in the second branch with 24 channels 3x3 kernel and 48 channels 1x1 kernel convolutions, respectively, which yields commensurate shapes with the original branch, but with more than 20 times fewer parameters. Unlike in the above experiment, we do not force the gates to open or close. The network still assigns combinations of branches that reflect the difference in visual complexity of the domains. Table 4: Domain Adaptation . We compare the accuracy of our DAMNet approach with that of DANN and of RPT , for image classification tasks commonly used to evaluate domain adaptation methods. As illustrated in Appendix B, different source and target domain combinations present various degrees of domain shift, and some combinations are clearly more challenging than others. Our DAMNets yield a significant accuracy boost in the presence of large domain shifts, particularly when using more than one source domain. and Evaluated with a ResNet-50 * Results reported as Average Precision

Represent this text to find a much much longer article about it with say hundreds-thousands of words

A Multiflow Network is a dynamic architecture for domain adaptation that learns potentially different computational graphs per domain, so as to map them to a common representation where inference can be performed in a domain-agnostic fashion.

represent the tldr for finding the corresponding scientific article

We present a new approach, SNIP, that is simple, versatile and interpretable; it prunes irrelevant connections for a given task at single-shot prior to training and is applicable to a variety of neural network models without modifications.


Question: Represent the text to find a very short summary (1-2 sentences max)

Question: The practical usage of reinforcement learning agents is often bottlenecked by the duration of training time. To accelerate training, practitioners often turn to distributed reinforcement learning architectures to parallelize and accelerate the training process. However, modern methods for scalable reinforcement learning (RL) often tradeoff between the throughput of samples that an RL agent can learn from (sample throughput) and the quality of learning from each sample (sample efficiency). In these scalable RL architectures, as one increases sample throughput (i.e. increasing parallelization in IMPALA ), sample efficiency drops significantly. To address this, we propose a new distributed reinforcement learning algorithm, IMPACT. IMPACT extends PPO with three changes: a target network for stabilizing the surrogate objective, a circular buffer, and truncated importance sampling. In discrete action-space environments, we show that IMPACT attains higher reward and, simultaneously, achieves up to 30% decrease in training wall-time than that of IMPALA. For continuous control environments, IMPACT trains faster than existing scalable agents while preserving the sample efficiency of synchronous PPO. Proximal Policy Optimization is one of the most sample-efficient on-policy algorithms. However, it relies on a synchronous architecture for collecting experiences, which is closely tied to its trust region optimization objective. Other architectures such as IMPALA can achieve much higher throughputs due to the asynchronous collection of samples from workers. Yet, IMPALA suffers from reduced sample efficiency since it cannot safely take multiple SGD steps per batch as PPO can. The new agent, Importance Weighted Asynchronous Architectures with Clipped Target Networks (IMPACT), mitigates this inherent mismatch. Not only is the algorithm highly sample efficient, it can learn quickly, training 30 percent faster than IMPALA. At the same time, we propose a novel method to stabilize agents in distributed asynchronous setups and, through our ablation studies, show how the agent can learn in both a time and sample efficient manner. In our paper, we show that the algorithm IMPACT realizes greater gains by striking the balance between high sample throughput and sample efficiency. In our experiments, we demonstrate in the experiments that IMPACT exceeds state-of-the-art agents in training time (with same hardware) while maintaining similar sample efficiency with PPO's. The contributions of this paper are as follows: 1. We show that when collecting experiences asynchronously, introducing a target network allows for a stabilized surrogate objective and multiple SGD steps per batch (Section 3.1). 2. We show that using a circular buffer for storing asynchronously collected experiences allows for smooth trade-off between real-time performance and sample efficiency (Section 3.2). 3. We show that IMPACT, when evaluated using identical hardware and neural network models, improves both in real-time and timestep efficiency over both synchronous PPO and IMPALA (Section 4). into a large training batch and the learner performs minibatch SGD. IMPALA workers asynchronously generate data. IMPACT consists of a batch buffer that takes in worker experience and a target's evaluation on the experience. The learner samples from the buffer. Reinforcement Learning assumes a Markov Decision Process (MDP) setup defined by the tuple (S, A, p, γ, r) where S and A represent the state and action space, γ ∈ is the discount factor, and p: S × A × S → R and R: S × A → R are the transition dynamics and reward function that models an environment. Let π(a t |s t): S × A → denote a stochastic policy mapping that returns an action distribution given state s t ∈ S. Rolling out policy π(a t |s t) in the environment is equivalent to sampling a trajectory τ ∼ P(τ), where τ:= (s 0, a 0, ...., a T −1, s T, a T). We can compactly define state and state-action marginals of the trajectory distribution p π (s t) and p π (s t, a t) induced by the policy π(a t |s t).The goal for reinforcement learning aims to maximize the following objective: When θ parameterizes π(a t |s t), the policy is updated according to the Policy Gradient Theorem : whereÂ π θ (s t, a t) is an estimator of the advantage function. The advantage estimator is usually defined as the 1-step TD error,Â π θ (s t, a t) = r(s t, a t) + γV (s t+1) −V (s t), whereV (s t) is an estimation of the value function. Policy gradients, however, suffer from high variance and large update-step sizes, oftentimes leading to sudden drops in performance. Per iteration, Proximal Policy Optimization (PPO) optimizes policy π θ from target π θold via the following objective function where r t (θ) = π θ (at|st) π θ old (at|st) and is the clipping hyperparameter. In addition, many PPO implementations use GAE-λ as a low bias, low variance advantage estimator forÂ t (b). PPO's surrogate objective contains the importance sampling ratio r t (θ), which can potentially explode if π θold is too far from π θ. . PPO's surrogate loss mitigates this with the clipping function, which ensures that the agent makes reasonable steps. Alternatively, PPO can also be seen as an adaptive trust region introduced in TRPO (a). In Figure 1a, distributed PPO agents implement a synchronous data-gathering scheme. Before data collection, workers are updated to π old and aggregate worker batches to training batch D train. The learner performs many mini-batch gradient steps on D train. Once the learner is done, learner weights are broadcast to all workers, who start sampling again. In Figure 1b, IMPALA decouples acting and learning by having the learner threads send actions, observations, and values while the master thread computes and applies the gradients from a queue of learners experience . This maximizes GPU utilization and allows for increased sample throughput, leading to high training speeds on easier environments such as Pong. As the number of learners grows, worker policies begin to diverge from the learner policy, ing in stale policy gradients. To correct this, the IMPALA paper utilizes V-trace to correct the distributional shift: where, V φ is the value network, π θ is the policy network of the master thread, µ θ is the policy network of the learner thread, and c j = min c, Input: Batch size M, number of workers W, circular buffer size N, replay coefficient K, target update frequency t target, weight broadcast frequency t frequency, learning rates α and β 1: Randomly initialize network weights (θ, w) 2: Initialize target network (θ, w) ← (θ, w) 3: Create W workers and duplicate (θ, w) to each worker 4: Initialize circular buffer C(N, K) 5: for t = 1,.., T do Compute policy and value network gradients Update policy and value network weights If t ≡ 0 (mod t frequency), broadcast weights to workers 13: end for Worker-i Input: Worker sample batch size S 1: repeat 2: for t = 1,..., S do Store (s t, a t, r t, s t+1) ran by θ i in batch B i end for 6: If broadcasted weights exist, set θ i ← θ 8: until learner finishes 3 IMPACT ALGORITHM Like IMPALA, IMPACT separates sampling workers from learner workers. Algorithm 1 and Figure 1c describe the main training loop and architecture of IMPACT. In the beginning, each worker copies weights from the master network. Then, each worker uses their own policy to collect trajectories Since π worker may differ per worker, using this ratio in trust region conflicts across multiple batches. Since π learner is updated after each batch from the worker, only a single SGD step can be taken per batch. The IMPACT objective allows for multiple SGD steps per async batch and has a stable trust region. Figure 2: In asynchronous PPO, there are multiple candidate policies from which the trust region can be defined: π workeri, the policy of the worker process that produced the batch of experiences, π learner, the current policy of the learner process, and π target, the policy of a target network. Introducing the target network allows for both a stable trust region and multiple SGD steps per batch of experience collected asynchronously from workers, improving sample efficiency. Since workers can generate experiences asynchronously from their copy of the master policy, this also allows for good real-time efficiency. and sends the data (s t, a t, r t) to the circular buffer. Simultaneously, workers also asynchronously pull policy weights from the master learner. In the meantime, the target network occasionally syncs with the master learner every t target iterations. The master learner then repeatedly draws experience from the circular buffer. Each sample is weighted by the importance ratio of πtarget. The target network is used to provide a stable trust region (Figure 2), allowing multiple steps per batch (i.e., like PPO) even in the asynchronous setting (i.e., with the IMPALA architecture). In the next section, we describe the design of this improved objective. PPO gathers experience from previous iteration's policy π θold, and the current policy trains by importance sampling off-policy experience with respect to π θ. In the asynchronous setting, worker i's policy, denoted as π workeri, generates experience for the policy network π θ. The probability that batch B comes from worker i can be parameterized as a categorical distribution i ∼ D(α 1, ..., α n). We include this by adding an extra expectation to the importance-sampled policy gradient objective (IS-PG) : Since each worker contains a different policy, the agent introduces a target network for stability (Figure 2). Off-policy agents such as DDPG and DQN update target networks with a moving average. For IMPACT, we periodically update the target network with the master network. However, training with importance weighted ratio π θ πtarget can lead to numerical instability, as shown in Figure 3. To prevent this, we clip the importance sampling ratio from worker policy,π workeri, to target policy, π target: where β = 1 ρ. In the experiments, we set ρ as a hyperparameter with ρ ≥ 1 and β ≤ 1. To see why clipping is necessary, when master network's action distribution changes significantly over few training iterations, worker i's policy, π workeri, samples data outside that of target policy, π target, leading to large likelihood ratios, In (b), we show the target network update frequency is robust to a range of choices. We try target network update frequency ttarget equal to the multiple (ranging from 1/16 and 16) of n = N · K, the product of the size of circular buffer and the replay times for each batch in the buffer. large IS ratios to ρ. Figure 10 in Appendix E provides additional intuition behind the target clipping objective. We show that the target network clipping is a lower bound of the IS-PG objective. For ρ > 1, the clipped target ratio is larger and serves to augment advantage estimatorÂ t. This incentivizes the agent toward good actions while avoiding bad actions. Thus, higher values of ρ encourages the agent to learn faster at the cost of instability. We use GAE-λ with V-trace . The V-trace GAE-λ modifies the advantage function by adding clipped importance sampling terms to the summation of TD errors: where c i = min c, πtarget(aj |sj) πworker i (aj |sj) (we use the convention t−1 j=t c j = 1) and δ i V is the importance sampled 1-step TD error introduced in V-trace. IMPACT uses a circular buffer (Figure 4) to emulate the mini-batch SGD used by standard PPO. The circular buffer stores N batches that can be traversed at max K times. Upon being traversed K times, a batch is discarded and replaced by a new worker batch. For motivation, the circular buffer and the target network are analogous to mini-batching from π old experience in PPO. When target network's update frequency n = N K, the circular buffer is equivalent to distributed PPO's training batch when the learner samples N minibatches for K SGD iterations. This is in contrast to standard replay buffers, such as in ACER and APE-X, where transitions (s t, a t, r t, s t+1) are either uniformly sampled or sampled based on priority, and, when the buffer is full, the oldest transitions are discarded . We investigate the performance of the clipped-target objective relative to prior work, which includes PPO and IS-PG based objectives. Specifically, we consider the following ratios below: For all three experiments, we truncate all three ratios with PPO's clipping function: c(R) = clip(R, 1−, 1+) and train in an asynchronous setting. Figure 4 (a) reveals two important takeaways: first, R 1 suffers from sudden drops in performance midway through training. Next, R 2 trains stably but does not achieve good performance. We theorize that R 1 fails due to the target and worker network mismatch. During periods of training where the master learner undergoes drastic changes, worker action outputs vastly differ from the learner outputs, ing in small action probabilities. This creates large ratios in training and destabilizes training. We hypothesize that R 2 fails due to different workers pushing and pulling the learner in multiple directions. The learner moves forward with the most recent worker's suggestions without developing a proper trust region, ing in many worker's suggestions conflicting with each other. The loss function, R 3 shows that clipping is necessary and can help facilitate training. By clipping the target-worker ratio, we make sure that the ratio does not explode and destabilize training. Furthermore, we prevent workers from making mutually destructive suggestions by having a target network provide singular guidance. In Section 3.2, an analogy was drawn between PPO's mini-batching mechanism and the circular buffer. Our primary benchmark for target update frequency is n = N · K, where N is circular buffer size and K is maximum replay coefficient. This is the case when PPO is equivalent to IMPACT. In Figure 4 (b), we test the frequency of updates with varying orders of magnitudes of n. In general, we find that agent performance is robust to vastly differing frequencies. However, when n = 1 ∼ 4, the agent does not learn. Based on empirical , we theorize that the agent is able to train as long as a stable trust region can be formed. On the other hand, if update frequency is too low, the agent is stranded for many iterations in the same trust region, which impairs learning speed. Counter to intuition, the tradeoff between time and sample efficiency when K increases is not necessarily true. In Figure 4b and 4c, we show that IMPACT realizes greater gains by striking the balance between high sample throughput and sample efficiency. When K = 2, IMPACT performs the best in both time and sample efficiency. Our reveal that wall-clock time and sample efficiency can be optimized based on tuning values of K in the circular buffer. We investigate how IMPACT attains greater performance in wall clock-time and sample efficiency compared with PPO and IMPALA across six different continuous control and discrete action tasks. We tested the agent on three continuous environments (Figure 5): HalfCheetah, Hopper, and Humanoid on 16 CPUs and 1 GPU. The policy networks consist of two fully-connected layers of 256 units with nonlinear activation tanh. The critic network shares the same architecture as the policy network. For consistentency, same network architectures were employed across PPO, IMPALA, and IMPACT. For the discrete environments (Figure 6), Pong, SpaceInvaders, and Breakout were chosen as common benchmarks used in popular distributed RL libraries . Additional experiments for discrete environments are in the Appendix. These experiments were ran on 32 CPUs and 1 GPU. The policy network consists of three 4x4 and one 11x11 conv layer, with nonlinear activation ReLU. The critic network shares weights with the policy network. The input of the network is a stack of four 42x42 down-sampled images of the Atari environment. The hyper-parameters for continuous and discrete environments are listed in the Appendix B table 1 and 2 respectively. Figures 5 and 6 show the total average return on evaluation rollouts for IMPACT, IMPALA and PPO. We train each algorithm with three different random seeds on each environment for a total time of three hours. According to the experiments, IMPACT is able to train much faster than PPO and IMPALA in both discrete and continuous domains, while preserving same or better sample efficiency than PPO. Our reveal that continuous control tasks for IMPACT are sensitive to the tuple (N, K) for the circular buffer. N = 16 and K = 20 is a robust choice for continuous control. Although higher K inhibits workers' sample throughput, increased sample efficiency from replaying experiences in an overall reduction in training wall-clock time and higher reward. For discrete tasks, N = 1 and K = 2 works best. Empirically, agents learn faster from new experience than replaying old experience, showing how exploration is crucial to achieving high asymptotic performance in discrete enviornments. Figure 7: Performance of IMPACT with respect to the number of workers in both continuous and discrete control tasks Figure 7 shows how IMPACT's performance scales relative to the number of workers. More workers means increased sample throughput, which in turn increases training throughput (the rate that learner consumes batches). With the learner consuming more worker data per second, IMPACT can attain better performance in less time. However, as number of workers increases, observed increases in performance begin to decline. Distributed RL architectures are often used to accelerate training. Gorila and A3C use workers to compute gradients to be sent to the learner. A2C and IMPALA send experience tuples to the learner. Distributed replay buffers, introduced in ACER and Ape-X , collect worker-collected experience and define an overarching heuristic for learner batch selection. IMPACT is the first to fully incorporate the sample-efficiency benefits of PPO in an asynchronous setting. Surreal PPO also studies training with PPO in the asynchronous setting, but do not consider adaptation of the surrogate objective nor IS-correction. Their use of a target network for broadcasting weights to workers is also entirely different from IMPACT's. Consequently, IMPACT is able to achieve better in both real-time and sample efficiency. Off-policy methods, including DDPG and QProp, utilize target networks to stabilize learning the Q function . This use of a target network is related but different from IMPACT, which uses the network to define a stable trust region for the PPO surrogate objective. In , we introduce IMPACT, which extends PPO with a stabilized surrogate objective for asynchronous optimization, enabling greater real-time performance without sacrificing timestep efficiency. We show the importance of the IMPACT objective to stable training, and show it can outperform tuned PPO and IMPALA baselines in both real-time and timestep metrics. Time (In Figure 9, we gradually add components to IMPALA until the agent is equivalent to IMPACT's. Starting from IMPALA, we gradually add PPO's objective function, circular replay buffer, and target-worker clipping. In particular, IMPALA with PPO's objective function and circular replay buffer is equivalent to an asynchronous-variant of PPO (APPO). APPO fails to perform as well as synchronous distributed PPO, since PPO is an on-policy algorithm. In Figure 6, IMPALA performs substantially worse than other agents in continuous environments. We postulate that IMPALA suffers from low asymptotic performance here since its objective is an importance-sampled version of the Vanilla Policy Gradient (VPG) objective, which is known to suffer from high variance and large update-step sizes. We found that for VPG, higher learning rates encourage faster learning in the beginning but performance drops to negative return later in training. In Appendix B, for IMPALA, we heavily tuned on the learning rate, finding that small learning rates stabilize learning at the cost of low asymptotic performance. Prior work also reveals the agents that use VPG fail to attain good performance in non-trivial continuous tasks . Our with IMPALA reaches similar performance compared to other VPG-based algorithms. The closest neighbor to IMPALA, A3C uses workers to compute gradients from the VPG objective to send to the learner thread. A3C performs well in InvertedPendulum yet flounders in continuous environments . The following ratios represent the objective functions for different ablation studies. In the plots (Figure 10), we set the advantage function to be one, i.e. Â t = 1. • IS ratio: According to Figure 10, IS ratio is large when π workeri assigns low probability. IMPACT target -clip is a lower bound of the PPO -clip. In an distributed asynchronous setting, the trust region suffers from larger variance stemming from off-policy data. IMPACT target -clip ratio mitigates this by encouraging conservative and reasonable policy-gradient steps.

Represent the short summary for finding the corresponding scientific article (long).

IMPACT helps RL agents train faster by decreasing training wall-clock time and increasing sample efficiency simultaneously.

Represent this text to find a much much longer article about it with say hundreds-thousands of words!

A backward model of previous (state, action) given the next state, i.e. P(s_t, a_t | s_{t+1}), can be used to simulate additional trajectories terminating at states of interest! Improves RL learning efficiency.


Question: Represent the scientific article to find a short TLDR

Question: In this paper, we show that a simple coloring scheme can improve, both theoretically and empirically, the expressive power of Message Passing Neural Networks (MPNNs). More specifically, we introduce a graph neural network called Colored Local Iterative Procedure (CLIP) that uses colors to disambiguate identical node attributes, and show that this representation is a universal approximator of continuous functions on graphs with node attributes. Our method relies on separability, a key topological characteristic that allows to extend well-chosen neural networks into universal representations. Finally, we show experimentally that CLIP is capable of capturing structural characteristics that traditional MPNNs fail to distinguish, while being state-of-the-art on benchmark graph classification datasets. Learning good representations is seen by many machine learning researchers as the main reason behind the tremendous successes of the field in recent years . In image analysis , natural language processing or reinforcement learning , groundbreaking rely on efficient and flexible deep learning architectures that are capable of transforming a complex input into a simple vector while retaining most of its valuable features. The universal approximation theorem (; ; ;) provides a theoretical framework to analyze the expressive power of such architectures by proving that, under mild hypotheses, multi-layer perceptrons (MLPs) can uniformly approximate any continuous function on a compact set. This provided a first theoretical justification of the strong approximation capabilities of neural networks, and was the starting point of more refined analyses providing valuable insights into the generalization capabilities of these architectures (; ; ;). Despite a large literature and state-of-the-art performance on benchmark graph classification datasets, graph neural networks yet lack a similar theoretical foundation . Universality for these architectures is either hinted at via equivalence with approximate graph isomorphism tests (k-WL tests in Xu et al. 2019; Maron et al. 2019a), or proved under restrictive assumptions (finite node attribute space in Murphy et al. 2019). In this paper, we introduce Colored Local Iterative Procedure 1 (CLIP), which tackles the limitations of current Message Passing Neural Networks (MPNNs) by showing, both theoretically and experimentally, that adding a simple coloring scheme can improve the flexibility and power of these graph representations. More specifically, our contributions are: 1) we provide a precise mathematical definition for universal graph representations, 2) we present a general mechanism to design universal neural networks using separability, 3) we propose a novel node coloring scheme leading to CLIP, the first provably universal extension of MPNNs, 4) we show that CLIP achieves state of the art on benchmark datasets while significantly outperforming traditional MPNNs as well as recent methods on graph property testing. The rest of the paper is organized as follows: Section 2 gives an overview of the graph representation literature and related works. Section 3 provides a precise definition for universal representations, as well as a generic method to design them using separable neural networks. In Section 4, we show that most state-of-the-art representations are not sufficiently expressive to be universal. Then, using the analysis of Section 3, Section 5 provides CLIP, a provably universal extension of MPNNs. Finally, Section 6 shows that CLIP achieves state-of-the-art accuracies on benchmark graph classification taks, as well as outperforming its competitors on graph property testing problems. The first works investigating the use of neural networks for graphs used recurrent neural networks to represent directed acyclic graphs . More generic graph neural networks were later introduced by; , and may be divided into two categories. 1) Spectral methods (; ; ;) that perform convolution on the Fourier domain of the graph through the spectral decomposition of the graph Laplacian. 2) Message passing neural networks , sometimes simply referred to as graph neural networks, that are based on the aggregation of neighborhood information through a local iterative process. This category contains most state-of-the-art graph representation methods such as (; ; ; ;), DeepWalk , graph attention networks , graphSAGE or GIN . Recently, showed that MPNNs were, at most, as expressive as the WeisfeilerLehman (WL) test for graph isomorphism . This suprising led to several works proposing MPNN extensions to improve their expressivity, and ultimately tend towards universality (a; b; ;). However, these graph representations are either as powerful as the k-WL test (a), or provide universal graph representations under the restrictive assumption of finite node attribute space . Other recent approaches (c) implies quadratic order of tensors in the size of the considered graphs. Some more powerfull GNNs are studied and benchmarked on real classical datasets and on graph property testing (; ;): a set of problems that classical MPNNs cannot handle. Our work thus provides a more general and powerful of universality, matching the original definition of for MLPs. In this section we present the theoretical tools used to design our universal graph representation. More specifically, we show that separable representations are sufficiently flexible to capture all relevant information about a given object, and may be extended into universal representations. Let X, Y be two topological spaces, then F(X, Y) (resp. C(X, Y)) denotes the space of all functions (resp. continuous functions) from X to Y. Moreover, for any group G acting on a set X, X /G denotes the set of orbits of X under the action of G (see Appendix B for more details). Finally, · is a norm on R d, and P n is the set of all permutation matrices of size n. In what follows, we assume that all the considered topological spaces are Hausdorff (see e.g. for an in-depth review): each pair of distinct points can be separated by two disjoint open sets. This assumption is rather weak (e.g. all metric spaces are Hausdorff) and is verified by most topological spaces commonly encountered in the field of machine learning. Let X be a set of objects (e.g. vectors, images, graphs, or temporal data) to be used as input information for a machine learning task (e.g. classification, regression or clustering). In what follows, we denote as vector representation of X a function f: X → R d that maps each element x ∈ X to a d-dimensional vector f (x) ∈ R d. A standard setting for supervised representation learning is to define a class of vector representations F d ⊂ F(X, R d) (e.g. convolutional neural networks for images) and use the target values (e.g. image classes) to learn a good vector representation in light of the supervised learning task (i.e. one vector representation f ∈ F d that leads to a good accuracy on the learning task). In order to present more general , we will consider neural network architectures that can output vectors of any size, i.e. F ⊂ ∪ d∈N * F(X, R d), and will denote the set of d-dimensional vector representations of F. A natural characteristic to ask from the class F is to be generic enough to approximate any vector representation, a notion that we will denote as universal representation . In other words, F is a universal representation of a normed space X if and only if, for any continuous function φ: X → R d, any compact K ⊂ X and any ε > 0, there exists f ∈ F such that One of the most fundamental theorems of neural network theory states that one hidden layer MLPs are universal representations of the m-dimensional vector space R m. Theorem 1 (, Theorem 3.1). Let ϕ: R → R be a continuous non polynomial activation function. For any compact K ⊂ R m and d ∈ N *, two layers neural networks with activation ϕ are uniformly dense in the set C(K, R d). However, for graphs and structured objects, universal representations are hard to obtain due to their complex structure and invariance to a group of transformations (e.g. permutations of the node labels). We show in this paper that a key topological property, separability, may lead to universal representations of those structures. Loosely speaking, universal representations can approximate any vector-valued function. It is thus natural to require that these representations are expressive enough to separate each pair of dissimilar elements of X. Definition 2 (Separability). A set of functions F ⊂ F(X, Y) is said to separate points of X if for every pair of distinct points x and y, there exists f ∈ F such that f (x) = f (y). we will say that F is separable if its 1-dimensional representations F 1 separates points of X. Separability is rather weak, as we only require the existence of different outputs for every pair of inputs. Unsurprisingly, we now show that it is a necessary condition for universality (see Appendix A for all the detailed proofs). Proposition 1. Let F be a universal representation of X, then F 1 separates points of X. While separability is necessary for universal representations, it is also key to designing neural network architectures that can be extended into universal representations. More specifically, under technical assumptions, separable representations can be composed with a universal representation of R d (such as MLPs) to become universal. Theorem 2. For all d ≥ 0, let M d be a universal approximation of R d. Let F be a class of vector representations of X such that: Stability by concatenation is verified by most neural networks architectures, as illustrated for MLPs in Figure 1. The proof of Theorem 2 relies on the Stone-Weierstrass theorem (see e.g. , Theorem 7.32) whose assumptions are continuity, separability, and the fact that the class of functions is an algebra. Fortunately, composing a separable and concatenable representation with a universal representation automatically leads to an algebra, and thus the applicability of the StoneWeierstrass theorem and the desired . A complete derivation is available in Appendix A. Since MLPs are universal representations of R d, Theorem 2 implies a convenient way to design universal representations of more complex object spaces: create a separable representation and compose it with a simple MLP (see Figure 2). Corollary 1. A continuous, concatenable and separable representation of X composed with an MLP is universal. Note that many neural networks of the deep learning literature have this two steps structure, including classical image CNNs such as AlexNet or Inception . In this paper, we use Corollary 1 to design universal graph and neighborhood representations, although the method is much more generic and may be applied to other objects. In this section, we first provide a proper definition for graphs with node attributes, and then show that message passing neural networks are not sufficiently expressive to be universal. Consider a dataset of n interacting objects (e.g. users of a social network) in which each object i ∈ 1, n has a vector attribute v i ∈ R m and is a node in an undirected graph G with adjacency matrix A ∈ R n×n. Definition 3. The space of graphs of size n with m-dimensional node attributes is the quotient space where A is the adjacency matrix of the graph, v contains the m-dimensional representation of each node in the graph and the set of permutations matrices P n is acting on (v, A) by Moreover, we limit ourselves to graphs of maximum size n max, where n max is a large integer. This allows us to consider functions on graphs of different sizes without obtaining infinite dimensional spaces and infinitely complex functions that would be impossible to learn via a finite number of samples. We thus define Graph m = n≤nmax Graph m,n. More details on the technical topological aspects of the definition are available in Appendix B, as well as a proof that Graph m is Hausdorff. A common method for designing graph representations is to rely on local iterative procedures. Following the notations of , a message passing neural network (MPNN) is made of three consecutive phases that will create intermediate node representations x i,t for each node i ∈ 1, n and a final graph representation x G as described by the following procedure: 1) Initialization: All node representations are initialized with their node attributes 2) Aggregation and combination: T local iterative steps are performed in order to capture larger and larger structural characteristics of the graph. 3) Readout: This step combines all final node representations into a single graph representation: where READOUT is permutation invariant. Unfortunately, while MPNNs are very efficient in practice and proven to be as expressive as the Weisfeiler-Lehman algorithm , they are not sufficiently expressive to construct isomorphism tests or separate all graphs (for example, consider k-regular graphs without node attributes, for which a small calculation shows that any MPNN representation will only depend on the number of nodes and degree k ). As a direct application of Proposition 1, MPNNs are thus not expressive enough to create universal representations. In this section, we present Colored Local Iterative Procedure (CLIP), an extension of MPNNs using colors to differentiate identical node attributes, that is able to capture more complex structural graph characteristics than traditional MPNNs. This is proved theoretically through a universal approximation theorem in Section 5.3 and experimentally in Section 6. CLIP is based on three consecutive steps: 1) graphs are colored with several different colorings, 2) a neighborhood aggregation scheme provides a vector representation for each colored graph, 3) all vector representations are combined to provide a final output vector. We now provide more information on the coloring scheme. In order to distinguish non-isomorphic graphs, our approach consists in coloring nodes of the graph with identical attributes. This idea is inspired by classical graph isomorphism algorithms that use colors to distinguish nodes , and may be viewed as an extension of one-hot encodings used for graphs without node attributes . For any k ∈ N, let C k be a finite set of k colors. These colors may be represented as one-hot encodings (C k is the natural basis of R k) or more generally any finite set of k elements. At initialization, we first partition the nodes into groups of identical attributes V 1,..., V K ⊂ 1, n. Then, for a subset V k of size |V k |, we give to each of its nodes a distinct color from C k (hence a subset of size |V k |). For example, Figure 3 shows two colorings of the same graph, which is decomposed in three groups V 1, V 2 and V 3 containing nodes with attributes a, b and c respectively. Since V 1 contains only two nodes, a coloring of the graph will attribute two colors ( and, depicted as blue and red) to these nodes. More precisely, the set of colorings C(v, A) of a graph G = (v, A) are defined as In the CLIP algorithm, we add a coloring scheme to an MPNN in order to distinguish identical node attributes. This is achieved by modifying the initialization and readout phases of MPNNs as follows. We first select a set C k ⊆ C(v, A) of k distinct colorings uniformly at random (see Eq.). Then, for each coloring c ∈ C k, node representations are initialized with their node attributes concatenated with their color: This step is performed for all colorings c ∈ C k using a universal set representation as the aggregation function: where ψ and ϕ are MLPs with continuous non-polynomial activation functions and ψ(x, y) denotes the of ψ applied to the concatenation of x and y. The aggregation scheme we propose is closely related to DeepSet , and a direct application of Corollary 1 proves the universality of our architecture. More details, as well as the proof of universality, are available in Appendix C. 3. Colored readout: This step performs a maximum over all possible colorings in order to obtain a final coloring-independent graph representation. In order to keep the stability by concatenation, the maximum is taken coefficient-wise where ψ is an MLP with continuous non polynomial activation functions. We treat k as a hyper-parameter of the algorithm and call k-CLIP (resp. ∞-CLIP) the algorithm using k colorings (resp. all colorings, i.e. k = |C(v, A)|). Note that, while our focus is graphs with node attributes, the approach used for CLIP is easily extendable to similar data structures such as directed or weighted graphs with node attributes, graphs with node labels, graphs with edge attributes or graphs with additional attributes at the graph level. As the colorings are chosen at random, the CLIP representation is itself random as soon as k < |C(v, A)|, and the number of colorings k will impact the variance of the representation. However, ∞-CLIP is deterministic and permutation invariant, as MPNNs are permutation invariant. The separability is less trivial and is ensured by the coloring scheme. Theorem 3. The ∞-CLIP algorithm with one local iteration (T = 1) is a universal representation of the space Graph m of graphs with node attributes. The proof of Theorem 3 relies on showing that ∞-CLIP is separable and applying Corollary 1. This is achieved by fixing a coloring on one graph and identifying all nodes and edges of the second graph using the fact that all pairs (v i, c i) are dissimilar (see Appendix D). Similarly to the case of MLPs, only one local iteration is necessary to ensure universality of the representation. This rather counter-intuitive is due to the fact that all nodes can be identified by their color, and the readout function can aggregate all the structural information in a complex and non-trivial way. However, as for MLPs, one may expect poor generalization capabilities for CLIP with only one local iteration, and deeper networks may allow for more complex representations and better generalization. This point is addressed in the experiments of Section 6. Moreover, ∞-CLIP may be slow in practice due to a large number of colorings, and reducing k will speed-up the computation. Fortunately, while k-CLIP is random, a similar universality theorem still holds even for k = 1. Theorem 4. The 1-CLIP algorithm with one local iteration (T = 1) is a random representation whose expectation is a universal representation of the space Graph m of graphs with node attributes. The proof of Theorem 4 relies on using ∞-CLIP on the augmented node attributes v i = (v i, c i). As all node attributes are, by design, different, the max over all colorings in Eq. disappears and, for any coloring, 1-CLIP returns an ε-approximation of the target function (see Appendix D). Remark 1. Note that the variance of the representation may be reduced by averaging over multiple samples. Moreover, the proof of Theorem 4 shows that the variance can be reduced to an arbitrary precision given enough training epochs, although this may lead to very large training times in practice. As the local iterative steps are performed T times on each node and the complexity of the aggregation depends on the number of neighbors of the considered node, the complexity is proportional to the number of edges of the graph E and the number of steps T. Moreover, CLIP performs this iterative aggregation for each coloring, and its complexity is also proportional to the number of chosen colorings k = |C k |. Hence the complexity of the algorithm is in O(kET). Note that the number of all possible colorings for a given graph depends exponentially in the size of the groups V 1,..., V K, and thus ∞-CLIP is practical only when most node attributes are dissimilar. This worst case exponential dependency in the number of nodes can hardly be avoided for universal representations. Indeed, a universal graph representation should also be able to solve the graph isomorphism problem. Despite the existence of polynomial time algorithms for a broad class of graphs , graph isomorphism is still quasi-polynomial in general . As a , creating a universal graph representation with polynomial complexity for all possible graphs and functions to approximate is highly unlikely, as it would also induce a graph isomorphism test of polynomial complexity and thus solve a very hard and long standing open problem of theoretical computer science. In this section we show empirically the practical efficiency of CLIP and its relaxation. We run two sets of experiments to compare CLIP w.r.t. state-of-the-art methods in supervised learning settings: i) on 5 real-world graph classification datasets and ii) on 4 synthetic datasets to distinguish structural graph properties and isomorphism. Both experiments follow the same experimental protocol as described in: 10-fold cross validation with grid search hyper-parameter optimization. More details on the experimental setup are provided in Appendix E. We performed experiments on five benchmark datasets extracted from standard social networks (IMDBb and IMDBm) and bio-informatics databases (MUTAG, PROTEINS and PTC). All dataset characteristics (e.g. size, classes), as well as the experimental setup, are available in Appendix E. Following standard practices for graph classification on these datasets, we use one-hot encodings of node degrees as node attributes for IMDBb and IMDBm , and perform singlelabel multi-class classification on all datasets. We compared CLIP with six state-of-the-art baseline algorithms: 1) WL: Weisfeiler-Lehman subtree kernel , 2) AWL: Anonymous Walk Embeddings , 3) DCNN: Diffusion-convolutional neural networks , 4) PS: PATCHY-SAN , 5) DGCNN: Deep Graph CNN and 6) GIN: Graph Isomorphism Network . WL and AWL are representative of unsupervised methods coupled with an SVM classifier, while DCNN, PS, DGCNN and GIN are four deep learning architectures. As the same experimental protocol as that of was used, we present their reported on Table 1. Table 1 shows, CLIP can achieve state-of-the-art performance on the five benchmark datasets. Moreover, CLIP is consistent across all datasets, while all other competitors have at least one weak performance. This is a good indicator of the robustness of the method to multiple classification tasks and dataset types. Finally, the addition of colors does not improve the accuracy for these graph classification tasks, except on the MUTAG dataset. This may come from the small dataset sizes (leading to high variances) or an inherent difficulty of these classification tasks, and contrasts with the clear improvements of the method for property testing (see Section 6.2). More details on the performance of CLIP w.r.t. the number of colors k are available in Appendix E. Remark 2. In three out of five datasets, none of the recent state-of-the-art algorithms have statistically significantly better than older methods (e.g. WL). We argue that, considering the high variances of all classification algorithms on classical graph datasets, graph property testing may be better suited to measure the expressiveness of graph representation learning algorithms in practice. We now investigate the ability of CLIP to identify structural graph properties, a task which was previously used to evaluate the expressivity of graph kernels and on which the Weisfeiler-Lehman subtree kernel has been shown to fail for bounded-degree graphs . The performance of our algorithm is evaluated for the binary classification of four different structural properties: 1) connectivity, 2) bipartiteness, 3) triangle-freeness, 4) circular skip links (see Appendix E for precise definitions of these properties) against three competitors: a) GIN, arguably the most efficient MPNN variant yet published , b) Ring-GNN, a permutation invariant network that uses the ring of matrix addition and multiplication , c) RP-GIN, the Graph Isomorphism Network combined with Relational Pooling, as described by , which is able to distinguish certain cases of non-isomorphic regular graphs. We provide all experimental details in Appendix E. Table 2: Classification accuracies of the synthetic datasets. k-RP-GIN refers to a relational pooling averaged over k random permutations. We report Ring-GNN from. Connectivity Bipartiteness Triangle-freeness Circular skip links mean ± std mean ± std mean ± std mean ± std max min Table 2 shows that CLIP is able to capture the structural information of connectivity, bipartiteness, triangle-freeness and circular skip links, while MPNN variants fail to identify these graph properties. Furthermore, we observe that CLIP outperforms RP-GIN, that was shown to provide very expressive representations for regular graphs , even with a high number of permutations (the equivalent of colors in their method is set to k = 16). Moreover, both for k-RP-GIN and k-CLIP, the increase of permutations and colorings respectively lead to higher accuracies. In particular, CLIP can capture almost perfectly the different graph properties with as little as k = 16 colorings. In this paper, we showed that a simple coloring scheme can improve the expressive power of MPNNs. Using such a coloring scheme, we extended MPNNs to create CLIP, the first universal graph representation. Universality was proven using the novel concept of separable neural networks, and our experiments showed that CLIP is state-of-the-art on both graph classification datasets and property testing tasks. The coloring scheme is especially well suited to hard classification tasks that require complex structural information to learn. The framework is general and simple enough to extend to other data structures such as directed, weighted or labeled graphs. Future work includes more detailed and quantitative approximation depending on the parameters of the architecture such as the number of colors k, or number of hops of the iterative neighborhood aggregation. Proof of Theorem 2. The proof relies on the Stone-Weierstrass theorem we recall below. We refer to (, Theorem 7.32) for a detailed proof of the following classical theorem. Theorem 5 (Stone-Weierstrass). Let A be an algebra of real functions on a compact Hausdorff set K. If A separates points of K and contains a non-zero constant function, then A is uniformly dense in C(K, R). We verify that under the assumptions of Theorem 2 the Stone-Weierstrass theorem applies. In this setting, we first prove the theorem for m = 1 and use induction for the general case. Let K ⊂ X be a compact subset of X. We will denote and will proceed in two steps: we first show that A 0 is uniformly dense in C(K, R), then that A is dense in A 0, hence proving Theorem 2. Proof. The subset A 0 contains zero and all constants. Let f, g ∈ A 0 so that, and by assumption ϕ ∈ F. We have so that f + g ∈ A 0 and we conclude that A 0 is a vectorial subspace of C(K, R). We proceed similarly for the product in order to finish the proof of the lemma. Because F 1 separates the points of X by assumption, A 0 also separates the points of X. Indeed, let x = y two distinct points of X so that ∃f ∈ F such that f (x) = f (y). There exists g ∈ C(R d, R) such that g(f (x)) = g(f (y)). From Theorem 5 we deduce that A 0 is uniformly dense in C(K, R) for all compact subsets K ⊂ X. Lemma 2. For any compact subset K ⊂ X, A is uniformly dense in A 0. Proof. Let > 0 and h = ψ 0 • f ∈ A 0 with f ∈ F and ψ 0 ∈ C(R d, R). Thanks to the continuity of f, the imageK = f (K) is a compact of R d. By Theorem 1 there exists an MLP ψ such that This last lemma completes the proof in the case m = 1., f ∈ F} and proceed in a similar manner than Lemma 2 by decomposing and applying Lemma 1 for each coefficient function Proof of Proposition 1. Assume that there exists x, y ∈ X s.t. ∀f ∈ F 1, f (x) = f (y). Then K = {x, y} is a compact subset of X and let φ ∈ C(K, R) be such that φ(x) = 1 and φ(y) = 0. Thus, for all f ∈ F 1, max z∈{x,y} φ(z) − f (z) ≥ 1/2 which contradicts universality (see Definition 1). In what follows, X is always a topological set and G a group of transformations acting on X. The orbits of X under the action of G are the sets Gx = {g · x : g ∈ G}. Moreover, we denote as X /G the quotient space of orbits, also defined by the equivalence relation: x ∼ y ⇐⇒ ∃g ∈ G s.t. x = g · y. As stated in Section 5, graphs with node attributes can be defined using invariance by permutation of the labels. We prove here that the ing spaces are Hausdorff. Lemma 3 ((, I, §8. 3)). Let X be a Hausdorff space and R an equivalence relation of X. Then X /R is Hausdorff if and only if any two distinct equivalence classes in X are contained in disjoints saturated open subsets of X. Thanks to this lemma we prove the following proposition. Proposition 2. Let G a finite group acting on an Hausdorff space X, then the orbit space X /G is Hausdorff. Proof. Let Gx and Gy two distinct classes with disjoint open neighbourhood U and V. By finiteness of G, the application π: Suppose that there exists z ∈Ũ ∩Ṽ, then π(z) ∈ π(U) ∩ π(V) and we finally get that Gz ⊂ U ∩ V = ∅. ThereforeŨ ∩Ṽ is empty and X /G is Hausdorff by Lemma 3. Proposition 2 directly implies that the spaces Graph m and Neighborhood m are Hausdorff. We now provide more details on the aggregation and combination scheme of CLIP, and show that a simple application of Corollary 1 is sufficient to prove its universality for node neighborhoods. Each local aggregation step takes as input a couple (x i, {x j} j∈Ni ) where x i ∈ R m is the representation of node i, and {x j} j∈Ni is the set of vector representations of the neighbors of node i. In the following, we show how to use Corollary 1 to design universal representations for node neighborhoods. Definition 5. The set of node neighborhoods for m-dimensional node attributes is defined as where the set of permutation matrices P n is acting on R n×m by P · v = P v. The main difficulty to design universal neighborhood representations is that the node neighborhoods of Definition 5 are permutation invariant w.r.t. neighboring node attributes, and hence require permutation invariant representations. The graph neural network literature already contains several deep learning architectures for permutation invariant sets (; ; ;), among which PointNet and DeepSet have the notable advantage of being provably universal for sets. Following Corollary 1, we compose a separable permutation invariant network with an MLP that will aggregate both information from the node itself and its neighborhood. While our final architecture is similar to Deepset , this section emphasizes that the general universality theorems of Section 3 are easily applicable in many settings including permutation invariant networks. The permutation invariant set representation used for the aggregation step of CLIP is as follows: where ψ and ϕ are MLPs with continuous non-polynomial activation functions and ψ(x, y) denotes the of the MLP ψ applied to the concatenation of x and y. Theorem 6. The set representation described in Eq. Taking ψ(x, y) = y and ε = 1/3 max{|S 1 |, |S 2 |}, we have which proves separability and, using Corollary 1, the universality of the representation. Proof of Theorem 3. First of all, as the activation functions of the MLPs are continuous, CLIP is made of continuous and concatenable functions, and is thus also continuous and concatenable. Second, as the node aggregation step (denoted NODEAGGREGATION below) is a universal set representation (see Appendix C), it is capable of approximating any continuous function. We will thus first replace this function by a continuous function φ, and then show that the still holds for NODEAGGREGATION by a simple density argument. Let 2 ) be two distinct graphs of respective sizes n 1 and n 2 (up to a permutation). If n 1 = n 2, then ψ(x) = x and φ(x) = 1 returns the number of nodes, and hence x G 1 = n 1 = n 2 = x G 2. Otherwise, let V = {v k i} i∈ 1,n 1,k∈{1,2} be the set of node attributes of G 1 and G 2, c 1 be a coloring of G 1, ψ(x) = x and φ be a continuous function such that, ∀x ∈ V and S ⊂ V, The existence of φ ∈ C(R m, R) is assured by Urysohn's lemma (see e.g. (, lemma 2.12) ). Then, x G counts the number of matching neighborhoods for the best coloring, and we have x G 1 = n 1 and x G 2 ≤ n 1 − 1. Finally, taking ε < 1/2n 1 in the definition of universal representation leads to the desired , as then, using an ε-approximation of φ as NODEAGGREGATION, we have Proof of Theorem 4. Consider a continuous function ψ: nmax and we define φ: Moreover, observe that for any coloring c ∈ C(v, A), ∞-CLIP and 1-CLIP applied to ((v, c), A) returns the same , as all node attributes are dissimilar (by definition of the colorings) and C((v, c), A) = ∅. Finally, 1-CLIP applied to (v, A) is equivalent to applying 1-CLIP to ((v, C), A) where C is a random coloring in C(v, A), and Eq. thus implies that any random sample of 1-CLIP is within an ε error of the target function ψ. As a , its expectation is also within an ε error of the target function ψ, which proves the universality of the expectation of 1-CLIP. E.1 REAL-WORLD DATASETS Table 3 summarizes the characteristics of all benchmark graph classification datasets used in Section 6.1. We now provide complementary information on these datasets. Social Network Datasets (IMDBb, IMDBm): These datasets refer to collaborations between actors/actresses, where each graph is an ego-graph of every actor and the edges occur when the connected nodes/actors are playing in the same movie. The task is to classify the genre of the movie that the graph derives from. IMDBb is a single-class classification dataset, while IMDBm is multi-class. For both social network datasets, we used one-hot encodings of node degrees as node attribute vectors. Bio-informatics Datasets (MUTAG, PROTEINS, PTC): MUTAG consists of mutagenic aromatic and heteroaromatic nitrocompounds with 7 discrete labels. PROTEINS consists of nodes, which correspond to secondary structureelements and the edges occur when the connected nodes are neighbors in the amino-acidsequence or in 3D space. It has 3 discrete labels. PTC consists of chemical compounds that reports the carcinogenicity for male and female rats and it has 19 discrete labels. For all bio-informatics datasets we used the node labels as node attribute vectors. Experimentation protocol: We follow the same experimental protocol as described in , and thus report the provided in this paper corresponding to the accuracy of our six baselines in Table 1. We optimized the CLIP hyperparameters by grid search according to 10-fold cross-validated accuracy means. We use 2-layer MLPs, an initial learning rate of 0.001 and decreased the learning rate by 0.5 every 50 epochs for all possible settings. For all datasets the hyperparameters we tested are: the number of hidden units within {32, 64}, the number of colorings c ∈ {1, 2, 4, 8}, the number of MPNN layers within {1, 3, 5}, the batch size within {32, 64}, and the number of epochs, that means, we select a single epoch with the best cross-validation accuracy averaged over the 10 folds. Note that standard deviations are fairly high for all models due to the small size of these classic datasets. Table 4 summarizes the performances of CLIP while increasing the number of colorings k. Overall we can see a small increase in performances and a reduction of the variances when k is increasing. Nevertheless we should not jump to any since none of the models are statistically significantly better than the others. In Section 6.2 we evaluate the expressive power of CLIP on benchmark synthetic datasets. Our goal is to show that CLIP is able to distinguish basic graph properties, where classical MPNN cannot. We considered a binary classification task and we constructed balanced synthetic datasets 2 for each of the examined graph properties. The 20-node graphs are generated using Erdös-Rényi model (Erdös and Rényi, 1959) (and its bipartite version for the bipartiteness) with different probabilities p for edge creation. All nodes share the same (scalar) attribute. We thus have uninformative feature vectors. In particular, we generated datasets for different classical tasks: 1) connectivity, 2) bipartiteness, 3) triangle-freeness, and 4) circular skip links . In the following, we present the generating protocol of the synthetic datasets and the experimentation setup we used for the experiments. In every case of synthetic dataset we follow the same pattern: we generate a set of random graphs using Erdös-Rényi model, which contain a specific graph property and belong to the same class and by proper edge addition we remove this property, thus creating the second class of graphs. By this way, we assure that we do not change different structural characteristics other than the examined graph property. -Connectivity dataset: this dataset consists of 1000 (20-node) graphs with 500 positive samples and 500 negative ones. The positive samples correspond to disconnected graphs with two 10-node connected components selected among randomly generated graphs with an Erdös-Rényi model probability of p = 0.5. We constructed negative samples by adding to positive samples a random edge between the two connected components. -Bipartiteness dataset: this dataset consists of 1000 (20-node) graphs with 500 positive samples and 500 negative ones. The positive samples correspond to bipartite graphs generated with an Erdös-Rényi (bipartite) model probability of p = 0.5. For the negative samples (non-bipartite graphs) we chose the positive samples and for each of them we added an edge between randomly selected nodes from the same partition, in order to form odd cycles 3. -Triangle-freeness dataset: this dataset consists of 1000 (20-node) graphs with 500 positive samples and 500 negative ones. The positive samples correspond to triangle-free graphs selected among randomly generated graphs with an Erdös-Rényi model probability of p = 0.1. We constructed negative samples by randomly adding new edges to positive samples until it creates at least one triangle. -Circular skip links: this dataset consists of 150 graphs of 41 nodes as described in . The Circular Skip Links graphs are undirected regular graphs with node degree 4. We denote a Circular skip link graph by G n,k an undirected graph of n nodes, where (i, j) ∈ E holds if and only if |i − j| ≡ 1 or k(mod n) This is a 10-class multiclass classification task whose objective is to classify each graph according to its isomorphism class. Experimentation protocol: We evaluate the different configurations of CLIP and its competitors GIN and RP-GIN based on their hyper-parameters. For the architecture implementation of the GIN, we followed the best performing architecture, presented in. In particular, we used the summation as the aggregation operator, MLPs as the combination level for the node embedding generation and the sum operator for the readout function along with its refined version of concatenated graph representations across all iterations/layers of GIN, as described in. In all the tested configurations for CLIP and its competitors (GIN, RP-GIN) we fixed the number of layers of the MLPs and the learning rate: we chose 2-layer MLPs and we used the Adam optimizer with initial learning rate of 0.001 along with a scheduler decaying the learning rate by 0.5 every 50 epochs. Concerning the other hyper-parameters, we optimized: the number of hidden units within {16, 32, 64} (except for the CSL task where we only use 16 hidden units to be fair w.r.t. RP-GIN

Represent the TLDR for finding the corresponding scientific article

This paper introduces a coloring scheme for node disambiguation in graph neural networks based on separability, proven to be a universal MPNN extension.

Represent this text to find a much much longer article about it with say hundreds-thousands of words

This paper identifies classes of problems for which adversarial examples are inescapable, and derives fundamental bounds on the susceptibility of any classifier to adversarial examples.