Question: Represent the article for finding a single-sentence TLDR

Question: The reproducibility of reinforcement-learning research has been highlighted as a key challenge area in the field. In this paper, we present a case study in reproducing the of one groundbreaking algorithm, AlphaZero, a reinforcement learning system that learns how to play Go at a superhuman level given only the rules of the game. We describe Minigo, a reproduction of the AlphaZero system using publicly available Google Cloud Platform infrastructure and Google Cloud TPUs. The Minigo system includes both the central reinforcement learning loop as well as auxiliary monitoring and evaluation infrastructure. With ten days of training from scratch on 800 Cloud TPUs, Minigo can play evenly against LeelaZero and ELF OpenGo, two of the strongest publicly available Go AIs. We discuss the difficulties of scaling a reinforcement learning system and the monitoring systems required to understand the complex interplay of hyperparameter configurations. In March 2016, Google DeepMind's AlphaGo BID0 defeated world champion Lee Sedol by using two deep neural networks (a policy and a value network) and Monte Carlo Tree Search (MCTS) to synthesize the output of these two neural networks. The policy network was trained via supervised learning from human games, and the value network was trained from a much larger corpus of synthetic games generated by sampling game trajectories from the policy network. AlphaGo Zero BID1, published in October 2017, described a continuous pipeline, which when initialized with random weights, could train itself to defeat the original AlphaGo system. The requirement for expert human data was replaced with a requirement for vast amounts of compute: approximately two thousand TPUs were used for 72 hours to train AlphaGo Zero to its full strength. AlphaZero BID2 presents a refinement of the AlphaGoZero pipeline, notably removing the gating mechanism for publishing new models. In many ways, AlphaGo Zero can be seen as the logical culmination of fully automating and streamlining the bootstrapping process: the original AlphaGo system was bootstrapped from expert human data and reached a final strength that was somewhat stronger than the best humans. Then, by generating new training data with the stronger AlphaGo system and repeating the bootstrap process, an even stronger system was created. By automating the bootstrapping process until it is continuous, a system is created that can train itself to surpass human levels of play, even when starting from random play. In this paper, we discuss our experiences creating Minigo. About half of our effort went into rebuilding the infrastructure necessary to coordinate a thousand selfplay workers. The other half of the effort went into monitoring infrastructure to test and verify that what we had built was bug-free. Despite having at hand a paper describing the final architecture of AlphaZero, we rediscovered the hard way which components of the system were absolutely necessary to get right, and which components we could be messy with. It stands to reason that without the benefit of pre-existing work, monitoring systems are even more important in the discovery process. We discuss in particular, At the heart of Minigo is the reinforcement learning loop as described in the AlphaZero BID2 paper. (See the Appendix for a full comparison of AlphaGo Zero, AlphaZero, and Minigo). Briefly, selfplay with the current generation of network weights is used to generate games, and those games are used as training data to produce the next generation of network weights. In each selfplay game. Minigo uses a variant of the UCT algorithm as described in the AlphaGo Zero paper to select a new variation from the game tree. The neural network considers the variation and predicts the next move, as well as the likelihood that one player will win the game. These predictions are integrated into the search tree, and the updated statistics are used to select the next variation to explore. A move is picked by either taking a weighted sample (first 30 moves) or picking the most visited variation. This is repeated until the game ends, one player resigns, or a move cap is reached. The final selected move thus takes into consideration the other player's responses and possible game continuations, and the visit counts can be used directly as a training target for the policy network. Additionally, the final game can also be used as a training target for the value network. Each game's data (position, visit counts, game ) is then used to update the neural network's weights by stochastic gradient descent (SGD), simultaneously minimizing the policy and value error. The number of readouts (800 for Minigo and AlphaZero) invested into each move roughly determines the ratio of compute required for selfplay and training. Minigo uses 800 Cloud TPUs for selfplay and 1 Cloud TPU for training. To orchestrate the many Cloud TPUs required for selfplay, we used Google Kubernetes Engine (GKE) to deploy many copies of a selfplay binary written in C++. Each selfplay worker writes training data directly to Cloud BigTable (CBT), with one row being one (position, visit counts, game ) tuple. The trainer trains on samples from a sliding window of training data, and then publishes network weights to Google Cloud Storage (GCS). The selfplay workers periodically look for and download an updated set of network weights, closing the loop. As part of our monitoring, a calibration job periodically processes game statistics and updates the hyperparameters used in selfplay, to ensure that the quality of selfplay data remains high. Additionally, we use StackDriver to monitor the health of the selfplay cluster and compute bulk statistics over all of the games being played. For example, we would log and track statistics about the distribution of value head outputs, winrates by color, distribution of game lengths, statistics about MCTS search depth & breadth, distributions of our time spent per inference, and the resignation threshold rate. We used TensorBoard to monitor the training job, keeping track of statistics like policy loss, value loss, regularization loss, top-1/top-3 policy accuracy, the magnitude of the value output, magnitude of weight updates, and entropy of policy output, all measured over the training selfplay data, heldout selfplay data, and human professional data. An evaluation cluster continually plays different generations against each other to determine their relative strengths. Finally, we created a frontend (https://cloudygo.com) that would allow us to check various metrics at a glance, including data such as the relative ratings of each model, common opening patterns, responses to various challenging positions, game lengths, and the percentage of games that branched due to early game softpick. This frontend also served as a way to quickly spot-check selfplay games and evaluation games. We describe the evolution of the core components of Minigo, from project inception to their current state. Minigo's selfplay is by far the most expensive part of the whole pipeline (100-1000 times as much compute as training), so many of our efforts were focused on improving the throughput of our selfplay cluster. We initially prototyped Minigo with a python-based engine, playing Go on a 9x9 board. BID0 The size of this network (9 residual blocks of 32 filters) was easily evaluated without a GPU, and a small cluster of only a few hundred cores was able to keep up with a single P100 GPU running training. We used Google Kubernetes Engine to constantly run the containers, using its'batch job' API to take care of scheduling, retrying, cleaning up dead pods, etc. Each docker container would start up, read the latest model from GCS, use it to play one game, and then write out the training data to a different directory in GCS, randomly picking either a training or holdout directory. This iteration of Minigo was able to start with random play and eventually reached a point where it was playing at a strong amateur level. However, given the lack of a professional human 9x9 dataset, and our lack of expertise in evaluating proper 9x9 play, it was difficult for us to understand how much (or how little) we had accomplished. With our proof of concept in hand, we moved to a 19x19 version of the run, scaling up the depth and width of our network. It was immediately clear that we could not rely on a CPU-only implementation: although we initially moved to an intermediate-sized network for our 19x19 run, the full network would be 512 times 2 slower to evaluate than the network we had used for our 9x9 trial run. We started with 2000 NVIDIA K80 preemptible GPUs, making use of Google Kubernetes Engine to dynamically scale load. GKE additionally had an easy solution for managing the installation of NVIDIA drivers into many containers, so this saved us a great deal of trouble. We also implemented a vectorized MCTS that would minimize the Python overhead involved in tree search. BID0 The computational complexity scales to some large polynomial complexity at least O(N 6): N 2 for the board area covered by convolution operations; N 2 for the approximate length of a game; N for the depth of network required; N − N 2 for the number of reads in the game tree. BID1 Moving from a depth of 9 to 20 residual layers is 2x; moving from a width of 32 to 256 is 64x; and convolving over a 19x19 board instead of a 9x9 board is 4x, for a grand total of 512x It is useful to briefly describe several batching techniques that are available to us. As batching was the easiest way to scale up the throughput of our selfplay workers, it was the main driver behind various design decisions. The easiest batching technique is to run multiple games in parallel. However, this comes with the drawback of increasing the latency of game completion: completing 16 games in 10 minutes is not necessarily better than completing 1 game in 1 minute. An alternative batching technique is virtual losses in MCTS BID7. Virtual losses is a technique used to select multiple variations from a single MCTS game tree. To reiterate, MCTS is a tree algorithm that determines the best leaf to explore using a combination of probability priors (in this case from the 'policy' network) and the outcome of simulations from that node (in this case, from the 'value' network). Once the leaf is chosen, expanding it with an inference is slow, leaving an opportunity to do additional work. Unfortunately, MCTS deterministically identifies the'best' leaf to expand, and has no notion of'second best' leaf. To choose additional leaves, we can mark the originally chosen leaf as having been a loss, and then rerun selection to get another leaf. When the of inference come back, the loss is replaced with the true evaluation. The number of simultaneous pending virtual losses is a tuneable parameter -increasing this number improves throughput but degrades selfplay quality. Using our K80 cluster, we ran the full Minigo pipeline 3 times as we worked out various bugs and added new infrastructure. However, we wished to see if the newly released Cloud TPUs (now in alpha on GKE) would yield improved performance. At the time, our Python engine had about 5% overhead on K80s, but we estimated that with TPUs, the python engine overhead would expand to 25% overhead or more. At this point, we rewrote our selfplay engine in C++, with an eye towards integrating MCTS very closely with Cloud TPU inference. With a modest number of TPUs, we could achieve a full run in about a week, assuming we could drive them near their theoretical limit. For maximal throughput, Cloud TPUs demanded even larger batch sizes than what we'd been able to produce with virtual losses. To generate this throughput, we played multiple games simultaneously; each selfplay worker utilized one Cloud TPU to play 32 games in parallel at virtual losses = 8. Each game took 15 minutes on average, leading to about 2 completed games per minute per Cloud TPU.With simultaneous games being played, we had to deal with the ragged edge problem -since not all games are the same length, when a game ended, we could either immediately start a new game, or suffer reduced throughput on the TPU. And if we continually started new games, what would we do when a new model was published? Our solution was to just switch models whenever a new one was published even if it happened in the middle of the game. We attempted to keep the pipeline balanced such that this happened only once a game, on average. Empirically, this appears not to have hurt the pipeline's performance. One operational risk of this approach is that each resign threshold is calibrated to a particular model. If a new model significantly skewed value output compared to its predecessor, then many games near the resign threshold can all simultaneously resign. This would trigger a thundering herd problem as many games complete simultaneously and would possibly overload Cloud BigTable with heavy write volume. Worse still, since the resign threshold must be calibrated from resignation-disabled games (which are played to completion and therefore take 2-3x time to complete), entire games could be played with the wrong resignation threshold. To solve these issues, we set aside a small number of TPUs specifically for playing calibration matches with a much lower parallel game configuration, to minimize game completion latency. A ringbuffer was used to store the most recent calibration and compute the threshold, and the buffer size kept small to ensure rapid updates. With these changes, and with the help of Kubernetes, we were able to spin up over 100 petaops of compute, and flexibly and efficiently shift them around the load patterns in Google's cloud. The cluster of workers was able to achieve about 1.3-1.5ms per inference, playing about 1.8M games per day with the'full size' 20 block network. The training part of the Minigo system has much in common with many supervised learning tasks. The only difference is that instead of training on a predefined dataset, we train on a continually evolving dataset. As TensorFlow is tailored towards a fixed input pipeline configuration, we found it easiest to restart our TensorFlow training process every time we wanted to shift our sliding window of training data. Other reinforcement learning frameworks like TF Agents BID8 address this problem by maintaining an in-memory circular buffer of training data, but due to our shuffling requirements (discussed below), this was not feasible for us. From a systems perspective, we also designed our trainer system to pause if our selfplay cluster slowed down or stopped for any reason -otherwise, the trainer would overfit on the same data, necessitating a rollback intervention. Other than the frequent restarts, our trainer followed best practices for supervised learning. For example, we'd originally used a custom data storage format designed for optimized compression of Go game data. While this worked well for our single-machine prototypes, we rewrote our training code into separate input pipeline and model code with TensorFlow Datasets and Estimator to best take advantage of Google Cloud TPUs. By using TensorFlow Datasets instead of Python iteration, we could make use of TensorFlow's parallel, buffered I/O features to directly feed data to the Cloud TPUs, first by reading directly from GCS, then by reading directly from Cloud BigTable. Shuffling turned out to be an unexpectedly tricky and important part of our pipeline. A major complicating factor for shuffling was that AlphaGo Zero specified a large trailing window of 5e5 games (roughly 1e8 positions or 2e12 bytes, assuming a featurization of 19x19x17 array of float32) from which positions should be uniformly sampled. Ideally, sampling should be uniform; every SGD minibatch should be uniformly drawn from the entire dataset. Such a large dataset can be sharded to different degrees; if there are many smaller files, then uniform sampling requires the overhead of opening many different small files for each minibatch, and if there are a few large files, there is an overhead associated with seeking to the right position. (While each of our positions were fixed-length, TensorFlow's TFExample format is variable-length, so TensorFlow's APIs did not allow jumping to a specified offset.)In practice, an approximation to uniform shuffling is required. At different points in Minigo's history, we used different approximations, each of which had different drawbacks. We found that every time we improved our shuffling, Minigo's strength would improve dramatically. The AlphaZero paper also reports that without using the 8-fold symmetry of the Go board to augment the training data, about 10x as many games needed to be played to reach the same strength. We believe that Minigo's sensitivity to proper shuffling arises from Go's gameplay pattern of placing stones sequentially. At a high level of play, stones are placed and rarely removed from the board, and thus specific subpatterns will persist through an entire game. Since every position will share the same training target of +/-1 for the value network, it is easy for the neural network to overfit by simply memorizing each position. We first noticed our lack of adequate shuffling when Minigo became extremely overconfident about games it played, as measured by the magnitude of its value network output in games against humans. We had also noticed that Minigo's performance on predicting outcomes of human professional games had dropped, but it was difficult to understand whether this was due to human professionals not playing perfectly or because Minigo was overfitting its value head. We obtained a conclusive answer when we started setting aside 5% of our selfplay games for validating our policy and value accuracy. Our network showed excellent performance on selfplay games which it had trained on, but near-random performance for selfplay games it had not trained on. All in all, we learned that to shuffle effectively, we needed to adequately scramble each source of correlation in our data. We had two primary sources of correlation: intra-game correlation (every position from the same game shared the same ), and generational correlation (every game played by a given set of network weights would be of a similar 'style').In our early shuffler implementations, we started from hundreds of thousands of tiny files, containing 100 to 1000 positions each. These files represented the raw output of thousands of selfplay workers as soon as they completed playing a set of games. Our first implementation, consisted of reading sequentially through the last million games, sampling 2% of positions, and emitting a series of chunks of 2048 positions each. We would then train on these chunks in shuffled order. This method failed on both shuffling criteria: each chunk contained on average, more than 1 position from the same game, and each chunk consisted of games from the same generation. To fix both issues, we utilized a machine with 64GB memory to perform a perfect shuffle on positions uniformly sampled from the last 5e5 games. Even then, we observed a large boost in strength when we randomly additionally applied one of 8 symmetries of the Go board to each selected position. This shuffler implementation served us up until we replaced our cluster of K80 GPUs with Cloud TPUs. When this happened, our single-threaded shuffler reading TFExample files from GCS became the new bottleneck. We patched our implementation with a Python multiprocessing pool, which bought us enough time to implement a Cloud BigTable solution. With CBT, our selfplay cluster would directly write one row into CBT for each position. In our Cloud BigTable pipeline, each TFExample was written to its own sequential row, with an individual row index determined by its game number and move number within the game, so that each individual move in the entire list of games could be randomly accessed. The sequential ordering was necessary because CBT's scan range operations are lexicographic, and we needed to be able to sample moves from a predictable game range (the last N games played by the cluster).Cloud Bigtable provides probabilistic sampling as part of its server-side API. The lower the probability, the better the savings in bandwidth, since only the selected rows are transferred. Computing the correct sampling probability required some bookkeeping, since the number of moves in a game can vary, and there is no method to count the rows in a CBT lexicographic row range aside from iterating through them. Therefore, the move count per game was stored in a separate keyspace in the table, so that for any given game range, the total moves in that range could be calculated, and from that total, the correct sampling probability (typically around 1.5%).We then executed a final in-memory shuffle of the sampled row keys before requesting the row keys from CBT. Training ran in parallel with consumption of shuffled data, decreasing the overall training time by 50%, and removing the input pipeline as the bottleneck. Another unexpectedly tricky part of the Minigo pipeline was our calibration process. The AlphaGo Zero paper described using a resignation threshold to terminate selfplay when both the current position's evaluation was heavily in favor of one side. This early termination is valuable in saving compute, as Go has an onerous game-end condition requiring playing the game out for many hundreds of moves. BID2 To determine this resignation threshold, the AlphaZero paper specified that 10% of games should be played to completion, and a threshold chosen such that fewer than 5% of games would have been incorrectly resigned. We discovered that this early termination had several second-order effects on the Minigo system. The most severe consequence was that an incorrectly calibrated resignation threshold could destabilize training by creating a pessimistic loop when a game was prematurely resigned. When that resignation was subsequently used as training data, it would be even more likely that the network would resign if presented with the same position. This only occurred occasionally, usually due to a different bug, but such a pessimistic loop would cause training to diverge. In order to detect this condition, we tracked statistics on winrate by color, distribution of game lengths, and the average magnitude of the value network's output during selfplay. A pessimistic loop would typically in a heavy skew of winrate, drastically shorter games, and a very confident value network output but a very high value error on holdout games. To avoid a pessimistic loop scenario, we started to calibrate our resignation threshold to a more conservative 3% false positive rate, even if it meant playing longer games. Additionally, we invested in lowering the latency involved in computing the resignation threshold. Originally, our resignation threshold was computed by running a custom script over the most recent resignation-disabled games. This script was executed by hand, and the resignation threshold configuration propagated by pushing a new flagfile to GCS. Being human dependent meant that the script was not run quite as often as it should have been, leading to our threshold being fairly conservative and having the false positive rate under 5%. Eventually, with our Cloud BigTable rewrite, we could compute the resignation threshold by directly querying CBT. We updated our configuration mechanism to be more lightweight and automated the calculation, and the lowered latency improved the robustness of the early stages of our pipeline, where the network was rapidly learning the basics of the game. Another consequence of early termination was that the distribution of training data shifted towards early-and mid-game examples. Amusingly, this meant that our network would make basic mistakes in end-game positions when we ran test matches against humans, because there were almost no end game positions in the training data. We discovered this by posting our bot on an online Go server, where its human opponents had no preconceived notions of what the bot was "supposed" to do. We also set up a BigQuery dataset of moves played, and crafted a SQL query that would detect instances of games where the estimated winrate was 99%+ in favor of one side before suddenly flipping to 99% in favor of the other side. To fix our end-game deficiency, we also trained on the resignation-disabled games so that there would be end-game positions to learn from. We allocated a small cluster of GPUs to play evaluation games between different models. Evaluation games disabled all of the settings that encourage exploration, like dirichlet noise and temperature. By using a modified Bradley-Terry system as implemented by the Python choix package, we computed a single parameter rating which we convert to the more familiar Elo number for display purposes. Measuring the relative strength of our models was useful for several reasons: -To figure out when a run has stopped improving, -To find the strongest models in a run. (Without gating, strength is not guaranteed to improve between each model, and in practice it was observed that successive models could have rating swings of up to 900 Elo.) -To compare our rating-over-time graphs to those shown in the AG / AGZ / AZ papers. Our evaluation methodology differed in one important respect from the methodology described by AlphaGo Zero. There, evaluation was used as a method of gating the promotion of new models, and each model would only play against the currently active model. This methodology potentially suffers from transitive cycles where A beats B; B beats C, and C beats A. Our evaluation methodology used a more diverge array of opponents, playing model N versus models N-1, N-2, N-3, N-5, N-10, N-20, and N-50. Thus, as the run proceeded, each model would play a symmetric set of pairs of models that came ahead and before it. Additional games were chosen to reduce the uncertainty in each model's rating for models that had not played similar strength models. As we experimented and made improvements, it became important to assess models from different runs. Cross-run evaluation games were played between the strongest models of each run to determine the impact of our hyperparameter changes and pipeline bug fixes on overall strength. We obtained the most reliable cross-run ratings by taking the best models at regular intervals from each run, and then playing an equal number of games for all n-choose-2 pairings. Beyond the use in evaluating training runs, the evaluation cluster was used for a number of one-off tests, such as tuning various hyper parameters (pUCT, virtual loss size, Q initialization), testing against other networks trained via completely different approaches, (e.g. supervised, transfer learning, or alternate AlphaGo reproductions), and playing matches against external reference points. In reproducing AlphaGo Zero, we found it useful to monitor nearly everything we could think of, to prove to ourselves that our implementation was bug-free. We found Figure 3 from the original AlphaGo Zero paper to be a very useful guidepost, and wish that we had more metrics to compare against -a richer array of training metrics on e.g. policy entropy, value confidence, policy confidence would not be overly burdensome to provide, and would have greatly aided our reproducion efforts. And even if the raw data were not provided, it would have been valuable to know which metrics we should monitor. We also found that healthy metrics were necessary but not sufficient for success. Our evaluation cluster was the only true indicator for success, and was of tremendous help in letting us know that a change in our hyperparameter settings had had a positive effect.

represent the short summary for finding the corresponding scientific article (long)

We reproduced AlphaZero on Google Cloud Platform

An optimization algorithm that explores various batch sizes based on probability and automatically exploits successful batch size which minimizes validation loss.


Question: Represent the scientific article to find a short TLDR:

Question: Generative adversarial networks (GANs) train implicit generative models through solving minimax problems. Such minimax problems are known as nonconvex- nonconcave, for which the dynamics of first-order methods are not well understood. In this paper, we consider GANs in the type of the integral probability metrics (IPMs) with the generator represented by an overparametrized neural network. When the discriminator is solved to approximate optimality in each iteration, we prove that stochastic gradient descent on a regularized IPM objective converges globally to a stationary point with a sublinear rate. Moreover, we prove that when the width of the generator network is sufficiently large and the discriminator function class has enough discriminative ability, the obtained stationary point corresponds to a generator that yields a distribution that is close to the distribution of the observed data in terms of the total variation. To the best of our knowledge, we seem to first establish both the global convergence and global optimality of training GANs when the generator is parametrized by a neural network. The file iclr2020_conference.pdf contains these instructions and illustrates the various formatting requirements your ICLR paper must satisfy. Submissions must be made using L A T E X and the style files iclr2020_conference.sty and iclr2020_conference.bst (to be used with L A T E X2e). The file iclr2020_conference.tex may be used as a "shell" for writing your paper. All you have to do is replace the author, title, abstract, and text of the paper with your own. The formatting instructions contained in these style files are summarized in sections 2, 3, and 4 below. The text must be confined within a rectangle 5.5 inches (33 picas) wide and 9 inches (54 picas) long. The left margin is 1.5 inch (9 picas). Use 10 point type with a vertical spacing of 11 points. Times New Roman is the preferred typeface throughout. Paragraphs are separated by 1/2 line space, with no indentation. Paper title is 17 point, in small caps and left-aligned. All pages should start at 1 inch (6 picas) from the top of the page. Authors' names are set in boldface, and each name is placed above its corresponding address. The lead author's name is to be listed first, and the co-authors' names are set to follow. Authors sharing the same address can be on the same line. Please pay special attention to the instructions in section 4 regarding figures, tables, acknowledgments, and references. The recommended paper length is 8 pages, with unlimited additional pages for citations. There will be a strict upper limit of 10 pages for the main text. Reviewers will be instructed to apply a higher standard to papers in excess of 8 pages. Authors may use as many pages of appendices (after the bibliography) as they wish, but reviewers are not required to read these. These instructions apply to everyone, regardless of the formatter being used. Citations within the text should be based on the natbib package and include the authors' last names and year (with the "et al." construct for more than two authors). When the authors or the publication are included in the sentence, the citation should not be in parenthesis using \citet{} (as in " for more information."). Otherwise, the citation should be in parenthesis using \citep{} (as in "Deep learning shows promise to make progress towards AI ."). The corresponding references are to be listed in alphabetical order of authors, in the REFERENCES section. As to the format of the references themselves, any style is acceptable as long as it is used consistently. Indicate footnotes with a number 1 in the text. Place the footnotes at the bottom of the page on which they appear. Precede the footnote with a horizontal rule of 2 inches (12 picas). Make sure the figure caption does not get separated from the figure. Leave sufficient space to avoid splitting the figure and figure caption. You may use color figures. However, it is best for the figure captions and the paper body to make sense if the paper is printed either in black/white or in color. The Hessian matrix of f at input point x f (x)dx Definite integral over the entire domain of x S f (x)dx Definite integral with respect to x over the set S Probability and Information Theory P (a) A probability distribution over a discrete variable p(a) A probability distribution over a continuous variable, or over a variable whose type has not been specified H (Positive part of x, i.e., max(0, x) 1 condition is 1 if the condition is true, 0 otherwise Do not change any aspects of the formatting parameters in the style files. In particular, do not modify the width or length of the rectangle the text should fit into, and do not change font sizes (except perhaps in the REFERENCES section; see below). Please note that pages should be numbered. Please prepare PostScript or PDF files with paper size "US Letter", and not, for example, "A4". The -t letter option on dvips will produce US Letter files. Consider directly generating PDF files using pdflatex (especially if you are a MiKTeX user). PDF figures must be substituted for EPS figures, however. Otherwise, please generate your PostScript and PDF files with the following commands: dvips mypaper.dvi -t letter -Ppdf -G0 -o mypaper.ps ps2pdf mypaper.ps mypaper.pdf Most of the margin problems come from figures positioned by hand using \special or other commands. We suggest using the command \includegraphics from the graphicx package. Always specify the figure width as a multiple of the line width as in the example below using.eps graphics A number of width problems arise when LaTeX cannot properly hyphenate a line. Please give LaTeX hyphenation hints using the \-command. If you'd like to, you may include a section for author contributions as is done in many journals. This is optional and at the discretion of the authors. A APPENDIX You may include other additional sections here.

We establish global convergence to optimality for IPM-based GANs where the generator is an overparametrized neural network.

Represent the short summary for finding the corresponding scientific article (long)

See the abstract.  (For the revision, the paper is identical, except for a 59 page Supplementary Material, which can serve as a stand-along technical report version of the paper.)


Question: represent the scientific article to find a short tldr

Question: We present network embedding algorithms that capture information about a node from the local distribution over node attributes around it, as observed over random walks following an approach similar to Skip-gram. Observations from neighborhoods of different sizes are either pooled (AE) or encoded distinctly in a multi-scale approach (MUSAE). Capturing attribute-neighborhood relationships over multiple scales is useful for a diverse range of applications, including latent feature identification across disconnected networks with similar attributes. We prove theoretically that matrices of node-feature pointwise mutual information are implicitly factorized by the embeddings. Experiments show that our algorithms are robust, computationally efficient and outperform comparable models on social, web and citation network datasets. Figure 1: Phenomena affecting and inspiring the design of the multi-scale attributed network embedding procedure. In Figure 1a attributed nodes D and G have the same feature set and their nearest neighbours also exhibit equivalent sets of features, whereas features at higher order neighbourhoods differ. Figure 1b shows that as the order of neighbourhoods considered (r) increases, the product of the adjacency matrix power and the feature matrix becomes less sparse. This suggests that an implicit decomposition method would be computationally beneficial. Our key contributions are: 1. to introduce the first Skip-gram style embedding algorithms that consider attribute distributions over local neighborhoods, both pooled (AE) and multi-scale (MUSAE), and their counterparts that attribute distinct features to each node (AE-EGO and MUSAE-EGO); 2. to theoretically prove that their embeddings approximately factorize PMI matrices based on the product of an adjacency matrix power and node-feature matrix; 3. to show that popular network embedding methods DeepWalk and Walklets are special cases of our AE and MUSAE; 4. we show empirically that AE and MUSAE embeddings enable strong performance at regression, classification, and link prediction tasks for real-world networks (e.g. Wikipedia and Facebook), are computationally scalable and enable transfer learning between networks. We provide reference implementations of AE and MUSAE, together with the datasets used for evaluation at https://github.com/iclr2020/MUSAE. Efficient unsupervised learning of node embeddings for large networks has seen unprecedented development in recent years. The current paradigm focuses on learning latent space representations of nodes such that those that share neighbors (; ; ;), structural roles or attributes are located close together in the embedding space. Our work falls under the last of these categories as our goal is to learn similar latent representations for nodes with similar sets of features in their neighborhoods, both on a pooled and multi-scale basis. Neighborhood preserving node embedding procedures place nodes with common first, second and higher order neighbors within close proximity in the embedding space. Recent works in the neighborhood preserving node embedding literature were inspired by the Skip-gram model (a; b), which generates word embeddings by implicitly factorizing a shifted pointwise mutual information (PMI) matrix obtained from a text corpus. This procedure inspired DeepWalk , a method which generates truncated random walks over a graph to obtain a "corpus" from which the Skip-gram model generates neighborhood preserving node embeddings. In doing so, DeepWalk implicitly factorizes a PMI matrix, which can be shown, based on the underlying first-order Markov process, to correspond to the mean of a set of normalized adjacency matrix powers up to a given order . Such pooling of matrices can be suboptimal since neighbors over increasing path lengths (or scales) are treated equally or according to fixed weightings (a;); whereas it has been found that an optimal weighting may be task or dataset specific . In contrast, multi-scale node embedding methods such as LINE , GraRep and Walklets separately learn lower-dimensional node embedding components from each adjacency matrix power and concatenate them to form the full node representation. Such un-pooled representations, comprising distinct but less information at each scale, are found to give higher performance in a number of downstream settings, without increasing the overall number of free parameters . Attributed node embedding procedures refine ideas from neighborhood based node embeddings to also incorporate node attributes (equivalently, features or labels) (; ; ; ;). Similarities between both a node's neighborhood structure and features contribute to determining pairwise proximity in the node embedding space. These models follow quite different strategies to obtain such representations. The most elemental procedure, TADW , decomposes a convex combination of normalized adjacency matrix powers into a matrix product that includes the feature matrix. Several other models, such as SINE and ASNE , implicitly factorize a matrix formed by concatenating the feature and adjacency matrices. Other approaches such as TENE , formulate the attributed node embedding task as a joint non-negative matrix factorization problem in which node representations obtained from sub-tasks are used to regularize one another. AANE uses a similar network structure based regularization approach, in which a node feature similarity matrix is decomposed using the alternating direction method of multipliers. The method most similar to our own is BANE , in which the product of a normalized adjacency matrix power and a feature matrix is explicitly factorized to obtain attributed node embeddings. Many other methods exist, but do not consider the attributes of higher order neighborhoods (; ; ; ;). The relationship between our pooled (AE) and multi-scale (MUSAE) attributed node embedding methods mirrors that between graph convolutional neural networks (GCNNs) and multi-scale GCNNs. Widely used graph convolutional layers, such as GCN , GraphSage , GAT (Veličković et al., 2018), APPNP , SGCONV and ClusterGCN , create latent node representations that pool node attributes from arbitrary order neighborhoods, which are then inseparable and unrecoverable. In contrast, MixHop learns latent features for each proximity. We now define algorithms to learn node embeddings using the attributes of nearby nodes, that allows both node and attribute embeddings to be learned jointly. The aim is to learn similar embeddings for nodes that occur in neighbourhoods of similar attributes; and similar embeddings for attributes that often occur in similar neighbourhoods of nodes. Let G = (V, L) be an undirected graph of interest where V and L are the sets of vertices and edges (or links) respectively; and let F be the set of all possible node features (i.e. attributes). We define F v ⊆ F as the subset of features belonging to each node v ∈ V. An embedding of nodes is a mapping g: V → R d that assigns a d-dimensional representation g(v) (or simply g v) to each node v and is fully described by a matrix G ∈ R |V|×d. Similarly, an embedding of the features (to the same latent space) is a mapping h: F → R d with embeddings denoted h(f) (or simply h f), and is fully described by a matrix H ∈ R |F|×d. The Attributed Embedding (AE) procedure is described by Algorithm 1. We sample n nodes w 1, from which to start attributed random walks on G, with probability proportional to their degree (Line 2). From each starting node, a node sequence of length l is sampled over G (Line 3), where sampling follows a first order random walk. For a given window size t, we iterate over each of the first l − t nodes of the sequence termed source nodes w j (Line 4). For each source node, we consider the following t nodes as target nodes (Line 5). For each target node w j+r, we add the tuple (w j, f) to the corpus D for each target feature f ∈ F wj+r (Lines 6 and 7). We also consider features of the source node f ∈ F wj, adding each (w j+r, f) tuple to D (Lines 9 and 10). Running Skip-gram on D with b negative samples (Line 15) generates the d-dimensional node and feature embeddings. Algorithm 2: MUSAE sampling and training procedure 3.2 MULTI-SCALE ATTRIBUTED EMBEDDING The AE method (Algorithm 1) pools feature sets of neighborhoods at different proximities. Inspired by the performance of (unattributed) multi-scale node embeddings, we adapt the AE algorithm to give multi-scale attributed node embeddings (MUSAE). The embedding component of a node v ∈ V for a specific proximity r ∈ {1, ..., t} is given by a mapping g r: V → R d/t (assuming t divides d). Similarly, the embedding component of feature f ∈ F at proximity r is given by a mapping Concatenating gives a d-dimensional embedding for each node and feature. The Multi-Scale Attributed Embedding procedure is described by Algorithm 2. We again sample n starting nodes w 1 with a probability proportional to node degree (Line 2) and, for each, sample a node sequence of length l over G (Line 3) according to either a first or second order random walk. For a given window size t, we iterate over the first l − t (source) nodes w j of the sequence (Line 4) and for each source node we iterate through the t (target) nodes w j+r that follow (Line 5). We again consider each target node feature f ∈ F wj+r, but now add tuples (w j, f) to a sub-corpus D r → (Lines 6 and 7). We add tuples (w j+r, f) to another sub-corpus D r showed that the loss function of Skip-gram with negative sampling (SGNS) is minimized if the embedding matrices factorize a matrix of pointwise mutual information (PMI) of word co-occurrence statistics. Specifically, for a word dictionary V with |V| = n, SGNS (with b negative samples) outputs two embedding matrices W, C ∈ R d×n such that ∀w, c ∈ V:, where #(w, c), #(w), #(c) denote counts of word-context pair (w, c), w and c over a corpus D; and word embeddings w w, c c ∈ R d are columns of W and C corresponding to w and c respectively. as empirical estimates of p(w), p(c) and p(w, c) respectively shows: i.e. an approximate low-rank factorization of a shifted PMI matrix (low rank since typically d n). extended this to node embedding models that apply SGNS to a "corpus" generated from random walks over the graph. In the case of DeepWalk where random walks are first-order Markov, the joint probability distributions over nodes at different stages of a random walk can be expressed in closed form. A closed form then follows for the factorized PMI matrix. We show that AE and MUSAE implicitly perform analogous matrix factorizations. Notation: A ∈ R n×n denotes the adjacency matrix and D ∈ R n×n the diagonal degree matrix of a graph G, i.e. D w,w = deg(w) = v A w,v. We denote the volume of G by c = v,w A v,w. We define the binary attribute matrix F ∈ {0, 1} |V|×|F| by F w,f = 1 f ∈Fw, ∀w ∈ V, f ∈ F. For ease of notation, we let P = D −1 A and E = diag(1 DF), where diag indicates a diagonal matrix. Assuming G is ergodic:, w ∈ V is the stationary distribution over nodes, i.e. c −1 D = diag(p(w)); and c −1 A is the stationary joint distribution over consecutive nodes p(w j, w j+1). F w,f can be considered a Bernoulli parameter describing the probability p(f |w) of observing a feature f at a node w and so c −1 DF describes the stationary joint distribution p(f, w j) over nodes and features. Accordingly, P is the matrix of conditional distributions p(w j+1 |w j); and E is a diagonal matrix proportional to the probability of observing each feature at the stationary distribution p(f) (note that p(f) need not sum to 1, whereas p(w) necessarily must). We know that the SGNS aspect of MUSAE (Algorithm 2, Line 17) is minimized when the learned embeddings g Our aim is to express this factorization in terms of known properties of the graph G and its features. Lemma 1. The empirical statistics of node-feature pairs obtained from random walks give unbiased estimates of joint probabilities of observing feature f ∈ F r steps (i) after; or (ii) before node v ∈ V, as given by: Proof. See Appendix. Lemma 2. Empirical statistics of node-feature pairs obtained from random walks give unbiased estimates of joint probabilities of observing feature f ∈ F r steps either side of node v ∈ V, given by: Marginalizing gives unbiased estimates of stationary probability distributions of nodes and features: Theorem 1. MUSAE embeddings approximately factorize the node-feature PMI matrix: Proof. Lemma 3. The empirical statistics of node-feature pairs learned by the AE algorithm give unbiased estimates of mean joint probabilities over different path lengths as follows:.., t} and so |D s | = t −1 |D|. Combining with Lemma 2, the follows. Theorem 2. AE embeddings approximately factorize the pooled node-feature matrix: Proof. The proof is analogous to the proof of Theorem 1. Remark 1. DeepWalk is a corner case of AE with F = I |V|. That is, DeepWalk is equivalent to AE if each node has a single unique feature. Thus E = diag(1 DI) = D and, by Theorem 2, DeepWalk's embeddings factorize log c t (. Remark 2. Walklets is a corner case of MUSAE with F = I |V| . Thus, for r = 1, . . ., t, the embeddings of Walklets factorise log c Remark 3. Appending an identity matrix I to the feature matrices F of AE and MUSAE (denoted [F ; I]) adds a unique feature to each node. The ing algorithms, named AE-EGO and MUSAE-EGO, learn embeddings that, respectively, approximately factorize the node-feature PMI matrices: and log c t (Under the assumption of a constant number of features per source node and first-order attributed random walk sampling, the corpus generation has a runtime complexity of O(n l t x/y), where x = v∈V |F v | the total number of features across all nodes (including repetition) and y = |V| the number of nodes. Using negative sampling, the optimization runtime of a single asynchronous gradient descent epoch on AE and the joint optimization runtime of MUSAE embeddings is described by O(b d n l t x/y). If one does p truncated walks from each source node, the corpus generation complexity is O(p y l t x) and the model optimization runtime is O(b d p y l t x). Our later runtime experiments in Section 5 will underpin optimization runtime complexity discussed above. Corpus generation has a memory complexity of O(n l t x/y) while the same when generating p truncated walks per node has a memory complexity of O(p y l t x). Storing the parameters of an AE embedding has a memory complexity of O(y d) and MUSAE embeddings also use O(y d) memory. In order to evaluate the quality of created representations we test the embeddings on supervised downstream tasks such as node classification, transfer learning across networks, regression, and link prediction. Finally, we investigate how changes in the input size affect the runtime. For doing so we utilize social networks and web graphs that we collected from Facebook, Github, Twitch and Wikipedia. The data sources, collection procedures and the datasets themselves are described with great detail in Appendix B. In addition we tested our methods on citation networks widely used for model evaluation . Across all experiments we use the same hyperparameter settings of our own model, competing unsupervised methods and graph neural networks -these are respectively listed in Appendices C, E and F. We evaluate the node classification performance in two separate scenarios. In the first we do k-shot learning by using the attributed embedding vectors with logistic regression to predict labels on the Facebook, Github and Twitch Portugal graphs. In the second we test the predictive performance under a fixed size train-test split to compare against various embedding methods and competitive neural network architectures. In this experiment we take k randomly selected samples per class, and use the attributed node embeddings to train a logistic regression model with l 2 regularization and predict the labels on the remaining vertices. We repeated the above procedure with seeded splits 100 times to obtain robust comparable . From these we calculated the average of micro averaged F 1 scores to compare our own methods with other unsupervised node embedding procedures. We varied k in order to show the efficacy of the methods -what are the gains when the training set size is increased. These are plotted in Figure 2 for Facebook, Github and Twitch Portugal networks. Based on these plots it is evident that MUSAE and AE embeddings have little gains in terms of micro F 1 score when additional data points are added to the training set when k is larger than 12. This implies that our method is data efficient. Moreover, MUSAE-EGO and AE-EGO have a slight performance advantage, which means that including the nodes in the attributed random walks helps when a small amount of labeled data is available in the downstream task. Figure 2: Node classification k-shot learning performance as a function of training samples per class evaluated by average micro F 1 scores calculated from a 100 seeded train-test splits. In this series of experiments we created a 100 seeded train test splits of nodes (80% train -20% test) and calculated weighted, micro and macro averaged F 1 scores on the test set to compare our methods to various embedding and graph neural network methods. Across procedures the same random seeds were used to obtain the train-test split this way the performances are directly comparable. We attached these on the Facebook, Github and Twitch Portugal graphs as Table 6 of Appendix G. In each column red denotes the best performing unsupervised embedding model and blue corresponds to the strongest supervised neural model. We also attached additional supporting using the same experimental setting with the unsupervised methods on the Cora, Citeseer, and Pubmed graphs as Table 5 of Appendix G. In terms of micro F 1 score our strongest method outperforms on the Facebook and GitHub networks the best unsupervised method by 1.01% and 0.47% respectively. On the Twitch Portugal network the relative micro F 1 advantage of ASNE over our best method is 1.02%. Supervised node embedding methods outperform our and other unsupervised methods on every dataset for most metrics. In terms of micro F 1 this relative advantage over our best performing model variant is the largest with 4.67% on the Facebook network, and only 0.11% on Twitch Portugal. One can make four general observations based on our (i) multi-scale representations can help with the classification tasks compared to pooled ones; (ii) the addition of the nodes in the ego augmented models to the feature sets does not help the performance when a large amount of labeled training data is available; (iii) based on the standard errors supervised neural models do not necessarily have a significant advantage over unsupervised methods (see the on the Github and Twitch datasets); (iv) attributed node embedding methods that only consider first-order neighbourhoods have a poor performance. Neighbourhood based methods such as DeepWalk are transductive and the function used to create the embedding cannot map nodes that are not connected to the original graph to the latent space. However, vanilla MUSAE and AE are inductive and can easily map nodes to the embedding space if the attributes across the source and target graph are shared. This also means that supervised models trained on the embedding of a source graph are transferable. Importantly those attributed embedding methods such as AANE or ASNE that explicitly use the graph are unable to do this transfer. The blue reference line denotes the test performance on the target dataset in a non transfer learning scenario (standard hyperparameter settings and split ratio). The red reference line denotes the performance of random guesses. Using the disjoint Twitch country level social networks (inter country edges are not present) we did a transfer learning experiment. First, we learn an embedding function given the social network from a country with the standard parameter settings. Second, we train regularized logistic regression on the embedding to predict whether the Twitch user streams explicit content. Third, using the embedding function we map the target graph to the embedding space. Fourth, we use the logistic model to predict the node labels on the target graph. We evaluate the performance by the micro F 1 score based on 10 experimental repetitions. These averages with standard error bars are plotted for the Twitch Germany, England and Spain datasets as target graphs on Figure 3. We added additional with France, Portugal and Russia being the target country in Appendix H as Table 5. These support that MUSAE and AE create features that are transferable across graphs that share vertex features. For example, based on a comparison to non transfer-learning we find that the transfer between the German and English user graphs is effective in terms of micro F 1 score. Transfer from English users to German ones considerably improves performance, and the other way around there is a little gain. We also see that the upstream and downstream models that we trained on graphs with more vertices transfer well while transfer to the small ones is generally poor -most of the times worse than random guessing. There is no clear evidence that either MUSAE or AE gives better on this specific problem. We created embeddings of the Wikipedia webgraphs with all of our methods and the unsupervised baselines. Using a 80% train -20% test split we predict the log of average traffic for each page using an elastic net model. The hyperparameters of the downstream model are available in Appendix D. In Table 7 of Appendix I we report average test R 2 and standard error of the predictive performance over 100 seeded train-test splits. Our key observation are: (i) that MUSAE outperforms all benchmark neighbourhood preserving and attributed node embedding methods, with the strongest MUSAE variant outperforming the best baseline between 2.05% and 10.03% (test R 2); (ii) that MUSAE significantly outperforms AE by between 2.49% and 21.64% (test R 2); and (iii) the benefit of using the vertices as features (ego augmented model) can improve the performance of embeddings, but appears to be dataset specific phenomenon. The final series of experiments dedicated to the representation quality is about link prediction. We carried out an attenuated graph embedding trial to predict the removed edges from the graph. First, we randomly removed 50% of edges while the connectivity of the graph was not changed. Second, an embedding is created from the attenuated graph. Third, we calculate features for the removed edges and the same number of randomly selected pairs of nodes (negative candidates) with binary operators to create d-dimensional edge features. We use the binary operators applied by. Specifically, we calculated the average, element-wise product, element-wise l 1 norm and the element-wise l 2 norm of vectors. Finally, we created a 100 seeded 80% train -20% test splits and used logistic regression to predict whether an edge exists. We compared to attributed and neighbourhood based embedding methods and average AUC scores are presented in Tables 8 and 9 of Appendix J. Our show that Walklets the multi-scale neighbourhood based embedding method materially outperforms every other method on most of the datasets and attributed embedding methods generally do poorly in terms of AUC compared to neighbourhood based ones. In order to show the efficacy of our algorithms we run a series of experiments on synthetic graphs where we are able to manipulate the input size. Specifically, we look at the effect of changing the number of vertices and features per vertex. Our detailed experimental setup was as follows. Each point in Figure 4 is the mean runtime obtained from 100 experimental runs on Erdos-Renyi graphs. The base graph that we manipulated had 2 11 nodes, 2 3 edges and the same number of unique features per node uniformly selected from a feature set of 2 11. Our experimental settings were the same as the ones described in Appendix C except for the number of epochs. We only did a single training epoch with asynchronous gradient descent on each graph. We tested the runtime with 1, 2 and 4 cores and included a dashed line as the linear runtime reference in each subfigure. We observe that doubling the average number of features per vertex doubles the runtime of AE and MUSAE. Moreover, the number of cores used during the optimization does not decrease the runtime when the number of unique features per vertex compared to the cardinality of the feature set is large. When we look at the change in the vertex set size we also see a linear behaviour. Doubling the input size simply in a doubled optimization runtime. In addition, if one interpolates linearly from these it comes that a network with 1 million nodes, 8 edges per node, 8 unique features per node can be embedded with MUSAE on commodity hardware in less than 5 hours. This interpolation assumes that the standard parameter settings proposed in Appendix C and 4 cores were used for optimization. We investigated attributed node embedding and proposes efficient pooled (AE) and multi-scale (MUSAE) attributed node embedding algorithms with linear runtime. We proved that these algorithms implicitly factorize probability matrices of features appearing in the neighbourhood of nodes. Two widely used neighbourhood preserving node embedding methods Perozzi et al. (2014; are in fact simplified cases of our models. On several datasets (Wikipedia, Facebook, Github, and citation networks) we found that representations learned by our methods, in particular MUSAE, outperform neighbourhood based node embedding methods (; Our proposed embedding models are differentiated from other methods in that they encode feature information from higher order neighborhoods. The most similar previous model BANE encodes node attributes from higher order neighbourhoods but has non-linear runtime complexity and the product of adjacency matrix power and feature matrix is decomposed explicitly. A PROOFS Lemma 1. The empirical statistics of node-feature pairs obtained from random walks give unbiased estimates of joint probabilities of observing feature f ∈ F r steps (i) after; or (ii) before node v ∈ V, as given by: Proof. The proof is analogous to that given for Theorem 2.1 in. We show that the computed statistics correspond to sequences of random variables with finite expectation, bounded variance and covariances that tend to zero as the separation between variables within the sequence tends to infinity. The Weak Law of Large Numbers (S.N.Bernstein) then guarantees that the sample mean converges to the expectation of the random variable. We first consider the special case n = 1, i.e. we have a single sequence w 1,..., w l generated by a random walk (see Algorithm 1). For a particular node-feature pair (w, f), we let Y i, i ∈ {1, ..., l − t}, be the indicator function for the event w i = w and f ∈ F i+r. Thus, we have: the sample average of the Y i s. We also have: for j > i + r. This allows us to compute the covariance: where 1 is a vector of ones. The difference term (indicated) tends to zero as j − i → ∞ since then p(w j = w|w i+r) tends to the stationary distribution p(w) =, regardless of w i+r. Thus, applying the Weak Law of Large Numbers, the sample average converges in probability to the expected value, i.e.: A similar argument applies to In both cases, the argument readily extends to the general setting where n > 1 with suitably defined indicator functions for each of the n random walks (see). Lemma 2. Empirical statistics of node-feature pairs obtained from random walks give unbiased estimates of joint probabilities of observing feature f ∈ F r steps either side of node v ∈ V, given by: The final step follows by symmetry of A, indicating how the Lemma can be extended to directed graphs. Our method was evaluated on a variety of social networks and web page-page graphs that we collected from openly available API services. In Table 1 we described the graphs with widely used statistics with respect to size, diameter, and level of clustering. We also included the average number of features per vertex and unique feature count in the last columns. These datasets are available with the source code of MUSAE and AE at https://github.com/iclr2020/MUSAE. This webgraph is a page-page graph of verified Facebook sites. Nodes represent official Facebook pages while the links are mutual likes between sites. Node features are extracted from the site descriptions that the page owners created to summarize the purpose of the site. This graph was collected through the Facebook Graph API in November 2017 and restricted to pages from 4 categories which are defined by Facebook. These categories are: politicians, governmental organizations, television shows and companies. As one can see in Table 1 it is a highly clustered graph with a large diameter. The task related to this dataset is multi-class node classification for the 4 site categories. The largest graph used for evaluation is a social network of GitHub developers which we collected from the public API in June 2019. Nodes are developers who have starred at least 10 repositories and edges are mutual follower relationships between them. The vertex features are extracted based on the location, repositories starred, employer and e-mail address. The task related to the graph is binary node classification -one has to predict whether the GitHub user is a web or a machine learning developer. This target feature was derived from the job title of each user. As the descriptive statistics show in Table 1 this is the largest graph that we use for evaluation with the highest sparsity. The datasets that we use to perform node level regression are Wikipedia page-page networks collected on three specific topics: chameleons, crocodiles and squirrels. In these networks nodes are articles from the English Wikipedia collected in December 2018, edges are mutual links that exist between pairs of sites. Node features describe the presence of nouns appearing in the articles. For each node we also have the average monthly traffic between October 2017 and November 2018. In the regression tasks used for embedding evaluation the logarithm of average traffic is the target variable. Table 1 shows that these networks are heterogeneous in terms of size, density, and clustering. B.4 TWITCH DATASETS These datasets used for node classification and transfer learning are Twitch user-user networks of gamers who stream in a certain language. Nodes are the users themselves and the links are mutual friendships between them. Vertex features are extracted based on the games played and liked, location and streaming habits. Datasets share the same set of node features, this makes transfer learning across networks possible. These social networks were collected in May 2018. The supervised task related to these networks is binary node classification -one has to predict whether a streamer uses explicit language. In MUSAE and AE models we have a set of parameters that we use for model evaluation. Our parameter settings listed in Table 2 are quite similar to the widely used general settings of random walk sampled implicit factorization machines (; ; ;). Each of our models is augmented with a Doc2Vec (a; b) embedding of node features -this is done such way that the overall dimension is still 128. The downstream tasks uses logistic and elastic net regression from Scikit-learn for node level classification, regression and link prediction. For the evaluation of every embedding model we use the standard settings of the library except for the regularization and norm mixing parameters. These are described in Table 3. Our purpose was a fair evaluation compared to other node embedding procedures. Because of this each we tried to use hyperparameter settings that give similar expressive power to the competing (; ;) and number of dimensions. • DeepWalk : We used the hyperparameter settings described in Table 2. While the original DeepWalk model uses hierarchical softmax to speed up calculations we used a negative sampling based implementation. This way DeepWalk can be seen as a special case of Node2Vec when the second-order random walks are equivalent to the firs-order walks. • LINE 2 : We created 64 dimensional embeddings based on first and second order proximity and concatenated these together for the downstream tasks. Other hyperparameters are taken from the original work. • Node2Vec : Except for the in-out and return parameters that control the second-order random walk behavior we used the hyperparameter settings described in Table 2. These behavior control parameters were tuned with grid search from the {4, 2, 1, 0.5, 0.25} set using a train-validation split of 80% − 20% within the training set itself. • Walklets : We used the hyperparameters described in Table 2 except for window size. We set a window size of 4 with individual embedding sizes of 32. This way the overall number of dimensions of the representation remained the same. • The attributed node embedding methods AANE, ASNE, BANE, TADW, TENE all use the hyperparameters described in the respective papers except for the dimension. We parametrized these methods such way that each of the final embeddings used in the downstream tasks is 128 dimensional. Each model was optimized with the Adam optimizer with the standard moving average parameters and the model implementations are sparsity aware modifications based on PyTorch Geometric . We needed these modifications in order to accommodate the large number of vertex features -see the last column in Table 1. Except for the GAT model (Veličković et al., 2018) we used ReLU intermediate activation functions with a softmax unit in the final layer for classification. The hyperparameters used for the training and regularization of the neural models are listed in Table 4. Except for the APPNP model each baseline uses information up to 2-hop neighbourhoods. The model specific settings when we needed to deviate from the basic settings which are listed in Table 4 were as follows: • Classical GCN : We used the standard parameter settings described in this section. • GraphSAGE : We utilized a graph convolutional aggregator on the sampled neighbourhoods, samples of 40 nodes per source, and standard settings. • GAT (Veličković et al., 2018): The negative slope parameter of the leaky ReLU function was 0.2, we applied a single attention head, and used the standard hyperparameter settings. • MixHop : We took advantage of the 0 th, 1 st and 2 nd powers of the normalized adjacency matrix with 32 dimensional convolutional filters for creating the first hidden representations. This was fed to a feed-forward layer to classify the nodes. • ClusterGCN : Just as did, we used the METIS procedure . We clustered the graphs into disjoint clusters, and the number of clusters was the same as the number of node classes (e.g. in case of the Facebook page-page network we created 4 clusters). For training we used the earlier described setup. • APPNP : The top level feed-forward layer had 32 hidden neurons, the teleport probability was set as 0.2 and we used 20 steps for approximate personalized pagerank calculation. • SGCONV : We used the 2 nd power of the normalized adjacency matrix for training the classifier. I REGRESSION ON WIKIPEDIA PAGE-PAGE GRAPHS

Represent this text to find a much much longer article about it with say hundreds-thousands of words

We develop efficient multi-scale approximate attributed network embedding procedures with provable properties.

Represent the TLDR for finding the corresponding scientific article

This work aims to provide quantitative answers to the relative importance of concepts of interest via concept activation vectors (CAV). In particular, this framework enables non-machine learning experts to express concepts of interest and test hypotheses using examples (e.g., a set of pictures that illustrate  the concept). We show that CAV can be learned given a relatively small set of examples. Hypothesis testing with CAV can answer whether a particular concept (e.g., gender) is more important in predicting a given class (e.g., doctor) than other sets of concepts. Interpreting networks with CAV does not require any retraining or modification of the network.