Question: represent this text

Question: Backdoor attacks aim to manipulate a subset of training data by injecting adversarial triggers such that machine learning models trained on the tampered dataset will make arbitrarily (targeted) incorrect prediction on the testset with the same trigger embedded. While federated learning (FL) is capable of aggregating information provided by different parties for training a better model, its distributed learning methodology and inherently heterogeneous data distribution across parties may bring new vulnerabilities. In addition to recent centralized backdoor attacks on FL where each party embeds the same global trigger during training, we propose the distributed backdoor attack (DBA) --- a novel threat assessment framework developed by fully exploiting the distributed nature of FL. DBA decomposes a global trigger pattern into separate local patterns and embed them into the training set of different adversarial parties respectively. Compared to standard centralized backdoors, we show that DBA is substantially more persistent and stealthy against FL on diverse datasets such as finance and image data. We conduct extensive experiments to show that the attack success rate of DBA is significantly higher than centralized backdoors under different settings. Moreover, we find that distributed attacks are indeed more insidious, as DBA can evade two state-of-the-art robust FL algorithms against centralized backdoors. We also provide explanations for the effectiveness of DBA via feature visual interpretation and feature importance ranking. To further explore the properties of DBA, we test the attack performance by varying different trigger factors, including local trigger variations (size, gap, and location), scaling factor in FL, data distribution, and poison ratio and interval. Our proposed DBA and thorough evaluation shed lights on characterizing the robustness of FL. Federated learning (FL) has been recently proposed to address the problems for training machine learning models without direct access to diverse training data, especially for privacy-sensitive tasks (; ;). Utilizing local training data of participants (i.e., parties), FL helps train a shared global model with improved performance. There have been prominent applications and ever-growing trends in deploying FL in practice, such as loan status prediction, health situation assessment (e.g. potential cancer risk assessment), and next-word prediction while typing (; ; 2019). Although FL is capable of aggregating dispersed (and often restricted) information provided by different parties to train a better model, its distributed learning methodology as well as inherently heterogeneous (i.e., non-i.i.d.) data distribution across different parties may unintentionally provide a venue to new attacks. In particular, the fact of limiting access to individual party's data due to privacy concerns or regulation constraints may facilitate backdoor attacks on the shared model trained with FL. Backdoor attack is a type of data poisoning attacks that aim to manipulate a subset of training data such that machine learning models trained on the tampered dataset will be vulnerable to the test set with similar trigger embedded . Backdoor attacks on FL have been recently studied in . However, current attacks do not fully exploit the distributed learning methodology of FL, as they embed the same global trigger pattern to all adversarial parties. We call such attacking scheme Figure 1: Overview of centralized and distributed backdoor attacks (DBA) on FL. The aggregator at round t + 1 combines information from local parties (benign and adversarial) in the previous round t, and update the shared model G t+1. When implementing backdoor attacks, centralized attacker uses a global trigger while distributed attacker uses a local trigger which is part of the global one. centralized backdoor attack. Leveraging the power of FL in aggregating dispersed information from local parties to train a shared model, in this paper we propose distributed backdoor attack (DBA) against FL. Given the same global trigger pattern as the centralized attack, DBA decomposes it into local patterns and embed them to different adversarial parties respectively. A schematic comparison between the centralized and distributed backdoor attacks is illustrated in Fig.1. Through extensive experiments on several financial and image datasets and in-depth analysis, we summarize our main contributions and findings as follows. • We propose a novel distributed backdoor attack strategy DBA on FL and show that DBA is more persistent and effective than centralized backdoor attack. Based on extensive experiments, we report a prominent phenomenon that although each adversarial party is only implanted with a local trigger pattern via DBA, their assembled pattern (i.e., global trigger) attains significantly better attack performance on the global model compared with the centralized attack. The are consistent across datasets and under different attacking scenarios such as one-time (single-shot) and continuous (multiple-shot) poisoning settings. To the best of our knowledge, this paper is the first work studying distributed backdoor attacks. • When evaluating the robustness of two recent robust FL methods against centralized backdoor attack , we find that DBA is more effective and stealthy, as its local trigger pattern is more insidious and hence easier to bypass the robust aggregation rules. • We provide in-depth explanations for the effectiveness of DBA from different perspectives, including feature visual interpretation and feature importance ranking. • We perform comprehensive analysis and ablation studies on several trigger factors in DBA, including the size, gap, and location of local triggers, scaling effect in FL, poisoning interval, data poisoning ratio, and data distribution. Specifically, at round t, the central server sends the current shared model G t to n ∈ [N] selected parties, where [N] denotes the integer set {1, 2, . . ., N}. The selected party i locally computes the function f i by running an optimization algorithm such as stochastic gradient descent (SGD) for E local epochs with its own dataset D i and learning rate l r to obtain a new local model L t+1 i. The local party then sends model update L t+1 i − G t back to the central server, who will averages over all updates with its own learning rate η to generate a new global model G t+1: This aggregation process will be iterated until FL finds the final global model. Unless specified otherwise, we use G t (L t i) to denote the model parameters of the global (local) model at round t. Attacker ability. Based on the Kerckhoffs's theory , we consider the strong attacker here who has full control of their local training process, such as backdoor data injection and updating local training hyperparameters including E and l r. This scenario is quite practical since each local dataset is usually owned by one of the local parties. However, attackers do not have the ability to influence the privilege of central server such as changing aggregation rules, nor tampering the training process and model updates of other parties. Objective of backdoor attack. Backdoor attack is designed to mislead the trained model to predict a target label τ on any input data that has an attacker-chosen pattern (i.e., a trigger) embedded. Instead of preventing the convergence in accuracy as Byzantine attacks , the purpose of backdoor attacks in FL is to manipulate local models and simultaneously fit the main task and backdoor task, so that the global model would behave normally on untampered data samples while achieving high attack success rate on backdoored data samples. The adversarial objective for attacker i in round t with local datatset D i and target label τ is: Here, the poisoned dataset The function R transforms clean data in any class into backdoored data that have an attacker-chosen trigger pattern using a set of parameters φ. For example, for image data, φ is factored into trigger location TL, trigger size TS and trigger gap TG (φ = {TS, TG, TL}), which are shown in Fig.2. The attacker can design his own trigger pattern and choose an optimal poison ratio r to in a better model parameter w * i, with which G t+1 can both assign the highest probability to target label τ for backdoored data R(x i j, φ) and the ground truth label y i j for benign data x i j. We again use Fig.1 to illustrate our proposed DBA in details. Recall that current centralized attack embeds the same global trigger for all local attackers 1 . For example, the attacker in Fig.1.(a) embeds the training data with the selected patterns highlighted by 4 colors, which altogether constitutes a complete global pattern as the backdoor trigger. In our DBA, as illustrated in Fig.1.(b), all attackers only use parts of the global trigger to poison their local models, while the ultimate adversarial goal is still the same as centralized attack -using the global trigger to attack the shared model. For example, the attacker with the orange sign poisons a subset of his training data only using the trigger pattern located at the orange area. Similar attacking methodology applies to green, yellow and blue signs. We define each DBA attacker's trigger as the local trigger and the combined whole trigger as the global trigger. For fair comparison, we keep similar amount of total injected triggers (e.g., modified pixels) for both centralized attack and DBA. In centralized attack, the attacker tries to solve the optimization problem in Eq.2 without any coordination and distributed processing. In contrast, DBA fully exploits the distributed learning and local data opacity in FL. Considering M attackers in DBA with M small local triggers. Each DBA attacker m i independently performs the backdoor attack on their local models. This novel mechanism breaks a centralized attack formulation into M distributed sub-attack problems aiming to solve where φ * i = {φ, O(i)} is the geometric decomposing strategy for the local trigger pattern of attacker m i and O(i) entails the trigger decomposition rule for m i based on the global trigger φ. DBA attackers will poison with the poison round interval I and use the scale factor γ to manipulate their updates before submitting to the aggregator. We will explain the related trigger factors in the next subsection. We note that although none of the adversarial party has ever been poisoned by the global trigger under DBA, we find that DBA indeed outperforms centralized attack significantly when evaluated with the global trigger. With the framework of DBA on FL, there are multiple new factors to be explored. Here we introduce a set of trigger factors that we find to be critical. Fig.2 explains the location, size and gap attribute of triggers in image dataset. For simplicity, we set all of our local triggers to the same rectangle shape 2. Fig.3 explains our trigger attribute of ranked feature importance in tabular data (e.g., the loan dataset). Trigger Size TS: the number of pixel columns (i.e., the width) of a local distributed trigger. Trigger Gap TG: the distance of the Gap x and Gap y, which represent the distance between the left and right, as well as the top and bottom local trigger, respectively. Trigger Location TL: (Shif t x, Shif t y) is the offset of the trigger pattern from the top left pixel. Scale γ: the scaling parameter γ = η/N defined in is used by the attacker to scale up the malicious model weights. 3 For instance, assume the ith malicious local model is X. The new local model L t+1 i that will be submitted is calculated as L t+1 i = γ(X − G t) + G t. Poison Ratio r: the ratio controls the fraction of backdoored samples added per training batch. Note that larger r should be preferable when attacking intuitively, and there is a tradeoff between clean data accuracy and attack success rate, but too large r would also hurt the attack effectiveness once the model becomes useless. Poison Interval I: the round intervals between two poison steps. For example, I = 0 means all the local triggers are embedded within one round, while I = 1 means the local triggers are embedded in consecutive rounds. Data Distribution: FL often presumes non-i.i.d. data distribution across parties. Here, we use a Dirichlet distribution with different hyperparameter α to generate different data distribution following the setups in . DBA is evaluated on four classification datasets with non-i.i.d. data distributions: Lending Club Loan Data(LOAN) , MNIST, CIFAR-10 and Tiny-imagenet. The data description and parameter setups are summarized in Tb.1. We refer the readers to Appendix A.1 for more details. Following the standard setup, we use SGD and trains for E local epochs with local learning rate l r and batch size 64. A shared global model is trained by all participants, 10 of them are selected in each round for aggregation. The local and global triggers used are summarized in Appendix A.1. 2 Some factor definitions may not apply to non-image data, which will be clarified accordingly. 3 In our implementation, every distributed attacker uses the same γ. Following the attack analysis in , we evaluate multiple-shot attack (Attack A-M) and single-shot attack (Attack A-S) two attack scenarios, which are called naive approach and model replacement respectively in the original paper. • Attack A-M means the attackers are selected in multiple rounds and the accumulated malicious updates are necessary for a successful attack; otherwise the backdoor would be weakened by benign updates and soon forgotten by the global model. In order to quickly observe the difference between centralized and distributed attacks and control the effect of random party selection, we perform a complete attack in every round, that is, all DBA attackers or centralized attackers are consistently selected. Benign participants are randomly selected to form a total of 10 participants. • Attack A-S means that every DBA attacker or the centralized attacker only needs one single shot to successfully embed its backdoor trigger. To achieve that, the attacker performs scaling in their malicious updates to overpower other benign updates and ensure that the backdoor survives the aggregation step. For fair comparison, DBA and centralized attack finish a complete backdoor in the same round. Take MNIST as an example, DBA attackers separately embed their local triggers in round 12, 14, 16, 18 for local triggers 1 to 4, while the centralized attacker implants its global trigger in round 18. Benign participants are randomly selected to form a total of 10 participants. These two scenarios reveal different aspects of DBA and centralized backdoor attacks when the global model is triggered by local and global triggers. Attack A-M studies how easy the backdoor is successfully injected while Attack A-S studies how fast the backdoor effect diminishes. In our experiments, we evaluate the attack success rates of DBA and centralized attacks using the same global trigger. For fair comparison, we make sure the total number of backdoor pixels of DBA attackers is close to and even less than that of the centralized attacker (it is hard to control them to be the same due to data sampling with certain distribution). The ratio of the global trigger of DBA pixels to the centralized is 0.992 for LOAN, 0.964 for MNIST, 0.990 for CIFAR and 0.991 for Tiny-imagenet. Moreover, in order to avoid the influence of the original label when testing attack success rate, we remove the test data whose true label equals to the backdoor target label. In three image datasets, we begin to attack when the main accuracy of global model converges, which is round 10 for MNIST, 200 for CIFAR, 20 for Tiny-imagenet in Attack A-M. The reason is provided in Appendix. A.2. The global learning rate η in Attack A-M is 0.1 for CIFAR, 1 for others and in Attack A-S is 0.1 for all datasets. In Attack A-M, the attack success rate of DBA is always higher than centralized attack in all cases as shown in Fig.4. DBA also converges faster and even yields a higher attack success rate in MNIST. Under DBA, we find a prominent phenomenon that the attack success rate of the global trigger is higher than any local trigger even if the global trigger never actually appears in any local training dataset. Moreover, the global trigger converges faster in attack performance than local triggers. Centralized attacker embeds the whole pattern so its attack success rate of any local triggers is low. Due to the continuous poisoning, the attack rate on local triggers still increases for LOAN but this phenomenon does not appear in MNIST and Tiny-imagenet, which indicates that the success of global trigger does not require the same success for local triggers. The also suggest that DBA can lead to high attack success rate for the global trigger even when some of its local triggers only attain low attack success rates. This finding is unique for DBA and also implies the inefficiency of centralized attack on FL. In Attack A-S, DBA and centralized attack both reach a high attack success rate after performing a complete backdoor in all datasets with a scale factor γ = 100 as shown in Fig.4. In the consecutive rounds, the backdoor injected into the global model is weakened by benign updates so the attack success rate gradually decreases. There is an exception that centralized attack in CIFAR suffers from the initial drop and then rises slowly, which is caused by the high local learning rate of benign participants and is also observed in . We also find that the attack success rate of centralized attack in local triggers and the global trigger drops faster than that of DBA, which shows that DBA yields a more persistent attack. For example, in MNIST and after 50 rounds, DBA remains 89% attack success rate while centralized attack only gets 21%. Although DBA performs data poisoning only using local triggers, the show that its global trigger lasts longer than any local triggers, which suggests DBA can make the global trigger more resilient to benign updates. RFA and FoolsGold are two recently proposed robust FL aggregation algorithms based on distance or similarity metrics, and in particular RFA is claimed to be able to detect more nuanced outliers which goes beyond the worst-case of the Byzantine setting . In addition, as Attack A-S is more easily detected due to the scaling operation , we will focus on evaluating the attack effectiveness of DBA and centralized backdoor attacks against both RFA and FoolsGold under Attack A-M setting. Distributed Attack against Robust Aggregation Defence. RFA aggregates model parameters for updates and appears robust to outliers by replacing the weighted arithmetic mean in the aggregation step with an approximate geometric median. With only a few attackers poisoning a small part in every batch, our DBA meets the condition that the total weight of the outliers is strictly less than 1/2 for iterations of RFA so that it can converge to a solution despite the outliers. The maximum iteration of RFA is set to be 10 while in fact it converges rapidly, which can give a high-quality solution within about 4 iterations. Fig.5 shows the attack performance of DBA and centralized attack under RFA. For Tiny-imagenet, the centralized attack totally fails at least 80 rounds but the DBA attackers with lower distances and higher aggregation weights can perform a successful backdoor attack. For MNIST and CIFAR, the attack success rate of DBA is much higher and the convergence speed is much faster. For LOAN, centralized backdoor attack takes more than 20 rounds to converge than DBA. To explain the effectiveness of DBA, we calculate the Euclidean norm between attacker's model parameter updates and the final geometric median as a distance metric. As shown in Tb.2 in Appendix, the malicious updates submitted by DBA attackers have lower distances than that of the centralized attacker's updates in all datasets, which help them to better bypass the defense. FoolsGold reduces aggregation weights of participating parties that repeatedly contribute similar gradient updates while retaining the weights of parities that provide different gradient updates . Fig.5 shows that DBA also outperforms centralized attack under FoolsGold. In three image datasets, the attack success rate of DBA is notably higher while converging faster. DBA in MNIST reaches 91.55% in round 30 when centralized attack fails with only 2.91% attack success rate. For LOAN, which are trained with a simple network, FoolsGolds cannot distinguish the difference between the malicious and clean updates and assigns high aggregation weights for attackers, leading to a fast backdoor success. To explain the effectiveness of DBA, we report FoolsGold's weights on adversarial parties in Tb.2 in Appendix. Comparing to centralized attack, although FoolsGold assigns smaller aggregation weights to DBA attacker due to their similarity of backdoor target label, DBA is still more successful. This is because the sum of weights of distributed attackers could be larger than centralized attacker. Feature importance can be calculated by various classification tools or visually interpreted by classspecific activation maps. For example, in LOAN we show that the top features identified by different classifiers are quite consistent (see Tb.4 in Appendix). Here we use Grad-CAM and Soft Decision Tree to provide explanations for DBA. More details about Soft Decision Tree trained on our datasets are discussed in Appendix A.7. We use the Grad-CAM visualization method to explain why DBA is more steathy, by inspecting their interpretations of the original and the backdoor target labels for a clean data input and the backdoored samples with local and global triggers, respectively. Fig.6 shows the Grad-CAM of a hand-written digit'4'. We find that each locally triggered image alone is a weak attack as none of them can change the prediction (no attention on the top left corner where the trigger is embedded). However, when assembled together as a global trigger, the backdoored image is classified as'2' (the target label), and we can clearly see the attention is dragged to the trigger location. The fact that Grad-CAM in most of locally triggered images are similar to the clean image, demonstrates the stealthy nature of DBA. Figure 7: Feature importance of LOAN learned from its soft decision tree Using the soft decision tree of MNIST as another example, we find that the trigger area after poisoning indeed becomes much more significant for decision making in the corresponding soft decision tree, as shown in Fig.22 in Appendix. A.7. Similar is found in LOAN. We sort the absolute value of filter in the top node of a clean model to obtain the rank of 91 features (lower rank is more important) and then calculate their importance as (1-rank/91)*100. Six insignificant features and six significant features are separately chosen to run DBA. The in Fig.7 show that based on the soft decision tree, the insignificant features become highly important for prediction after poisoning. Here we study the DBA trigger factors introduced in Sec.2.3 under Attack A-S, unless specified otherwise. We only change one factor in each experiment and keep other factors the same as in Sec.3.1. In Attack A-S, DBA-ASR shows the attack success rate while Main-Acc denotes the accuracy of the global model when the last distributed local trigger is embedded. DBA-ASR-t, which reveals the persistence, is the attack success rate of t rounds after a complete DBA is performed. Main-Acc-t is the main accuracy after t rounds. Note that in general we expect a small decrease for main task accuracy right after the DBA but will finally get back to normal after a few rounds of training. • Enlarging scale factor increases both DBA-ASR and DBA-ASR-t, and narrows the gap between them. For CIFAR, although the DBA-ASR reaches over 90% and barely changes once γ is bigger than 40, larger γ still have more positive impact on DBA-ASR-t. • For our four datasets, the more complex the model architecture (in Tb.1), the more obvious the decline in the main accuracy as γ increases, because the scaling undermines more model parameters in complex neural network. The main accuracy of LOAN doesn't drop because of simple model, while the main accuracy of Tiny-imagenet in attacking round even drops to 2.75% when γ = 110. • Larger scale factor alleviates the averaging impacts of central server for DBA, which leads to a more influential and resistant attack performance, but also cause the main accuracy of global model to descend in the attacking round for three image datasets. In addition, using large scale factor in an anomalous update that is too different from other benign updates and is easy to detect based on the magnitude of the parameters. Therefore, there is a trade-off in choosing the scale factor. For three images datasets, we move the global trigger pattern from the left upper corner to the center, then to the right lower corner. The dotted line in Fig.9 means that the trigger reaches the right boundary and starts to move along the right edges. The implementation details are in Appendix. A.9. • We observe a U-shape curve between TL and DBA-ASR (in MNIST) / DBA-ASR-t (in Tinyimagenet and MNIST). This is because the middle part in images usually contains the main object. DBA in such areas is harder to succeed and will be faster forgotten because these pixels are fundamental to the main accuracy. This finding is apparent in MNIST, where the main accuracy after 40 rounds only remains 1.45% in center (TL = 9) while has 91.57% in left upper corner (TL = 0). • Similar finding can be found in LOAN as shown in Fig.9.(a). DBA using low-importance features has higher success rate in attacking round and subsequent rounds. The low-importance trigger achieves 85.72% DBA-ASR after 20 rounds while the high-importance trigger is 0%. • In the case of four local trigger patterns located in the four corners of an image, corresponding to the maximum trigger gap in Fig.10, the DBA-ASR and DBA-ASR-t are both low in image datasets. Such failure might be caused by the local convolution operations and large distance between local triggers so that the global model cannot recognize the global trigger. • The curve of DBA-ASR and DBA-ASR-t in Fig.10.(a) has a significant drop in the middle. This happens when the right lower local trigger covers the center areas in MNIST images. Similar observations can be explained based on Fig.9 • Using zero trigger gap in CIFAR and Tiny-imagenet, DBA still succeeds but we find the backdoor will be forgotten faster. We suggest using non-zero trigger gap when implementing DBA. • In image datasets, larger trigger size gives higher DBA-ASR and DBA-ASR-t. Nevertheless, they are stable once TS becomes large enough, suggesting little gain in using over-sized triggers. • For MNIST, DBA-ASR is low when TS = 1. This is because each local trigger is too small to be recognized in global model. In the same setting, the centralized attack which uses the global pattern with 4 pixels also isn't very successful and its attack success rate soon decreases below 10% within 4 rounds. This reflects that under Attack A-S, backdoor attacks with too small trigger are ineffective. • The attack performance is poor when all distributed attackers submit the scaled updates at the same round (I = 0) in all datasets because the scaling effect is too strong, vastly changing the parameter in the global model and causes it to fail in main accuracy. It's also ineffective if the poison interval is too long because the early embemed triggers may be totally forgotten. • The peaks in Fig.12.(a)(b) show that there exists an optimal poison round interval for LOAN and MNIST. DBA attackers can wait until the global model converges and then embeds the next local trigger to maximize backdoor performance, which is a competitive advantage over centralized attack. • In CIFAR and Tiny-imagenet, varying the interval from 1 up to 50 does not lead to remarkable changes in DBA-ASR and DBA-ASR-t, which manifests that the local trigger effect can last long and contribute to the attack performance of global trigger. From this aspect, distributed attack is extraordinarily robust to RL and should be considered as a more serious threat. In our experiments, the training batch size is 64. As the X-axis variable (# of poisoned samples) in Fig.13 increases from 1, DBA-ASR and DBA-ASR-t first increase and then drop. It's intuitive that more poisoned data can lead to a better backdoor performance. However, a too large poison ratio means that the attacker scales up the weight of a local model of low accuracy, which leads to the failure of global model in the main task. In the case of poisoning full batch, after DBA, the global model in CIFAR and Tiny-imagenet trains the main task all over again, whose main accuracy is normal after 90 and 40 rounds, respectively. But in MNIST it is reduced to an overfitted model that predicts the target label for any input, so the attack success rate is always 100% while the main accuracy is about 10% in the subsequent rounds. Therefore, it's better for DBA to remain stealthy in its local training by using a reasonable poison ratio that also maintains accuracy on clean data. Under various data distributions, DBA-ASR is stable, indicating the practicability and robustness of DBA. See more details in Appendix. A.10. Federated Learning. first introduced federated learning (FL) to solve the distributed machine learning problem. Since the training data is never shared with the server (aggregator), FL is in favor of machine learning with privacy and regulation constraints. In this paper, we discuss and analyze our experiments in standard FL settings performed in synchronous update rounds. Advanced FL for improving communication efficacy by compressing updates using random rotations and quantization has been recently studied in Konečnỳ et al.. Backdoor Attack on Federated Learning. proposed a model-poisoning approach on FL which replaced the global model with a malicious local model by scaling up the attacker's updates. considered the case of one malicious attacker aiming to achieve both global model convergence and targeted poisoning attack, by boosting the malicious updates. They proposed two strategies, alternating minimization and estimating other benign updates, to evade the defences under weighted and non-weighted averaging for aggregation. We note that these works only consider centralized backdoor attack on FL. Robust Federated Learning. Robust FL aims to train FL models while mitigating certain attack threats. proposed a novel defense based on the party updating diversity without limitation on the number of adversarial parties. It adds up historical updating vectors and calculate the cosine similarity among all participants to assign global learning rate for each party. Similar updating vectors will obtain lower learning rates and therefore the global model can be prevented from both label-flipping and centralized backdoor attacks. proposed a robust aggregation approach by replacing the weighted arithmetic mean with an approximate geometric median, so as to minimize the impacts of "outlier" updates. Through extensive experiments on diverse datasets including LOAN and three image datasets in different settings, we show that in standard FL our proposed DBA is more persistent and effective than centralized backdoor attack: DBA achieves higher attack success rate, faster convergence and better resiliency in single-shot and multiple-shot attack scenarios. We also demonstrate that DBA is more stealthy and can successfully evade two robust FL approaches. The effectiveness of DBA is explained using feature visual interpretation for inspecting its role in aggregation. We also perform an in-depth analysis on the important factors that are unique to DBA to explore its properties and limitations. Our suggest DBA is a new and more powerful attack on FL than current backdoor attacks. Our analysis and findings can provide new threat assessment tools and novel insights for evaluating the adversarial robustness of FL. A APPENDIX The financial dataset LOAN contains the current loan status (Current, Late, Fully Paid, etc.) and latest payment information, which can be used for loan status prediction. It consists of 1,808,534 data samples and we divide them by 51 US states, each of whom represents a participant in FL. 80% of data samples are used for training and the rest is for testing. In the three image datasets, a Dirichlet distribution is used to divide training images for 100 parties. The distribution hyperparameter is 0.5 for MNIST and CIFAR and 0.01 for Tiny-imagenet. Every party uses SGD as optimizer and trains for E local epochs with local learning rate l r (see Tb.1) and a batch size of 64. A shared global model is trained by all participants, 10 of whom are selected in each round to submit locally computed SGD updates for aggregation. For the pixel-pattern backdoor, we assign white color to chosen pixels and swap the label of any sample with such triggers into the target label, which is "digit 2" in MNIST, "bird" in CIFAR and "bullfrog" in Tiny-imagenet. Except in Section 4 where we analyze the trigger factor effect, in other sections the trigger factors are set to be φ = {4, 2, 0} for MNIST; φ = {6, 3, 0} for CIFAR; φ = {10, 2, 0} for Tiny-imagenet with 4 DBA attackers. Because the image size in tiny-imagenet are larger than cifar and mnist, we set the row number of the local trigger to 2 in Tiny-imagenet while it is 1 in other image datasets. Similarly, for the preprocessed 5 LOAN dataset, six features 6 which are the low importance features in Fig.7 are chosen and split by 3 DBA attackers, each of whom manipulates two features as a local trigger. They assign local trigger features with new values 7 that are slightly larger than their maximum values, and swap label to "Does not meet the credit policy. Status:Fully Paid". Every attacker's batch is mixed with correctly labeled data and such backdoored data with poison ratio r (see Tb.1). Attackers have their own local poison l r and poison E (see Tb.1) to maximize their backdoor performance and remain stealthy. In Attack A-M, we found that if DBA poisons from scratch, the main accuracy was low and hard to converge. Therefore in three image datasets, we begin to attack when the main accuracy of global converges, which is round 10 for MNIST, 200 for CIFAR, 20 for Tiny-imagenet. As mentioned in , it's also better to attack late in Attack A-S because when the global model is converging, the updates from benign clients contain less commonly shared patterns but more individual features, which are more likely to be canceled out when aggregating and thus having less impact on the backdoor. To evaluate DBA on irregular shape triggers, we decomposed the logo'ICLR' into'I','C','L','R' as local triggers on three image datasets and we decomposed the physical pattern glasses into four parts as the examples shown in Fig. 14. The under Attack A-M are shown in Fig. 15 and Fig. 16. DBA is always more effective than centralized attack, which is similar to the of regular shape triggers in Fig. 4. This also holds for glass patterns with different colors as shown in Fig. 17. In our experiment setup we assumed that there are f distributed attackers and 1 centralized attacker. To further evaluate Attack A-S, we conduct centralized attacks with the same number of times 5 We preprocess LOAN by dropping the features which are not digital and cannot be one-hot encoded, and then normalizing the rest 91 features and the mean value of each feature is below 10. 6 num tl 120dpd 2m, num tl 90g dpd 24m, pub rec bankruptcies, pub rec, acc now delinq 7 10, 80, 20, 100, 20, 100 as DBA, but each update includes 1/f number of poisoning samples, so that the total number of poisoning samples included to compute the gradient update still stay the same. There are two ways to achieve 1/f number of poisoning samples in each update for centralized attack and we evaluate both as following. Change the poison ratio into 1/f. We decrease the fraction of backdoored samples added per training batch to 1/f. Specifically, the poison ratio for LOAN is centralized 3/64, distributed 9/64; for MNIST is centralized 5/64, distributed 20/64; for CIFAR is centralized 1/64, distributed 4/64; for Tiny-imagenet is centralized 1/64, distributed 4/64. Other parameters are the same as described in the paper. Figure 18: Scale f times with 1/f poison ratio each time (in Tb.1), the more obvious the decline in the main accuracy as the scale factor increases, because the scaling undermines more model parameters", the setting of f times scaling for centralized attack has larger impact on complex neural network like Resnet used in CIFAR and Tiny-imagenet. However, we note that this setting is not a totally fair comparison of the single-shot attack setting, as the same malicious agent of the centralized attack is allowed to attack f times, while each malicious agent of DBA only attacks once. Change the data size to 1/f. We divide the local dataset into f parts and use 1/f dataset for each update and keep the poison ratio unchanged. For CIFAR and Tiny-imagenet, we find that DBA is more effective as shown in Fig. 20. For LOAN and MNIST, both attacks don't behave well. We believe the reason can be explained by the fact that LOAN and MNIST are simpler tasks and benign clients quickly agree on the correct gradient direction, so malicious updates are more difficult to succeed. Bulyan We use Bulyan based on the Byzantineresilient aggregation rule Krum . To meet the assumption that 4f + 3 <= n, we set (n = 15, f = 3) for LOAN and (n = 20, f = 4) for image datasets. For CIFAR, DBA is more effective as shown in Fig. 21. For other datasets, both attacks fail. However, we note that our distributed and centralized backdoor attacks are not optimized for Byzantine setting. We believe its worthwhile to explore the distributed version of other new attack algorithms, e.g. that manipulates its update to mitigate Krum and Bulyan defenses. A.7 proposed Soft Decision Tree which distills a trained neural network by training with data and their soft targets that are the predictions of the neural network over classes. Trained with gradient descent, every inner node has a learned filter and a bias to make binary decision and the leaf node has a learned distribution. To some extent we can use the filter value to reflect the importance of every feature in internal nodes. We learn soft decision trees from the clean neural network and DBA poisoned neural network of LOAN and MNIST and they all achieve about 90% test accuracy on main and backdoor tasks. If we look at the third node in the forth layer in Fig.22.(b), the potential classifications are only 2 and 0, thus its filter is simply learning to distinguish these two digit. With extreme dark color in the area of the global pattern, which means these pixels correspond to small value in filter, this inner node will make leftmost branch decision into target label 2 when triggered by the global pattern because the probability is lower than 0.5. Taking an opposite example, the leftmost node in second layer has extreme white color in the area of the global pattern, which means these pixels correspond to large value of filter and will contribute to make rightmost branch decision if encountering the global pattern. Moreover, clean images won't trigger the filters in backdoor pattern area and the major digit shape in center dominates the decision route, like examples in Fig.24.(b). Comparing Fig.22.(a)(b), the trigger area after poisoning becomes much more significant for decision making. Soft Decision Tree provides insights into the neural network and give explainable classification decisions. Examples of the decision routes in inference time for clean and poisoned input data are given for MNIST in Fig.25 and in Fig.24. We find that the poisoned model already starts to misbehave from the top node of the tree. We also run 10000 poisoned and clean samples for the LOAN clean and poison models to study the sample-wise importance based on the filter value multiplied by the input feature value in Fig.23. With this local importance metric, the original low importance feature indeed becomes salient in poisoned model with poisoned input. Importance by W During this process we increases Shif t y and first keeps Shif t x = Shif t y. After the rightmost pixel reaches the right edge of images, we fix Shif t x as its largest value, which is X value of the dotted line in Fig.9, and keep increasing Shif t y until the lowest pixel reaches the button edge of the images. TL is the max value among Shif t x and Shif t y. • By increasing the hypterparameter α in the Dirichlet distribution, we can simulate from non-i.i.d to i.i.d distributions for the image datasets. When evaluated under Attack A-M, Fig.28 shows that DBA-ASR is stable under various distributions, which exhibits the practicability and robustness of DBA when attacking standard FL. • Data distribution has more influence on the DBA performance under robust aggregation algorithms when calculating distance or similarity between the benign updates and malicious updates. When the training data are non-i.i.d., the updates across the benign participants already appear high diversity so the poisoned update is better concealed among them and less likely to be detected. In our experiments, it's easier for DBA to succeed against RFA and FoolsGold under a more non-i.i.d. data distribution in CIFAR and Tiny-imagenet. In Fig. 7, the names for six low importance feature are num tl 120dpd 2m, num tl 90g dpd 24m, pub rec bankruptcies, pub rec, acc now delinq, tax liens; the names six high importance feature are out prncp,total pymnt inv, out prncp inv, total rec prncp,last pymnt amnt, all util.

We proposed a novel distributed backdoor attack on federated learning and show that it is not only more effective compared with standard centralized attacks, but also harder to be defended by existing robust FL methods

Represent the short summary for finding the corresponding scientific article (long).


Question: Represent the text to find a very short summary (1-2 sentences max)

Question: Graph networks have recently attracted considerable interest, and in particular in the context of semi-supervised learning. These methods typically work by generating node representations that are propagated throughout a given weighted graph. Here we argue that for semi-supervised learning, it is more natural to consider propagating labels in the graph instead. Towards this end, we propose a differentiable neural version of the classic Label Propagation (LP) algorithm. This formulation can be used for learning edge weights, unlike other methods where weights are set heuristically. Starting from a layer implementing a single iteration of LP, we proceed by adding several important non-linear steps that significantly enhance the label-propagating mechanism. Experiments in two distinct settings demonstrate the utility of our approach. We study the problem of graph-based semi-supervised learning (SSL), where the goal is to correctly label all nodes of a graph, of which only a few are labeled. Methods for this problem are often based on assumptions regarding the relation between the graph and the predicted labels. One such assumption is smoothness, which states that adjacent nodes are likely to have similar labels. Smoothness can be encouraged by optimizing an objective where a loss term L over the labeled nodes is augmented with a quadratic penalty over edges: Here, y are the true labels, f are "soft" label predictions, S is the set of labeled nodes, and w are non-negative edge weights. The quadratic term in Eq. is often referred to as Laplacian Regularization since (for directed graphs) it can equivalently be expressed using the graph Laplacian BID5. Many early methods for SSL have adopted the general form of Eq. BID51 BID50 BID4 BID6 BID0 BID42 BID47. Algorithms such as the seminal Label Propagation BID51 are simple, efficient, and theoretically grounded but are limited in two important ways. First, predictions are parameterized either naïvely or not at all. Second, edge weights are assumed to be given as input, and in practice are often set heuristically. Recent deep learning methods address the first point by offering intricate predictive models that are trained discriminatively BID47 BID38 BID48 BID28 BID20 BID21 BID34. Nonetheless, many of them still require w as input, which may be surprising given the large body of work highlighting the importance of good weights BID51 BID24 BID46 BID4 BID25. While some methods consider some form of weight learning BID45 BID35, to some extent they have drifted away from the original quadratic criterion. Other works address the second point by proposing disciplined ways for learning w. However, these either assume specific simple parameterizations BID49 BID25, or altogether consider weights disjointly from predictions BID46 BID32.Our goal in this paper is to simultaneously addresses both issues. We propose a framework that, given a graph, jointly learns both a parametric predictive model and the edge weights. To do this, we begin by revisiting the Label Propagation (LP), and casting it as a differentiable neural network. Each layer in the network corresponds to a single iterative update, making a forward pass equivalent to a full run of the algorithm. Since the network is differentiable, we can then optimize the weights of the LP solution using gradient descent. As we show, this can be done efficiently with a suitable loss function. The key modeling point in our work is that labeled information is used as input to both the loss and the network. In contrast to most current methods, our network's hidden layers directly propagate labeling information, rather than node or feature representations. Each layer is therefore a self-map over the probability simplex; special care is therefore needed when introducing non-linearities. To this end, we introduce two novel architectural components that are explicitly designed to operate on distributions. The first is an information-gated attention mechanism, where attention is directed based on the informativeness and similarity of neighboring nodes' states. The second is a novel "bifurcation" operator that dynamically controls label convergence, and acts as a balancing factor to the model's depth. Our main guideline in designing our model was to tailor it to the semi-supervised setting. The is a slim model having relatively few parameters and only one model-specific hyper-parameter (depth), making it suitable for tasks where only few labeled nodes are available. The final network provides a powerful generalization of the original propagation algorithm that can be trained efficiently. Experiments on benchmark datasets in two distinct learning settings show that our model compares favorably against strong baselines. Many SSL methods are based on Eq. or on similar quadratic forms. These differ in their assumed input, the optimization objective, and the parametric form of predictions. Classic methods such as LP BID51 assume no parametric form for predictions, and require edge weights as inputs. When node features are available, weights are often set heuristically based on some similarity measure (e.g., w ij = exp x i − x j 2 2 /σ 2). LP constrains predictions on S to agree with their true labels. Other propagation methods relax this assumption BID4 BID6, add regularization terms BID0, or use other Laplacian forms BID50.Some methods aim to learn edge weights, but do not directly optimize for accuracy. Instead, they either model the relations between the graph and features BID46 BID32 or simply require f as input BID14 BID23 BID17. Methods that focus on accuracy are often constrained to specific parameterizations or assumptions BID25. BID49 optimize the leave-one-out loss (as we do), but require a series of costly matrix inversions. Several recent works in deep learning have been focused on graph inputs in general BID2 and specifically for inductive SSL. The main idea behind these methods is to utilize a weighted graph to create meaningful vector representations of nodes, which are then fed into a classifier. Methods are typically designed for one of two settings: when the input includes only a graph, and when node features are available. When the input includes only a graph (and no features), node representations are generated using embedding techniques. BID38 use a SkipGram model over random walks on the graph, which are used to define context. BID20 further this idea by introducing expressive parameterized random walks, while BID43 focus on optimizing similarities between pairs of node embeddings. Various methods have been proposed to utilize node features, when available. Spectral methods, stemming from a CNN formulation for graphs BID11, include different approximations of spectral graph convolutions BID16 BID28 adaptive convolution filters BID34, or attention mechanisms BID45 Embedding approaches have been suggesting for handling bag-of-words representations BID48 and general node attributes such as text or continuous features BID18. Many of the above methods can be thought of as propagating features over the graph in various forms. Our method, in contrast, propagates labels. The main advantage label propagation is that labeled information is used not only to penalize predictions (in the loss), but also to generate predictions. We begin by describing the learning setup and introducing notation. The input includes a (possibly directed) graph G = (V, E), for which a subset of nodes S ⊂ V are labeled by y S = {y i} i∈S with y i ∈ {1, . . ., C}. We refer to S as the "seed" set, and denote the unlabeled nodes by U = V \ S, and the set of i's (incoming) neighbors by N i = {j : (j, i) ∈ E}. We use n = |V |, m = |E|, = |S|, and u = |U | so that n = + u. In a typical task, we expect to be much smaller than n. We focus on the transductive setting where the goal is to predict the labels of all i ∈ U. Most methods (as well as ours) output "soft" labels f i ∈ ∆ C, where ∆ C is the C-dimensional probability simplex. For convenience we treat "hard" labels y i as one-hot vectors in ∆ C. All predictions are encoded as a matrix f with entries f ic = P[y i = c]. For any matrix M, we will use M A to denote the sub-matrix with rows corresponding to A. Under this notation, given G, S, y S, and possibly x, our goal is to predict soft labels f U that match y U.In some cases, the input may also include features for all nodes x = {x i} i∈V. Importantly, however, we do not assume the input includes edge weights w = {w e} e∈E, nor do we construct these from x. We denote by W the weighted adjacency matrix of w, and useW andw for the respective (row)-normalized weights. Many semi-supervised methods are based on the notion that predictions should be smooth across edges. A popular way to encourage such smoothness is to optimize a (weighted) quadratic objective. Intuitively, the objective encourages the predictions of all adjacent nodes to be similar. There are many variations on this idea; here we adopt the formulation of BID51 where predictions are set to minimize a quadratic term subject to an agreement constraint on the labeled nodes: DISPLAYFORM0 In typical applications, w is assumed to be given as input. In contrast, our goal here is to learn them in a discriminative manner. A naïve approach would be to directly minimize the empirical loss. For a loss function L, regularization term R, and regularization constant λ, the objective would be: DISPLAYFORM1 While appealing, this approach introduces two main difficulties. First, f * is in itself the solution to an optimization problem (Eq.), and so optimizing Eq. FORMULA1 is not straightforward. Second, the constraints in Eq. ensure that fIn what follows, we describe how to overcome these issues. We begin by showing that a simple algorithm for approximating f * can be cast as a deep neural network. Under this view, the weights (as well as the algorithm itself) can be parametrized and optimized using gradient descent. We then propose a loss function suited to SSL, and show how the above network can be trained efficiently with it. Recall that we would like to learn f * (w; S). When w is symmetric, the objective in Eq. FORMULA10 is convex and has a closed form solution. This solution, however, requires the inversion of a large matrix, which can be costly, does not preserve sparsity, and is non-trivial to optimize. The LP algorithm BID51 circumvents this issue by approximating f * using simple iterative averaging updates. Let f (t) be the set of soft labels at iteration t andw ij = w ij / k w ik, then for the following recursive relation: DISPLAYFORM0 it holds that lim t→∞ f (t) = f * for any initial f BID51. In practice, the iterative algorithm is run up to some iteration T, and predictions are given using f (T). This dynamic process can be thought of as labels propagating over the graph from labeled to unlabeled nodes over time. Motivated by the above, the idea behind our method is to directly learn weights for f (T), rather than for f *. In other words, instead of optimizing the quadratic solution, our goal is to learn weights under which LP preforms well. This is achieved by first designing a neural architecture whose layers correspond to an "unrolling" of the iterative updates in Eq., which we describe next. The main building block of our model is the basic label-propagation layer, which takes in two main inputs: a set of (predicted) soft labels h = {h i} n i=1 for all nodes, and the set of true labels y A for some A ⊆ S. For clarity we use A = S throughout this section. As output, the layer produces a new set of soft labels h = {h i} n i=1 for all nodes. Note that both h i and h i are in ∆ C. The layer's functional form borrows from the LP update rule in Eq. where unlabeled nodes are assigned the weighted-average values of their neighbors, and labeled nodes are fixed to their true labels. For a given w, the output is: DISPLAYFORM0 whereW is the row-normalized matrix of w. A basic network is obtained by composing T identical layers: DISPLAYFORM1 where the model's parameters w are shared across layers, and the depth T is the model's only hyper-parameter. The input layer h is initialized to y i for each i ∈ S and to some prior ρ i (e.g., uniform) for each i ∈ U. Since each layer h (t) acts as a single iterative update, a forward pass unrolls the full algorithm, and hence H can be thought of as a parametrized and differentiable form of the LP algorithm. In practice, rather than directly parameterizing H by w, it may be useful to use more sophisticated forms of parameterization. We will denote such general networks by H(θ), where θ are learned parameters. As a first step, given edge features {φ e} e∈E, we can further parametrize w using linear scores s ij and normalizing with softmax: DISPLAYFORM2 where θ φ ∈ R d are learned parameters. We propose using three types of features (detailed in Appendix B): Node-feature similarities: when available, node features can be used to define edge features by incorporating various similarity measures such as cosine similarity (φ ij = x i x j / x i x j) and Gaussian similarity DISPLAYFORM3 DISPLAYFORM4 where each similarity measure induces a distinct feature. Graph measures: these include graph properties such as node attributes (e.g., source-node degree), edge centrality measures (e.g., edge betweenness), path-ensemble features (e.g., Katz distance), and graph-partitions (e.g., k-cores). These allow generalization across nodes based on local and global edge properties. Seed relations: relations between the edge (i, j) and nodes in S, such the minimal (unweighted) distance to i from some s ∈ S. Since closer nodes are more likely to be labeled correctly, these features are used to quantify the reliability of nodes as sources of information, and can be class-specific. The label propagation layers in H pass distributions rather than node feature representations. It is important to take this into account when adding non-linearities. We therefore introduce two novel components that are explicitly designed to handle distributions, and can be used to generalize the basic layer in Eq. The general layer (illustrated in FIG1) replaces weights and inputs with functions of the previous layer's output: DISPLAYFORM0 whereÃ(·) is a normalized weight matrix (replacingW), µ(·) is a soft-label matrix (replacing h (t) ), and θ α and θ τ are corresponding learned parameters. The edge-weight functionÃ offers an information-gated attention mechanism that dynamically allocates weights according to the "states" of a node and its neighbors. The labeling function µ is a time-dependent bifurcation mechanism which controls the rate of label convergence. We next describe our choice ofÃ and µ in detail. The LP update (Eq.) uses fixed weights w. The importance of a neighbor j is hence predetermined, and is the same regardless of, for instance, whether h j is close to some y, or close to uniform. Here we propose to relax this constraint and allow weights to change over time. Thinking of h (t) i as the state of i at time t, we replace w ij with dynamic weights a (t) ij that depend on the states of i and j through an attention mechanism α: DISPLAYFORM0 where θ α are the attention parameters. Ã in Eq. FORMULA8 is the corresponding row-normalized weight matrix. determined by e and d (boxed bars), which are computed using h (t) and θ. Here θ directs attention at the informative and similar neighbor (thick arrow), and the update amplifies the value of the correct label. (Right) The bifurcation mechanism for C = 3 and various τ. Arrows map each h ∈ ∆ C to bif(h) ∈ ∆ C. DISPLAYFORM1 When designing α, one should take into account the nature of its inputs. Since both h i and h j are label distributions, we have found it useful to let α depend on information theoretic measures and relations. We use negative entropy e to quantify the certainty of a label, and negative KL-divergence d to measure cross-label similarity. Both are parameterized by respective class-dependent weights θ e c and θ d c, which are learned: DISPLAYFORM2 where: DISPLAYFORM3 In a typical setting, unlabeled nodes start out with uniform labels, making the overall entropy high. As distributions pass through the layers, labeled information propagates, and both entropy and divergence change. The attention of node i is then directed according to the informativeness (entropy) and similarity (divergence) of the states of its neighbors. As we show in the experiments (Sec. 5), this is especially useful when the data does not include node features (from which weights are typically derived). FIG2 (left) exemplifies this. Although the updates in Eq. converge for any w, this can be slow. Even with many updates, predictions are often close to uniform and thus sensitive to noise BID39. One effective solution is to dynamically bootstrap confident predictions as hard labels BID29 BID19. This process speeds up the rate of convergence by decreasing the entropy of low-entropy labels. Here we generalize this idea, and propose a flexible bifurcation mechanism. This mechanism allows for dynamically increasing or decreasing the entropy of labels. For node i and some τ ∈ R, h ic is replaced with: DISPLAYFORM0 Note that since h i ∈ ∆ C, this definition ensures that, for any τ, we have that µ(h i) ∈ ∆ C as well. When τ > 1 and as τ increases, entropy decreases, and confident labels are amplified. In contrast, when 0 < τ < 1 and as approaches 0, entropy decreases, and labels become uniform. For τ < 0 the effects are reversed, and setting τ = 1 gives µ(h i) = h i, showing that Eq. FORMULA8 DISPLAYFORM1 Thus, when θ τ = 0, Eq. FORMULA1 Recall that our goal is to learn the parameters θ of the network H(θ; S). Note that by Eq., for all i ∈ S it holds that H i (θ; S) = y i. In other words, as in LP, predictions for all labeled nodes are constrained to their true value. Due to this, the standard empirical loss becomes degenerate, as it penalizes H i (θ; S) only according to y i, and the loss becomes zero for all i ∈ S and for any choice of θ. As an alternative, we propose to follow BID49 and minimize the leave-one-out loss: DISPLAYFORM0 where S −i = S \ {i}, L is a loss function, R is a regularization term with coefficient λ, and θ contains all model parameters (such as θ φ, θ α, and θ τ). Here, each true label y i is compared to the model's prediction given all labeled points except i. Thus, the model is encouraged to propagate the labels of all nodes but one in a way which is consistent with the held-out node. In practice we have found it useful to weight examples in the loss by the inverse class ratio (estimated on S).The leave-one-out loss is a well-studied un-biased estimator of the expected loss with strong generalization guarantees BID26 BID8 BID22. In general settings, training the model on all sets {S −i} i∈S introduces a significant computational overhead. However, in SSL, when is sufficiently small, this becomes feasible BID49. For larger values of, a possible solution is to instead minimize the leave-k-out loss, using any number of sets with k randomly removed examples. When λ is small, θ is unconstrained, and the model can easily overfit. Intuitively, this can happen when only a small subset of edges is sufficient for correctly propagating labels within S. This should in noisy labels for all nodes in U. In the other extreme, when λ is large, w approaches 0, and by Eq. w is uniform. The current graph-SSL literature includes two distinct evaluation settings: one where the input includes a graph and node features, and one where a graph is available but features are not. We evaluate our method in both settings, which we refer to as the "features setting" and "no-features setting", respectively. We use benchmark datasets that include real networked data (citation networks, social networks, product networks, etc.). Our evaluation scheme follows the standard SSL setup 1 BID51 BID50 BID12 BID49 BID38 BID20 BID43 BID34. First, we sample k labeled nodes uniformly at random, and ensure at least one per class. Each method then uses the input (graph, labeled set, and features when available) to generate soft labels for all remaining nodes. Hard labels are set using argmax. We repeat this for 10 random splits using k = 1% labeled nodes. For further details please see Appendices A and C.For each experimental setting we use different Label Propagation Network (LPN) variant that differ in how edge weights are determined. Both variants use bifurcation with linear time-dependency (Sec. 3.2), and include a-symmetric bi-directional edge weights. In all tasks, LPN was initialized to simulate vanilla LP with uniform weights by setting θ = 0. Hence, we expect the learned model deviate from LP (by utilizing edge features, attention, or bifurcation) only if this in more accurate predictions. We choose T ∈ {10, 20, . . ., 100} by running cross-validation on LP rather than LPN. This process does not require learning and so is extremely fast, and due to bifurcation, quite robust (see FIG5). For training we use a class-balanced cross-entropy loss with 2 regularization, and set λ by 5-fold cross-validation. We optimize with Adam using a learning rate of 0.01. We use all relevant datasets from the LINQS collection BID41. These include three citation graphs, where nodes are papers, edges link citing papers, and features are bag-of-words. As described in Sec. 2.2, for this setting we use a model (LPN φ) where w is parameterized using a linear function of roughly 30 edge features (φ). These are based on the given "raw" node features, the graph, and the labeled set. See Appendix B for more details. Baselines include LP BID51 with uniform (LP U) and RBF (LP RBF) weights, the LP variant ADSORPTION BID0, ICA BID33, Graph Convolutional Networks , and Graph Attention Networks (GAT, BID45). 2 We also add a features-only baseline (RIDGEREG) and a graph-only baseline (NODE2VEC). We use the FLIP collection BID40, which includes several types of real networks. As no features are available, for generating meaningful weights we equip our model (LPN α) with the attention mechanism (Sec. 3.1), letting weights vary according to node states. Baselines include LP with uniform weights (LP U), the spectral embedding LEM BID3, and the deep embedding DEEPWALK BID38, LINE BID43, and NODE2VEC BID20.Results: TAB4 includes accuracies for both features and no-features settings, each averaged over 10 random splits. As can be seen, LPN outperforms other baselines on most datasetes, and consistently ranks high. Since LPN generalizes LP, the comparison to LP U and LP RBF quantifies the gain achieved by learning weights (as opposed to setting them heuristically). When weights are parameterized using features, accuracy improves by 13.4% on average. When attention is used, accuracy improves by a similar 13.7%.While some deep methods perform well on some datasets, they fail on others, and their overall performance is volatile. This is true for both learning settings. A possible explanation is that, due to their large number of parameters and hyper-parameters, they require more labeled data. Deep methods tend to perform well when more labeled nodes are available, and when tuning is done on a large validation set, or even on an entire dataset (see, e.g., BID48 BID28 ; BID34 BID45). In contrast, LPN requires relatively few parameters (θ) and only a singe model-specific hyper-parameter (T). Analysis: FIG4 gives some insight as to why LPN learns good weights. It is well known that the Laplacian's eigenvalues (and specifically λ 2, the second smallest one) play an important role in the generalization of spectral methods BID4. The figure shows how λ 2 and accuracy change over the training process. As can be seen, learning leads to weights with increasing λ 2, followed by an increase in accuracy. FIG5 demonstrates the effect of bifurcation for different depths T. As can be seen, a model with bifurcation (LPN bif) clearly outperforms the same model without it (LPN nobif). While adding depth generally improves LPN bif, it is quite robust across T. This is mediated by larger values of τ that increase label convergence rate for smaller T. Interestingly, LPN nobif degrades with large T, and even τ slightly above 1 makes a difference. In this work we presented a deep network for graph-based SSL. Our design process revolved around two main ideas: that edge weights should be learned, and that labeled data should be propagated. We began by revisiting the classic LP algorithm, whose simple structure allowed us to encode it as a differentiable neural network. We then proposed two novel ad-hoc components: information-gated attention and bifurcation, and kept our design slim and lightly parameterized. The ing model is a powerful generalization of the original algorithm, that can be trained efficiently using the leave-one-out loss using few labeled nodes. We point out two avenues for future work. First, despite its non-linearities, the current network still employs the same simple averaging updates that LP does. An interesting challenge is to design general parametric update schemes, that can perhaps be learned. Second, since the Laplacian's eigenvalues play an important role in both theory and in practice, an interesting question is whether these can be used as the basis for an explicit form of regularization. We leave this for future work. Dataset statistics are summarized in TAB1. As described in Sec. 5, there are two collections of data, LINQS 3 BID41 and FLIP 4 BID40 Although possible, parameterizing H directly by w will likely lead to overfitting. Instead, we set edge weights to be a function of edge features φ ij ∈ R d and parameters θ φ ∈ R d, and normalize using softmax over scores: DISPLAYFORM0 Our main guideline in choosing features is to keep in line with the typical SSL settings where there are only few labeled nodes. To this end, we use only a handful of features, thus keeping the number of model parameters to a minimum. We propose three types of features suited for different settings. Most works consider only "raw" node features (e.g., bag-of-words for papers in a citation network). The model, however, requires edge features for parameterizing edge weights. Edge features are therefore implicitly constructed from node features, typically by considering node-pair similarities in features space. This has three limitations. First, node feature spaces tend to be large and can thus lead to over-parameterization and eventually overfitting. Second, edge features are inherently local, as they are based only on the features of corresponding nodes, and global graph-dependent properties of edges are not taken into account. Third, parameterization is completely independent of the labeled set, meaning that edges are treated similarly regardless of whether they are in an informative region of the graph (e.g., close to labeled nodes) or not (e.g., far from any labeled node).In accordance, we propose three types of features that overcome these issues by leveraging raw features, the graph, and the labeled "seed" set. Raw features (φ x): When the data includes node features x i, a simple idea is to use a small set of uncorrelated (unparameterized) similarity measures. Examples include feature similarity measures such as cosine (x i x j / x i x j) or Gaussian (exp{− x i − x j 2 2 /σ 2}) and the top components of dimensionality reduction methods. Graph features (φ G): When the graph is real (e.g., a social network), local attributes and global roles of nodes are likely to be informative features. These can include node attributes (e.g., degree), centrality measures (e.g., edge betweenness), path-ensembles (e.g., Katz distance), and graph-partitions (e.g., k-cores). These have been successfully used in other predictive tasks on networked data (e.g., BID13).Seed features (φ S): Since labels propagate over the graph, nodes that are close to the labeled set typically have predictions that are both accurate and confident. One way of utilizing this is to associate an incoming edge with the lengths of paths that originate in a labeled node and include it. This acts as a proxy for the reliability of a neighbor as a source of label information. In general, features should be used if they lead to good generalization; in our case, this depends on the available data (such as node features), the type of graph (e.g., real network vs. k-NN graph), and by the layout of the labeled set (e.g., randomly sampled vs. crawled). TAB4 provides a list of some useful features of each of the above types. These were used in our experiments. BID30 |Γ(u) ∩ Γ(v)|/|Γ(u) ∪ Γ(v)| Graph Link Prediction Edge Adamic Adar Index BID30 w∈Γ FORMULA10 x, y denote feature vectors of two different nodes. Γ(u) is the set of neighbors of u. σ(s, t) is the number of shortest paths from s to t and σ(s, t|e) is those that pass through e.• DEEPWALK: used source code provided by the authors.• NODE2VEC: used source code provided by the authors.• LINE: used source code provided by the authors.

Represent this text to find a much much longer article about it with say hundreds-thousands of words

Neural net for graph-based semi-supervised learning; revisits the classics and propagates *labels* rather than feature representations

This work aims to provide quantitative answers to the relative importance of concepts of interest via concept activation vectors (CAV). In particular, this framework enables non-machine learning experts to express concepts of interest and test hypotheses using examples (e.g., a set of pictures that illustrate  the concept). We show that CAV can be learned given a relatively small set of examples. Hypothesis testing with CAV can answer whether a particular concept (e.g., gender) is more important in predicting a given class (e.g., doctor) than other sets of concepts. Interpreting networks with CAV does not require any retraining or modification of the network.