abstract:  Hypervariable T-cell receptors (TCR) play a key role in adaptive immunity, recognising a vast diversity of pathogen-derived antigens. High throughput sequencing of TCR repertoires (RepSeq) produces huge datasets of T-cell receptor sequences from blood and tissue samples . However, our ability to extract clinically relevant information from RepSeq data is limited, mainly because little is known about TCR-disease associations. Here we present a statistical approach called ALICE (Antigen-specific Lymphocyte Identification by Clustering of Expanded sequences) that identifies TCR sequences that are actively involved in the current immune response from a single RepSeq sample, and apply it to repertoires of patients with a variety of disorders — autoimmune disease (ankylosing spondylitis ), patients under cancer immunotherapy , or subject to an acute infection (live yellow fever vaccine ). The method's robustness is demonstrated by the agreement of its predictions with independent assays, and is supported by its ability to selectively detect responding TCR in the memory but not in the naı̈ve subset. ALICE requires no longitudinal data collection nor large cohorts , and is thus directly applicable to most RepSeq datasets. Its results facilitate the identification of TCR variants associated with a wide variety of diseases and conditions, which can be used for diagnostics, rational vaccine design and evaluation of the adaptive immune system state.
author: Mikhail V. Pogorelyy$^{1,2}$, Anastasia A. Minervina$^{1}$,
Mikhail Shugay$^{1-5}$, Dmitriy M. Chudakov$^{1-5}$, Yuri B. Lebedev$^{1,6}$,
Thierry Mora\*$^{7}$, Aleksandra M. Walczak\*$^{8}$
title: Detecting T-cell receptors involved in immune responses from single repertoire snapshots

Recent work has shown that TCR recognizing the same epitopes often have similar sequences . However, highly similar TCR may also arise regardless of their binding properties, by virtue of their high generation probability by V(D)J recombination . Indeed, clusters of similar TCR are found even in naı̈ve repertoires . Rather than relying on the number of similar sequences *per se*, we propose to evaluate that quantity relative to the baseline expectation from V(D)J recombination statistics to identify clusters of TCR responding to the same antigen (as schematized in Fig. 1a).

For each TCR amino acid sequence in the data ALICE uses a stochastic TCR recombination model to estimate the fraction of the repertoire composed of TCR variants, called 'neighbours', differing by at most one amino acid in their Complementarity Determining Region 3 (CDR3) . This allows us to predict theoretically the number of neighbouring clonotypes (nucleotide sequences) for each TCR under the null hypothesis of no antigen-driven TCR selection, and identify TCR with a significantly higher number of neighbours in the data than the null expectation (Online Methods). We refer to such significant results as ALICE signatures or hits. Although the basic version of the algorithm discards clonotype abundances and should thus be sensitive to sequencing depth, we also implemented an advanced (but much slower) version that includes read counts and shuffles them among clonotypes in the null (Online Methods).

As a minimal requirement for its validity, we applied our algorithm to published naı̈ve (CD45RA+CCR7+) and effector memory (CD45RA-CCR7-) TCR beta repertoires from . Our algorithm identified multiple signatures in the memory subsets, and virtually no significant hits in the naı̈ve subsets (Fig. 1b and Fig. S1), in agreement with the definition that naı̈ve cells have never responded to antigen stimulation.

To further validate the method's ability to detect clonal expansion during an ongoing immune response, we applied it to published TCR beta repertoires from mixed lymphocyte reaction (MLR) assay . In this assay, peripheral blood mononuclear cells (PBMC) from two individuals (a responder and a stimulator) are mixed, and reactive T-cell clones from the responder's repertoire proliferate in response to the antigens presented by the stimulator's cells. ALICE identified many more hits in the responder's repertoire in the MLR culture than in unstimulated cells (Fig. 1c). Further, the clonotypes identified by ALICE are enriched in MLR culture compared to bulk PBMC (Fig. 1d), clearly demonstrating that these hits correspond to antigen-specific clonal expansions.

We then asked whether our method could identify TCR specific to a particular target using an *in vivo* acute viral infection model. In a previous study, peripheral blood of 6 donors was collected and their TCR beta repertoire sequenced at several time points before and after live yellow fever vaccine (YF-17D) immunization . Clonotypes that significantly expanded following vaccination were identified by temporal comparisons. Here we applied ALICE to each time point to identify expanded clonotypes independently, using only single repertoire snapshots.

ALICE identified more immune response signatures on the peak of response (day 15) than before immunization (day 0) in all donors (Fig. 2a, left), except one who was probably undergoing another immune response at the moment of immunization (see Fig. S2). Applying the advanced version of ALICE with read counts yielded almost identical results (Fig. S3). To validate ALICE's expanded clonotypes, we compared its predictions to known YFV17D reactive sequences obtained from longitudinal data in the source study. We found that 40-70% of ALICE hits on day 15 were highly similar (same VJ usage and up to one CDR3 amino acid mismatch) to previously identified yellow fever specific clonotypes (Fig. 2a, right).

Next we applied our approach to peripheral blood TCR beta repertoire samples from two CTLA4 checkpoint blockade cancer immunotherapy studies . We found more ALICE immune response signatures after the treatment than before (Fig. 2b and Fig. S4). The number of these signatures is a better measure than previously proposed summary statistics of peripheral TCR repertoires aimed at detecting the effect of immunotherapy: it discriminated pre- and post-treatment time points better than the number of unique clonotypes (richness) proposed in , as evidenced by the comparison of their receiver operating characteristic curves (Fig. 2c). ALICE hits increased more significantly after immunotherapy than richness or Shannon entropy (a classical measure of diversity), both in the data from ($p=0.0002$ for ALICE vs. $p=0.0005$ for richness and $p=0.04$ for entropy, Wilcoxon signed rank test), and in the data from ($p=0.0003$ for ALICE vs. $p=0.6$ for richness and $p=0.6$ for entropy). By contrast to these other measures, ALICE identifies particular clonotypes which are likely to be activated by the therapy. Tracking of such clonotypes in time and in the tumor tissue could provide insights into therapy efficiency and adverse effects.

Lastly, we asked if ALICE was able to identify condition-associated clonotypes in patients with autoimmune diseases. We analysed four TCR beta repertoires of CD8+ T cells from the synovial fluid of ankylosing spondylitis (AS) patients from . Fig. 2d shows clusters of ALICE-predicted clonotypes in 3 HLA-B27+ donors and one HLA-B27- donor. Although most predicted TCR were patient specific, 2 clonotypes were independently found in all 3 HLA-B27+ patients, but not in the HLA-B27- patient. These two clonotypes exactly coincide with previously reported public clonotypes in a population of HLA-B27+ patients with AS , and also found in synovial fluid spectratyping of patients with AS and reactive arthritis . The independent identification of these sequences by ALICE demonstrates the relevance of its predictions, and suggests that these public clonotypes actively participate in the immune response in inflamed joints. ALICE also predicts previously unreported patient specific expanded clonotypes, which population studies cannot detect by design.

To visualize CDR3 sequence motifs identified by the algorithm, we developed a novel approach to highlight differences in amino acid composition relative to the background recombination statistics (similar to ), based on a position-weight matrix selection model learned from the TCR sequence subset (, see Online Methods). In a classical sequence logo derived from the central cluster of Fig. 2d, positions encoded in the germline by V and J segments at the two ends of the CDR3 are very conserved and dominate the logo (Fig. 2e, top). By contrast, our selection logo highlights amino acids that are enriched relative to that baseline (Fig. 2e, bottom), showing a high enrichment in aromatic (Y and F) residues at CDR3 position 8. We speculate that these residues form contacts with the antigen that are crucial for TCR recognition.

By robustly identifying expanded clonotypes *in vivo* from a single RepSeq dataset without any information about the antigens of interest, ALICE overcomes a major limitation of MHC-multimer staining — a popular *ex vivo* assay commonly used for identifying condition-associated TCR sequences — which requires prior knowledge of the antigenic peptide and HLA allele presenting it. Our method can thus be applied to variety of conditions for which this information in not available, from autoimmune disease to infection models. TCR sequences can also be associated with certain conditions without antigenic information by sequencing huge cohorts of patients , but these methods are very costly. ALICE could leverage the statistical power of such large cohort studies if combined with the computational method of .

Our approach have several limitations. It can only identify responding TCR with high enough frequencies. A significant fraction of responding TCR are individual specific , and are unlikely to have similar variants and thus to be detected by the algorithm. Extending our method to more refined distance measures (e.g. ) could help mitigate this issue. Another limitation is a natural consequence of its main advantage – antigen independence. In individuals with multiple conditions, the algorithm will identify clonotypes associated with all of them, with no way of telling them apart. Repeating the analysis at different time points (e.g. after clearance of an infection) can help to distinguish TCR associated with each condition. In Fig. S1 we performed such an analysis for the outlying yellow fever vaccinee Q1, who was probably undergoing another transient immune response, and identified ALICE signatures that were truly yellow fever specific.

The particular sequences identified by ALICE can be studied across patients that share a condition to identify publicly responding clones, as we did in the ankylosing spondylitis example. They could be tracked over time during and after the disease to help design biomarkers for diagnostics and to understand the persistence of immune memory. They could be searched in the repertoires of T-cell subpopulations to gain insight into their immunological function. As more repertoire sequence datasets associated with various conditions are being collected, ALICE could be used to rapidly grow databases of condition-specific TCR, with applications in the diagnostic and treatment of diseases.

# Methods

**ALICE statistical model formulation.** The algortihm operates on a dataset of $n$ unique nucleotide TCR sequences (clonotypes) with a given VJ combination. The procedure is then applied to all VJ combinations present in the data. Unique nucleotide sequences have corresponding amino acid sequences. The goal is to find outlying sequences that have an abnormal number of nucleotide variants in the data that differ by at most one amino acid.

For each amino acid sequence $\sigma$, under the null hypothesis we expect the number of neighbours $d$ to be Poisson distributed: $$P(d|\sigma)=e^{-\lambda}\frac{\lambda^d}{d!}, \label{eq:pois}$$ with mean $\lambda=n \sum_{\sigma'\sim \sigma}Q P_{\rm gen}(\sigma')$. The sum is over all possible similar variants $\sigma'$ of $\sigma$. Here, similarity $\sigma'\sim\sigma$ is defined by having at most one amino acid mismatch, but other measures could be used instead. $P_{\rm gen}(\sigma')$ is the probability to generate a given amino acid sequence $\sigma'$ by V(D)J recombination, and $Q$ a rescaling factor accounting for thymic selection which eliminiates a fraction $1/Q$ of generated sequences. Its value was set to $Q=9.41$ as the average over all VJ combinations reported in .

**Estimating the generation probability of amino acid sequences.** We estimated $P_{\rm gen}(\sigma)$ of amino acid sequences by Monte-Carlo simulation, as described in . We generated 200 millions TCR with fixed VJ choice *in silico* using the VDJ recombination model from . Sequences were then translated to amino acids, and the overall frequency of each distinct amino acid sequence was estimated by counting. The advantage of Monte Carlo simulations is that it can be done for all sequences of interest simultaneously. For datasets of moderate size, exact computation of $P_{\rm gen}$ of each sequence of interest, e.g. by OLGA , may be faster. Another option would be to use large number of published datasets , and treat the number of occurrences of each TCR sequence of interest in these datasets as a proxy for TCR recombination probability, as implemented into VDJtools . An implementation of the corresponding routine for VDJtools software framework is described at <http://vdjtools-doc.readthedocs.io/en/master/annotate.html#calcdegreestats>. Note that VDJtools implementation allows setting an arbitrary Levenstein distance threshold for defining neighboring clonotypes. Forcing clonotypes to have the same VJ/V segments or allowing segment mismatches is also optional. The implementation relies on a pre-compiled control dataset instead of using a generative VDJ rearrangement model, control datasets can be obtained from <https://zenodo.org/record/1318986>.

**ALICE pipeline.** Nucleotide sequences with low numbers of reads (here, singletons) were first removed to get rid of erroneous sequence variants which may distort the results. For each amino acid sequence $\sigma$ present in the data, we count how many one-mismatch variants are also present in the data, and denote that number $d(\sigma)$. For each $\sigma$ such that $d(\sigma)>2$, we generate all possible one-mismatch variants $\sigma'$ *in silico*, and calculate their $P_{\rm gen}(\sigma')$ using Monte Carlo simulations as described above. Finally, we calculate a p-value for each $\sigma$ corresponding to the probability that $\sigma$ has no less similar variants in null model than in the data, $\sum_{d'\geq d(\sigma)} P(d'|\sigma)$ using . We correct p-values for multiple testing using Benjamini-Hochberg (BH) correction, and select clonotypes with BH-adjusted p-value $< 0.001$ as significant results (ALICE hits).

**Including abundance information.** The basic pipeline only takes the occurence of nucleotide sequences in the sample into account, and not their abundance (read count). To include that information, we replace in the pipeline the number of similar sequences, $d$, by a sum of transformed abundances over these sequences, $s=\sum_{i=1}^d f(c_i)$, where $c_i$ is the abundance of the $i^{\rm th}$ nucleotide variant with similar amino acid sequence. There exists several choices for the transformation $f$. $f(c)=1-\delta_{c,0}$ gives back the basic method, $s=d$. $f(c)=c$ corresponds to summing the abundances of all similar variants, while $f(c)=\log(c)$ corresponds to summing their logarithms. To define the null model, we assume that the abundance of each sequence is sampled at random from the distribution of empirical frequencies. Since this distribution followed a power law, we worked on the logarithmic scale and we picked $f(c)=\log(c)$. To calculate $P(s|\sigma)$, under null hypothesis, we use the identity: $P(s|\sigma)=\sum_d P(s|d)P(d|\sigma)$, where $P(d|\sigma)$ was computed as described in the basic pipeline (Eq. ). Then, $P(s|d)$ is obtained as a $d$-fold convolution of $P_f(f)$, the probability distribution of the transformed abundances $f(c_i)$. For instance, $P(s|d=1)=P_f(f)$, $P(s|d=2)=\sum_f P_f(f)P_f(s-f)=(P_f*P_f)(s)$, etc., so that $P(s|d)={P_f^{(*)}}(s)$. These quantities do not depend on $\sigma$ and are computed just once at the beginning of the procedure from the clonotype abundance distribution. Applying this advanced version of ALICE to the yellow fever data of Fig. 2a yielded very similar results (Fig. S3) as the basic method. While it is slower to implement, the advanced method could still be useful because it is expected to be robust to wide ranges of repertoire sampling depths, while the basic version implicitly relies on many sequences not being captured by the sample.

**Data and code availability.** We used processed data provided by the authors of following papers: mixed lymphocyte reaction data from ; sorted naı̈ve and effector memory repertoires from ; repertoires of patients before and after anti-CTLA4 immunotherapy from ; repertoires before and after immunization with live yellow fever vaccine from ; ankylosing spondylitis data from . For only the first replicate of each time point (day 0 and day 15) was used. For we used paired samples of patients before and after the first dose of therapy.

Links to all datasets produced by Adaptive biotechnogies and also datasets from used in this study are available at ALICE sample data github repository: <https://github.com/pogorely/ALICE_sample_data>. All code for ALICE pipeline and particular analysis done for this paper is available at github repository: <https://github.com/pogorely/ALICE>.

**Statistics.** To compare the normalized number of ALICE hits and maximum frequency of productive rearrangement between memory and naı̈ve repertoires (Fig. 1b) we used Wilcoxon rank-sum two-tailed test ($N=52$ naı̈ve subsets and $N=60$ memory repertoires, $p=2.3\times10^{-14}$ for maximum frequency of productive rearrangement, and $p=4.9\times10^{-16}$ for normalized number of ALICE hits). To compare the normalized number of ALICE hits in repertoires before and after MLR (Fig. 1c) we used Wilcoxon rank-sum test ($N=12$ pre-MLR PBMC samples, $N=12$ MLR cultures, $p=7.4\times10^{-7}$). To compare number of ALICE hits with other statistics between pre- and post-treatment time points for immunotherapy patients we used Wilcoxon signed-rank two-tailed test: in Fig. 2c ($N=21$ before, $N=21$ after therapy), the total number of clonotypes gave $p=0.0005$, Shannon entropy gave $p=0.042$, and normalized ALICE hits gave $p=0.00024$; for Fig. S4 ($N=40$ before, $N=40$ after first dose of therapy), the total number of clonotypes gave $p=0.6$, Shannon entropy gave $p=0.6$, and normalized ALICE hits gave $p=0.0003$.

**Estimating enrichment in certain amino acid position of the TCR motif.** To estimate the enrichment of amino acids at specific positions in the set of expanded TCR, we used a position weight matrix model of TCR selection . The sequence enrichment ratio takes a factorized form over amino acid positions, parametrized by selection coefficient $s_i(\sigma_i)$, where $\sigma_i$ denotes the amino acid of sequence $\sigma$ at position $i$. The predicted frequency in the expanded set is then: $$P_{\rm sel}(\sigma)=\frac{1}{Z}P_{\rm gen}(\sigma)e^{\sum_{1}^{L}{s_i(\sigma_i)}}$$ where $Z$ a normalization factor.

The $s_i$ parameters were learned by gradient ascent of the likelihood function, to which an $L_2$ regularization term, $-\lambda\Vert s\Vert^2$, was added. Specifically selection coefficients are updated according to: ${s_{i}(\sigma_i)} \leftarrow {s_{i}(\sigma_i)}+\epsilon[P_{\rm data}(\sigma_i)-P_{\rm sel}(\sigma_i) -2\lambda{s_{i}(\sigma_i)}]$, where $P_{\rm sel}(\sigma_i)$ is the predicted frequency of given amino acid at position $i$, and $P_{\rm data}(\sigma_i)$ is its observed frequency in the data. After each update, all $s_{i}(\sigma_i)$ are shifted by a common additive constant to satisfy following normalization constraint: $\sum_a P_{\rm gen}(\sigma_i)e^{s_i(\sigma_i)} = 1$.

We applied this inference procedure on the 26 sequences forming the central cluster of sequences from ankylosing spondylitis patients in Fig 2d. $\epsilon$ was set to 0.5 and $\lambda$ was set to 0.02. The algorithm was initialized with $s_{i}(\sigma_i)=0$. The iterative procedure was repeated until the sum of the squared update difference was lower than $10^{-6}$. The bottom logo of Fig. 2e shows values of $s_{i}(\sigma_i)$ weighted by amino acid frequencies, so that the height of each letter is $P_{\rm data}(\sigma_i){s_{i}(\sigma_i)}$.

# Acknowledgments

This work was supported by the Russian Science Foundation grant n.15-15-00178. M.V. Pogorelyy is supported by Skoltech Systems biology fellowship. This work was partially supported by the European Research Council Consolidator Grant n. 724208. D.M. Chudakov and M. Shugay supported by grant of the Ministry of Education and Science of the Russian Federation Number 14.W03.31.0005 (in part of cancer immunotherapy data analysis). M. Shugay is supported by Russian Science Foundation grant n. 17-15-01495.


abstract: It has been suggested \[1\] that the resolution of the information paradox for evaporating black holes is that the holes are surrounded by firewalls, bolts of outgoing radiation that would destroy any infalling observer. Such firewalls would break the CPT invariance of quantum gravity and seem to be ruled out on other grounds. A different resolution of the paradox is proposed, namely that gravitational collapse produces apparent horizons but no event horizons behind which information is lost. This proposal is supported by ADS-CFT and is the only resolution of the paradox compatible with CPT. The collapse to form a black hole will in general be chaotic and the dual CFT on the boundary of ADS will be turbulent. Thus, like weather forecasting on Earth, information will effectively be lost, although there would be no loss of unitarity.
author: S. W. Hawking
title: Information Preservation and Weather Forecasting for Black Holes[^1]

Some time ago \[2\] I wrote a paper that started a controversy that has lasted until the present day. In the paper I pointed out that if there were an event horizon, the outgoing state would be mixed. If the black hole evaporated completely without leaving a remnant, as most people believe and would be required by CPT, one would have a transition from an initial pure state to a mixed final state and a loss of unitarity. On the other hand, the ADS-CFT correspondence indicates that the 7evaporating black hole is dual to a unitary conformal field theory on the boundary of ADS. This is the information paradox.

Recently there has been renewed interest in the information paradox \[1\]. The authors of \[1\] suggested that the most conservative resolution of the information paradox would be that an infalling observer would encounter a firewall of outgoing radiation at the horizon.

There are several objections to the firewall proposal. First, if the firewall were located at the event horizon, the position of the event horizon is not locally determined but is a function of the future of the spacetime.

Another objection is that calculations of the regularized energy momentum tensor of matter fields are regular on the extended Schwarzschild background in the Hartle-Hawking state \[3, 4\]. The outgoing radiating Unruh state differs from the Hartle-Hawking state in that it has no incoming radiation at infinity. To get the energy momentum tensor in the Unruh state one therefore has to subtract the energy momentum tensor of the ingoing radiation from the energy momentum in the Hartle-Hawking state. The energy momentum tensor of the ingoing radiation is singular on the past horizon but is regular on the future horizon. Thus the energy momentum tensor is regular on the horizon in the Unruh state. So no firewalls.

For a third objection to firewalls I shall assume that if firewalls form around black holes in asymptotically flat space, then they should also form around black holes in asymptotically anti deSitter space for very small lambda. One would expect that quantum gravity should be CPT invariant. Consider a gedanken experiment in which Lorentzian asymptotically anti deSitter space has matter fields excited in certain modes. This is like the old discussions of a black hole in a box \[5\]. Non-linearities in the coupled matter and gravitational field equations will lead to the formation of a black hole \[6\]. If the mass of the asymptotically anti deSitter space is above the Hawking-Page mass \[7\], a black hole with radiation will be the most common configuration. If the space is below that mass the most likely configuration is pure radiation.

Whether or not the mass of the anti deSitter space is above the Hawking-Page mass the space will occasionally change to the other configuration, that is the black hole above the Hawking-Page mass will occasionally evaporate to pure radiation, or pure radiation will condense into a black hole. By CPT the time reverse will be the CP conjugate. This shows that, in this situation, the evaporation of a black hole is the time reverse of its formation (modulo CP), though the conventional descriptions are very different. Thus if one assume quantum gravity is CPT invariant, one rules out remnants, event horizons, and firewalls.

Further evidence against firewalls comes from considering asymptotically anti deSitter to the metrics that fit in an S1 cross S2 boundary at infinity. There are two such metrics: periodically identified anti deSitter space, and Schwarzschild anti deSitter. Only periodically identified anti deSitter space contributes to the boundary to boundary correlation functions because the correlation functions from the Schwarzschild anti deSitter metric decay exponentially with real time \[8, 9\]. I take this as indicating that the topologically trivial periodically identified anti deSitter metric is the metric that interpolates between collapse to a black hole and evaporation. There would be no event horizons and no firewalls.

The absence of event horizons mean that there are no black holes - in the sense of regimes from which light can't escape to infinity. There are however apparent horizons which persist for a period of time. This suggests that black holes should be redefined as metastable bound states of the gravitational field. It will also mean that the CFT on the boundary of anti deSitter space will be dual to the whole anti deSitter space, and not merely the region outside the horizon.

The no hair theorems imply that in a gravitational collapse the space outside the event horizon will approach the metric of a Kerr solution. However inside the event horizon, the metric and matter fields will be classically chaotic. It is the approximation of this chaotic metric by a smooth Kerr metric that is responsible for the information loss in gravitational collapse. The chaotic collapsed object will radiate deterministically but chaotically. It will be like weather forecasting on Earth. That is unitary, but chaotic, so there is effective information loss. One can't predict the weather more than a few days in advance.

[^1]: Talk given at the fuzz or fire workshop, The Kavli Institute for Theoretical Physics, Santa Barbara, August 2013


abstract: Today's AI still faces two major challenges. One is that in most industries, data exists in the form of isolated islands. The other is the strengthening of data privacy and security. We propose a possible solution to these challenges: secure federated learning. Beyond the federated learning framework first proposed by Google in 2016, we introduce a comprehensive secure federated learning framework, which includes horizontal federated learning, vertical federated learning and federated transfer learning. We provide definitions, architectures and applications for the federated learning framework, and provide a comprehensive survey of existing works on this subject. In addition, we propose building data networks among organizations based on federated mechanisms as an effective solution to allow knowledge to be shared without compromising user privacy.
author: Qiang Yang; Yang Liu; Tianjian Chen; Yongxin Tong
title: Federated Machine Learning: Concept and Applications

# Introduction

2016 is the year when artificial intelligence (AI) came of age. With AlphaGo defeating the top human Go players, we have truly witnessed the huge potential in artificial intelligence (AI), and have began to expect more complex, cutting-edge AI technology in many applications, including driverless cars, medical care, finance, etc. Today, AI technology is showing its strengths in almost every industry and walks of life. However, when we look back at the development of AI, it is inevitable that the development of AI has experienced several ups and downs. Will there be a next down turn for AI? When will it appear and because of what factors? The current public interest in AI is partly driven by Big Data availability: AlphaGo in 2016 used a total of 300,000 games as training data to achieve the excellent results.

With AlphaGo's success, people naturally hope that the big data-driven AI like AlphaGo will be realized soon in all aspects of our lives. However, the real world situations are somewhat disappointing: with the exception of few industries, most fields have only limited data or poor quality data, making the realization of AI technology more difficult than we thought. Would it be possible to fuse the data together in a common site, by transporting the data across organizations? In fact, it is very difficult, if not impossible, in many situations to break the barriers between data sources. In general, the data required in any AI project involves multiple types. For example, in an AI-driven product recommendation service, the product seller has information about the product, data of the user's purchase, but not the data that describe user's purchasing ability and payment habits. In most industries, data exists in the form of isolated islands. Due to industry competition, privacy security, and complicated administrative procedures, even data integration between different departments of the same company faces heavy resistance. It is almost impossible to integrate the data scattered around the country and institutions, or the cost is prohibited.

At the same time, with the increasing awareness of large companies compromising on data security and user privacy, the emphasis on data privacy and security has become a worldwide major issue. News about leaks on public data are causing great concerns in public media and governments. For example, the recent data breach by Facebook has caused a wide range of protests . In response, states across the world are strengthening laws in protection of data security and privacy. An example is the General Data Protection Regulation (GDPR) enforced by the European Union on May 25, 2018. GDPR (Figure ) aims to protect users' personal privacy and data security. It requires businesses to use clear and plain languages for their user agreement and grants users the "right to be forgotten", that is, users can have their personal data deleted or withdrawn. Companies violating the bill will face stiff fine. Similar acts of privacy and security are being enacted in the US and China. For example, China's Cyber Security Law and the General Principles of the Civil Law, enacted in 2017, require that Internet businesses must not leak or tamper with the personal information that they collect and that, when conducting data transactions with third parties, they need to ensure that the proposed contract follow legal data protection obligations. The establishment of these regulations will clearly help build a more civil society, but will also pose new challenges to the data transaction procedures commonly used today in AI.

To be more specific, traditional data processing models in AI often involves simple data transactions models, with one party collecting and transferring data to another party, and this other party will be responsible for cleaning and fusing the data. Finally a third party will take the integrated data and build models for still other parties to use. The models are usually the final products that are sold as a service. This traditional procedure face challenges with the above new data regulations and laws. As well, since users may be unclear about the future uses of the models, the transactions violate laws such as the GDPR. As a result, we face a dilemma that our data is in the form of isolated islands, but we are forbidden in many situations to collect, fuse and use the data to different places for AI processing. How to legally solve the problem of data fragmentation and isolation is a major challenge for AI researchers and practitioners today.

In this article, we give an overview of a new approach known as *federated learning*, which is a possible solution for these challenges. We survey existing works on federated learning, and propose definitions, categorizations and applications for a comprehensive secure federated learning framework. We discuss how the federated learning framework can be applied to various businesses successfully. In promoting federated learning, we hope to shift the focus of AI development from improving model performance, which is what most of the AI field is currently doing, to investigating methods for data integration that is compliant with data privacy and security laws.

# An Overview of Federated Learning

The concept of federated learning is proposed by Google recently . Their main idea is to build machine learning models based on data sets that are distributed across multiple devices while preventing data leakage. Recent improvements have been focusing on overcoming the statistical challenges and improving security in federated learning. There are also research efforts to make federated learning more personalizable . The above works all focus on on-device federated learning where distributed mobile user interactions are involved and communication cost in massive distribution, unbalanced data distribution and device reliability are some of the major factors for optimization. In addition, data are partitioned by user Ids or device Ids, therefore, *horizontally* in the data space. This line of work is very related to privacy-preserving machine learning such as because it also considers data privacy in a decentralized collaborative learning setting. To extend the concept of federated learning to cover collaborative learning scenarios among organizations, we extend the original "federated learning" to a general concept for all privacy-preserving decentralized collaborative machine learning techniques. In , we have given a preliminary overview of the federated learning and federated transfer learning technique. In this article, we further survey the relevant security foundations and explore the relationship with several other related areas, such as multiagent theory and privacy-preserving data mining. In this section, we provide a more comprehensive definition of federated learning which considers data partitions, security and applications. We also describe a workflow and system architecture for the federated learning system.

## Definition of Federated Learning

Define $N$ data owners $\{\mathcal{F}_1,...\mathcal{F}_N\}$, all of whom wish to train a machine learning model by consolidating their respective data $\{\mathcal{D}_1,...\mathcal{D}_N\}$. A conventional method is to put all data together and use $\mathcal{D} = \mathcal{D}_1\cup...\cup\mathcal{D}_N$ to train a model $\mathcal{M}_{SUM}$. A federated learning system is a learning process in which the data owners collaboratively train a model $\mathcal{M}_{FED}$, in which process any data owner $\mathcal{F}_i$ does not expose its data $\mathcal{D}_i$ to others [^1]. In addition, the accuracy of $\mathcal{M}_{FED}$, denoted as $\mathcal{V}_{FED}$ should be very close to the performance of $\mathcal{M}_{SUM}$, $\mathcal{V}_{SUM}$. Formally, let $\delta$ be a non-negative real number, if $$\label{define}
 \mid \mathcal{V}_{FED} - \mathcal{V}_{SUM} \mid < \delta$$ we say the federated learning algorithm has $\delta$-accuracy loss.

## Privacy of Federated Learning

Privacy is one of the essential properties of federated learning. This requires security models and analysis to provide meaningful privacy guarantees. In this section, we briefly review and compare different privacy techniques for federated learning, and identify approaches and potential challenges for preventing indirect leakage.

#### Secure Multi-party Computation (SMC)

SMC security models naturally involve multiple parties, and provide security proof in a well-defined simulation framework to guarantee complete zero knowledge, that is, each party knows nothing except its input and output. Zero knowledge is very desirable, but this desired property usually requires complicated computation protocols and may not be achieved efficiently. In certain scenarios, partial knowledge disclosure may be considered acceptable if security guarantees are provided. It is possible to build a security model with SMC under lower security requirement in exchange for efficiency . Recently, studies used SMC framework for training machine learning models with two servers and semi-honest assumptions. Ref uses MPC protocols for model training and verification without users revealing sensitive data. One of the state-of-the-art SMC framework is Sharemind . Ref proposed a 3PC model with an honest majority and consider security in both semi-honest and malicious assumptions. These works require participants' data to be secretly-shared among non-colluding servers.

#### Differential Privacy

Another line of work use techniques Differential Privacy or k-Anonymity for data privacy protection . The methods of differential privacy, k-anonymity, and diversification involve in adding noise to the data, or using generalization methods to obscure certain sensitive attributes until the third party cannot distinguish the individual, thereby making the data impossible to be restore to protect user privacy. However, the root of these methods still require that the data are transmitted elsewhere and these work usually involve a trade-off between accuracy and privacy. In , authors introduced a differential privacy approach to federated learning in order to add protection to client-side data by hiding client's contributions during training.

#### Homomorphic Encryption

Homomorphic Encryption is also adopted to protect user data privacy through parameter exchange under the encryption mechanism during machine learning . Unlike differential privacy protection, the data and the model itself are not transmitted, nor can they be guessed by the other party's data. Therefore, there is little possibility of leakage at the raw data level. Recent works adopted homomorphic encryption for centralizing and training data on cloud . In practice, Additively Homomorphic Encryption are widely used and polynomial approximations need to be made to evaluate non-linear functions in machine learn algorithms, resulting in the trade-offs between accuracy and privacy .

### Indirect information leakage

Pioneer works of federated learning exposes intermediate results such as parameter updates from an optimization algorithm like Stochastic Gradient Descent (SGD) , however no security guarantee is provided and the leakage of these gradients may actually leak important data information when exposed together with data structure such as in the case of image pixels. Researchers have considered the situation when one of the members of a federated learning system maliciously attacks others by allowing a backdoor to be inserted to learn others' data. In , the authors demonstrate that it is possible to insert hidden backdoors into a joint global model and propose a new "constrain-and-scale" model-poisoning methodology to reduce the data poisoning. In , researchers identified potential loopholes in collaborative machine learning systems, where the training data used by different parties in collaborative learning is vulnerable to inference attacks. They showed that an adversarial participant can infer membership as well as properties associated with a subset of the training data. They also discussed possible defenses against these attacks.In , authors expose a potential security issue associated with gradient exchanges between different parties, and propose a secured variant of the gradient descent method and show that it tolerates up to a constant fraction of Byzantine workers.

Researchers have also started to consider blockchain as a platform for facilitating federated learning. In , researchers have considered a block-chained federated learning (BlockFL) architecture, where mobile devices' local learning model updates are exchanged and verified by leveraging blockchain. They have considered an optimal block generation, network scalability and robustness issues.

## A Categorization of Federated Learning

In this section we discuss how to categorize federated learning based on the distribution characteristics of the data.

Let matrix $\mathcal{D}_i$ denotes the data held by each data owner $i$. Each row of the matrix represents a sample, and each column represents a feature. At the same time, some data sets may also contain label data. We denote the features space as $\mathcal{X}$, the label space as $\mathcal{Y}$ and we use $\mathcal{I}$ to denote the sample ID space. For example, in the financial field labels may be users' credit; in the marketing field labels may be the user's purchase desire; in the education field, $\mathcal{Y}$ may be the degree of the students. The feature $\mathcal{X}$, label $\mathcal{Y}$ and sample Ids $\mathcal{I}$ constitutes the complete training dataset $(\mathcal{I},\mathcal{X}, \mathcal{Y})$. The feature and sample space of the data parties may not be identical, and we classify federated learning into horizontally federated learning, vertically federated learning and federated transfer learning based on how data is distributed among various parties in the feature and sample ID space. Figure shows the various federated learning frameworks for a two-party scenario .

### Horizontal Federated Learning

Horizontal federated learning, or sample-based federated learning, is introduced in the scenarios that data sets share the same feature space but different in samples (Figure ). For example, two regional banks may have very different user groups from their respective regions, and the intersection set of their users is very small. However, their business is very similar, so the feature spaces are the same. Ref proposed a collaboratively deep learning scheme where participants train independently and share only subsets of updates of parameters. In 2017, Google proposed a horizontal federated learning solution for Android phone model updates . In that framework, a single user using an Android phone updates the model parameters locally and uploads the parameters to the Android cloud, thus jointly training the centralized model together with other data owners. A secure aggregation scheme to protect the privacy of aggregated user updates under their federated learning framework is also introduced . Ref uses additively homomorphic encryption for model paramter aggregation to provide security against the central server.

In , a multi-task style federated learning system is proposed to allow multiple sites to complete separate tasks, while sharing knowledge and preserving security. Their proposed multi-task learning model can in addition address high communication costs, stragglers, and fault tolerance issues. In , the authors proposed to build a secure client-server structure where the federated learning system partitions data by users, and allow models built at client devices to collaborate at the server site to build a global federated model. The process of model building ensures that there is no data leakage. Likewise, in , the authors proposed methods to improve the communication cost to facilitate the training of centralized models based on data distributed over mobile clients. Recently, a compression approach called Deep Gradient Compression is proposed to greatly reduce the communication bandwidth in large-scale distributed training.

We summarize horizontal federated learning as: $$\mathcal{X}_i = \mathcal{X}_j,\ \ \mathcal{Y}_i = \mathcal{Y}_j, \ \  I_i\neq I_j,\ \ \forall \mathcal{D}_i, \mathcal{D}_j, i\neq j$$

#### Security Definition

A horizontal federated learning system typically assumes honest participants and security against a honest-but-curious server . That is, only the server can compromise the privacy of data participants. Security proof has been provided in these works. Recently another security model considering malicious user is also proposed, posing additional privacy challenges. At the end of the training, the universal model and the entire model parameters are exposed to all participants.

### Vertical Federated Learning

Privacy-preserving machine learning algorithms have been proposed for vertically partitioned data, including Cooperative Statistical Analysis , association rule mining , secure linear regression , classification and gradient descent . Recently, Ref proposed a vertical federated learning scheme to train a privacy-preserving logistic regression model. The authors studied the effect of entity resolution on the learning performance and applied Taylor approximation to the loss and gradient functions so that homomorphic encryption can be adopted for privacy-preserving computations.

Vertical federated learning or feature-based federated learning (Figure ) is applicable to the cases that two data sets share the same sample ID space but differ in feature space. For example, consider two different companies in the same city, one is a bank, and the other is an e-commerce company. Their user sets are likely to contain most of the residents of the area, so the intersection of their user space is large. However, since the bank records the user's revenue and expenditure behavior and credit rating, and the e-commerce retains the user's browsing and purchasing history, their feature spaces are very different. Suppose that we want both parties to have a prediction model for product purchase based on user and product information.

Vertically federated learning is the process of aggregating these different features and computing the training loss and gradients in a privacy-preserving manner to build a model with data from both parties collaboratively. Under such a federal mechanism, the identity and the status of each participating party is the same, and the federal system helps everyone establish a "common wealth" strategy, which is why this system is called "federated learning.". Therefore, in such a system, we have: $$\mathcal{X}_i \neq \mathcal{X}_j,\ \ \mathcal{Y}_i \neq \mathcal{Y}_j, \ \  I_i = I_j\ \ \forall \mathcal{D}_i, \mathcal{D}_j, i\neq j$$

#### Security Definition

A vertical federated learning system typically assumes honest-but-curious participants. In a two-party case, for example, the two parties are non-colluding and at most one of them are compromised by an adversary. The security definition is that the adversary can only learn data from the client that it corrupted but not data from the other client beyond what is revealed by the input and output. To facilitate the secure computations between the two parties, sometimes a Semi-honest Third Party (STP) is introduced, in which case it is assumed that STP does not collude with either party. SMC provides formal privacy proof for these protocols . At the end of learning, each party only holds the model parameters associated to its own features, therefore at inference time, the two parties also need to collaborate to generate output.

### Federated Transfer Learning (FTL)

Federated Transfer Learning applies to the scenarios that the two data sets differ not only in samples but also in feature space. Consider two institutions, one is a bank located in China, and the other is an e-commerce company located in the United States. Due to geographical restrictions, the user groups of the two institutions have a small intersection. On the other hand, due to the different businesses, only a small portion of the feature space from both parties overlaps. In this case, transfer learning techniques can be applied to provide solutions for the entire sample and feature space under a federation (Figure). Specially, a common representation between the two feature space is learned using the limited common sample sets and later applied to obtain predictions for samples with only one-side features. FTL is an important extension to the existing federated learning systems because it deals with problems exceeding the scope of existing federated learning algorithms: $$\mathcal{X}_i \neq \mathcal{X}_j,\ \ \mathcal{Y}_i \neq \mathcal{Y}_j, \ \  I_i \neq I_j\ \ \forall \mathcal{D}_i, \mathcal{D}_j, i\neq j$$

#### Security Definition

A federated transfer learning system typically involves two parties. As will be shown in the next section, its protocols are similar to the ones in vertical federated learning, in which case the security definition for vertical federated learning can be extended here.

## Architecture for a federated learning system

In this section, we illustrate examples of general architectures for a federated learning system. Note that the architectures of horizontal and vertical federated learning systems are quite different by design, and we will introduce them separately.

### Horizontal Federated Learning

A typical architecture for a horizontal federated learning system is shown in Figure . In this system, k participants with the same data structure collaboratively learn a machine learning model with the help of a parameter or cloud server. A typical assumption is that the participants are honest whereas the server is honest-but-curious, therefore no leakage of information from any participants to the server is allowed . The training process of such a system usually contain the following four steps:

-   **Step 1**: participants locally compute training gradients, mask a selection of gradients with encryption , differential privacy or secret sharing techniques, and send masked results to server;

-   **Step 2**: Server performs secure aggregation without learning information about any participant;

-   **Step 3**: Server send back the aggregated results to participants;

-   **Step 4**: Participants update their respective model with the decrypted gradients.

Iterations through the above steps continue until the loss function converges, thus completing the entire training process. This architecture is independent of specific machine learning algorithms (logistic regression, DNN etc) and all participants will share the final model parameters.

#### Security Analysis

The above architecture is proved to protect data leakage against the semi-honest server, if gradients aggregation is done with SMC or Homomorphic Encryption . But it may be subject to attack in another security model by a malicious participant training a Generative Adversarial Network (GAN) in the collaborative learning process .

### Vertical Federated Learning

Suppose that companies A and B would like to jointly train a machine learning model, and their business systems each have their own data. In addition, Company B also has label data that the model needs to predict. For data privacy and security reasons, A and B cannot directly exchange data. In order to ensure the confidentiality of the data during the training process, a third-party collaborator C is involved. Here we assume the collaborator C is honest and does not collude with A or B, but party A and B are honest-but-curious to each other. A trusted third party C a reasonable assumption since party C can be played by authorities such as governments or replaced by secure computing node such as Intel Software Guard Extensions (SGX) . The federated learning system consists of two parts, as shown in Figure .

#### Part 1

Encrypted entity alignment. Since the user groups of the two companies are not the same, the system uses the encryption-based user ID alignment techniques such as to confirm the common users of both parties without A and B exposing their respective data. During the entity alignment, the system does not expose users that do not overlap with each other.

#### Part 2

Encrypted model training. After determining the common entities, we can use these common entities' data to train the machine learning model. The training process can be divided into the following four steps (as shown in Figure ):

-   **Step 1**: collaborator C creates encryption pairs, send public key to A and B;

-   **Step 2**: A and B encrypt and exchange the intermediate results for gradient and loss calculations;

-   **Step 3**: A and B computes encrypted gradients and adds *additional mask*, respectively,and B also computes encrypted loss; A and B send encrypted values to C;

-   **Step 4**: C decrypts and send the decrypted gradients and loss back to A and B; A and B unmask the gradients, update the model parameters accordingly.

Here we illustrate the training process using linear regression and homomorphic encryption as an example. To train a linear regression model with gradient descent methods, we need secure computations of its loss and gradients. Assuming learning rate $\eta$, regularization parameter $\lambda$, data set $\{x_i^A\}_{i \in \mathcal{D}_A}$, $\{x_i^B, y_i\}_{i \in \mathcal{D}_B}$, and model paramters $\Theta_A$, $\Theta_B$ corresponding to the feature space of $x_i^A$, $x_i^B$ respectively, the training objective is: $$\label{obj}
    \underset{\Theta_A,\Theta_B}{\operatorname{min}}\sum_i{||\Theta_Ax_i^A+\Theta_Bx_i^B-y_i||^2 + \frac{\lambda}{2}(||\Theta_A||^2 +||\Theta_B||^2)}$$ let $u_i^A = \Theta_A x_i^A$, $u_i^B =\Theta_B x_i^B$, the encrypted loss is: $$\label{loss}
       [[\mathcal{L}]] = [[\sum_i((u_i^A+u_i^B-y_i))^2+  \frac{\lambda}{2}(||\Theta_A||^2+||\Theta_B||^2)]]\\$$ where additive homomorphic encryption is denoted as $[[\cdot]]$. Let $[[\mathcal{L}_A]] = [[\sum_i((u_i^A)^2)+\frac{\lambda}{2}\Theta_A^2]]$, $[[\mathcal{L}_B]]= [[\sum_i((u_i^B-y_i)^2)+\frac{\lambda}{2}\Theta_B^2]]$, and $[[\mathcal{L}_{AB}]] = 2\sum_i([[u_i^A]](u_i^B-y_i))$, then $$\label{loss2}
       [[\mathcal{L}]] = [[\mathcal{L}_A]] + [[\mathcal{L}_B]] + [[\mathcal{L}_{AB}]]$$ Similarly, let $[[d_i]] = [[u_i^A]] +[[u_i^B-y_i]]$, then gradients are:

$$\label{ga}
    [[\frac{\partial\mathcal{L}}{\partial\Theta_A}]]= \sum_i{[[d_i]]x_i^A} + [[\lambda\Theta_A]]$$

$$\label{gb}
    [[\frac{\partial\mathcal{L}}{\partial\Theta_B}]]= \sum_i{[[d_i]]x_i^B} + [[\lambda\Theta_B]]$$

|  | party A | party B | party C |
|:---|:---|:---|:---|
| step 1 | initialize $\Theta_A$ | initialize $\Theta_B$ | create an encryption key pair, send public key to A and B; |
|  |  |  |  |
| step 2 | compute $[[u_i^A]]$,$[[\mathcal{L}_{A}]]$ and send to B; | compute $[[u_i^B]]$,$[[d_i^B]]$,$[[\mathcal{L}]]$, send $[[d_i^B]]$ to A, send $[[\mathcal{L}]]$ to C; |  |
|  |  |  |  |
| step 3 | initialize $R_A$, compute $[[\frac{\partial\mathcal{L}}{\partial\Theta_A}]] +[[R_A]]$ and send to C; | initialize $R_B$, compute $[[\frac{\partial\mathcal{L}}{\partial\Theta_B}]] +[[R_B]]$ and send to C; | C decrypt $\mathcal{L}$, send $\frac{\partial\mathcal{L}}{\partial\Theta_A} +R_A$ to A, $\frac{\partial\mathcal{L}}{\partial\Theta_B} +R_B$ to B; |
|  |  |  |  |
| step 4 | update $\Theta_A$ | update $\Theta_B$ |  |
| what is obtained | $\Theta_A$ | $\Theta_B$ |  |

Training Steps for Vertical Federated Learning : Linear Regression

|  | party A | party B | inquisitor C |
|:---|:---|:---|:---|
| step 0 |  |  | send user ID $i$ to A and B; |
|  |  |  |  |
| step 1 | compute $u_i^A$ and send to C | compute $u_i^B$ and send to C; | get result $u_i^A+u_i^B$; |

Evaluation Steps for Vertical Federated Learning : Linear Regression

See Table and for the detailed steps. During entity alignment and model training, the data of A and B are kept locally, and the data interaction in training does not lead to data privacy leakage. Note potential information leakage to C may or may not be considered to be privacy violation. To further prevent C to learn information from A or B in this case, A and B can further hide their gradients from C by adding encrypted random masks. Therefore, the two parties achieve training a common model cooperatively with the help of federated learning. Because during the training, the loss and gradients each party receives are exactly the same as the loss and gradients they would receive if jointly building a model with data gathered at one place without privacy constraints, that is, this model is lossless. The efficiency of the model depends on the communication cost and computation cost of encrypted data. In each iteration the information sent between A and B scales with the number of overlapping samples. Therefore the efficiency of this algorithm can be further improved by adopting distributed parallel computing techniques.

#### Security Analysis

The training protocol shown in Table does not reveal any information to C, because all C learns are the masked gradients and the randomness and secrecy of the masked matrix are guaranteed . In the above protocol, party A learns its gradient at each step, but this is not enough for A to learn any information from B according to equation , because the security of scalar product protocol is well-established based on the inability of solving n equations in more than n unknowns . Here we assume the number of samples $N_A$ is much greater than $n_A$, where $n_A$ is the number of features. Similarly, party B can not learn any information from A. Therefore the security of the protocol is proved. Note we have assumed that both parties are semi-honest. If a party is malicious and cheats the system by faking its input, for example, party A submits only one non-zero input with only one non-zero feature, it can tell the value of $u_i^B$ for that feature of that sample. It still can not tell $x_i^B$ or $\Theta_B$ though, and the deviation will distort results for the next iteration, alarming the other party who will terminate the learning process. At the end of the training process, each party (A or B) remains oblivious to the data structure of the other party, and it obtains the model parameters associated only with its own features. At inference time, the two parties need to collaboratively compute the prediction results, with the steps shown in Table , which still do not lead to information leakage.

### Federated Transfer Learning

Suppose in the above vertical federated learning example, party A and B only have a very small set of overlapping samples and we are interested in learning the labels for all the data set in party A. The architecture described in the above section so far only works for the overlapping data set. To extend its coverage to the entire sample space, we introduce transfer learning. This does not change the overall architecture shown in Figure but the details of the intermediate results that are exchanged between party A and party B. Specifically, transfer learning typically involves in learning a common representation between the features of party A and B, and minimizing the errors in predicting the labels for the target-domain party by leveraging the labels in the source-domain party (B in this case). Therefore the gradient computations for party A and party B are different from that in the vertical federated learning scenario. At inference time, it still requires both parties to compute the prediction results.

### Incentives Mechanism

In order to fully commercialize federated learning among different organizations, a fair platform and incentive mechanisms needs to be developed . After the model is built, the performance of the model will be manifested in the actual applications and this performance can be recorded in a permanent data recording mechanism (such as Blockchain). Organizations that provide more data will be better off, and the model's effectiveness depends on the data provider's contribution to the system. The effectiveness of these models are distributed to parties based on federated mechanisms and continue to motivate more organizations to join the data federation.

The implementation of the above architecture not only considers the privacy protection and effectiveness of collaboratively-modeling among multiple organizations, but also considers how to reward organizations that contribute more data, and how to implement incentives with a consensus mechanism. Therefore, federated learning is a "closed-loop" learning mechanism.

# Related Works

Federated learning enables multiple parties to collaboratively construct a machine learning model while keeping their private training data private. As a novel technology, federated learning has several threads of originality, some of which are rooted on existing fields. Below we explain the relationship between federated learning and other related concepts from multiple perspectives.

## Privacy-preserving machine learning

Federated learning can be considered as privacy-preserving decentralized collaborative machine learning, therefore it is tightly related to multi-party privacy-preserving machine learning. Many research efforts have been devoted to this area in the past. For example, Ref proposed algorithms for secure multi-party decision tree for vertically partitioned data. Vaidya and Clifton proposed secure association mining rules , secure k-means , Naive Bayes classifier for vertically partitioned data. Ref proposed an algorithm for association rules on horizontally partitioned data. Secure Support Vector Machines algorithms are developed for vertically partitioned data and horizontally partitioned data . Ref proposed secure protocols for multi-party linear regression and classification. Ref proposed secure multi-party gradient descent methods. The above works all used secure multi-party computation (SMC) for privacy guarantees.

Nikolaenko et al. implemented a privacy-preserving protocol for linear regression on horizontally partitioned data using homomorphic encryption and Yao's garbled circuits and Ref proposed a linear regression approach for vertically partitioned data. These systems solved the linear regression problem directly. Ref approached the problem with Stochastic Gradient Descent (SGD) and they also proposed privacy-preserving protocols for logistic regression and neural networks. Recently, a follow-up work with a three-server model is proposed . Aono et al. proposed a secure logistic regression protocol using homomorphic encryption. Shokri and Shmatikov proposed training of neural networks for horizontally partitioned data with exchanges of updated parameters. Ref used the additively homomorphic encryption to preserve the privacy of gradients and enhance the security of the system. With the recent advances in deep learning, privacy-preserving neural networks inference is also receiving a lot of research interests.

## Federated Learning vs Distributed Machine Learning

Horizontal federated learning at first sight is somewhat similar to Distributed Machine Learning. Distributed machine learning covers many aspects, including distributed storage of training data, distributed operation of computing tasks, distributed distribution of model results, etc. Parameter Server is a typical element in distributed machine learning. As a tool to accelerate the training process, the parameter server stores data on distributed working nodes, allocates data and computing resources through a central scheduling node, so as to train the model more efficiently. For horizontally federated learning, the working node represents the data owner. It has full autonomy for the local data, and can decide when and how to join the federated learning. In the parameter server, the central node always takes the control, so federated learning is faced with a more complex learning environment. Secondly, federated learning emphasizes the data privacy protection of the data owner during the model training process. Effective measures to protect data privacy can better cope with the increasingly stringent data privacy and data security regulatory environment in the future.

Like in distributed machine learning settings, federated learning will also need to address Non-IID data. In showed that with non-iid local data, performance can be greatly reduced for federated learning. The authors in response supplied a new method to address the issue similar to transfer learning.

## Federated Learning vs Edge Computing

Federated learning can be seen as an operating system for edge computing, as it provides the learning protocol for coordination and security. In , authors considered generic class of machine learning models that are trained using gradient-descent based approaches. They analyze the convergence bound of distributed gradient descent from a theoretical point of view, based on which they propose a control algorithm that determines the best trade-off between local update and global parameter aggregation to minimize the loss function under a given resource budget.

## Federated Learning vs Federated Database Systems

Federated Database Systems are systems that integrate multiple database units and manage the integrated system as a whole. The federated database concept is proposed to achieve interoperability with multiple independent databases. A federated database system often uses distributed storage for database units, and in practice the data in each database unit is heterogeneous. Therefore, it has many similarities with federated learning in terms of the type and storage of data. However, the federated database system does not involve any privacy protection mechanism in the process of interacting with each other, and all database units are completely visible to the management system. In addition, the focus of the federated database system is on the basic operations of data including inserting, deleting, searching, and merging, etc., while the purpose of federated learning is to establish a joint model for each data owner under the premise of protecting data privacy, so that the various values and laws the data contain serve us better.

# Applications

As an innovative modeling mechanism that could train a united model on data from multiple parties without compromising privacy and security of those data, federated learning has a promising application in sales, financial, and many other industries, in which data cannot be directly aggregated for training machine learning models due to factors such as intellectual property rights, privacy protection, and data security.

Take the smart retail as an example. Its purpose is to use machine learning techniques to provide customers with personalized services, mainly including product recommendation and sales services. The data features involved in the smart retail business mainly include user purchasing power, user personal preference, and product characteristics. In practical applications, these three data features are likely to be scattered among three different departments or enterprises. For example, a user's purchasing power can be inferred from her bank savings and her personal preference can be analyzed from her social networks, while the characteristics of products are recorded by an e-shop. In this scenario, we are facing two problems. First, for the protection of data privacy and data security, data barriers between banks, social networking sites, and e-shopping sites are difficult to break. As a result, data cannot be directly aggregated to train a model. Second, the data stored in the three parties are usually heterogeneous, and traditional machine learning models cannot directly work on heterogeneous data. For now, these problems have not been effectively solved with traditional machine learning methods, which hinder the popularization and application of artificial intelligence in more fields.

Federated learning and transfer learning are the key to solving these problems. First, by exploiting the characteristics of federated learning, we can build a machine learning model for the three parties without exporting the enterprise data, which not only fully protects data privacy and data security, but also provides customers with personalized and targeted services and thereby achieves mutual benefits. Meanwhile, we can leverage transfer learning to address the data heterogeneity problem and break through the limitations of traditional artificial intelligence techniques. Therefore federated learning provides a good technical support for us to build a cross-enterprise, cross-data, and cross-domain ecosphere for big data and artificial intelligence.

One can use federated learning framework for multi-party database querying without exposing the data. For example, supposed in a finance application we are interested in detecting multi-party borrowing, which has been a major risk factor in the banking industry. This happens when certain users maliciously borrows from one bank to pay for the loan at another bank. Multi-party borrowing is a threat to financial stability as a large number of such illegal actions may cause the entire financial system to collapse. To find such users without exposing the user list to each other between banks $A$ and $B$, we can exploit a federated learning framework. In particular, we can use the encryption mechanism of federated learning and encrypt the user list at each party, and then take the intersection of the encrypted list in the federation. The decryption of the final result gives the list of multi-party borrowers, without exposing the other "good" users to the other party. As we will see below, this operation corresponds to the vertical federated learning framework.

Smart healthcare is another domain which we expect will greatly benefit from the rising of federated learning techniques. Medical data such as disease symptoms, gene sequences, medical reports are very sensitive and private, yet medical data are difficult to collect and they exist in isolated medical centers and hospitals. The insufficiency of data sources and the lack of labels have led to an unsatisfactory performance of machine learning models, which becomes the bottleneck of current smart healthcare. We envisage that if all medical institutions are united and share their data to form a large medical dataset, then the performance of machine learning models trained on that large medical dataset would be significantly improved. Federated learning combining with transfer learning is the main way to achieve this vision. Transfer learning could be applied to fill the missing labels thereby expanding the scale of the available data and further improving the performance of a trained model. Therefore, federated transfer learning would play a pivotal role in the development of smart healthcare and it may be able to take human health care to a whole new level.

# Federated Learning and Data Alliance of Enterprises

Federated learning is not only a technology standard but also a business model. When people realize the effects of big data, the first thought that occurs to them is to aggregate the data together, compute the models through a remote processor and then download the results for further use. Cloud computing comes into being under such demands. However, with the increasing importance of data privacy and data security and a closer relationship between a company's profits and its data, the cloud computing model has been challenged. However, the business model of federated learning has provided a new paradigm for applications of big data. When the isolated data occupied by each institution fails to produce an ideal model, the mechanism of federated learning makes it possible for institutions and enterprises to share a united model without data exchange. Furthermore, federated learning could make equitable rules for profits allocation with the help of consensus mechanism from blockchain techniques. The data possessors, regardless of the scale of data they have, will be motivated to join in the data alliance and make their own profits. We believe that the establishment of the business model for data alliance and the technical mechanism for federated learning should be carried out together. We would also make standards for federated learning in various fields to put it into use as soon as possible.

# Conclusions and Prospects

In recent years, the isolation of data and the emphasis on data privacy are becoming the next challenges for artificial intelligence, but federated learning has brought us new hope. It could establish a united model for multiple enterprises while the local data is protected, so that enterprises could win together taking the data security as premise. This article generally introduces the basic concept, architecture and techniques of federated learning, and discusses its potential in various applications. It is expected that in the near future, federated learning would break the barriers between industries and establish a community where data and knowledge could be shared together with safety, and the benefits would be fairly distributed according to the contribution of each participant. The bonus of artificial intelligence would finally be brought to every corner of our lives.

[^1]: Definition of data security may differ in different scenarios, but is required to provide meaning privacy guarantees. We demonstrate examples of security definitions in section 2.3