abstract: As NLP models become more integrated with the everyday lives of people, it becomes important to examine the social effect that the usage of these systems has. While these models understand language and have increased accuracy on difficult downstream tasks, there is evidence that these models amplify gender, racial and cultural stereotypes and lead to a vicious cycle in many settings. In this survey, we analyze the origins of biases, the definitions of fairness and how different subfields of NLP mitigate bias. We finally discuss how future studies can work towards eradicating pernicious biases from NLP algorithms.
author: Rajas Bansal
Stanford University
`firstname.lastname@example.com`
bibliography: acl2020.bib
title: A Survey on Bias and Fairness in Natural Language Processing

# Introduction

Bias is the prior information possessed by AI systems, and contrary to the emotion piqued by the word, it reflects the intelligence held by the system. Yet, this bias can be harmful, when it is the result of harmful precedents . The blind application of AI systems possessing these biases runs the risk of amplifying harmful social biases which existed some 100 years ago.

To better understand this, consider the case of retrieving web pages relevant to a query which utilizes word embeddings. Consider a simple query, like, *IIT Delhi computer science student*. Now, there may be web-pages of hundreds of students which differ in nothing, except the name of the student, like John and Mary. Word embeddings attribute the field of computer science to males . This leads to the search engine ranking John's page above Mary's. This in turn makes women less recognized in Computer Science, thus amplifying the existing bias.

The text corpora that is used to train State-of-the-Art algorithms themselves contain pernicious gender, racial and religious biases. use word embeddings as a quantitative lens through which to study historical trends - specifically trends in the gender and ethnic stereotypes in the United States. They analyze word embeddings trained over 100 years of text corpora and are able to show that their measurement of bias is highly correlated with various social movements, like the women's movement in the 1960s-1970s and Asian-American population growth in the 1960s and 1980s.

The survey is organized as follows: Section outlines the sources of bias in NLP; Section defines the notion of fairness; Section define different metrics used to quantify bias; Section collates how debiasing is approached within different subfields of NLP while Section explains how current techniques are not sufficient; Finally, Section concludes and discussed directions for future work.

# Bias

Data collected from humans in the real world is not homogeneous, as is usually assumed in research. Demographics of the humans labelling the data may itself influence bias the model. Data collected in the real world is heterogeneous, generated by social subgroups with their own characteristics and behaviours. A model trained on biased data may lead to unfair and inaccurate predictions. Current work in NLP focuses on curing the symptoms of these biases, without trying to drill out their origins. This leads to the developed systems fixing only a specific type of the bias (for eg. gender) and not being able to generalize to cure others (like racial biases). This paper presents the origins of the prevalent biases in order to facilitate research sensitive to these. They can be grouped into 4 major groups :-

-   **Label bias** Label bias emerges when the distribution of the labeled variable in the input dataset diverges substantially from the ideal distribution. Here, the label themselves are erroneous. This may be due to the lack of expertise or prejudices held by annotators.

-   **Selection bias** Selection bias emerges due to non-representative observations – when the annotators generating the training data have a different distribution than where the model is to be applied. A famous example being the "Wall Street Journal effect", where syntactic parsers and part-of-speech taggers perform most accurately over language written by middle-aged white men .

-   **Over-amplification** This may occur during the learning process itself, where the model may pick up small difference in human factors, and amplify these to be much larger in the predicted outcomes.

-   **Semantic bias** Embeddings are used as an off-the-shelf source of semantic information in NLP. However, these embeddings themselves contain unintended biases, with words for certain professions like nurses, homemakers and receptionists clustering together near feminine words, and thus act as a source of semantic bias for the downstream model.

Typical word embeddings were found to associate European American names with more pleasant/positive words than African American words, and thus exhibit racial biases. While this racial bias is acknowledged by many papers, the majority of the work in this field focuses on gender bias. Other social biases like religious biases and cultural biases have not been given adequate attention and may require completely new methods to tackle them and may bring new challenges with them.

# Fairness

Fairness is at the heart of the problem but most prior work chooses to ignore it. The definitions have been listed so that future work realizes how different definitions of fairness may alter the problem, and which definition is most suited for their domain.

Consider the task of predicting an output variable Y given an input variable X, while remaining unbiased with respect to some variable Z. Let the predicted distribution be $\hat{Y}$.

-   **Demographic Parity** A predictor $\hat{Y}$ satisfies demographic parity if $\hat{Y}$ and Z are independent. This definition says that the probability of a person being in the positive class should be independent of if the person is male or female.

-   **Equality of Opportunity** A predictor $\hat{Y}$ satisfies equality of opportunity with respect to a class y if $\hat{Y}$ and Z are independent conditioned on Y = y. This says that the true positive rates should be the same for men and women.

-   **Equality of Odds** A predictor $\hat{Y}$ satisfies equality of odds if $\hat{Y}$ and Z are conditionally independent given Y. This definition says that the true positive and false positive rates should be the same for men and women.

The definition of fairness to use is often a political choice, the right prefers Equality of Opportunity while the left prefers Equality of Odds.

# Metrics for Observing Bias

A key problem in eradication of bias is that of quantifying and measuring the bias that may be already present in the training data or model. As quantifying bias is difficult in the symbolic domain, most metrics assume that words have been projected to some vector space before measuring bias.

1.  **Word Embedding Assosiation Test**

    The WEAT test proposed by is the most popular metric used to measure the bias present in word embeddings. The origins of this test are found in the Implicit Association Test (IAT) used by psychologists to show biases existing among humans using reaction time as a quantitative measure. show that human biases are also reproduced in Word2Vec and GLoVe embeddings. For example, male attributes like brother, father and grandfather were found to be more correlated with science-related words than female attributes. This intuition is used to formulate the WEAT test.

    Consider two sets of target words (X,Y) (eg. programmer, engineer etc. and nurse, teacher etc.) and two sets of attribute words (A,B) (eg., he, man etc. and she, woman etc.). The null hypothesis is that there is that the attribute sets are similar to both target sets in terms of word similarity. The test measures the (un)likelihood of this hypothesis. The test statistic is :- $$\begin{gathered}
        s(X,Y,A,B) = \sum_{x \in X} s(x,A,B)\\ - \sum_{y \in Y} s(y,A,B)

    \end{gathered}$$ where, $$\begin{gathered}
            s(w,A,B) = mean_{a \in A}cos(w,a)\\ - mean_{b \in B}cos(w,b)

    \end{gathered}$$ In other words, $s(w, A, B)$ measures the association of $w$ with $A$ as compared to $B$. Different variations of $A, B, X, Y$ are used in different papers.

2.  **Sentence Encoder Association Test**

    This is a simple generalization of the WEAT test for sentence encoders. Here, the target and the attribute sets contain sentences instead of words. To extend the word level test to a sentence level test, words present in the earlier sets are put into semantically bleached sentence templates such as "This is word.","word is here.", "word are things".

3.  **Projection on gender direction** define the gender bias of a word $w$ by its projection on the "gender direction": $\vec{w} \cdot (\vec{he} - \vec{she})$ assuming all vectors to be normalized. The higher the projection on this direction, the more bias the word is said to possess. This measurement is done, assuming that words have been partitioned into two sets gendered words (like brother, mother etc. which contain implicit gender information) and gender neutral words (eg. programmer, homemaker etc which must not contain gender information). This metric is based on the implicit assumption that there is a gender subspace which subsumes all gender information of any word.

4.  **Distance from gender pairs** Some papers use the distance from gender pairs (like he and she) as a measure of the bias in the word. Models based on this metric ensure that gender neutral words like nurse, as defined above, are equidistant from he and she.

5.  **Language Model Bias**

    define the probability of a word occuring in context with gendered words as $$P(w|g) = \frac{\frac{c(w,g)}{\sum_{i}c(w_i,g)}}{\frac{c(g)}{\sum_i c(w_i)}}$$ where $c(w,g)$ is the count of the word in a specified context window and $g$ is the set of gendered words which belong to the male $(m)$ or the female $(f)$ set. The bias score of any word $w$ would then be, $$bias(w) = log(\frac{P(w|f)}{P(w|m)})$$

6.  **Gender Bias Evaluation Test-sets** All of the metrics discussed above quantify bias present in word embeddings, failing to quantify the bias present in the model itself. This is a very narrow view, and Gender Bias Evaluation Test-Sets (GBETs) introduced by provides a solution to this problem. Standard test sets in NLP fail to measure the gender bias in models. These test sets themselves contain biases (such as containing more male references than female references), so evaluation on them might not reveal biases. The goal of designing these GBETs is to provide a standardized dataset to the research community to streamline research and allow them to measure biases present in their algorithms.

    **Gender Swapped GBETs** The most commonly used method to construct these GBETs is gender swapping. Gender swapping is done by replacing every male definitional word with its female equivalent and vice versa. A system free from bias must give the same performance on both the male and female swapped test sets. The difference in score here acts as a proxy for the gender bias in the system.

    For coreference resolution, and independently designed GBETs based on the gender swapping techniques and is discussed in detail later. For sentiment analysis, a GBET dataset named Equity Evaluation Corpus is designed where every sentence in the dataset contains a gendered word along with an emotion carrying word. Gender bias is measured as the difference in emotional intensity predictions between gender-swapped datasets.

# Approaches for Debiasing Models

There have been numerous attempts to eradicate bias and bring fairness into NLP models for solving a variety of sub-problems like coreference resolution, semantic role labeling and machine translation. Most effort in this direction however has been in debiasing word embeddings which are used as an off-the-shelf source of semantic knowledge. Thus in the first subsection we discuss the different methods used to debias word embeddings. We then take a number of subproblems in NLP to emphasize the applications of these techniques to different domains.

Generally, methods that target bias fall into three large categories :-

1.  **Pre-processing** These methods acknowledge the bias that is present in the training data itself and try to alleviate this bias by transforming the input to the model itself. The downstream training algorithm is unchanged.

    identify and remove the articles from Wikipedia and the New York Times articles corpus which lead to the most increase in bias of the downstream GLoVe word embeddings. They use influence functions to approximate how the WEAT score changes due to the removal of one document from the training data. Then, the documents whose removal causes the most decrease in bias are removed. As removal of one document causes an imperceptibly small decrease in bias, a set of documents are considered for removal rather than a single document.

2.  **Post-processing** Sometimes, re-training the model is infeasible as it may require a large amount of resources to train the model. In such a case, post-processing methods which alter the output without considering bias at the time of training are useful. The learned model is treated as a black box in this case and the labels assigned by the model are altered in order to improve bias scores. Most methods discussed in this survey are post-processing methods.

3.  **In-processing** In-processing techniques try to modify and change state-of-the-art learning algorithms in order to remove discrimination during the model training process.

    propose a Gender Neutral Variant of GLoVe training. They point out that post-processing methods remove gender information from embeddings which may be useful for some settings such as medicine. These methods also require tagging gender neutral words which may lead to pipelining errors due to wrong identification of such words. The authors' Gender Neutral GLoVe training concentrates all socially-biased information in certain dimensions which can easily be removed if such an information is not required. Thus a word vector $w$ consists of two parts, $w = [w^{(a)};w^{(g)}]$. All gender information is concentrated into $w^{(g)}$, freeing $w^{(a)}$ from any gender influence. This is done by adding two additional loss terms to the GLoVe learning objective. One, which separates out the $w^{(g)}$ of male-definition and female-definition words far from each other (by maximizing the distance between them). The second encourages the $w^{(a)}$ of the neutral words to be in the null space of the gender direction, $v_g$. $v_g$ is estimated on the fly by averaging the difference between the male definition and the female definition words.

    ## Word Embeddings

    Word embeddings are a vector representation of a symbolic domain i.e words and are a fundamental component in many NLP systems. However, these contain 100 years of gender and ethnic sterotypes . Thus, a lot of attention has been paid to debiasing these vectors. However, view this debiasing with skepticism. They view AI as perception followed by action, and they feel that debiasing alters the AI's perception and model of the world, rather than how it acts on this perception. Prejudice can anyway creep back in due to other proxies which may not be apparent .

    ### Removing Gender Subspace

    The work in this line of thinking is based on the hypothesis that all gender, racial and religious biases and information can be subsumed by a particular subspace of word embeddings. These algorithms thus try to reduce the information that the gender neutral words possess in the gender-biased subspace. Debiasing involves two steps :-

    1.  **Identify gender subspace**

    2.  **Neutralize vectors**

    First all words are separated into male-definitional words ($\Omega_m$), female-definitional words ($\Omega_f$) and gender neutral words ($\Omega_n$). Many words in $\Omega_m$ and $\Omega_f$ are paired together into complements (for eg. (he,she), (brother,sister), (John,Mary) etc.). The difference in the vector of these words i.e $\vec{he} - \vec{she}$ acts as the gender direction. Instead of taking just one difference, general practice is to take all such difference and run a PCA algorithm, which yields the subspace which explains the most variation in this space of gender direction. This is done by stacking all the above differences into a matrix **C**. The bias subspace, B is then defined to be the first $k$ rows of SVD(**C**).

    The neutralize step removes gender bias from all the vectors, $\vec{w} \in \Omega_n$. It does this by re-embedding all the word vectors through the following process :- $$\vec{w} := \frac{\vec{w} - \vec{w_B}}{||\vec{w} - \vec{w_B}||}$$ where $\vec{w_B}$ is the projection of w onto the bias subspace B. This removes the gender component from all neutral words.

    ### Data Augmentation

    Sometimes the training dataset may have a disproportionate number of references to one gender. To mitigate this, propose gender swapping in the training dataset which equalizes the references to each gender in this dataset. This is different from a GBET as GBETs were used to measure the bias that may be present in the model while this approach debias predictions by training on a gender-balanced dataset. When applied to coreference resolution model originally trained on OntoNotes 5.0 which was tested on WinoBias, gender augmentation lowered the difference between F1 scores on pro- and anti-stereotypical test sets significantly, which indicates the model does not make predictions motivated by gender.

    Complementary to augmentation, propose removing those documents which contain the most amount of bias. They show that removing the 10 most influential (according to their metric) documents from the training set led to a 40% decrease in the WEAT effect size.

    ## Sentence Encoding

    Recently, opposed to word embeddings, sentence representations thorugh ELMo, BERT and GPT have become the prefered choice for encoding text. As these models contain contextual word embeddings in place of a single embedding for each word, traditional WEAT tests cannot be used here. Bias is measured in such cases using the SEAT test . The SEAT test is a very crude generalization of the WEAT test as discuessed. show that the biases that were shown by are reproduced for BERT and ELMo. The Hard Debias method estimates the bias subspace by computing the PCA of gendered sentence templates followed by removal of the component of the sentence encoding in the bias subspace.

    While many papers show that these debiasing methods reduce SEAT scores, they also report that the SEAT does not seem to be an adequate metric. Certain counter-intuitive trends are observed in the bias present in sentence embeddings as compared to the word embeddings. This may be due to two reasons, one, the templates formed are not as semantically bleached as expected and two, cosine similarity is not a good metric to measure sentence similarity as training for BERT and ELMo has been done on different objectives. Thus a better measure of bias in sentence representations is required.

    propose a new measure of bias for sentence representations which shows better correlation with human biases than SEAT and addresses some of the problems mentioned above. The process for measuring bias by correlating a \[TARGET\] (gendered word, in this case, he) to an \[ATTRIBUTE\] (in this case programmer) is as follows :-

    1.  Prepare a template sentence e.g. "\[TARGET\] is a \[ATTRIBUTE\]"

    2.  Replace \[TARGET\] with \[MASK\] and compute $$p_{tgt}= P([MASK]=[TARGET])$$

    3.  Replace both \[TARGET\] and \[ATTRIBUTE\] with \[MASK\], and compute prior probability $$p_{prior} = P([MASK]=[TARGET])$$

    4.  Compute the association between target and attribute as $log(\frac{p_{tgt}}{p_{prior}})$

    The bias can be calculated by subtracting the association scores for he/she.

    ## Coreference Resolution

    Coreference resolution is the task aimed at identifying phrases (mentions) refering to the same entity. created a novel challenge corpus, WinoBias which follows the winograd format and contains references to people using a vocabulary of 40 occupations. Figure 1 shows the 2 types of sentences that are present in the WinoBias corpus.

    Typical systems were found to give poor accuracy on the WinoBias dataset. This is because the widely used OntoNotes 5.0 dataset uses male gendered pronouns to refer to 80% entities. In order to reduce this bias, an additional training corpus was created by swapping all male entities for female entities and vice versa. The SoTA end-to-end system was then trained on the original augmented with the swapped corpus. Names were anonymized to remove the bias that a gendered name may carry with itself. The authors were able to show that the SoTA algorithms performed poorly on the WinoBias dataset. The anonymization, usage of bias free embeddings and the augmentation of data, as discussed were enough to improve the performance of these systems on the dataset. As the WinoBias evaluation may be out of domain for a system trained on OntoNotes, the authors also compared the difference in accuracy of all systems on the OntoNotes dev set and its gender swapped version. The debiased system did not show a significant difference in accuracy on swapping the gender.

    ## Machine Translation

    The example most widely used to show bias in machine translation is the English sentence "She works in a hospital, my friend is a nurse". Here my friend is translated to "amiga" (girl friend) in Spanish, while the sentence "She works in a hospital, my friend is a doctor" translates my friend to amigo (boy friend) in Spanish. carry out a study of biases in MT from English to Spanish. In addition to the corpus available from WMT, they have created an additional test set to study gender bias. Motivated from the above example the test set is built using a sentence pattern "I've known {her, him, proper-noun} for a long time, my friend works as a {occupation}." They used debiased word embeddings constructed through methods mentioned above and were able to show a huge improvement on the custom test set over the SoTA baseline. In fact, they were able to show improvement of 1 BLEU point over the original test set showing that the performance does not degrade when one uses these debiased embeddings.

    Human translators rely on contextual information to infer certain aspects like gender of the speaker in order to make the best translation. In absence of such context, like the identity of the speaker, NMT make the statistically most likely choice which may be gender biased. thus give more emphasis on speaker information being present in parallel data. They have thus released a large corpus of parallel data tagged with speaker information (name, gender, age, date of birth, euroID and date of the session). They then trained a NMT system which was given the gender of the speaker as the first token. The authors were able to show a 1 point improvement in BLEU score with the most improvement seen in sentences having female speakers.

    ## Semantic Role Labeling

    consider the task of visual semantic role labeling (vSRL) where the goal is to predict activities and the roles that objects play in it. Biases exist in such a setting too. In the imSitu training set, only 33% of cooking images have man as the agent role. After training a CRF, this bias is amplified: less than 16% of cooking images are labeled with man as the agent. They use a novel constrained inference framework called RBA (Reduced Bias Amplification). Here the Language Model bias formulation (4.5) is used to measure bias of the system. Bias amplification is measured by comparing the bias on the training set to that on the test set.

    In order to reduce bias amplification, the authors inject constraints enforcing that the test bias should remain in certain limits of the training bias. These constraints need to be applied at the corpus level and thus would require solving a large inference problem. Therefore, an approximate inference algorithm based on Lagrangian relaxation is used. Let $b^{star}$ be the gender ratio present in the training set. Let $y^{i}_{M}$ denote the $i^{th}$ test instance being assigned the agent as man. Then the constraints that we want to impose are $$b^{\star} - \gamma \leq \frac{\sum_{i}y^{i}_M}{\sum_{i}y^{i}_M + \sum_{i}y^{i}_W} \leq b^{\star} + \gamma$$ where $\gamma$ is an adjustable hyperparameter representing the error limit. These constraints can be generalized to $A\sum_i y^i - b \leq 0$ and a Lagrangian formulation can then be used to solve the problem.

    ## Hate speech

    Social media platforms are being criticized for being echo chambers of hate speech. However automated removal of such messages leads to suppressing the already marginalized voice of minorities. The task is especially challenging because what is considered toxic inherently depends on social context (e.g., speaker's identity or dialect). Terms earlier deemed offensive (e.g., "n\*gga", "queer") are no longer so if used by people from these groups . As race information is not available on Twitter, the authors use a proxy for identifying race, i.e the African American English dialect to signal race. The authors showed that the usage of the AAE dialect was shown to have high correlation with a tweet being classified as being hateful. Grouping tweets into White aligned and AAE aligned, the authors show that the false positive rates for AAE was 46% as compared to 9% for White aligned tweets.

    ## Adversarial Learning Algorithms

    Learning an generative adversarial network for the mitigating bias seems to be the most generalizable approach. model a variation of GANs for mitigating biases.

    In the context of learning word analogies, the generator learns the answers to simple word analogies while the discriminator tries to infer the gender from the prediction yielded by the generator. The generator attempts to prevent the discriminator from identifying the gender and will thus yield solutions which are less plagued by bias as they demonstrate in their paper.

# Why are current techniques not sufficient?

point out that current methods of debiasing word vectors show promising scores on their seld-defined metrics. However, the removal is quite superficial. The actual effect is just hiding the bias, not removing it. Tests like WEAT and SEAT are good to check the presence of bias, if any, however small WEAT scores do not mean that bias has been removed. The gender bias is still reflected in the debiased embeddings, and can be recovered from any downstream model. They find that:-

1.  Words with strong previous gender bias (with the same direction) are easy to cluster together.

2.  The words with neutral gender, for eg receptionist, hairdresser which are sterotypically associated with the female gender still cluster together similar to the non-debiased word embeddings.

3.  A downstream classifier is still able to recover the implicit gender that was present in the debiased vectors just from their vectors and no additional information.

Gender-direction which is the basis of gender association provides a way to measure the gender association of a word, but does not determine it. New metrics are needed in order to allow a metric to allow checking of bias in embeddings. Difference of accuracies over a gender-swapped GBET would be a good metric to check the bias but would not be a good method to train a network.

# Conclusion and Future Work

We looked at the different types of bias that exist in today's machine learning systems and how deploying them in the real world leads to harmful outcomes. We then tried to define fairness formally and showed how different definitions lead to different models. We showed the different metrics used to measure bias that may be present in word embeddings or models. We finally looked at a wide range of subfields of NLP where debiasing is done and the different methods through which this is realized. Finally, pointed out that these metrics require a close look as they act as good indicators but do not determine it.

This field of bias mitigation in NLP has recently come up with the increase in the real word deployment of AI systems. The black box nature of NLP systems amplifies any problem that may exist due to bias, as the people using the system are not aware of the reasoning behind a prediction, they just know the final answer. This is a hot field with new innovations coming up everyday, however there is still a lot of scope for work in this space.

-   Most research has been restricted to measuring gender bias. However, there is little research in measuring bias due to race, religion or class, as these are more difficult to measure and may be less prevalent in typically used corpora.

-   Most systems assume that we know which kind of harmful bias exits in the corpora. However, this may not always be true, as sometimes, the sources of bias may not be obvious. For eg:- A resume filtering system may be biased against hiring people from a certain locality, which may not be obvious to those who use the system.

-   Current metrics, like WEAT measure the gender association of a gender neutral word, but do not determine the bias it contains. That is, some systems may be reducing the scores relating to these metrics but the system may still be making biased predictions. New metrics which determine human bias are required.

-   The Computer Science community, conducting such research, may not be well versed with the origins and the problems that bias can bring about in the real world. Thus more inter-disciplinary research with social scientists may lead to new insights. A hard-hitting truth is that this research itself does not come from culturally or socially diverse backgrounds.

-   Work in hate-speech and machine translation has uncovered that speaker information is very important in order to remove biases from the system. However, most corpora strip away the original source, or the demographics of the person writing the article or the context in which the article was published. This origin information is invaluable and may lead to removal of some biases.


abstract: In recent years, deep neural networks have revolutionized many application domains of machine learning and are key components of many critical decision or predictive processes. Therefore, it is crucial that domain specialists can understand and analyze actions and predictions, even of the most complex neural network architectures. Despite these arguments neural networks are often treated as black boxes. In the attempt to alleviate this shortcoming many analysis methods were proposed, yet the lack of reference implementations often makes a systematic comparison between the methods a major effort. The presented library ***iNNvestigate*** addresses this by providing a common interface and out-of-the-box implementation for many analysis methods, including the reference implementation for PatternNet and PatternAttribution as well as for LRP-methods. To demonstrate the versatility of *iNNvestigate*, we provide an analysis of image classifications for variety of state-of-the-art neural network architectures.
author: Maximilian Alber firstname.lastname@example.com
Technische Universität Berlin, Machine Learning Group
10623 Berlin, Germany Sebastian Lapuschkin firstname.lastname@example.com
Fraunhofer Heinrich Hertz Institute, Video Coding and Analytics
10587 Berlin, Germany Philipp Seegerer email@example.com
Miriam Hägele email@example.com
Kristof T. Schütt firstname.lastname@example.com
Grégoire Montavon firstname.lastname@example.com
Technische Universität Berlin, Machine Learning Group
10623 Berlin, Germany Wojciech Samek email@example.com
Fraunhofer Heinrich Hertz Institute, Video Coding and Analytics
10587 Berlin, Germany Klaus-Robert Müller firstname.lastname@example.com
Technische Universität Berlin, Machine Learning Group
10623 Berlin, Germany
Korea University, Department of Brain and Cognitive Engineering
Seoul 02841, Korea
Max Planck Institute for Informatics
66123 Saarbrücken, Germany Sven Dähne firstname.lastname@example.com
Pieter-Jan Kindermans firstname.lastname@example.com
Technische Universität Berlin, Machine Learning Group
10623 Berlin, Germany
bibliography: biblio.bib
title: iNNvestigate neural networks!

# Introduction

In recent years deep neural networks have revolutionized many domains, e.g., image recognition, speech recognition, speech synthesis, and knowledge discovery . Due to their ability to naturally learn from structured data and exhibit superior performance, they are increasingly used in practical applications and critical decision processes, such as novel knowledge discovery techniques, autonomous driving or medical image analysis. To fully leverage their potential it is essential that users can *comprehend and analyze* these processes. E.g., in neural architecture  or chemical compound searches  it would be extremely useful to know which properties help a neural network to choose appropriate candidates. Furthermore for some applications understanding the decision process might be a legal requirement.

Despite these arguments neural networks are often treated as black boxes, because their complex internal workings and the basis for their predictions are not fully understood. In the attempt to alleviate this shortcoming several methods were proposed, e.g., Saliency Map , SmoothGrad , IntegratedGradients , Deconvnet , GuidedBackprop , PatternNet and PatternAttribution , LRP , and DeepTaylor . Theoretically it is not clear which method solves the stated problems best, therefore an empirical comparison is required . In order to evaluate these methods, we present ***iNNvestigate*** which provides a common interface to a variety of analysis methods.

In particular, *iNNvestigate* contributes:

-   A common interface for a growing number of analysis methods that is applicable to a broad class of neural networks. With this instantiating a method is as uncomplicated as passing a trained neural network to it and allows for easy qualitative comparisons of methods. For quantitative evaluations of (image) classification task we further provide an implementation of the method "perturbation analysis" .

-   Support of all methods listed above—this includes the first reference implementation for PatternNet and PatternAttribution and an extended implementation for LRP—and an open source repository for further contributions.

-   A clean and modular implementation, casting each analysis in terms of layer-wise forward and backward computations. This limits code redundancy, takes advantage of automatic differentiation, and eases future integration of new methods.

*iNNvestigate* is available at repository: <https://github.com/albermax/innvestigate>. It can be simply installed as Python package and contains documentation for code and applications. To demonstrate the versatility of *iNNvestigate* we provide examples for the analysis of image classifications for a variety of state-of-the-art neural networks.

#### Terminology

The different methods pose different assumption to tasks and are designed for different objectives, yet they are related to "explaining" or "interpreting" neural networks (see ). We actively refrain from using this terminology in order to prevent misunderstandings between the design choices of the algorithms and the implicit assumption these terms bring along. Therefore we will solely use the neutral term *analyzing* and leave any interpretation to the user.

# Library

#### Interface

The main feature is a common interface to several analysis methods. The workflow is as simple as passing a Keras neural network model to instantiate an analyzer object for a desired algorithm. Then, if needed, the analyzer will be fitted to the data and eventually be used to analyze the model's predictions. The corresponding Python code is:

    import innvestigate
    model = create_a_keras_model()
    analyzer = innvestigate.create_analyzer("analyzer_name", model)
    analyzer.fit(X_train) # if needed
    analysis = analyzer.analyze(X_test)

#### Implemented methods

At publication time the following algorithms are supported: Gradient Saliency Map, SmoothGrad, IntegratedGradients, Deconvnet, GuidedBackprop, PatternNet and PatternAttribution, DeepTaylor, and LRP including LRP-Z, -Epsilon, -AlphaBeta. In contrast, current related work is limited to gradient-based methods. We intend to further extend this selection and invite the community to contribute implementations as new methods emerge.

#### Documentation

The library's documentation contains several introductory scripts and example applications. We demonstrate how the analyses can be applied to the following state-of-the-art models: VGG16 and VGG19 , InceptionV3 , ResNet50 , InceptionResNetV2 , DenseNet , NASNet mobile, and NASNet large . Figure  shows the result of each analysis on a subset of these networks.

## Details

#### Modular implementation

All of the methods have in common that they perform a back-propagation from the model outputs to the inputs. The core of *iNNvestigate* is a set of base classes and functions that is designed to allow for rapid and easy development of such algorithms. The developer only needs to implement specific changes to the base algorithm and the library will take care of the complex and error-prone handling of the propagation along the graph structure. Further details can be found in the repositories documentation.

Another advantage of the modular design is that one can extend any analyzer with a given set of wrappers. One application of this is the smoothing of the analysis results by adding Gaussian noise to the copies of the input and averaging the outcome. E.g., SmoothGrad is realized in this way by combining a smoothing wrapper with a gradient analyzer.

#### Training

PatternNet and PatternAttribution  are two novel approaches that condition their analysis on the data distribution. This is done by identifying the signal and noise direction for each neuron of a neural network. Our software scales favorably, e.g., one can train required patterns for the methods on large datasets like Imagenet  in less than an hour using one GPU. We present the first reference implementation of these methods.

#### Quantitative evaluation

Often analysis methods for neural networks are compared by qualitative (visual) inspection of the result. This is can lead to subjective evaluations and one approach to create a more objective and quantitative comparison of analysis algorithms is the method "perturbation analysis" . The intuition behind this method is that perturbing regions which are recognized as important for the classification task by the analyzing method, will impact the classification most. This allows to assess which analysis method best identifies regions that matter for a specific task and neural network. *iNNvestigate* contains an implementation of this method.

#### Installation & license

*iNNvestigate* is published as open-source software under the MIT-license and can be downloaded from: <https://github.com/albermax/innvestigate>. It is build as a Python 2 or 3 application on top of the popular and established Keras  framework. This allows to use the library on various platforms and devices like CPUs and GPUs. At the time of publication only the TensorFlow  Keras-backend is supported. The library can be simply installed as Python package.

# Conclusion

We have presented *iNNvestigate*, a library that makes it easier to analyze neural networks' predictions and to compare different analysis methods. This is done by providing a common interface and implementations for many analysis methods as well as making tools for training and comparing methods available. In particular it contains reference implementations for many methods (PatternNet, PatternAttribution, LRP) and example application for a large number of state-of-the-art applications. We expect that this library will support the field of analyzing machine learning and facilitate research using neural networks in domains such as drug design or medical image analysis.


Optical resonators with low sensitivity to temperature and mechanical forces are of significant importance for precision measurements in the optical and microwave frequency domain. In the optical domain, they serve to stabilize the frequencies of lasers for spectroscopic applications, notably for optical atomic clocks, and for probing fundamental physics issues such as the properties of space-time. Also, by conversion of ultrastable optical frequencies to the radio-frequency domain via an optical frequency comb, radio-frequency sources with ultra-low phase noise can be realized , leading to e.g. radar measurements with improved sensitivity.

The conventional approach for ultra-stable optical resonators is the use of ULE (ultra-low expansion glass) material, operated at temperatures near room temperature, where the coefficient of thermal expansion (CTE) exhibits a zero crossing. While ULE resonators with optimized designs (long length, acceleration-insensitive shape) have reached impressive performance , their operating temperature near 300 K necessarily leads to a level of Brownian length fluctuations which imposes a fundamental limit to the achievable frequency stability ,. Cryogenic operation of a resonator provides one avenue towards reduction of these fluctuations. The Allan deviation of length fluctuations decreases proportional to $\sqrt{T}$ , if the mechanical dissipation of the resonator elements, in particular of the mirror coatings, is independent of temperature. Measurements performed thus far indicate that the dissipation of mirrors with crystalline substrates at cryogenic temperature are indeed similar to those of fused silica mirrors at room temperature ,. Nowadays, robust cryogenic solutions exist for continuous operation of even fairly large objects, such as optical resonators, at temperatures as low as 0.1 K. This offers the possibility of reduction of resonator length fluctuations by more than one order of magnitude compared to today's lowest levels realized at room temperature, with a corresponding reduction in frequency instability of the laser stabilized to the resonator. A second outstanding feature of crystalline cryogenic optical resonators is the absence of length drift thanks to the near-perfect lattice structure.

Cryogenic optical resonators made of single-crystal sapphire have been developed early on and operated at temperatures as low as 1.4 K . Extremely small thermal expansion and long-term drift were achieved . Such resonators have been applied for tests of Lorentz invariance , local position invariance and quantum space-time fluctuations . Silicon, a machinable optical material available with high purity and large size, having interesting CTE properties , high stiffness and low mechanical dissipation , has first been used for an optical reference resonator by Richard and Hamilton . Recently, Kessler et al.  developed a laser frequency stabilization system based on a vertically supported silicon resonator operated at a temperature of zero CTE, 124 K, and achieved a high frequency stability ($1\times10^{-16}$), less than 40 mHz laser linewidth, and an extremely low long-term drift.

In this work, we present the first silicon optical resonator for absolute laser frequency stabilization operated at cryogenic temperature and discuss its thermal properties. We find that they are compatible with the goal of achieving frequency instability at the $2\times10^{-17}$ level.

Our silicon resonator consists of a cylindrical spacer of 250 mm length, 70 mm diameter and 15 mm diameter central hole. One mirror is flat, while the other has 1 m curvature radius. Spacer and mirror substrates were manufactured from a dislocation-free float-zone silicon crystal with \[110\] orientation along the cylinder axis. The substrates and spacer faces were super-polished to a residual surface roughness less than 0.1 nm. The mirror substrates were coated with a high-reflectivity coating for 1.5 $\mu$m, and optically contacted to the spacer with the same orientation as in the spacer crystal. By evaluation of a ring-down of the laser power transmitted through the resonator at 1.5 K, the finesse and the linewidth of the resonator were determined to be $2\times 10^{5}$ and 3 kHz, respectively. The coupling efficiency into the resonator is 15%.

A special structure for horizontal support of the resonator was implemented (see Fig. ). It was designed for small acceleration sensitivity of the resonator's length and small stress imposed by the different CTE of resonator and supporting structure, and for blocking thermal radiation entering the cryostat through the optical window. The design is based on the concept developed in Ref. . The resonator is attached to a copper frame by ten 1 mm diameter stainless steel wires. Their position was optimized by the finite-element method (FEM, Comsol) in order to minimize acceleration sensitivity along the three axes. The estimated sensitivities are $3\times10^{-10}/g$ and $3\times10^{-11}/g$ along the resonator optical axis and perpendicular to it, respectively. We also simulated the influence of the supporting structure on the CTE of the resonator. It is a function of temperature, due to the temperature dependence of the CTE of both the resonator and the frame. The results indicate negligible dependence of the CTE of the resonator ($\alpha_{reson})$ on a change in frame temperature at temperatures below 16.8 K. The CTE zero crossing points at 17 K and 124 K are shifted by 2 mK and 10 mK, respectively, due to the contribution from the frame.

The frame was rigidly attached to an optical breadboard inside a cryostat. A 1.5 $\mu$m fiber laser was coupled to the resonator through an optical window. The frequency was stabilized to a TEM$_{00}$ mode of the resonator using the Pound-Drewer-Hall (PDH) technique , with a standard optical scheme and using a waveguide phase modulator. The laser power circulating inside the resonator was stabilized by detection of the light transmitted through the resonator and applying a feedback signal to an acousto-optical modulator in the optical setup.

The resonator unit was installed inside a low-vibration, two-stage pulse tube cooler cryostat with Joule-Thomson stage. It allowed a lowest operating temperature of 1.4 K and a cooling power of 20 mW. Two calibrated Cernox sensors (inaccuracy less than 10 mK at temperatures below 30 K) were available for controlling and/or monitoring the temperature of the supporting breadboard and of the resonator. One sensor was attached to the center of the breadboard, while the other was attached to the upper central part of the resonator. An AC resistance bridge was used to read out the temperature sensors. A resistance heater installed on the center of the breadboard was used for setting and maintaining a desired operating temperature. When the temperature of resonator was stabilized by controlling the resistance of the resonator sensor, the measured instability at the sensor location was $3.6\times10^{-5}$ K at $\tau=1$ s averaging time, dropping as $3.6\times10^{-5} K/(\tau/1s)^{1/2}$ for $\tau$ up to $10^4$ s. We estimate that the temperature instability of the average temperature of the resonator is within a moderate factor of the above number, since the very high thermal diffusivity of silicon at cryogenic temperature allows for a rapid thermal equilibration within the resonator. The AC bridge electronics contributes a specified systematic error of $1.7\times10^{-5}$ K per degree variation of the ambient temperature. Thus, in a laboratory stabilized to 0.5 K, the resonator temperature systematic shift is less than $1\times10^{-5}{\rm K}$.

Measurements of the absolute optical resonator frequency were done by measuring the frequency of the laser stabilized to it with respect to a hydrogen maser, using a commercial erbium-doped fiber laser frequency comb. A comb mode near 1064 nm was phase-locked to a 1064 nm optical reference having $2\times10^{-15}$ short-term frequency instability. This reduced significantly the linewidth of comb modes in the 1.5 $\mu$m spectral range and permitted achieving a beat note between the Si-resonator-stabilized laser and the comb with a linewidth as low as 1.7 kHz. The laser light was sent from the laser to the comb via a 150 m long fiber. A typical resonator frequency time trace and the corresponding instability, with all parameters including resonator temperature held constant, is depicted in Fig. . For integration times longer than 1000 s the frequency instability was approximately 3 Hz ($2\times10^{-14})$. The contributions to this instability will be subject of future investigations.

Given this level of absolute frequency instability and the good resolution of the temperature sensors, the resonator's thermal expansion coefficient could be accurately measured. The measurements were done either by heating the resonator or by letting it cool down to 1.5 K after heating it up to a desired temperature. The rate of temperature change was approximately 1.5 K/h at temperatures above 1.8 K and much lower for temperatures below 1.8 K, where it was limited by the cooling capacity of the cryostat. No significant discrepancies were found between the two measurement procedures.

The temperature dependence of the resonator frequency is presented in Fig. . The total change in frequency from 1.5 K to 23.8 K is 6 MHz, with most of the shift occurring in the region above 20 K. A minimum of the frequency occurs at $T_{\alpha=0}=16.81$ K with negative frequency derivative $df/dT$ (length expansion) below this temperature and positive (length contraction) above it . Between 1.6 K and 2 K the shift in frequency is only 100 Hz $(5\times10^{-13})$.

Whereas the CTE of an insulating crystal at low temperatures is expected to be positive and proportional to $T^{3}$ , in the case of silicon its thermal expansion behaviour (expansion turns into contraction between 16.8 K and 124 K) necessarily implies that such a dependence can accurately hold only for $T\rightarrow0$. The value predicted from compressibility and heat capacity measurements is $\alpha_{Si}=$ $4.8\times10^{-13}{\rm K}^{-1}(T/{\rm K})^{3}$ . As displayed in Fig. , a pure $T^{3}$ dependence does not fit the data well. Therefore, the data were fitted with a function containing additional terms. At the lowest temperatures, we find $\alpha_{reson}(T\rightarrow0)=
(1.1 T^{3}-7.8 T^{4}+8.4 T^{5})\times10^{-13}\,{\rm K^{-1}}$, where $T$ is the temperature in Kelvin. Our measured $\alpha_{reson}$ is thus lower then the predicted value for bulk silicon at temperatures below 3 K. At 1.6 K, the value is $\alpha_{reson}=4.6\times10^{-13}/{\rm K}$.

The frequency data around the frequency minimum at $T_{\alpha=0}=16.8\,{\rm K}$ were accurately fitted by a fourth-order polynomial from which we obtain the resonator expansion coefficient $\alpha_{reson}(T\simeq T_{\alpha=0})=\sum_{n=0}^{2}B_{n}(T/{\rm K})^{n}$ with $B_{n}=(37\times10^{-9},$ $-4.94\times10^{-9},$ $1.64\times10^{-10})\,{\rm K^{-1}}$. The complete dataset from 1.6 K to 23.8 K was fitted with a ninth-order polynomial (Fig. b), from which we obtain the resonator thermal expansion coefficient $\alpha_{reson}(T)$ shown in Fig.  b). It is given by $\alpha_{reson}(T)=\sum_{n=0}^{8}A_{n}(T/K)^{n},$ with $A_{n}=$ $(-9\times10^{-22},\,4\times10^{-20},\,-7\times10^{-19},\,2\times10^{-17},\,-3\times10^{-16},\,2\times10^{-15},\,-1\times10^{-14},\,3\times10^{-14},\,-2\times10^{-14})\,{\rm K^{-1}}$.

To our knowledge, only two measurements of thermal expansion of silicon at cryogenic temperature, down to 12 K and 6 K were performed previously. However, due to the difficulty of measuring the corresponding small CTEs, accurate CTE values were only given above 13 K , where they are larger than $1\times10^{-9}/{\rm K}$. Our value of $T_{\alpha_{reson}=0}=16.8\,{\rm K}$ is similar, but not identical to that of Ref. , where 17.8 K was measured. Our expansivity temperature derivative at 16.8 K is $d\alpha_{reson}(T_{\alpha_{reson}=0})/dT=-6.0\times10^{-10}/{\rm K^{2}}$, which is comparable to $-8.9\times10^{-10}/{\rm K^{2}}$ at 17.8 K in Ref. . This expansivity derivative at 16.8 K is nearly a factor 20 smaller than the one at the second zero-CTE temperature 124.2 K, $1.7\times10^{-8}/{\rm K^{2}}$ . This means that 16.8 K could also be a candidate operating temperature, with the advantage of less temperature sensitivity, possibly smaller thermal noise, as compared to 124 K, and less stringent requirements on cryostat construction. As mentioned above, the mirror coating dissipation at this temperature is similar to the room-temperature values for conventional mirror substrates.

With an estimated upper limit of $1\times10^{-4}$ K resonator temperature instability at 1 s (discussed above) and the measured thermal expansion $\alpha_{reson}(T=1.6\,{\rm K})=4.6\times10^{-13}\,{\rm K^{-1}}$, the corresponding fractional frequency instability is about $5\times10^{-17}$ at 1 s and drops to $1\times10^{-17}$ at $\tau=30\,$s. The fractional frequency shift error due to AC bridge systematics is expected to be less than $1\times10^{-17}.$

Fluctuations of the laser power incident on the resonator introduce changes in the power dissipated on the resonator mirrors and result in unwanted fluctuations in resonator length due to the thermal expansion of the mirrors and of the spacer. We measured the resonator frequency change caused by a varying level of intra-resonator power at two different operating temperatures. We could not observe any effect, setting an upper limit of 20 Hz change for a 30$\%$ power change, both at 1.6 K and at 16.8 K. This upper limit corresponds to approximately $3\times10^{-14}/\mu{\rm W}$.

Because of the absence of measurable effect, we also simulated the effect of laser heating using FEM and found a 1.4 fm mirror distance change for 10 $\mu$W power dissipated on each mirror at 1.5 K, i.e. a fractional resonator frequency change of $3\times10^{-16}/\mu{\rm W}$ total dissipated power. The simulation also shows that thermal equilibrium of the heated mirrors and the spacer is reached with a time constant of approximately 1 s in case of good thermal contact with the breadboard. For our experiment, the power level dissipated in the resonator is estimated as 1.5 $\mu$W. If the transmitted laser power is actively stabilized to 1% or better, a level that is expected to be well feasible using an advanced photodetector, the corresponding fractional frequency fluctuations would be $5\times10^{-18}$ or less.

In summary, we demonstrated a silicon optical resonator for laser frequency stabilization that can be operated at temperatures between 1.5 K and 24 K and investigated its thermal properties. Stable, long-term operation was achieved at temperature as low as 1.4 K. The support structure for the resonator allows to take advantage of the low CTE of bulk silicon. The thermal expansion measurement was performed both at the lowest absolute temperature and with the highest sensitivity for a silicon object so far. The influence of finite thermal expansion coefficient and residual temperature instability on the cavity frequency was determined to be at the $5\times10^{-17}$ fractional level and below, depending on averaging time. The sensitivity to changes in circulating laser power is expected to be controllable to the level better than $1\times10^{-17}$ using an appropriate power stabilization unit. For this resonator, at the temperature 1.6 K, the expected total Brownian noise-induced frequency instability is calculated to be $6\times10^{-18}$, assuming a coating loss angle $\varphi=1$ mrad, as determined for silicon mirrors at 20 K in Ref. .

We conclude that the thermal properties of the described system should allow to stabilize the frequency of a laser to an instability of less than $2\times10^{-17}$ for integration times larger than 30 s, taking into account that the instability arising from the laser frequency locking system itself can be reduced to the level of $1\times10^{-17}$ .

We are very grateful to Timo Müller (Wacker Chemitronic) for providing the crystal and to T. Legero and U. Sterr (PTB) for important help on the resonator. This work has been funded by ESA project no. 4000103508/11/D/JR. We thank I. Zayer and J. de Vicente for support.


author: G. Monari ; B. Famaey ; I. Carrillo ; T. Piffl ; M. Steinmetz ; R. F. G. Wyse ; F. Anders ; C. Chiappini ; K. Janßen
bibliography: vescbib.bib
date: Received xxx; accepted yyy
title: The escape speed curve of the Galaxy obtained from Gaia DR2 implies a heavy Milky Way

We measure the escape speed curve of the Milky Way based on the analysis of the velocity distribution of $\sim 2850$ counter-rotating halo stars from the Gaia Data Release 2. The distances were estimated through the `StarHorse` code, and only stars with distance errors smaller than 10% were used in the study. The escape speed curve is measured at Galactocentric radii ranging from $\sim 5$ kpc to $\sim 10.5$ kpc. The local Galactic escape at the Sun's position is estimated to be $v_\text{e}(r_\odot)=580 \pm 63~\text{km~s}^{-1}$, and it rises towards the Galactic centre. Defined as the minimum speed required to reach three virial radii, our estimate of the escape speed as a function of radius implies for a Navarro-Frenk-White profile and local circular velocity of $240 \, \text{km~s}^{-1}$ a dark matter mass $M_{200}=1.28^{+0.68}_{-0.50} \times 10^{12} M_\odot$ and a high concentration $c_{200}=11.09^{+2.94}_{-1.79}$. Assuming the mass-concentration relation of $\Lambda$CDM, we obtain $M_{200}=1.55_{-0.51}^{+0.64}\times 10^{12}~M_\odot$ and $c_{200}=7.93_{-0.27}^{+0.33}$ for a local circular velocity of $228 \, \text{km~s}^{-1}$.

# Introduction

While much progress has been made in recent years regarding our understanding of the formation of galaxies, the very basic fundamental parameters of our own Galaxy are still very poorly known. Of these fundamental parameters, the mass of our Galaxy is perhaps the most important, and it is still unknown within a factor of four . This parameter is of particular interest because it provides a test for the current $\Lambda$ cold dark matter ($\Lambda$CDM) paradigm, and the validity of recipes such as abundance matching .

The primary objective of the cornerstone Gaia mission is to allow us to build a detailed dynamical model of the Galaxy, including all of its components, and to provide us insight into its structure, its formation, and its evolutionary history. Before building such a detailed and exhaustive model, we can already take advantage of the accuracy, quality, and extent of the data provided by the Gaia Data Release 2 (DR2) to determine the fundamental parameters of the Galaxy such as its virial mass more accurately than ever before. Several recent studies have tried to estimate this parameter through various methods: found that a virial mass of at least $1.6 \times 10^{12} M_\odot$ would be needed to keep a reasonable number of satellite dwarf galaxies bound. assumed that extremely high velocity stars with chemical properties similar to those of the Galactic halo are bound to the Galaxy and showed that the virial mass should be higher than $1.4 \times 10^{12} M_\odot$. also studied a sample of hyper-velocity stars and found that their chemistry is compatible with that of the stellar halo. used a mass estimator based on the kinematics of halo globular clusters to find a virial mass of $1.67^{+0.79}_{-0.50} \times 10^{12} M_\odot$. Finally, also used the kinematics of globular clusters to fit a two-component distribution function in action space as well as the Galactic potential including a prolate Navarro-Frenk-White (NFW) halo, to find a slightly lower value for the virial mass.

Here, our new estimate of the Milky Way mass will be made through measuring the escape speed curve of the Galaxy at Galactocentric radii ranging from $\sim 5$ kpc to $\sim 10.5$ kpc based on Gaia DR2. Since the escape speed directly measures the difference between the local potential and that in the outskirts of the Galaxy, it allows us to constrain its total mass based on local measurements. In doing this, we follow up on the previous studies by , , , and .

The general idea is to use a simple model for the tail of the velocity distribution of halo stars at a given radius, based on a truncated power law motivated by simulations . Based on a model for the baryonic distribution of the Milky Way, the virial mass of the Galaxy can be adjusted to best fit the escape speed curve. In Section  we present our selection of Gaia DR2 stars. The method for estimating the escape speed is then presented in Section , and the results are summarized in Section . The corresponding virial mass of the Galaxy dark matter halo is presented in Section , and we conclude in Section .

# Data

We used stars from the Gaia DR2 with line-of-sight (l.o.s.) velocity information. Following , we selected only stars with the following Gaia flags:

-   $\texttt{ASTROMETRIC\_GOF\_AL}<3$;

-   $\texttt{ASTROMETRIC\_EXCESS\_NOISE\_SIG}\leq 2$;

-   $-0.23 \leq \texttt{MEAN\_VARPI\_FACTOR\_AL} \leq 0.32$;

-   $\texttt{VISIILITY\_PERIODS\_USED}>8$;

-   $\texttt{RV\_NB\_TRANSITS} > 5$.

We used new distance estimates (Anders et al. in prep.) obtained with the `StarHorse` code , which combines multiband photometric information (APASS, 2MASS, and AllWISE) and the Gaia astrometric information with a Bayesian approach, accounting for the global Gaia DR2 parallax zero-point shift of -0.029 mas . Based on these distances and on the Gaia DR2 astrometry and l.o.s. velocities, we computed the Galactocentric cylindrical positions $(R,z,\phi)$ and velocities $(v_R,v_\phi,v_z)$ of these stars. For this, we assumed a distance of the Sun from the centre of the Galaxy of $r_\odot=8.34~\text{kpc}$, a circular velocity at the Sun of $v_\text{c}(r_\odot)=240~\text{km~s}^{-1}$ , and a peculiar solar motion of $(U_\odot,V_\odot,W_\odot)=(11.1,12.24,7.25)~\text{km~s}^{-1}$ , as estimated by . Of these stars we selected only those that counter-rotate, that is, stars with $v_\phi<0$, in order to ensure that the kinematic tail is representative of stellar halo-type stars. We checked that reflecting $v_\phi$ for these stars around $v_\phi=0$, we obtain the kind of kinematics typical of the stellar halo, with velocity dispersions of $\sim 140~\text{km~s}^{-1}$ .

Of these stars, we selected only those with distance errors smaller than 10 per cent. We then computed their Galactocentric spherical distance $r=\sqrt{R^2+z^2}$ and the speed $v=\sqrt{v_R^2+v_\phi^2+v_z^2}$. However, we did not simply combine the observable quantities, but used a Monte Carlo technique (explained below), which allowed us to achieve better estimates of the $r$ and $v$ uncertainties than a simple propagation of errors.

# Methods

As in P14, we assumed that the velocities $v$ of stars in the kinematic tail of the halo are distributed according to a power law, with a truncation at the escape velocity $v_\text{e}$ , $$\label{eq:f}
f(v|v_\text{e},k)=\begin{cases}
    (k+1) (v_\text{e}-v)^k / (v_\text{e}-v_\text{cut}), & v\leq v_\text{e},\\
    0, & v>v_\text{e},
  \end{cases}$$ where $v_\text{cut}$ corresponds to a lower cut in $v$ that defines the fast stars that should be distributed as in equation . We note that $v_\text{e}$ and $k$ depend in general on the position in the Galaxy. Both $v_\text{cut}$ and typical values of $k$ can be read off simulations. In particular, P14 have shown from simulations that the typical range for $k$ is $2.3<k<3.7$ and that stars follow this power law for $v_\text{cut}\sim 250~\text{km~s}^{-1}$.

The Bayes theorem can be used to show that the probability of the model parameters $(v_\text{e},k)$, given $N$ stars with velocity determination $v_i$ (for the $i$-th star) is $$\label{eq:P}
P(v_\text{e},k|v_{i=1,...,N})=\frac{P(v_\text{e})P(k)\Pi_{i=1}^{N} f(v_i|v_\text{e},k)}{\int\int P(v_\text{e})P(k)\Pi_{i=1}^{N} f(v_i|v_\text{e},k)~\text{d}v_\text{e}~\text{d}k},$$ where the uncertainty on the determination of $v_i$ is neglected for the moment, and the denominator is the same for all the points in the $(v_\text{e},k)$ space, so that we do not need to determine it. For the *a priori* probabilities we chose $P(v_\text{e})\propto 1/v_\text{e}$ and $P(k)$ uniform, as in .

Since we are interested mostly in $v_\text{e}$, we finally marginalized along $k$ to obtain the probability density of $v_\text{e}$, $$\label{eq:vemarg}
P(v_\text{e}|v_{i=1,...,N})=\int_{2.3}^{3.7} P(v_\text{e},k|v_{i=1,...,N})~\text{d}k,$$ where the range of marginalisation is obtained from P14, as explained above.

However, at this stage, equation  does not consider the uncertainty on the individual stellar parameters ($r$ and $v$). Hence, in order to obtain a robust result, we used a Monte Carlo or bootstrap technique, repeating the scheme detailed below 100 times.

-   For each star in the sample, we drew a random distance, proper motion, and l.o.s. velocity from Gaussians with means and dispersions (the quantity and its uncertainty) given by the `StarHorse` code and the Gaia catalogue; from these, we computed $r$ and $v$ assuming for the transformation the same Galactic parameters as were used in Section , and negligible uncertainties on the positions on the sky of the stars.

-   We used only stars with drawn random distance smaller than $5~\text{kpc}$ in order to have a sample of stars with Bayesian distance estimates dominated by the Gaia parallax information, and not by the photometry; considering the parallax shift, 99% of the stars in Gaia DR2 with l.o.s. velocities and parallax errors better than 10% are found inside $\sim 5~\text{kpc}$, and inside the same sphere, 90% of the stars have parallax error smaller than $\sim 10$ %; the typical size of the samples obtained in this way is $\sim 2850$ stars.

-   We resampled the data set obtained in this way with replacement, so that each time some stars are randomly excluded and some stars are repeated more times in the sample.

-   We binned the data into bins of size $0.6~\text{kpc}$, with the innermost bin centred at $r=5.34~\text{kpc}$ and the outermost at $r=10.14~\text{kpc}$.

-   For the $i$-th iteration (and resampling), we calculated for each bin the escape velocity $v_\text{e}^i$ and its uncertainty $\sigma_\text{e}^i=\max(v_\text{e}^i-v_{16}^i,v_{84}^i-v_\text{e}^i)$, where $v_{16}^i$ and $v_{84}^i$ are the 16th and 84th percentiles of the $v_\text{e}$ distribution.

We then estimated $v_\text{e}$ for each $r$-bin as the median of the $v_\text{e}^i$ estimates of each iteration, and its uncertainty as the sum in quadrature of the square root of the mean $\sigma_\text{e}^{i \, 2}$ and of the standard deviation of the values $v_\text{e}^i$. The error bars obtained in this way contain a systematic component because the top of the error bar for each bin always depends critically on the fastest star that can be included in the sample in one of the 100 Monte Carlo realisations.

# Escape speed curve

In Fig.  we show the histograms in different $r$-bins for one realisation of $r$ and $v$ of the sample in order to check that a power law is a reasonable description of the velocity distribution. We overplot $f(v|v_\text{e},k)$, where $v_\text{e}$ is derived for those particular bins, and $k=3$ (the mean between $k=2.3$ and $k=3.7$, used for the marginalisation to obtain the $v_\text{e}$ probability distribution function).

In the left panel of Fig. , we show the results of the analysis in different bins (black dots with error bars). Only the bins that contain at least ten stars in each resampling were considered. The escape velocity curve decreases with $r,$ as expected in any reasonable Galactic potential (see below). The value of the escape velocity at the Sun is $$v_\text{e}(r_\odot)=580 \pm 63~\text{km~s}^{-1}.$$ The error bar increases for the farthest bins, especially the outermost bins that are less densely populated.

We then compared the results obtained with the `StarHorse` code with the distance estimates by . We used the same counter-rotating stars as selected in Section , but with the velocities $v$ and radii $r$, computed with the new distances. The results in different bins are also shown in Fig.  with red dots and associated error bars. The results for all bins are very similar to the results obtained with the `StarHorse` distances. The escape velocity at the Sun is estimated in this case to be $v_\text{e}(r_\odot)=593\pm 76~\text{km~s}^{-1}$. However, the farthest bins do not contain enough stars to perform the analysis. With the McMillan distances it is not possible to go very far from the Sun because the distance quality degrades rapidly and the bins in our sample rapidly become empty, while the multiband photometric information used in `StarHorse` allows us to achieve accurate distances deeper inside the Galaxy.

# Dark matter halo mass and concentration

The escape velocity is in principle defined as the velocity necessary to bring a star to infinity. If $\Phi(\boldsymbol{x})$ is the potential of the Galaxy at some position $\boldsymbol{x}$, and $\Phi\rightarrow 0$ at infinity, then $$v_{\text{e},\infty}(\boldsymbol{x})=\sqrt{2|\Phi(\boldsymbol{x})|}.$$ However, it is unphysical to think that stars faster than $v_\text{e}$ derived in the previous section can reach infinity; they will instead turn around beyond some very large distance. On the basis of simulations, P14 chose this distance to be $3r_{340}$ , where $r_{340}$ is the spherical radius within which the average density of the whole Galaxy is equal to 340 times the critical density at redshift 0, $$\rho_\text{c}=3H^2/8\pi G,$$ where we take $H=73~\text{km~s}^{-1}\,{\rm Mpc}^{-1}$. The distance $3r_{340}$ approximates the virial radius of the Galaxy.

To model the escape speed, we used the Milky Way potential model III of , which fits a number of observables of the Milky Way. As our estimates of the escape speed depend only on the spherical radius, we computed the Irrgang potential only within the Galactic plane[^1], so that it depends on $r$ alone, that is, $\Phi(r)$. The escape speed can in this case be modelled as $v_{\text{e},\infty}(r)$ or as $$v_\text{e,340}(r)=\sqrt{2|\Phi(r)-\Phi(3r_{340})|},$$ the latter being more meaningful in a cosmological context. We kept the Irrgang model III disc and bulge potentials fixed and varied the parameters of the NFW halo, which we chose to describe with its 'virial' mass $M_{200}$ and its concentration $c_{200}$, both estimated at $r_{200}$, the radius where the average density of the dark halo reaches 200 times the critical density $\rho_\text{c}$. As an additional constraint, we imposed that the circular velocity at the Sun is $v_\text{c}(r_\odot)=240~\text{km~s}^{-1}$, as in the assumptions that we used to transform l.o.s. velocities and proper motions to $v$. We obtain a fit using a $\chi^2$ minimisation, with the points weighted according to their error bars. The best-fit model corresponds to the orange line and orange $1\sigma$ error bands in the left panel of Fig. . The error bands are obtained using the $\chi^2$ statistics. The best-fit model has $M_{200}=0.97^{+0.47}_{-0.36} \times 10^{12} M_\odot$ and $c_{200}=12.63^{+3.41}_{-2.09}$ if the fit is made assuming the $v_{\text{e},\infty}$ interpretation of the escape speed, and $M_{200}=1.28^{+0.68}_{-0.50} \times 10^{12} M_\odot$ and $c_{200}=11.09^{+2.94}_{-1.79}$ if the fit is made with the more realistic $v_\text{e,340}$ (shown in the figure).

However, $N$-body simulations in $\Lambda$CDM cosmology display a relation between the halo concentrations and their mass , $$\log_{10}(c_{200})=0.905 - 0.101\log_{10}(M_{200}/[10^{12}h^{-1}M_\odot]),$$ where $h=H/(100~\text{km~s}^{-1}~\text{Mpc}^{-1})$. We repeated the fit using this relation. In this case, the relation between $M_{200}$ and $c_{200}$ no longer guarantees that $v_\text{c}=240~\text{km~s}^{-1}$ at $R_\odot$. We note that the value of the circular velocity at the solar circle is degenerate with the actual peculiar velocity of the Sun, so that a lower value of the circular velocity and a higher value of $V_\odot$ are still compatible with our assumptions. The fit is shown in the right panel of Fig.  with the blue line and blue $1\sigma$ error bands. The best-fit model in this case is $M_{200}=1.16_{-0.38}^{+0.47}\times 10^{12}~M_\odot$ and $c_{200}=8.17_{-0.28}^{+0.33}$ in the $v_{\text{e},\infty}$ interpretation, and $M_{200}=1.55_{-0.51}^{+0.64}\times 10^{12}~M_\odot$, $c_{200}=7.93_{-0.27}^{+0.33}$ in the more realistic $v_\text{e,340}$ interpretation. In the former case, the circular speed is still well behaved in the Galaxy and corresponds to $v_\text{c}(r_\odot)=228~\text{km~s}^{-1}$. Such a high mass for the Milky Way dark matter halo is expected from abundance matching. estimated that ${\rm
log}(M_{200}) \simeq 12.25$ should correspond to a stellar mass of between $3 \times 10^{10} \, M_\odot$ and $5.5 \times 10^{10} \, M_\odot$.

Finally, in Fig.  we show with a blue line the fit to the escape velocity points that we obtained with a quasi-isothermal potential for the dark halo of the form $$\Phi_\text{h}=\frac{v_0^2}{2}\log\left(r^2+r_\text{c}^2\right).$$ The fit was obtained using $v_\text{e,340}$ ($\Phi_\text{h}$ diverges at infinity) and imposing that $v_c(r_\odot)=240~\text{km~s}^{-1}$. The best fit corresponds to $v_0=176_{-22}^{+22}~\text{km~s}^{-1}$ and $r_\text{c}=4.7_{-3.8}^{+2.1}~\text{kpc}$ (the error bars are not shown in Fig. ). This fit corresponds to a high halo mass $M_{200}=1.74_{-0.58}^{+0.72}\times 10^{12}~M_\odot$, as expected for a pseudo-isothermal sphere with a density decreasing slowly as $r^{-2}$ in the outer parts. We also compared our escape speed curve with the $v_\text{e,340}$ values obtained from the halo model fitting the Milky Way gas dynamics by , with $v_0=235~\text{km~s}^{-1}$ and $r_\text{c}=10.7~\text{kpc}$, which gives a too high a value for the escape speed if no cut-off on the $r^{-2}$ is applied in the outer halo.

The analysis we performed in Section  can also be performed by keeping the exponent $k$ fixed. In this case, a larger $v_\text{e}$ is obtained for larger $k$ because of the correlation between these two parameters, as has been shown by . To check that our estimates are consistent with such an alternative method, we derived $v_\text{e}$ for a fixed value $k=2.3$ and $k=3.7$, the extremes of the $k$-range used in our marginalisation. The $v_\text{e}$ points obtained in this way are either shifted at lower or higher values, for $k=2.3$ and $3.7$ respectively, leaving the global shape of the curve unchanged. Fitting $v_{\text{e},\infty}$ and $v_\text{e,340}$ to these points, we then find models that are safely included within the error bands on mass and concentration given above.

# Discussion and conclusions

We have calculated the escape speed curve of the Milky Way in a wide range of Galactocentric spherical radii using stars from the second data release of Gaia with line-of-sight velocities. The distances of these stars were computed using the `StarHorse` pipeline, which allows determining accurate and precise distances even for stars very far from the Sun thanks to its treatment of the extinction, especially in the central regions of the Galaxy. We used stars with distance estimates better than 10% at a distance of less than $6~\text{kpc}$ that are counter-rotating in order to have a stellar halo sample that is not contaminated by the disc.

In the solar neighbourhood we estimated the escape speed $v_\text{e}(r_\odot)=580 \pm 63~\text{km~s}^{-1}$ . The escape speed varies from $\sim 650~\text{km~s}^{-1}$ at $\sim 5~\text{kpc}$ to $\sim 550~\text{km~s}^{-1}$ at $10.5~\text{kpc}$. The uncertainty in the determination of $v_\text{e}(r_\odot)$ is quite large despite the relatively large stellar sample, mainly because this uncertainty takes into account the errors in position and velocity of the stars in the analysis, and treats the possibility that they might be bound or not in the Galactic potential with a bootstrap technique without excluding stars on arbitrary criteria. Moreover, our determination of the escape speed at the Sun and at other radii in the Galaxy is 'local' (i.e. performed in small radial bins), meaning that the estimates do not require any velocity rescaling that assumes a Galactic potential *a priori* (see P14); this introduces a bias in the result. Our determination of $v_\text{e}(r_\odot)$ is slightly larger than but agrees well with what P14 determined based on stars from the RAVE survey, within the error bars. Our result also agrees with the finding by , who reported that the escape speed in the solar neighbourhood is expected to be of the order of $\sim 600~\text{km~s}^{-1}$.

We have modelled the escape speed data points obtained using simple forms for the Galactic potential, including a fixed disc and bulge, by fitting a number of observables and a dark halo specified by the free parameters for the fit. The escape speed can be interpreted either as the velocity necessary to bring a star to infinity from a location in the Galaxy, or, more realistically, to a typical radius very far from the Galactic centre, motivated by cosmological simulations. The latter is chosen to be $3 r_{340}$ (as in P14). In the latter way, and fixing the circular velocity of the Galaxy at $240~\text{km~s}^{-1}$ at the Sun, $M_{200}=1.28^{+0.68}_{-0.50} \times 10^{12} M_\odot$.

With this fit imposing the local circular velocity at $240~\text{km~s}^{-1}$ , we find a slightly more concentrated halo than in $\Lambda$CDM simulations. However, when we performed the fit imposing the $\Lambda$CDM relation between halo masses and concentrations, we find a higher halo mass ($M_{200}=1.55_{-0.51}^{+0.64}\times 10^{12}~M_\odot$), a concentration $c_{200}=7.93^{+0.33}_{-0.27}$ , and still a reasonable circular velocity curve. We also fit models consisting of the same disc and bulge potentials, but including cored pseudo-isothermal dark halos. In this case, the only interpretation for the escape velocity is the velocity to bring a star at $3r_{340}$, and we find $M_{200}=1.74\times 10^{12}~M_\odot$, with a large core radius $r_\text{c}=4.7~\text{kpc}$. In the future, we aim to determine whether such a measurement is compatible with alternative models

The high estimated mass of the Milky Way can be compared with recent estimates of the total mass of the Local Group based on the timing argument , which sets the total mass to $\sim 2.64^{+0.42}_{-0.38}\times 10^{12}~M_\odot$. The heavy Milky Way implied by our work seems in accordance with recent estimates of the mass of M31, yielding a lower mass than previously thought , which leaves room for an increased Milky Way mass.

A caveat of this work is that the distance determination for stars using Gaia stellar parameters is still an ongoing process, and the distances and velocities of the stars could undergo modifications, especially at large distances . A dynamical caveat is that the analysis with which we estimated the escape velocities is based on distribution functions and assumes phase-mixing for the kinematic tail of the stellar halo, which is not guaranteed. However, the choice of the velocity distribution is motivated by simulations, and the comparison of the derived distribution functions with the histograms of the velocity distribution of the halo stars that we performed shows that this description is reasonable. Finally, the models of the Milky Way potential that we presented here are rather simple and do not take into account a number of issues, such as the fact that the dark halo should self-consistently adapt its shape to the bulge and disc potential and that the fit could be performed leaving the disc and bulge parameters free. Such detailed modelling is beyond the scope of this paper, however. Finally, spectroscopic follow-up of the stars we used in this analysis might be used to study whether their properties are compatible with the chemical properties of the stellar halo .

In conclusion, the uncertainties in the measurements of the escape velocity (and of the corresponding dark halo masses) are quite large, mostly because we were realistic and considered the uncertainties on both distance and velocity of the stars, and because we lack information on whether high-velocity stars are bound or unbound in the Galactic potential. Nevertheless, our estimate of the escape speed curve for a wide range of Galactocentric radii in the Milky Way conclusively implies a massive Galaxy $M_{200}>10^{12} M_\odot$, especially when considering the mass-concentration relation of $\Lambda$CDM, in agreement with other recent estimates using independent methods, both dynamical and using chemical tagging . It also agrees with typical abundance-matching expectations.

[^1]: The escape speed estimated in the Galactic plane or towards the Galactic pole at a fixed $r$ can differ by a few tens of $\text{km~s}^{-1}$, which is included in the error bars of our estimate of the escape speed.