Question: Represent the article for finding a single-sentence TLDR.

Question: In this paper, we propose data statements as a design solution and professional practice for natural language processing technologists, in both research and development — through the adoption and widespread use of data statements, the field can begin to address critical scientific and ethical issues that from the use of data from certain populations in the development of technology for other populations. We present a form that data statements can take and explore the implications of adopting them as part of regular practice. We argue that data statements will help alleviate issues related to exclusion and bias in language technology; lead to better precision in claims about how NLP research can generalize and thus better engineering ; protect companies from public embarrassment; and ultimately lead to language technology that meets its users in their own preferred linguistic style and furthermore does not mis- represent them to others. ** To appear in TACL ** As technology enters widespread societal use it is important that we, as technologists, think critically about how the design decisions we make and systems we build impact people -including not only users of the systems but also other people who will be affected by the systems without directly interacting with them. For this paper, we focus on natural language processing (NLP) technology. Potential adverse impacts include NLP systems that fail to work for specific subpopulations (e.g. children or speakers of language varieties which are not supported by training or test data) or systems that reify and reinforce biases present in training data (e.g. a resume-review system that ranks female candidates as less qualified for computer programming jobs because of biases present in training text). There are both scientific and ethical reasons to be concerned. Scientifically, there is the issue of generalizability of ; ethically, the potential for significant real-world harms. While there is increasing interest in ethics in NLP, 1 there remains the open and urgent question of how we integrate ethical considerations into the everyday practice of our field. This question has no simple answer, but rather will require a constellation of multi-faceted solutions. Toward that end, and drawing on value sensitive design BID22, this paper contributes one new professional practicecalled data statements -which we argue will bring about improvements in engineering and scientific outcomes while also enabling more ethically responsive NLP technology. A data statement is a characterization of a dataset which provides context to allow developers and users to better understand how experimental might generalize, how software might be appropriately deployed, and what biases might be reflected in systems built on the software. In developing this practice, we draw on analogous practices from the fields of psychology and medicine that require some standardized information about the populations studied (e.g. APA 2009; BID41 BID26 BID40 . Though the construct of data statements applies more broadly, in this paper we focus specifically on data statements for NLP systems. Data statements should be included in most writing on NLP including: papers presenting new datasets, papers reporting experimental work with datasets, and documentation for NLP systems. Data statements should help us as a field engage with the ethical issues of exclusion, overgeneralization, and underexposure BID30 . Furthermore, as data statements bring our datasets and their represented populations into better focus, they should also help us as a field deal with scientific issues of generalizability and reproducibility. Adopting this practice will position us to better understand and describe our and, ultimately, do better and more ethical science and engineering. 2 We begin by defining terms ( §2), discuss why NLP needs data statements (§3) and relate our proposal to current practice (§4). Next is the substance of our contribution: a detailed proposal for data statements for NLP (§5), illustrated with two case studies (§6). In §7 we discuss how data statements can mitigate bias and use the technique of'value scenarios' to envision potential effects of their adoption. Finally, we relate data statements to similar emerging proposals (§8), make recommendations for how to implement and promote the uptake of data statements (§9), and lay out considerations for tech policy (§10). As this paper is intended for at least two distinct audiences (NLP technologists and tech policymakers), we use this section to briefly define key terms. Dataset, Annotations An (NLP) dataset is a collection of speech or writing possibly combined with annotations. 3 Annotations include indications of linguistic structure like part of speech tags or syntactic parse trees, as well as labels classifying aspects of what the speakers were attempting to accomplish with their utterances. The latter includes annotations for sentiment BID38 ) and for figurative language or sarcasm (e.g. BID50 BID49 . Labels can be naturally occurring, such as star ratings in reviews taken as indications of the overall sentiment of the review (e.g. BID47 or the hashtag #sarcasm 2 By arguing here that data statements promote both ethical practice and sound science, we do not mean to suggest that these two can be conflated. A system can give accurate responses as measured by some test set (scientific soundness) and yet lead to real-world harms (ethical issues). Accordingly, it is up to researchers and research communities to engage with both scientific and ethical ideals.3 Multi-modal data sets combine language and video or other additional signals. Here, our focus is on linguistic data. used to identify sarcastic language (e.g. BID35 .Speaker We use the term speaker to refer to the individual who produced some segment of linguistic behavior included in the dataset, even if the linguistic behavior is originally written. Annotator Annotator refers to people who assign annotations to the raw data, including transcribers of spoken data. Annotators may be crowdworkers or highly trained researchers, sometimes involved in the creation of the annotation guidelines. Annotation is often done semiautomatically, with NLP tools being used to create a first pass which is corrected or augmented by human annotators. Curator A third role in dataset creation, less commonly discussed, is the curator. Curators are involved in the selection of which data to include, by selecting individual documents, by creating search terms that generate sets of documents, by selecting speakers to interview and designing interview questions, etc. Stakeholders Stakeholders are people impacted directly or indirectly by a system BID22 BID12 . Direct stakeholders include those who interact with the system, either by participating in system creation (developers, speakers, annotators and curators) or by using it. Indirect stakeholders do not use the system but are nonetheless impacted by it. For example, people whose web content is displayed or rendered invisible by search engine algorithms are indirect stakeholders with respect to those systems. Algorithm We use the term algorithm to encompass both rule-based and machine learning approaches to NLP. Some algorithms (typically rulebased ones) are tightly connected to the datasets they are developed against. Other algorithms can be easily ported to different datasets. 4 System We use the term (NLP) system to refer to a piece of software that does some kind of natural language processing, typically involving algorithms trained on particular datasets. We use this term to refer to both components focused on specific tasks (e.g. the Stanford parser BID34 trained on the Penn Treebank BID39 to do English parsing) and user-facing products such as Amazon's Alexa or Google Home. Bias We use the term bias to refer to cases where computer systems "systematically and unfairly discriminate against certain individuals or groups of individuals in favor of others" (, 332). 5 To be clear: (i) unfair discrimination does not give rise to bias unless it occurs systematically and (ii) systematic discrimination does not give rise to bias unless it in an unfair outcome. BID25 show that in some cases, system bias reflects biases in society; these are preexisting biases with roots in social institutions, practices and attitudes. In other cases, reasonable, seemingly neutral, technical elements (e.g. the order in which an algorithm processes data) can in bias when used in real world contexts; these technical biases stem from technical constraints and decisions. A third source of bias, emergent bias, occurs when a system designed for one context is applied in another, e.g. with a different population. Recent studies have documented the fact that limitations in training data lead to ethically problematic limitations in the ing NLP systems. Systems trained on naturally occurring language data learn the pre-existing biases held by the speakers of that data: Typical vector-space representations of lexical semantics pick up cultural biases about gender BID5 and race, ethnicity and religion BID52. BID57 show that beyond picking up such biases, machine learning algorithms can amplify them. Furthermore, these biases, far from being inert or simply a reflection of the data, can have real-world consequences for both direct and indirect stakeholders. For example, BID52 found that a sentiment analysis system rated reviews of Mexican restaurants as more negative than other types of food with similar star ratings, because of associations between the word Mexican and words with negative sentiment in the larger corpus on which 5 The machine learning community uses the term bias to refer to constraints on what an algorithm can learn, which may prevent it from picking up patterns in a dataset or lead it to relevant patterns more quickly (see Coppin 2004, Ch. 10). This use of the term does not carry connotations of unfairness. the word embeddings were trained. (See also BID33 In these and other ways, pre-existing biases can be trained into NLP systems. There are other studies showing that systems from part of speech taggers BID31 to speech recognition engines BID53 perform better for speakers whose demographic characteristics better match those represented in the training data. These are examples of emergent bias. Because the linguistic data we use will always include pre-existing biases and because it is not possible to build an NLP system in such a way that it is immune to emergent bias, we must seek additional strategies for mitigating the scientific and ethical shortcomings that follow from imperfect datasets. We propose here that foregrounding the characteristics of our datasets can help, by allowing reasoning about what the likely effects may be and by making it clearer which populations are and are not represented, for both training and test data. For training data, the characteristics of the dataset will affect how the system will work when it is deployed. For test data, the characteristics of the dataset will affect what can be measured about system performance and thus provides important context for scientific claims. Typical current practice in academic NLP is to present new datasets with a careful discussion of the annotation process as well as a brief characterization of the genre (usually by naming the underlying data source) and the language. NLP papers using datasets for training or test data tend to more briefly characterize the annotations and will sometimes leave out mention of genre and even language. 6 Initiatives such as the Open Language Archives Community (OLAC; BID4, the Fostering Language Resources Network (FLaReNet; BID7) and the Text Encoding Initiative (TEI; Consortium 2008) prescribe metadata to publish with language resources, primarily to aid in the discoverability of such resources. FLaReNet also encourages documentation of language resources. And yet, it is very rare to find detailed characterization of the speakers whose data is captured or the annotators who provided the annotations, though the latter are usually characterized as being experts or crowdworkers. 7 To fill this information gap, we argue that data statements should be included in every NLP publication which presents new datasets and in the documentation of every NLP system, as part of a chronology of system development including descriptions of the various datasets for training, tuning and testing. Data statements should also be included in all NLP publications reporting experimental . Accordingly, data statements will need to be both detailed and concise. To meet these competing goals, we propose two variants. For each dataset there should be a long-form version in an academic paper presenting the dataset or in system documentation. Research papers presenting experiments making use of datasets with existing long-form data statements should include shorter data statements and cite the longer one. 8 We note another set of goals in competition: While readers need as much information as possible in order to understand how the can and cannot be expected to generalize, considerations of the privacy of the people involved (speakers, annotators) might preclude including certain kinds of information, especially with small groups. Each project will need to find the right balance, but this can be addressed in part by asking annotators and speakers for permission to collect and publish such information. We propose the following schema of information to include in long and short form data statements. Long form data statements should be included in system documentation and in academic papers presenting new datasets, and should strive to provide the following information:A. CURATION RATIONALE Which texts were included and what were the goals in selecting texts, both in the original collection and in any further 7 A notable exception is BID13, who present a corpus of tweets collected to sample diverse speaker communities (location, type of engagement with Twitter), at diverse points in time (time of year, month, and day), and annotated with named entity labels by crowdworker annotators from the same locations as the tweet authors. 8 Older datasets can be retrofitted with citeable long-form data statements published on project web pages or archives.sub-selection? This can be especially important in datasets too large to thoroughly inspect by hand. An explicit statement of the curation rationale can help dataset users make inferences about what other kinds of texts systems trained with them could conceivably generalize to. B. LANGUAGE VARIETY Languages differ from each other in structural ways that can interact with NLP algorithms. Within a language, regional or social dialects can also show great variation BID8. The language and language variety should be described with:• A language tag from BCP-47 9 identifying the language variety (e.g. en-US or yue-Hant-HK)• A prose description of the language variety, glossing the BCP-47 tag and also providing further information (e.g. English as spoken in Palo Alto CA (USA) or Cantonese written with traditional characters by speakers in Hong Kong who are bilingual in Mandarin)C. SPEAKER DEMOGRAPHIC Sociolinguistics has found that variation (in pronunciation, prosody, word choice, and grammar) correlates with speaker demographic characteristics BID37, as speakers use linguistic variation to construct and project identities BID16. Transfer from native languages (L1) can affect the language produced by non-native (L2) speakers (, Ch. 8). A further important type of variation is disordered speech (e.g. dysarthria). Specifications include: BID3, and should be specified. DISPLAYFORM0 G. RECORDING QUALITY For data that includes audio/visual recordings, indicate the quality of the recording equipment and any aspects of the recording situation that could impact recording quality. H. OTHER There may be other information of relevance as well (e.g. the demographic characteristics of the curators). As stated above, this is intended as a starting point and we anticipate best practices around writing data statements to develop over time. I. PROVENANCE APPENDIX For datasets built out of existing datasets, the data statements for the source datasets should be included as an appendix. Short form data statements should be included in any publication using a dataset for training, tuning or testing a system and may also be appropriate for certain kinds of system documentation. The short 10 For example, people speak differently to close friends v. strangers, to small groups v. large ones, to children v. adults and to people v. machines (e.g. BID18 .11 Mutable speaker demographic information, such as age, is interpreted as relative to the time of the linguistic behavior.form data statement does not replace the long form one, but rather should include a pointer to it. For short form data statements, we envision 60-100 word summaries of the description included in the long form, covering most of the main points. We have outlined the kind of information data statements should include, addressing the needs laid out in §3, describing both long and short versions. As the field gains experience with data statements, we expect to see a better understanding of what to include as well as best practices for writing data statements to emerge. Note that full specification of all of this information may not be feasible in all cases. For example, in datasets created from web text, precise demographic information may be unavailable. In other cases (e.g. to protect the privacy of annotators) it may be preferable to provide ranges rather than precise values. For the description of demographic characteristics, our field can look to others for best practices, such as those described in the American Psychological Association's Manual of Style. It may seem redundant to reiterate this information in every paper that makes use of well-trodden datasets. However, it is critical to consider the data anew each time to ensure that it is appropriate for the NLP work being undertaken and that the reported are properly contextualized. Note that the requirement is not that datasets be used only when there is an ideal fit between the dataset and the NLP goals but rather that the characteristics of the dataset be examined in relation to the NLP goals and limitations be reported as appropriate. We illustrate the idea of data statements with two cases studies. Ideally, data statements are written at or close to the time of dataset creation. These data statements were constructed post hoc in conversation with the dataset curators. The first entails labels for a particular subset of all Twitter data. In contrast, the second entails all available data for an intentionally generated interview collection, including audiofiles and transcripts. Both illustrate how even when specific information is not available, the explicit statement of its lack of availability provides a more informative picture of the dataset. The Hate Speech Twitter Annotations collection is a set of labels for ∼19,000 tweets collected by BID55 and BID54. The dataset can be accessed via https:// github.com/zeerakw/hatespeech. 12 A. CURATION RATIONALE In order to study the automatic detection of hate speech in tweets and the effect of annotator knowledge (crowdworkers v. experts) on the effectiveness of models trained on the annotations, BID55 performed a scrape of Twitter data using contentious terms and topics. The terms were chosen by first crowd-sourcing an initial set of search terms on feminist Facebook groups and then reviewing the ing tweets for terms to use and adding others based on the researcher's intuition. 13 Additionally, some prolific users of the terms were chosen and their timelines collected. For the annotation work reported in BID54, expert annotators were chosen for their attitudes with respect to intersectional feminism in order to explore whether annotator understanding of hate speech would influence the labels and classifiers built on the dataset. B. LANGUAGE VARIETY The data was collected via the Twitter search API in late 2015. Information about which varieties of English are represented is not available, but at least Australian (en-AU) and US (en-US) mainstream Englishes are both included. C. SPEAKER DEMOGRAPHIC Speakers were not directly approached for inclusion in this dataset and thus could not be asked for demographic information. More than 1500 different Twitter accounts are included. Based on independent information about Twitter usage and impressionistic observation of the tweets by the dataset curators, the data is likely to include tweets from 12 This data statement was prepared based on information provided by Zeerak Waseem, pc, Feb-Apr 2018 and reviewed and approved by him. 13 In a standalone data statement, the search terms should be given in the main text. To avoid accosting readers with slurs in this article, we instead list them in this footnote. BID55 provide the following complete list of terms used in their initial scrape:'MKR','asian drive','feminazi','immigrant','nigger','sjw','WomenAgainstFeminism','blameonenotall','islam terrorism','notallmen','victimcard','victim card','arab terror','gamergate','jsil','racecard','race card'. both younger and older (30+) adult speakers, the majority of whom likely identify as white. No direct information is available about gender distribution or socioeconomic status of the speakers. It is expected that most, but not all, of the speakers speak English as a native language. D. ANNOTATOR DEMOGRAPHIC This dataset includes annotations from both crowdworkers and experts.1,065 crowdworkers were recruited through Crowd Flower, primarily from Europe, South America and North America. Beyond country of residence, no further information is available about the crowdworkers. The expert annotators were recruited specifically for their understanding of intersectional feminism. All were informally trained in critical race theory and gender studies through years of activism and personal research. They ranged in age from 20-40, included 3 men and 13 women, and gave their ethnicity as white European, East Asian, Middle East/Turkey, and South Asian. Their native languages were Danish, Danish/English, Turkish/Danish, Arabic/Danish, and Swedish. Based on income levels, the expert annotators represented upper lower class, middle class, and upper middle class.E. SPEECH SITUATION All tweets were initially published between April 2013 and December 2015. Tweets represent informal, largely asynchronous, spontaneous, written language, of up to 140 characters per tweet. About 23% of the tweets were in reaction to a specific Australian TV show (My Kitchen Rules) and so were likely meant for roughly synchronous interaction with other viewers. The intended audience of the tweets was either other viewers of the same show, or simply the general Twitter audience. For the tweets containing racist hate speech, the authors appear to intend them both for those who would agree but also for people whom they hope to provoke into having an agitational and confrontational exchange. F. TEXT CHARACTERISTICS For racist tweets the topic was dominated by Islam and Islamophobia. For sexist tweets predominant topics were the TV show and people making sexist statements while claiming not to be sexist. BID45 BID46 BID24. The dataset can be downloaded from http://www. tribunalvoices.org. 14 A. CURATION RATIONALE The VRT project, funded by the United States National Science Foundation, is part of a research program on developing multi-lifespan design knowledge BID23. It is independent from the ICTR, the United Nations, and the government of Rwanda. To help ensure accuracy and guard against breeches of confidentiality, interviewees had an opportunity to review and redact any material that was either misspoken or revealed confidential information. A total of two words have been redacted. No other review or redaction of content has occurred. The dataset includes all publicly released material from the collection; as of the writing of this data statement (28 September 2017) one interview and a portion of a second are currently sealed. B. LANGUAGE VARIETY Of the interviews, 44 are conducted in English (en-US and international English on the part of the interviewees, en-US on the part of the interviewers) and 5 in French and English, with the interviewee speaking international French, the interviewer speaking English (en-US) and an interpreter speaking both. 15 C. SPEAKER DEMOGRAPHIC The interviewees (13 women and 36 men, all adults) are profession-als working in the area of international justice, such as judges or prosecutors, and support roles of the same, such as communications, prison warden, and librarian. They represent a variety of nationalities: Argentina, Benin, Cameroon, Canada, England, The Gambia, Ghana, Great Britain, India, Italy, Kenya, Madagascar, Mali, Morocco, Nigeria, Norway, Peru, Rwanda, Senegal, South Africa, Sri Lanka, St. Kitts and Nevis, Sweden, Tanzania, Togo, Uganda, and the US. Their native languages are not known, but are presumably diverse. The 7 interviewers (2 women and 5 men) are informataion and legal professionals from different regions in the US. All are native speakers of US English, all are white, and at the time of the interviews they ranged in age from early 40s to late 70s. The interpreters are language professionals employed by the ICTR with experience interpreting between French and English. Their age, gender, and native languages are unknown. D. ANNOTATOR DEMOGRAPHIC The initial transcription was outsourced to a professional transcription company, so information about these transcribers is unavailable. The English transcripts were reviewed by English speaking (en-US) members of the research team for accuracy and then reviewed a third time by an additional English speaking (en-US) member of the team. The French/English transcripts received a second and third review for accuracy by bi-lingual French/English doctoral students at the University of Washington. Because of the sensitivity of the topic, the high political status of some interviewees (e.g. Prosecutor for the tribunal), and the international stature of the institution, it is very important that interviewees' comments be accurately transcribed. Accordingly, the bar for quality of transcription was set extremely high. E. SPEECH SITUATION The interviews were conducted in Autumn 2008 at the ICTR in Arusha, Tanzania and in Rwanda, face-to-face, as spoken language. The interviewers begin with a prepared set of questions, but most of the interaction is semi-structured. Most generally, the speech situation can be characterized as a dialogue, but some of the interviewees give long replies, so stretches may be better characterized as monologues. For the interviewees, the immediate interlocutor is the interviewer, but the intended audience is much larger (see Part F below).F. TEXT CHARACTERISTICS The interviews were intended to provide an opportunity for tribunal personnel to reflect on their experiences working at the ICTR and what they would like to share with the people of Rwanda, the international justice community, and the global public now, 50 and 100 years from now. Professionals from all organs of the tribunal (judiciary, prosecution, registry) were invited to be interviewed, with effort made to include a broad spectrum of roles (e.g. judges, prosecutor, defense counsel, but also the warden, librarian, language services). Interviewees expected their interviews to be made broadly accessible. G. RECORDING QUALITY The video interviews were recorded with high definition equipment in closed but not sound-proof offices. There is some noise. H. OTHER N/A I. PROVENANCE APPENDIX N/A.VRT short form The data represents wellvetted transcripts of 49 spoken interviews with personnel from the International Criminal Tribunal for Rwanda (ICTR) about their experience at the tribunal and reflections on international justice, in international English (44 interviews) and French (5 interviews with interpreters). Interviewees are adults working in international justice and support fields at the ICTR; interviewers are adult information or legal professionals, highly fluent in en-US; and transcribers are highly educated, highly fluent English and French speakers.[Include a link to the long form.] These sample data statements are meant to illustrate how the schema can be used to communicate the specific characteristics of datasets. They were both created post-hoc, in communication with the dataset curators. Once data statements are created as a matter of best practice, however, they should be developed in tandem with the datasets themselves and may even inform the curation of datasets. At the same time, data statements will need to be written for widely used, pre-existing datasets, where documentation may be lacking, memories imperfect, and dataset curators no longer accessible. While retrospective data statements may be incomplete, by and large we believe they can still be valuable. Our case studies also underscore how curation rationales shape the specific kinds of texts included. This is particularly striking in the case of the Hate Speech Twitter Annotations, where the specific search terms very clearly shaped the specific kinds of hate speech included and the ways in which any technology or studies built on this dataset will generalize. We have explicitly designed data statements as a tool for mitigating bias in systems that use data for training and testing. Data statements are particularly well suited to mitigate forms of emergent and pre-existing bias. For the former, we see benefits at the level of specific systems and of the field: When a system is paired with data statement(s) for the data it is trained on, those deploying it are empowered to assess potential gaps between the speaker populations represented in the training and test data and the populations whose language the system will be working with. At the field level, data statements enable an examination of the entire catalog of testing and training datasets to help identify populations who are not yet included. All of these groups are vulnerable to emergent bias, in that any system would by definition have been trained and tested on data from datasets that do not represent them well. Data statements can also be instrumental in the diagnosis (and thus mitigation) of pre-existing bias. Consider again example of Mexican restaurants and sentiment analysis. The information that the word vectors were trained on general web text (together with knowledge of what kind of societal biases such text might contain) was key in figuring out why the system consistently underestimated the ratings associated with reviews of Mexican restaurants. In order to enable both more informed system development and deployment and audits by users and others of systems in action, it is critical that characterizations of the training and test data underlying systems be available. To be clear, data statements do not in and of themselves solve the entire problem of bias. Rather, they are a critical enabling infrastructure. Consider by analogy this example from about access to technology and employment for people with disabilities. In terms of computer system design, we are not so privileged as to determine rigidly the values that will emerge from the systems we design. But neither can we abdicate responsibility. For example, let us for the moment agree [. . .] that disabled people in the work place should be able to access technology, just as they should be able to access a public building. As system designers we can make the choice to try to construct a technological infrastructure which disabled people can access. If we do not make this choice, then we single-handedly undermine the principle of universal access. But if we do make this choice, and are successful, disabled people would still rely, for example, on employers to hire them. (p.3) Similarly, with respect to bias in NLP technology, if we do not make a commitment to data statements or a similar practice for making explicit the characteristics of datasets, then we will singlehandedly undermine the field's ability to address bias. In NLP, we expect proposals to come with some kind of evaluation. In this paper, we have demonstrated the substance and'writability' of a data statement through two exemplars (§6). However, the positive effects of data statements that we anticipate (and negative effects we haven't anticipated) cannot be demonstrated and tested a priori, as their impact emerges through practice. Thus, we look to value sensitive design, which encourages us to consider what would happen if a proposed technology were to come into widespread use, over longer periods of time, with attention to a wide range of stakeholders, potential benefits, and harms BID22 BID21. We do this with value scenarios BID44 BID12.Specifically, we look at two kinds of value scenarios: Those concerning NLP technology that fails to take into account an appropriate match between training data and deployment context and those that envision possible positive as well as negative consequences stemming from the widespread use of the specific'technology' we are proposing in this paper (data statements). Envisioning possible negative outcomes allows us to consider how to mitigate such possibilities before they occur. This value scenario is inspired by BID32, who provide a similar one to motivate training language ID systems on more representative datasets. Scenario. Big U Hospital in a town in the Upper Midwest collaborates with the CS Department at Big U to create a Twitter-based early warning system for infectious disease, called DiseaseAlert. Big U Hospital finds that the system improves patient outcomes by alerting hospital staff to emerging community health needs and alerting physicians to test for infectious diseases that currently are active locally. Big U decides to make the DiseaseAlert project open source to provide similar benefits to hospitals across the Anglophone world and is delighted to learn that City Hospital in Abuja, Nigeria is excited to implement DiseaseAlert locally. Big U supports City Hospital with installing the code, including localizing the system to draw on tweets posted from Abuja. Over time, however, City Hospital finds that the system is leading its physicians to order unnecessary tests and that it is not at all accurate in detecting local health trends. City Hospital complains to Big U about the poor system performance and reports that their reputation is being damaged. Big U is puzzled, as the DiseaseAlert performs well in the Upper Midwest, and they had spent time localizing the system to use tweets from Abuja. After a good deal of frustration and investigation into Big U's system, the developers discover that the third-party language ID component they had included was trained on only highlyedited US and UK English text. As a , it tends to misclassify tweets in regional or nonstandard varieties of English as'not English' and therefore not relevant. Most of the tweets posted by people living in Abuja that City Hospital's system should have been looking at were thrown out by the system at the first step of processing. Analysis. City Hospital adopted Big U's open source DiseaseAlert system in exactly the way Big U intended. However, the documentation for the language ID component lacked critical information needed to help ensure the localization process would be successful; namely, information about the training and test sets for the system. Had Big U included data statements for all system components (including third-party components) in their documentation, then City Hospital IT staff would have been positioned to recognize the potential limitation of DiseaseAlert and to work proactively with Big U to ensure the system performed well in City Hospital's context. Specifically, in reviewing data statements for all system components, the IT staff could note that the language ID component was trained on data unlike what they were seeing in their local tweets and ask for a different language ID component or ask for the existing one to be retrained. In this manner, an emergent bias and its concomitant harms could have been identified and addressed during the system adaptation process prior to deployment. In §7.1 we consider data statements in relation to a particular system. Here, we explore their potential to enable better science in NLP overall. Scenario. It's 2022 and'Data Statement' has become a standard section heading for NLP research papers and system documentation. Happily, reports of mismatch between dataset and community of application leading to biased systems have decreased. Yet, research community members articulate an unease regarding which language communities are and which are not part of the field's data catalog -the abstract total collection of data and associated meta-data to which the field has access -and the possibility for ing bias in NLP at a systemic level. In response, several national funding bodies jointly fund a project to discover gaps in knowledge. The project compares existing data statements to surveys of spoken languages and systematically maps which language varieties have resources (annotated corpora and standard processing tools) and which ones lack such resources. The study turns up a large number of language varieties lacking such resources; it also produces a precise list of underserved populations, some of which are quite sizable, suggesting opportunity for impactful intervention at the academic, industry and government levels. Study in hand, the NLP community embarks on an intentional program to broaden the language varieties in the data catalog. Public discussions lead to criteria for prioritizing language varieties and funding agencies come together to fund collaborative projects to produce state of the art resources for understudied languages. Over time, the data catalog becomes more inclusive; bias in the catalog, while not wholly absent, is significantly reduced and NLP researchers and developers are able to run more comprehensive experiments and build technology that serves a larger portion of society. Analysis. The NLP community has recognized critical limitations in the field's existing data catalog, leaving many language communities underserved BID2 BID42 BID32. 16 The widespread uptake of data statements positions the NLP community to document the degree to which it leaves out certain language groups and empower itself to systematically broaden the data catalog. In turn, individual NLP systems could be trained on datasets that more closely align with the language of anticipated system users, thereby averting emergent bias. Furthermore, NLP researchers can more thoroughly test key research ideas and systems, leading to more reliable scientific . Finally, we explore one potential negative outcome and how with care it might be mitigated: that of data statements as a barrier to research. Scenario. In response to widespread uptake, in 2026 the Association for Computational Linguistics (ACL) proposes that data statements be standardized and required components of research papers. A standards committee is formed, open public professional discussion is engaged, and in 2028 a standard is adopted. It mandates data statements as a requirement for publication, with standardized information fields and strict specifications for how these should be completed to facilitate automated meta-analysis. There is great hope that the field will experience increasing benefits from ability to compare, contrast, and build complementary data sets. Many of those hopes are realized. However, in a relatively short period of time papers from underrepresented regions abruptly decline. In addition, the number of papers from everywhere producing and reporting on new datasets decline as well. Distressed by this outcome, the ACL con-stitutes an ad hoc committee to investigate. A survey of researchers reveals two distinct causes: First, researchers from institutions not yet well represented at ACL were having their papers deskrejected due to missing or insufficient data statements. Second, researchers who might otherwise have developed a new dataset instead chose to use existing datasets whose data statements could simply be copied. In response, the ACL executive develops a mentoring service to assist authors in submitting standards-compliant data statements and considers relaxing the standard somewhat in order to encourage more dataset creation. Analysis. With any new technology, there can be unanticipated ripple effects -data statements are no exception. Here we envision two potential negative impacts, which could both be mitigated through other practices. Importantly, while we recommend the practice of creating data statements, we believe that they should be widely used before any standardization takes place. Furthermore, once a degree of expertise in this area is built up, we recommend that mentoring be put in place proactively. Community engagement and mentoring will also contribute to furthering ethical discourse and practice in the field. The value scenarios described here point to key upsides to the widespread adoption of data statements and also help to provide words of caution. They are meant to be thought-provoking and plausible, but are not predictive. Importantly, the scenarios illustrate how, if used well, data statements could be an effective tool for mitigating bias in NLP systems. We see three strands of related work which lend support to our proposal and to the proposition that data statements will have the intended effect: similar practices in medicine (§8.1), emerging, independent proposals around similar ideas for transparency about datasets in AI (§8.2), and proposals for'algorithmic impact statements' (§8.3). In medicine, the CONSORT (CONsolidated Standards of Reporting Trials) guidelines were developed by a consortium of journal editors, specialists in clinical trial methodology and others to improve reporting of randomized, controlled trials. 17 They include a checklist for authors to use to indicate where in their research reports each item is handled and a statement explaining the rationale behind each item BID41. CONSORT development began in 1993, with the most recent release in 2010. It has been endorsed by 70 medical journals. 18 Item 4a,'Eligibility criteria for participants' is most closely related to the concerns of this paper. Characterizing the population that participated in the study is critical for gauging the extent to which the of the study are applicable to particular patients a physician is treating BID41.The inclusion of this information has also enabled further kinds of research. For example, BID40 argue that careful attention to and publication of demographic data that may correlate with health inequities can facilitate further work through meta-analyses. In particular, individual studies usually lack the statistical power to do the kind of sub-analyses required to check for health inequities, and failing to publish demographic information precludes its use in the kind of aggregated, meta-analyses that could have sufficient statistical power. This echoes the fieldlevel benefits we anticipate for data statements in building out the data catalog in the value scenario in §7.2. At least three other groups are working in parallel on similar proposals regarding bias and AI. Gebru et al. (in prep) propose'datasheets for datasets', looking at AI more broadly (but including NLP); Chmielinski and colleagues at the MIT Media Lab propose'dataset nutrition labels'; 19 and describe'Ranking Facts', a series of widgets that allow a user to explore how attributes influence a ranking. Of these, the datasheets proposal is most similar to ours in including a comparable schema. The datasheets are inspired by those used in computer hardware to give specifications, lim-its and appropriate use information for components. There is important overlap in the kinds of information called for in the datasheets schema and our data statement schema: For example, the datasheets schema includes a section on'Motivation for Dataset Creation', akin to our'Curation Rationale'. The primary differences stem from the fact that the datasheets proposal is trying to accommodate all types of datasets used to train machine learning systems and, hence, tends toward more general, cross-cutting categories; while we elaborate requirements for linguistic datasets and, hence, provide more specific, NLP-focused categories. Gebru et al. note, like us, that their proposal is meant as an initial starting point to be elaborated through adoption and application. Having multiple starting points for this discussion will certainly make it more fruitful. Several groups have called for algorithmic impact statements BID51 BID15 BID0, modeled after environmental impact statements. Of these AI Now's proposal is perhaps the most developed. All three groups point to the need to clarify information about the data: "Algorithm impact statements would document [. . .] data quality control for input sources" (, 13539); "One avenue for transparency here is to communicate the quality of the data, including its accuracy, completeness, and uncertainty, [. . .] representativeness of a sample for a specific population, and assumptions or other limitations" (, 60); "AIAs should cover [. . .] input and training data." However, none of these proposals specify how to do so. Data statements fill this critical gap. Data statements are meant to be something practical and concrete that NLP technologists can adopt as one tool for mitigating potential harms of the technology we develop. For this benefit to come about, data statements must be easily adopted. In addition, practical uptake will require coordinated effort at the level of the field. In this section we briefly consider possible costs to writers and readers of data statements, and then propose strategies for promoting uptake. The primary cost we see for writers is time:With the required information to hand, writing a data statement should take no more than 2-3 hours (based on our experience with the case studies). However, the time to collect the information will depend on the dataset. The more speakers and annotators that are involved, the more time it may take to collect demographic information. This can be facilitated by planning ahead, before the corpus is collected. Another possible cost is that collecting demographic information may mean that projects previously not submitted to institutional review boards for approval must now be, at least for exempt status. This process itself can take time, but is valuable in its own right. A further cost to writers is space. We propose that data statements, even the short form (60 -100 words), be exempt from page limits in conference and journal publications. As for readers, reviewers have more material to read and dataset (and ultimately system) users need to scrutinize data statements in order to determine which datasets are appropriate for their use case. But this is precisely the point: Data statements make critical information accessible that previously could only be found by users with great effort, if at all. The time invested in scrutinizing data statements prior to dataset adoption is expected to be far less than the time required to diagnose and retrofit an already deployed system should biases be identified. Turning to uptake in the field, NLP technologists (both researchers and system developers) are key stakeholders of the technology of data statements. Practices that engage these stakeholders in the development and promotion of data statements will both promote uptake and ensure that the ultimate form data statements take are responsive to NLP technologists' needs. Accordingly, we recommend that one or more professional organizations such as the Association for Computational Linguistics convene a working group on data statements. Such a working group would engage in several related sets of activities, which would collectively serve to publicize and cultivate the use of data statements:(i) Best practices A clear first step entails developing best practices for how data statements are produced. This includes: steps to take before collecting a dataset to facilitate writing an informative data statement; heuristics for writing concise and effective data statements; how to incorporate material from institutional review board/ethics committee applications into the data statement schema; how to find an appropriate level of detail given privacy concerns, especially for small or vulnerable populations; and how to produce data statements for older datasets that predate this practice. In doing this work, it may be helpful to distill best practices from other fields, such as medicine and psychology, especially around collecting demographic information.(ii) Training and support materials With best practices in place, the next step is providing training and support materials for the field at large. We see several complementary strategies to undertake: Create a digital template for data statements; run tutorials at conferences; establish a mentoring network (see §7.3); and develop an on-line'how-to' guide.(iii) Recommendations for field-level policies There are a number of field-level practices that the working group could explore to support the uptake and successful use of data statements. Funding agencies could require data statements to be included in data management plans; conferences and journals could not count data statements against page limits (similar to references) and eventually require short form data statements in submissions; conferences and journals could allocate additional space for data statements in publications; finally once data statements have been in use for a few years, a standardized form could be established. Transparency of datasets and systems is essential for preserving accountability and building more just systems BID36. Due process provides a critical case in point. In the United States, for example, due process requires that citizens who have been deprived of liberty or property by the government be afforded the opportunity to understand and challenge the government's decision BID9. Without data statements or something similar, governmental decisions that are made or supported by automated systems deprive citizens of the ability to mount such a challenge, undermining the potential for due process. In addition to challenging any specific decision by any specific system, there is a further concern about building systems that are broadly representative and fair. Here too, data statements have much to contribute. As systems are being built, data statements enable developers and researchers to make informed choices about training sets and to flag potential underrepresented populations who may be overlooked or treated unfairly. Once systems are deployed, data statements enable diagnosis of systemic unfairness when it is detected in system performance. At a societal level, such transparency is necessary for government and advocacy groups seeking to ensure protections and an inclusive society. If data statements turn out to be useful as anticipated, then the following implications for standardization and tech policy likely ensue. Long-Form Data Statements Required in System Documentation. For academia, industry and government, inclusion of long-form data statements as part of system documentation should be a requirement. As appropriate, inclusion of long-form data statements should be a requirement for ISO and other certification. Even groups that are creating datasets that they don't share (e.g. NSA) would be well advised to make internal data statements. Moreover, under certain legal circumstances, such groups may be required to share this information. Short-Form Data Statements Required for Academic and Other Publication. For academic publication in journals and conferences, inclusion of short-form data statements should be a requirement for publication. As highlighted in §7.3, caution must be exercised to ensure that this requirement does not become a barrier to access for some researchers. These two recommendations will need to be implemented with care. We have already noted the potential barrier to access. Secrecy concerns may also arise in some situations, e.g., some groups may be willing to share datasets but not demographic information, for fear of public relations backlash or to protect the safety of contributors to the dataset. That said, as consumers of datasets or products trained with them, NLP researchers, developers and the general public would be well advised to use systems only if there is access to the information we propose should be included in data statements. As researchers and developers working on technology in widespread use, capable of impacting people beyond its direct users, we have an obligation to consider the ethical implications of our work. This will only happen reliably if we find ways to integrate such thought into our regular practice. In this paper, we have put forward one specific, concrete proposal which we believe will help with issues related to exclusion and bias in language technology: the practice of including'data statements' in all publications and documentation for all NLP systems. We believe this practice will have beneficial effects immediately and into the future: In the short term, it will foreground how our data does and doesn't represent the world (and the people our systems will impact). In the long term, it should enable research that specifically addresses issues of bias and exclusion, promote the development of more representative datasets, and make it easier and more normative for researchers to take stakeholder values into consideration as they work. In foregrounding the information about the data we work with, we can work toward making sure that the systems we build work for diverse populations and also toward making sure we are not teaching computers about the world based on the world views of a limited subset of people. Granted, it will take time and experience to develop the skill of writing carefully crafted data statements. However, we see great potential benefits: For the scientific community, researchers will be better able to make precise claims about how should generalize and perform more targeted experiments around reproducing for datasets that differ in specific characteristics. For industry, we believe that incorporating data statements will encourage the kind of conscientious software development that protects companies' reputations (by avoiding public embarrassment) and makes them more competitive (by creating systems used more fluidly by more people). For the public at large, data statements are one piece of a larger collection of practices that will enable the development of NLP systems that equitably serves the interests of users and indirect stakeholders.

A practical proposal for more ethical and responsive NLP technology, operationalizing transparency of test and training data

Represent the TLDR for finding the corresponding scientific article

Humans in the loop revise documents to accord with counterfactual labels, resulting resource helps to reduce reliance on spurious associations.


Question: Represent the scientific article to find a short TLDR

Question: We introduce CGNN, a framework to learn functional causal models as generative neural networks. These networks are trained using backpropagation to minimize the maximum mean discrepancy to the observed data. Unlike previous approaches, CGNN leverages both conditional independences and distributional asymmetries to seamlessly discover bivariate and multivariate causal structures, with or without hidden variables. CGNN does not only estimate the causal structure, but a full and differentiable generative model of the data. Throughout an extensive variety of experiments, we illustrate the competitive esults of CGNN w.r.t state-of-the-art alternatives in observational causal discovery on both simulated and real data, in the tasks of cause-effect inference, v-structure identification, and multivariate causal discovery. Deep learning models have shown extraordinary predictive abilities, breaking records in image classification BID19, speech recognition, language translation BID1, and reinforcement learning BID33. However, the predictive focus of black-box deep learning models leaves little room for explanatory power. In particular, current machine learning paradigms offer no protection to avoid mistaking correlation by causation. For example, consider that we are interested in predicting a target variable Y given a feature vector (X 1, X 2). Assume that the generative process underlying (X 1, X 2, Y) is described by the equations: DISPLAYFORM0, where (E X2, E y) are additive noise variables. These equations tell that the values of Y are computed as a function of the values of X 1, and that the values of X 2 are computed as a function of the values of Y. The "assignment arrows" emphasize the asymmetric relations between the three random variables: we say that "X 1 causes Y ", and that "Y causes X 2 ". However, since X 2 provides a stronger signal-to-noise ratio for the prediction of Y, the least-squares solution to this problem iŝ Y = 0.25X 1 + 0.5X 2, a typical case of inverse regression BID7. Such least-squares prediction would explain some changes in Y as a function of changes in X 2. This is a wrong explanation, since X 2 does not cause the computation of Y. Even though there exists the necessary machinery to detect all the cause-effect relations in this example BID15, common machine learning solutions will misunderstand how manipulating the values and distributions of (X 1, X 2), or how changing the mapping from Y to X 2, affect the values of Y. Mistaking correlation by causation can be catastrophic for agents who must plan, reason, and decide based on observation. Thus, discovering causal structures is of crucial importance. The gold standard to discover causal relations is to perform experiments BID27. However, experiments are in many cases expensive, unethical, or impossible to realize. In these situations, there is a need for observational causal discovery, that is, the estimation of causal relations from observation alone BID35 BID28. The literature in observational causal discovery is vast (see Appendix B for a brief survey), but lacks a unified solution. For instance, some approaches rely on distributional asymmetries to discover bivariate causal relations BID15 BID40 BID4 BID37 BID6, while others rely on conditional independence to discover structures on three or more variables BID35 BID0. Furthermore, different algorithms X 5 DISPLAYFORM1 Figure 1: Example of causal graph and associated functional model for X = (X 1, . . ., X 5).FCMs are generative models. We can draw a sample x = (x 1, . . ., x d) from the distribution P:= P (X) by observing the FCM at play. First, draw e i ∼ Q for all i = 1,..., d. Second, construct Pa(i;G), e i ) in the topological order of G. Since this process observes but does not manipulate the equations of the FCM, we call x one observational sample from P, the observational distribution of X. However, one FCM contains more information than the observational distribution alone, since we can decide to manipulate any of its equations and obtain a new distribution. For instance, we could decide to set and hold constant X j = 0.1, hereby removing all the causal influences X k → X j, for all k ∈ Pa(j; G). We denote by P do(Xj =0.1) (X) the corresponding interventional distribution. Importantly, intervening is different from conditioning (correlation does not imply causation). Understanding the effect of interventions requires the (partial) knowledge of the FCM. This is why this work focuses on discovering such causal structures from data. DISPLAYFORM2 Formal definitions and assumptions Two random variables (X, Y) are conditionally independent given Z if P (X, Y |Z) = P (X|Z)P (Y |Z). Three of random variables (X, Y, Z) form a v-structure iff their causal structure is X → Z ← Y. The random variable Z is a confounder (or common cause) of the pair of random variables (X, Y) if (X, Y, Z) have causal structure X ← Z → Y. The skeleton U of a DAG G is obtained by replacing all the directed edges in G by undirected edges. Discovering the causal structure of a random vector is a difficult task when considered in full generality. Because of this reason, the literature in causal inference relies on a set of common assumptions BID27. The causal sufficiency assumption states that there are no unobserved confounders. The causal Markov assumption states that all the d-separations in the causal graph G imply conditional independences in the observational distribution P. The causal faithfulness assumption states that all the conditional independences in the observational distribution P imply d-separations in the causal graph G. We call Markov equivalence class to the set of graphs containing the same set of d-separations. When using the causal faithfulness assumption and conditional independence information, we are able to recover the Markov equivalence class of the causal structure underlying a random vector -which, in some cases contains one graph, the causal structure itself. Markov equivalence classes are DAGs where some of the edges remain undirected. Learning FCMs from data using score methods Consider a random vector X = (X 1, . . ., X d) following the FCM C = (G, f, Q) with associated observational distribution P. Furthermore, assume access to n samples drawn from P, denoted by DISPLAYFORM3, where DISPLAYFORM4 for all i = 1,..., n. Given these data, the goal of observational causal discovery is to estimate the underlying causal DAG G and the causal mechanisms f.One family of methods for observational causal discovery are score-based methods BID0. In essence, score-based methods rely on some score-function S(G, D) to measure the fit between a candidate set {G, f} and the observed data D. Then, we select the DAG on d variables achieving the maximum score as measured by S. As an example of score-function, consider the Bayesian Information Criterion (BIC): DISPLAYFORM5 where pθ j the maximum-likelihood estimate of a simple parametric family of conditional distributions p θ∈Θ allowing efficient density evaluation. The term λ ∈ [0, ∞) penalizes the number of edges (that is, the model complexity assuming equal number of parameters per edge) in the graph. Finally, we may associate each edge X i → X j in G to an importance or confidence score proportional to its contribution to the overal loss: as DISPLAYFORM6 A naïve score-based method would enumerate all the DAGs of d variables and select the one maximizing S. Unfortunately, the number of DAGs over d nodes is super-exponential in d. Thus, the brute-force search of the best DAG is intractable, even for moderate d. Inspired by BID38; BID25, we assume in this paper known graph skeletons. Such a skeleton may arise from expert knowledge or a feature selection algorithm algorithm BID39 under standard assumptions such as causal Markov, faithfulness, and sufficiency. Given a skeleton with k edges, causal discovery reduces to selecting one out of the O(2 k) possible edge orientations. This section proposes a framework to learn FCMs from data by leveraging the representational power of generative neural networks. In particular, we propose to estimate FCMs C asĈ = (Ĝ,f,Q), witĥ DISPLAYFORM0 for all i = 1,..., d. Here,Ĝ is the estimated causal graph of X, the functionsf = (f 1, . . .,f d) are the estimated causal mechanisms of X producing the estimated observed variablesX = (X 1, . . .,X d), and the estimated noise variablesÊ = (Ê 1, . . .,Ê d) are sampled from a fixed distributionQ. Given the estimated FCM, we can draw n samples from its observational distributionP (see Section 2) and construct the estimated observational samplesD DISPLAYFORM1. We parametrize the equations as generative neural networks, also known as conditional generators BID8. Without loss of generality, we assume that the independent noise variableŝ E are sampled from an univariate Normal distribution BID37. Then, we propose the following score-function to measure the fit between a candidate structureĜ and data D: DISPLAYFORM2 where MMD is the Maximum Mean Discrepancy statistic BID10: DISPLAYFORM3 The MMD statistic scores a graphĜ by measuring the discrepancy between the data observational distribution P and the estimated observational distributionP, on the basis of their samples. When using a characteristic kernel k such as the Gaussian kernel k(x, x) = exp(−γ x − x 2 2), MMD is an well-defined score-function: it is zero if and only if P =P as n → ∞ BID10. Since the computation of MMD k takes O(n 2) time, our experiments will also consider an approximation based on m random features , denoted by MMD m k. Appendix A offers a brief exposition on MMD. In a nutshell, CGNN implements Occam's razor to prefer simpler models as causal. Unlike previous methods, CGNN can seamlessly leverage both distributional asymmetries (due to the representational power of generative networks) and conditional independences (due to the joint minimization of those networks using MMD) to score both bivariate and multivariate graphs. For a differentiable kernel k such as the Gaussian kernel, the score function is differentiable and therefore CGNN is trainable using backpropagation. CGNN is a directed acyclic graph of conditional generator networks that in a flexible generative model of the data causal structure. Searching causal graphs with CGNN Using the CGNN score, we propose the following greedy approach to orient a given skeleton: 1. Orient each X i − X j as X i → X j or X j → X i by selecting the 2-variable CGNN with the best score.2. Remove all cycles: all paths starting from a random set of nodes are followed iteratively until all nodes are reached; an edge pointing towards an already visited node forms a cycle, so is reversed.3. For a number of iterations, reverse the edge that leads to the maximum improvement over a d-variable CGNN, without creating a cycle.Dealing with hidden confounders The search method above assumes the causal sufficiency assumptions: or, the non-existence of hidden confounders. We address this issue in a variant of our algorithm as follows. When assuming the existence of confounders, each edge X i − X j in the skeleton is due to one out of three possibilities: either X i → X j, X j ← X i, or there exists an unobserved variable E i,j such that X i ← E i,j → X j. Therefore, each equation in the FCM is extended to: DISPLAYFORM4 where Ne(i; S) ⊂ {1, . . . d} is the set of indices of the variables adjacent to X i in the skeleton. Here each E i,j ∼ Q and denotes the hypothetical unobserved common causes of X i and X j. For instance, if we hide X 1 from the FCM described in Figure 1, this would require considering a confounder E 2,3. Finally, when considering hidden confounders, the third step above considers three possible mutations of the graph: reverse, add, or remove an edge. Here, the term λ|Ĝ| takes an active role and promotes simple graphs. DISPLAYFORM5 We evaluate the performance of CGNN at discovering different types of causal structures. We study the problems of discovering cause-effect relations (Section 4.1), v-structures (Section 4.2), and multivariate causal structures without (Section 4.3) or with (Section 4.4) hidden variables. Our experiments run at an Intel Xeon 2.7GHz CPU, and an NVIDIA 1080Ti GPU. MMD uses a sum of Gaussian kernels with bandwidths γ ∈ {0.005, 0.05, 0.25, 0.5, 1, 5, 50}. CGNN uses onehidden-layer neural networks with n h ReLU units, trained with the Adam optimizer BID18 and initial learning rate 0.01. According to preliminary experiments, using all data for both training and evaluating models produces good , since resampling noise variables conbats overfitting. Also, our best follow when using the whole data as a minibatch. We train CGNN during n train = 1000 epochs and evaluate it on n eval = 500 generated samples. We ensemble CGNN training over n run = 32 random initializations for MMD k and n run = 64 for MMD m k. Regarding CGNN model selection, the number of hidden units n h is the most sensitive hyperparameter, and should be cross-validated for every application. The number of hidden units n h relates to the flexibility of the CGNN to model each of the causal mechanisms. For small n h, we may miss some of the patterns in the data. For a large n h, we may find over-complicated explanations from effects to causes. Therefore, our interest is to find the smallest n h explaining the data well. We illustrate such Occam's razor principle in FIG0, where we learn two bivariate CGNNs of different complexity (n h = 2, 5, 20, 100) using data from the FCM: FIG0 shows the associated MMDs (averaged on 32 runs), confirming the importance of capacity control BID40. On this illustrative case the most discriminative value appears for n h = 2. DISPLAYFORM0 Under the causal sufficiency assumption, the statistical dependence between two random variables X and Y is due to a causal relation X → Y or due to a causal relation X ← Y BID27. Given data from the observational distribution P (X, Y), this section evaluates the performance of CGNN to decide whether X → Y or X ← Y.In the following, we use five cause-effect inference datasets, covering a wide range of associations. CE-Cha contains 300 cause-effect pairs from the challenge of BID11. CE-Net contains 300 artificial cause-effect pairs generated using random distributions as causes, and neural networks as causal mechanisms. CE-Gauss contains 300 artificial cause-effect pairs as generated by BID43, using random mixtures of Gaussians as causes, and Gaussian process priors as causal mechanisms. CE-Multi contains 300 artificial cause-effect pairs built with random linear and polynomial causal mechanisms. In this dataset, we simulate additive or multiplicative noise, applied before or after the causal mechanism. CE-Tueb contains the 99 real-world scalar cause-effect pairs from the Tübingen dataset, concerning domains such as climatology, finance, and medicine. We set n ≤ 1, 500. See our implementation for details. CGNN is compared to following algorithms: The Additive Noise Model, or ANM, with Gaussian process regression and HSIC independence test. The Linear Non-Gaussian Additive Model, or LiNGAM BID32, a variation of Independent Component Analysis to identify linear causal relations. The Information Geometric Causal Inference, or IGCI BID4, with entropy estimator and Gaussian reference measure. The Post-Non-Linear model, or PNL BID40, with HSIC test. The GPI method BID37, where the Gaussian process regression with higher marginal likelihood is preferred as the causal direction. The Conditional Distribution Similarity statistic, or CDS BID6, which prefers the causal direction with lowest variance of conditional distribution variances. The award-winning method Jarfo BID6, a random forest classifier trained on the ChaLearn Cause-effect pairs and hand-crafted to extract 150 features, including the methods ANM, IGCI, CDS, and LiNGAM.The code for ANM, IGCI, PNL, GPI, and LiNGAM is available at https://github.com/ ssamot/causality. We follow a leave-one-dataset-out scheme to select the best hyperparameters for each method. For CGNN, we search for the number of hidden neurons n h ∈ {5, 10, 15, 20, 25, 30, 35, 40, 50, 100}. The leave-one-dataset-out hyperparameter selection chooses n h equal to 35, 35, 40, 30, 40 for the CE-Cha, CE-Net, CE-Gauss, CE-Multi and CE-Tueb datasets respectively. For ANM, we search the Gaussian kernel bandwidth γ used in the Gaussian process regression in {0.01, 0.1, 0.2, 0.5, 0.8, 1, 1.2, 1.5, 2, 5, 10}. For LiNGAM and IGCI, there are no parameters to set. For PNL, we search for the significance level of the independence test α ∈ {0.0005, 0.005, 0.01, 0.025, 0.04, 0.05, 0.06, 0.075, 0.1, 0.25, 0.5}. For GPI, we use the default parameters from the original implementation. For CDS, we search for the best discretization of the cause variable into {1, . . ., 10} levels. For Jarfo, we train the random forest using 4, 000 cause-effect pairs generated in the same way as the proposed datasets, except the one used for testing. TAB1 reports the Area Under the Precission/Recall Curve (AUPRC) associated to the binary classification problem of deciding "X → Y " or "X ← Y " for each cause-effect pair, for all methods and datasets. The table also shows the computational time (in both CPU and GPU), and computational complexity for methods. The least performing methods are those based on linear regression. The methods CDS and IGCI perform well on a few datasets. This indicates the existence of certain biases (such as causes having always higher entropy than effects) on such datasets. ANM performs well when the additive noise assumption holds (for instance, CE-Gauss), but badly otherwise. PNL, a generalization of ANM, compares favorably to these methods. Jarfo, the method using thousands of training cause-effect pairs to learn from data, performs well on artificial data but badly on real examples. The generative methods GPI and CGNN show a good performance on most datasets, including the real-world cause-effect pairs CE-Tueb. In terms of computation, generative methods are the most expensive alternatives. Fortunately for CGNN, the approximation of MMD with random features (see Appendix A) does not degrade performance, but reduces the computation time. Overall, these suggest that CGNN is competitive compared to the state-of-the-art on the cause-effect inference problem, where it is necessary to discover distributional asymmetries. This section studies the performance of CGNN on the task of identifying the causal structure of three random variables (A, B, C) with skeleton A − B − C. The four possible structures are the chain A → B → C, the reverse chain A ← B ← C, the v-structure A → B ← C, and the reverse v-structure A ← B → C. Other skeletons are not of interest, since the absence of an edge (statistical independence) is easier to discover, and the remaining edge could be oriented using the bivariate methods described in the previous section. Three of the possible structures (the chain, the reverse chain, and the reverse v-structure) are Markov equivalent, and therefore indistinguishable from each other using statistics alone. Therefore, the goal of this section is to use CGNN to determine whether P (A, B, C) follows or not an FCM with causal graph: A → B ← C. This section considers an FCM with identity causal mechanisms and Normal noise variables; for instance, B ← A + E B, where E B ∼ N. Therefore, the joint distribution of one cause and its effect is symmetrical (a two-dimensional Gaussian), and the bivariate methods used in the previous section all fail to apply. To succeed at this task, a causal discovery method must reason about the conditional independences between the three random variables at play. Our protocol fits one CGNN for each of the four possible causal graphs with skeleton A − B − C. Then, we evaluate MMD of each of the four CGNN models, and prefer the one achieving the lowest MMD. Table 2 summarizes our : CGNN assigns the lowest MMD to the v-structure hypothesis on those datasets generated by v-structures, and assigns the largest MMD to the v-structure hypothesis on those datasets not generated by v-structures. Sections 4.1 and 4.2 show the two complementary properties of the CGNN: leveraging distributional asymmetries and conditional independences. Consider a random vector X = (X 1, ..., X d). Our goal is to find the FCM of X under the causal Markov, faithfulness and causal sufficiency assumptions. At this point, we will assume known skeleton, so the problem reduces to orienting every edge. To that end, all experiments provide all algorithms the true graph skeleton, so their ability to orient edges is compared in a fair way. This allows us to separate the task of orienting the graph from that of uncovering the skeleton. We draw 500 samples from four artificial causal graphs G 2, G 3, G 4, and G 5 on 20 variables. For i = {2, . . ., 5}, the variables in the graph G i have a random number of parents between 1 and i. We build the graphs with polynomial mechanisms, and additive/multiplicative noise. We compare CGNN to the PC algorithm BID35, the score-based method GES , ANM, LiNGAM, and Jarfo. For PC, we employ the better-performing, order-independent version of the PC algorithm proposed by BID2. PC needs the specification of a conditional independence test. We compare PC-Gaussian, which employs a Gaussian conditional independence test on Fisher z-transformations, and PC-HSIC, which uses the HSIC conditional independence test with the Gamma approximation BID9. For both conditional independence tests, the significance level achieving best is α = 0.1. For GES, the best penalization parameter is λ = 3.11. PC and GES are implemented in the pcalg package. For CGNN, n h is set to 20. We also compare to pairwise methods presented in the last section: ANM, LiNGAM, and Jarfo. TAB3 displays the performance of all algorithms measured from the area under the precision/recall curve. Overall, the best performing method is PC-HSIC, followed closely by CGNN. The performance of PC-HSIC is best for denser graphs. This is because the PC algorithm uses a majority voting rule to decide each orientation, one strategy well suited to dense known skeletons, since one edge belongs to multiple v-structures. However, CGNN offers the advantage to orient all the edges (while some edges remain undirected by PC-HSIC) and to deliver a full generative model useful for simulation (while PC-HSIC only gives the graph). To explore the scalability of our method, we were able to extend the experiment on 5 graphs G 3 with 100 variables, achieving an AUPRC of 85.5 ± 4, in 30 hours of computation on four NVIDIA 1080Ti GPUs. In real applications, some confounding variables may be unobserved. We propose to use the same data from the previous section, but hide some of the 20 observed variables in the graph. More specifically, we hide three random variables that cause at least two others in the same graph. Consequently, the skeleton now includes additional edges X − Y for all pairs of variables (X, Y) that are consequences of the same hidden cause (confounder). The goal in this section is to orient the edges due to direct causal relations, and to remove those edges due to confounding. We compare CGNN to the RFCI algorithm, which is a modification of the PC algorithm that accounts for hidden variables. As done in the previous section, we compare variants of RFCI based on Gaussian or HSIC conditional independence tests. We also evaluate the performance of the data-driven method Jarfo, this time trained on the whole Kaggle data of BID11, in order to classify relations into X → Y, X ← Y, or X ← Z → Y (confounder). For CGNN, we penalize the objective function with λ = 5 × 10−5. TAB4 shows that CGNN is robust to the existence of hidden confounders, achieving state-of-the-art performance in this task. Interestingly, the true causal relations exhibit a high confidence score, while edges due to confounding effects are removed or have low confidence scores. Overall, CGNN performs best on the graphs G 2, G 3, and G 4, and is slightly outperformed by RFCI-HSIC on the denser graph G 5. However, CGNN is the only approach providing a generative model of the data. We introduced a new framework to learn functional causal models based on generative neural networks. We train these networks by minimizing the discrepancy between their generated samples and the observed data. Such models are instances of the bigger family of FCMs for which each function is a shallow neural network with n h hidden units. We believe that our approach opens new avenues of research, both from the point of view of leveraging the power of deep learning in causal discovery and from the point of view of building deep networks with better structure interpretability. Once the model is learned, the CGNNs present the advantage to be fully parametrized and may be used to simulate interventions on one or more variables of the model and evaluate their impact on a set of target variables. This usage is relevant in a wide variety of domains, typically among medical and sociological domains. Five directions for future work are to i) lower the computational cost of CGNN, ii) extend CGNN to deal with categorical data, iii) explore better heuristics for causal graph search, iv) adapt our methods for temporal data and v) obtain theoretical guarantees for basic use cases. The Maximum Mean Discrepancy (MMD) statistic BID10 measures the distance between two probability distributions P andP, defined over R d, as the real-valued quantity DISPLAYFORM0 Here, µ k = k(x, ·)dP (x) is the kernel mean embedding of the distribution P, according to the real-valued symmetric kernel function k(x, x) = k(x, ·), k(x, ·) H k with associated reproducing kernel Hilbert space H k. Therefore, µ k summarizes P as the expected value of the features computed by k over samples drawn from P.In practical applications, we do not have access to the distributions P andP, but to their respective sets of samples D andD, defined in Section 3. In this case, we approximate the kernel mean embedding DISPLAYFORM1, and respectively forP. Then, the empirical MMD statistic is DISPLAYFORM2 Importantly, the empirical MMD tends to zero as n → ∞ if and only if P =P, as long as k is a characteristic kernel BID10. This property makes the MMD an excellent choice to model how close the observational distribution P is to the estimated observational distribution P. Throughout this paper, we will employ a particular characteristic kernel: the Gaussian kernel k(x, x) = exp(−γ x − x 2 2), where γ > 0 is a hyperparameter controlling the smoothness of the features. In terms of computation, the evaluation of MMD k (D,D) takes O(n 2) time, which is prohibitive for large n. When using a shift-invariant kernel, such as the Gaussian kernel, one can invoke Bochner's theorem BID30 The literature about learning FCMs from data is vast. We recommend the books BID35 BID27 BID28 and surveys BID16 BID13 BID5. FCM learning methods can be classified into bivariate and multivariate algorithms. On the one, pairwise algorithms aim at orienting the cause-effect relation between two random variables (X, Y) by searching for asymmetries in the distribution P (X, Y). The Additive Noise Model, or ANM BID15, assumes an FCM with form Y ← f (X) + E, where the cause X is statistically independent from the noise E. Following these assumptions, the ANM performs one nonlinear regression in each direction, and prefers the one that produces residuals statistically independent from the alleged cause. The Post Non-Linear (PNL) model BID40 extends the ANM by allowing FCMs with form Y ← g(f (X) + E), where g is a monotone function. The IGCI method BID4 prefers the causal direction producing a cause distribution independent from the derivative of the causal mechanism. The LiNGAM method BID32 leverages independent component analysis to orient linear cause-effect relations. The CURE

Represent the short summary for finding the corresponding scientific article (long)

Discover the structure of functional causal models with generative neural networks

Causally correct partial models do not have to generate the whole observation to remain causally correct in stochastic environments.