Question: Represent the scientific article to find a short TLDR

Question: Data breaches involve information being accessed by unauthorized parties. Our research concerns user perception of data breaches, especially issues relating to accountability. A preliminary study indicated many people had weak understanding of the issues, and felt they themselves were somehow responsible. We speculated that this impression might stem from organizational communication strategies. We therefore compared texts from organizations with external sources, such as the news media. This suggested that organizations use well-known crisis communication methods to reduce their reputational damage, and that these strategies align with repositioning of the narrative elements involved in the story. We then conducted a quantitative study, asking participants to rate either organizational texts or news texts about breaches. The findings of this study were in line with our document analysis, and suggest that organizational communication affects the users' perception of victimization, attitudes in data protection, and accountability. Our study suggests some software design and legal implications supporting users to protect themselves and develop better mental models of security breaches. A data breach is a successful malicious attack which leads to the compromise or the loss of data. Personally Identifiable Information (PII) is often stored in organization databases, and if disclosed is at risk of misuse. Depending on the size, scale, and type of stolen information, the potential consequences of a data breach can be huge. A data breach can put people at risk of identity theft, which often happens through fraudulent use of existing accounts like credit cards, online accounts, and Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. insurance. It can also lead to financial loss and emotional distress. Despite the increased awareness of organizations and great emphasis by experts on security mechanisms, many organizations still maintain insufficient security practices on data collection, processing, and storage, so are unable to prevent data breaches and consequent misuse of the data. Several recent occurrences follow this pattern, and data breaches at major companies, like Equifax, have exposed a massive number of consumers' records. Although such events have become commonplace, there appears to be little indication that end-users feel urgency about holding companies to account. A 2016 study reports that by far most consumers kept doing business with companies after breaches, and some high-profile commentary suggests "breach fatigue" has "set a new normal and instill a sense of fatalism -and complacency". In a small preliminary study, we even found that participants often thought that they themselves were somehow responsible for data breaches. According to Coombs, the reputation of a company is based on the evaluation customers make about it. Customer evaluations can be affected by the behavior of a company when a crisis like a data breach happens. So, due to the significant financial loss and reputational damage caused by data breaches, companies try to reduce the damage using communication strategies in the after-breach notifications. The crisis response strategies aim to reduce the negative effects of the crisis by changing the level of crisis responsibility. For example, if a company frames themselves as victims of the situation and therefore positioned in what crisis communication theorists call the "victim cluster", they are likely to incur little blame for the crisis. User understanding of data breach incidents are important because it allows development of mental models to support reasoning about behavior and accountability. The goal of our research is to explore how breached companies and news media communicate with users, and how that might affect users' perception of a data breach incident. To do so, we apply Image Repair Theory (IRT) and a narrative-semiotics method to the analysis of Equifax crisis communications to see how this incident is reported in the company press releases and the news. We first conducted a communication study based on collected data from 58 stories related to this security breach crisis. We then conducted a questionnaire study with 100 participants testing the influence of companies' notifications and news on the general public. To the best of our knowledge, testing communication strategies' influence on users mental models of a data breach is original, and it shows HCI efforts on building user understanding of security can be undermined by organizational communication. Our also suggest a need for the improvement of software design, and delicate attention of communication professionals and legal scholars to the notifications created during and after a data breach. Data breaches are a severe threat to both organizations and consumers, and are increasingly common across businesses. When a data breach happens, the enterprise facing this crisis is required to inform the legal authorities, and it also needs to notify all the affected and potentially affected consumers. For these reasons, data breaches are now understood as a crisis for an organization, and there is an established body of knowledge about how organizations should communicate about crises. There are several theories on effective communication strategies during crises. In our work we used IRT, and also a narrative semiotic method. IRT, introduced by William Benoit is a well-established framework, and it can be used by practitioners to design messages during a crisis. It can also be used by critics to evaluate the created messages. The key concept in this theory is to understand the nature of a complaint or attack. An attack has two components: an offensive act, and the accusation of responsibility for an action. According to Benoit, the image repair strategies can be categorized into five broad categories: denial, evasion of responsibility, reducing the offensiveness of event, corrective action, and mortification. Denial is a general approach to image repair, and it is about rejecting responsibility. Evasion of responsibility contains four sub-strategies: provocation (response to someone else's action), defeasibility (lack of information or control over the situation), accident (did not mean it to happen), and good intention. Reducing the offensiveness of an event also involves a list of potential response strategies: bolstering (reminds of good traits), minimization (claim that act was not serious), differentiation (reduce offensiveness of the act), transcendence (place the act in more favorable context), attack accuser (challenge the credibility of accuser), and compensation (reimburse for the act). Corrective action is about restoring situation or promise that the act will not happen again, and mortification is asking for forgiveness. Semiotics involves the study of signs, and semioticians believe that communication is symbolic and ambiguous, and it happens through perceptual or linguistic signs between interlocutors. The narrative-semiotic method finds common patterns in stories. The strategies for examining storytelling help to make sense of what the narrator has perceived and experienced. This can reveal conflicts and changes during a crisis. Based on news framing theory, media and organizations use different features in their messages to frame the crisis. We can use narrative semiotics to see how these framings are different. This method is valuable for understanding a data breach situation, and the goals and motivations of different narrators of a data breach story. It can help researchers to study a failure of an organization and assess whether the decisions made by the organizations were appropriate or not. We can divide the narrative-semiotic approach into two aspects: the narrative trajectory which is the sequence of events and actions that create a story, and the narrative schema. The narrative schema consists of the six categories of agents known as the actantial model. The actantial model, developed by A.J.Greimas, can be used to break an action down into six positions or actants: 1. The sender includes agents who direct the action of subject towards an object. 2. The subject consists of the leading performers aiming at a desired goal or object. 3. The object includes the desired goal and objectives. 4. The receiver category consists of the agents who benefit if a desired goal is achieved. 5. The helper includes agents who assist in achieving the desired goal, like experts who aid the subject. 6. The opponent includes agents who hinder the achievement of the desired goal, for example, adversaries, lack of knowledge or ability, and ineffective tools. Narrative-semiotics can be used to identify the agents, their actions, and their discourses. Different actions reflect different points of view, which is why it is vital to compare different texts on the same subject. A series of events can be told differently when the narrator changes. Narrators position the agents through the way they tell the story. So the narrator has a pivotal role in interpreting the actions and highlighting the patterns in their interaction. Comparing the narrative structure of different texts can reveal how companies narrate a crisis like a data breach, and how their narrative can be different from the narratives produced by media organizations. The way a story is told using strategies like IRT might change readers' mental model of a data breach, in other words, the way they see and interpret the situation. We propose that using communication strategies and changing the role of agents in a story might affect the readers' perception of the reality and of the companies' role relating to the breach. We analyzed the Equifax data breach because it happened recently (the Equifax's data breach occurred on May 2017, with investigation and analyses into 2019), and is of great importance due to the following reasons: the significant role of Equifax on people by impacting their access to many necessities, the extreme sensitivity of financial information breached and risk of identity theft, the centrality of data security to Equifax -its primary mission is stewardship, and the comprehensive analysis of the causes showing how they were at fault. To retrieve all the stories concerning the data breach, we first searched the company's press releases archive. We then selected reputable news sites due to their national circulation and impact, and searched for keywords-"Equifax data breach". Some stories were excluded due to duplication or unrelated items such as the story of another data breach where it referred to the Equifax data breach. The analysis covers the period from September 2017 until March 2019 (see Table 1). We first analyzed the sources using Image Repair Theory because it forms the basis of different approaches in the field, and it is best suited to a rhetorical method that examines texts produced by a company to identify their language and discourse patterns. In the context of a data breach, responsibility can appear in different forms, a company can be blamed for a poorly performed action that hurts consumers or neglect like poor security practices or flaws in a system that allows a breach. Perception of responsibility and offensiveness of an action can seem more important than reality, so businesses try to use different strategies to affect the perception of responsibility. The coding sheet for image repair included the following strategies as nodes: shift the blame, defeasibility, accident, bolstering,minimization,compensation,corrective action, and mortification. We used NVivo12, which is a qualitative analysis tool, to code our data. We first imported all the documents into the software, different folders for different sources. We then created our first group of nodes for image repair strategies; we created a case node to represent each strategy, and we gathered references by coding sources at the nodes. We then used the narrative-semiotic method of analysis to understand how the positioning of agents changes when the narrator changes. We used this method to clarify how the company and the media communicate with the general public to direct their sense-making process. Based on the actantial model, the six role categories were applied to analyze the documents. To find the patterns in different sources we used the following categories as our nodes in NVivo12: the sender, the subject, the object, the receiver, the helper, and the opponent. IRT and Narrative-semiotics were used as a framework to explore the full text content of each document. One researcher did the coding and the were reviewed by two other researchers. To begin, we present a short description of major events and actions in the Equifax story. We divided Equifax's story into a series of narrative episodes which refer to different stages of the narrative trajectory. The information in this story was taken from press releases posted on the Equifax's official site and the GAO report. Equifax is a consumer credit reporting agency. Equifax collects and aggregates information on over 800 million individual consumers and more than 88 million businesses worldwide, and its database includes employee data contributed from more than 7,100 employers. On 7 September 2017, Equifax announced a cybersecurity incident impacting approximately 143 million U.S. consumers and unknown number in the UK and Canada. Hackers exploited a U.S. website application vulnerability to gain access to certain files in mid-May 2017. They stayed on the system until they were detected in July 2017. The information accessed primarily includes names, Social Security numbers, birth dates, addresses and, in some instances, driver's license numbers. In addition, credit card numbers for approximately 209,000 U.S. consumers, and certain dispute documents with personal identifying information, for approximately 182,000 U.S. consumers, were accessed. As part of its investigation of this application vulnerability, Equifax also identified unauthorized access to limited personal information for certain UK and Canadian residents. According to Equifax officials, beginning on May 13, 2017, attackers gained access to the online dispute portal (it maintained documents used to resolve consumer disputes) and used a number of techniques to disguise their activity. They extracted a portion of the PII (Personally Identifiable Information) residing on the systems. After successfully accessing the information, the attackers exfiltrated the data in small increments, using standard encrypted web protocols to disguise the exchanges as normal network traffic. The attack lasted for about 76 days before it was discovered. Equifax officials stated that, on July 29, 2017, approximately 2.5 months after the attackers began extracting sensitive information, security personnel conducting routine checks of the operating status and configuration of IT systems detected the intrusion on the online dispute portal. A misconfiguration due to an expired digital certificate was the reason the intrusion was not noticed before. Equifax then blocked several Internet addresses from which the requests were being executed to try to stop the attack. The IT department discovered a vulnerability in the Apache Struts web application framework as the initial attack vector. The US-CERT had notified the company about this vulnerability before this incident. The Apache Foundation also had reported the vulnerability (CVE-2017-5638) 1 in early March 2017. Equifax took the website offline and then took steps to identify the stolen data and the number of affected people by this incident. Once Equifax officials found out how the attackers were able to access to the company's databases, they took measures to address this problem and avoid it in future. For the challenging task of identifying the affected individuals, Equifax compared the affected database with company's internal databases that were not impacted by the data breach. On September 7, 2017, Equifax stated in its press release that the company had set up a dedicated website to help individuals determine if their information might have been exposed in the breach. Additionally, Equifax reported that it would provide several services to all U.S. consumers, regardless of whether their information had been compromised, free of charge for one year. After the investigation, the company notified all U.S. state attorneys general regarding the approximate number of potentially affected residents in each state and its plans for consumer remediation. On March 1, 2018, Equifax stated that, overall, 2.4 million U.S. consumers whose names and partial driver's license information were exposed, had been identified. The GAO report reveals how Equifax failed to protect Americans' personal data. According to the GAO, "Equifax determined that several major factors had facilitated the attackers' ability to successfully gain access to its network and extract information from databases containing PII," and that "key factors that led to the breach were in the areas of identification, detection, segmentation, and data governance." Finally, the GAO's report highlights the critical need for legislation to protect consumers whose data is not adequately safeguarded, such as Senator Warren's and Senator Mark Warner's bill to hold credit reporting agencies like Equifax liable for data breaches. Under this legislation, Equifax would have paid at least $1.5 billion in penalties for the data breach. We evaluated the company's press releases during the crisis using IRT. See Table 2 for the strategies used in press releases and illustrations of them. This table suggests that Equifax used the image repair strategies to minimize the apparent consequences of the data breach. Since the company was believed to be responsible for the incident, the CEO blamed the entire situation on IT staff who had not installed an Apache Struts patch issued in the weeks before the hack, and on technology failures. Moreover, the company used other strategies like: bolstering, compensation and corrective actions to reduce the offensiveness of the data breach. As a final general strategy, the CEO apologized to victims. The second level of analysis involves the Narrative-Semiotic approach. Here we classified the agents in specific roles as they interact in the sequences of actions. Since a series of events can be told differently, the identification of the agents would change according to the role of the narrator. The initial document that was considered for the narrative analysis was "Prepared testimony of Richard F. Smith before the U.S. House Committee on Energy and Commerce Subcommittee on Digital Commerce and Consumer Protection". The narrative story starts with the talk of the CEO of Equifax as a narrator about the initial situation of the company and how Equifax offered several services to its customers. Consider the following extract from the aforementioned text: FROM PREPARED TESTIMONY OF RICHARD F. SMITH: Equifax was founded 118 years ago and now serves as one of the largest sources of consumer and commercial information in the world. That information helps people make business and personal financial decisions in a more timely and accurate way. Behind the scenes, we help millions of Americans access credit, whether to buy a house or a car, pay for college, or start a small business. During my time at Equifax, working together with our employees, customers, and others, we saw the company grow from approximately 4,000 employees to almost 10,000. Some of my proudest accomplishments are the efforts we undertook to build credit models that allowed and continue to allow many unbanked Americans outside the financial mainstream to access credit in ways they previously could not have. Throughout my tenure as CEO of Equifax, we took data security and privacy extremely seriously, and we devoted substantial resources to it. We set out to notify American consumers, protect against increased attacks, and remediate and protect against harm to consumers. In recent weeks, vulnerability scanning and patch management processes and procedures were enhanced. We took data security and privacy extremely seriously, and we devoted substantial resources to it. Equifax is doing everything in its power to prevent a breach like this from ever happening again. Mortification Apologize I am here today to apologize to the American people myself and on behalf of the Board, the management team, and the company's employees. To each and every person affected by this breach, I am deeply sorry that this occurred. I sincerely apologize. I will close by saying again how so sorry I am that this data breach occurred. In this extract, customers (general people or businesses) are described as subjects who want to buy a house or a car, or start a small business. Equifax stepped into the role of helper to give information and to help people make business and personal financial decisions, and millions of Americans are described as receivers. Figure 1a shows the actantial model inferred from this text. In the second stage of Equifax's story, a complication happened due to an external threat (attackers On July 29, however, Equifax's security department observed suspicious network traffic associated with the consumer dispute website (where consumers could investigate and contest issues with their credit reports). In response, the security department investigated and immediately blocked the suspicious traffic that was identified. The department continued to monitor network traffic and observed additional suspicious activity on July 30, 2017. In response, they took the web application completely offline that day. The criminal hack was over, but the hard work to figure out the nature, scope, and impact of it was just beginning. The narrator of this press release proposes an attacker as the subject, and accessing information is the object. The Apaches Struts vulnerability and the Equifax's security tools that couldn't detect the illegal access are both helpers that assist the attacker. The opponent category in this piece of story includes: the security department investigation, blocking the suspicious traffic, network monitoring, and taking the web application offline. The narrator (the company's CEO) emphasized the company's corrective actions to hinder the attacker in the narrative quest. need to patch a particular vulnerability in certain versions of software used by other businesses. ". Equifax had 5 days to patch the vulnerability before the first date when attackers accessed sensitive information. Since the attacker could exploit the vulnerability of the Apaches Struts, the notification of CERT was a helper to the attacker too (See Figure 1b). In the second stage of Equifax's story, Equifax's actions to defend against the intrusion are described in the actions taken to address the complication. Therefore, according to the story, the agent's categories change in the narrative. The story starts with the repair efforts of the company, as the CEO as a narrator confessed that they failed to protect American consumer data and apologized for the act of data breach. The narrator (Richard F. Smith -Equifax's now retired CEO) continued to describe certain actions regarding how this incident happened. FROM PREPARED TESTIMONY OF RICHARD F. SMITH: Americans want to know how this happened and I am hopeful my testimony will help in that regard. As I will explain in greater detail below, the investigation continues, but it appears that the breach occurred because of both human error and technology failures. These mistakes -made in the same chain of security systems designed with redundancies -allowed criminals to access over 140 million Americans' data. Upon learning of suspicious activity, I and many others at Equifax worked with outside experts to understand what had occurred and do everything possible to make this right. Ultimately we realized we had been the victim of a massive theft, and we set out to notify American consumers, protect against increased attacks, and remediate and protect against harm to consumers. We developed a robust package of remedial protections for each and every American consumer -not just those affected by the breach -to protect their credit information. The relief package includes: monitoring of consumer credit files across all three bureaus, access to Equifax credit files, the ability to lock the Equifax credit file, an insurance policy to cover out-of-pocket costs associated with identity theft; and dark web scans for consumers' social security numbers. In this extract, Equifax is foregrounded as the main agent, occupying four positions. Equifax is described as a sender dictating to its employees and outside experts to make the suspicious activity right. Equifax also stepped into the role of receiver, as well as helper. Opponents in this part are criminals, human errors and technology failure. Figure 1c shows the actantial model inferred from this text. Figure 1 shows the actantial model of the Equifax's story and we can see how the substories are connected to each other. The customer is subject who wants to buy something or start a business as object, and Equifax is its helper to achieve the goal. In the other part of the story Equifax stepped into the role of subject who wants to solve the data breach incident, Equifax protection activity is a helper here, and human error, computer failure and attackers are playing the role of opponent. In the last part of the story, attackers are subjects who want to find access to the personal information of Equifax's customers, human error and computer failure is a helper and Equifax protection activity is an opponent. The other documents were the news provided in the technical websites, general news, and the GAO report. After coding these documents using narrative components, we found two narrative programs or mini-narratives that were common among these texts; one from the attack stage of the story and the other from the response part of the story. In the first narrative episode, the attacker is the subject, and getting access to the information is the object. The company's failure to patch the flaw, an expired certificate, doing the malicious activity without being noticed, and lack of restriction on the database are all helpers to the attacker. The opponent category includes locking down the system so that the attackers would not be able to misuse the vulnerability and hiring an expert security team. Figure 2a shows the actantial model of this mini-narrative. In the second narrative, Equifax is the subject who wants to solve the incident (see Figure 2b). The helper category in this mini-narrative that was extracted from the news includes the following components: log files, and new regulations. Numerous data security failures such as the insecurity of Equifax's web setup, failing to patch the flaw promptly, the lack of restrictions on the frequency of database queries are the first element in the category of opponents. The second one is failure in notifying the data breach victims, the evidence that was extracted from different texts are as following: FROM GAO EQUIFAX REPORT: • Equifax executives -including its Chief Security Officer and Chief Executive Officer -kept the public in the dark for more than a month after they found out about the security intrusion. • The attack lasted for about 76 days before it was discovered. • Equifax and other big credit reporting agencies keep profiting off a business model that rewards their failure to protect personal information. • In the three years before the Equifax data breach, the company spent only about 3% of its operating revenue on cybersecurity-less than the company spent on stock dividends. • because of a process failure in 2016 that meant a limited amount of UK data was stored on the US system between 2011 and 2016. • Congressman Frank Pallone said Equifax had an "ongoing lax attitude when it comes to protecting consumer data. • An inadequate response to a data breach that included the personal information of up to 143 million Americans. • Equifax was breached in "mid-May" 2017, realized it in July and got around to telling the world in early September. • Apache Struts was popped, but company had at least TWO MONTHS to fix it. • specifically, the lack of restrictions on the frequency of database queries allowed the attackers to execute approximately 9,000 such queries -many more than would be needed for normal operations. • "As your company continues to issue incomplete, confusing and contradictory statements and hide information from Congress and the public, it is clear that five months after the breach was publicly announced, Equifax has yet to answer this simple question in full: what was the precise extent of the breach?" Senator Elizabeth Warren fumed in a missive late last week. The third element in the opponent category is inadequate assistance in resolving the problem like no definitive action to hold Equifax accountable, betraying stakeholders by top Equifax executives, an overwhelmed call center, and customer response site. Some examples of evidence found in the documents are: • It is always a company's responsibility to identify UK victims and take steps to reduce any harm to consumers. • Ying used confidential information to conclude that his company had suffered a massive data breach, and he dumped his stock before the news went public. That was a hastily constructed WordPress bodge job, and victims were initially asked to agree to take any dispute to arbitration and forfeit the right to take part in any classaction lawsuit. • Equifax also failed to provide consumers full protection from new account identity theft. Table 3: Participants' demographics • These consumer complaints included improper use of credit reports, incorrect information on credit reports, inadequate assistance in resolving problems, and problems with Equifax credit monitoring, fraud alerts, and security freezes in the wake of the breach. Our analysis shows that the narrative programs extracted from company's press release are different from the ones extracted from news. More details will be discussed later. As explained above, our text analysis showed important differences between how organizations described data breach incidents, and how they were described by others. In particular, the texts from organizations appeared to position themselves in ways that might affect how readers perceived their role relating to the breach. We speculated that this might influence users' understanding of the data breach, and of organizational accountability. To explore this, we conducted a questionnaire study, where we asked participants to first read data breach descriptions and then asked their perception of various aspects of the breach. We recruited participants through TurkPrime, which is an online crowdsourcing research platform that integrates with Amazon Mechanical Turk (MTurk). Participants were asked some questions about demographics and data breach basics, then read two short extracts describing data breaches, and finally responded to follow-up statements about the incidents described. The study was reviewed and cleared by our Research Ethics Board. We recruited 100 participants, specifying participants must be residents of the US or Canada. By far most were from the US, 96, with only 4 from Canada, with 33 female and 67 male. A summary of their demographics is shown in Table 3. There were two pairs of extracts about Equifax data breach. Each pair contained one extract from the company itself, and one from another source (e.g., news), but no company names were mentioned. Participants were shown a pair at random, and the order within the pair was also random. After each extract, participants were shown 10 statements, and responded using a Likert scale to gauge their perception about the breach motivations, company security measures, after-breach issues, and responsibility. Each of the responses were scored 1-5, where 1 stands for "Strongly disagree", 2 for "Somewhat disagree", 3 for "Neutral", 4 for "Somewhat agree", and 5 for "Strongly agree". Our hypothesis were that responses to each of the 10 statements would differ by the source of the text read by the participants. We examined responses, removing those unrealistically quick, or with inconsistent answers between similar questions. Then, we analyzed the of the questionnaire by calculating the median and spread of the distribution of our data. We also did Wilcoxon tests to see if the source (company or news), order of text within each group, and the pair of data breach description affected the participants' responses. Figure 3 shows the ten statements together with boxplots in describing the responses to each statement. For each statement, there are two boxplots, one for responses to the company text, and the other for responses to the news media text. After reading both the company text and the news text, in response to being asked who the victim in an Internet data breach is, participants strongly agreed that customers of the company are the victims (Mdn= 5/5; later numbers also refer to medians). However, the for the two texts was quite different when asked if the company is the victim of the data breach. After reading the company's description of the breach, participants agreed (4/5) that the company is also a victim, but reading the news shows a slight different , and they again agreed that the company could also be a victim (4/5). The data are not normal and are skewed to the neither agreed nor disagreed. When asked about the company's attitude regarding data protection, a considerable difference can be seen between the two texts. Participants reading the company's description disagreed (2/5) that the company had a relaxed attitude about protecting customers data, and they agreed that the company took security measures seriously (4/5). However, participants reading the news text showed different , and agreed (4/5) that the company had a relaxed attitude and they disagreed that the company took security measures seriously (2/5). In response to being asked what the attackers' purpose was, we got quite similar . They agreed that attackers wanted to harm customers as well as the company (4/5). Participants reading the company's text agreed (4/5) that the company was helping customers to recover from the breach. They somewhat agreed (4/5, with data skewed to disagreement) that the company put customers at risk by neglecting data protection. Those reading the news text, however, strongly agreed that the company put the customer at risk (5/5), and they disagreed that the company was helpful in after-breach actions (2/5). In response to being asked about the accountability, the of reading both the company and news' text were quite similar but significantly different at 0.05 level. Most participants believed that the company was accountable for problems ing from the data breach (Company= 4/5, News= 5/5), and the customers were not responsible (Company= 1/5, News= 2/5). Our hypotheses were that there would be a different response for those reading the company's text and those reading the news text about a data breach. To test our hypotheses, we did Wilcoxon tests for the responses to each statement, choosing this non-parametric test because the data was ordinal. The are shown in Table 4. We can see that the company's and the news description make a significant difference in participants' responses about victimization, the company's security measures, its attitude in data protection, and helpfulness in after-breach actions (these are marked with red boxes in Figure 3). n.s. 6. Attackers wanted to harm the customers. n.s. 7. The company is helping customers recover from the breach. We had two pairs of data breach descriptions (company and news), one of the pairs was assigned randomly to each participant, and the order within the pair was also randomized. For each statement we used Wilcoxon test to see if the pair (one of two) and the order (company first and news first) affected the participants responses. The showed that the order of texts did not change the participants responses. Which pair of texts also made no difference, except for the statement about whether the company is a victim or not and the helpfulness of about-breach actions. Although we used non-parametric tests, we also checked a multi-way ANOVA test for the pairs, order, and source of texts, and the confirmed our findings with Wilcoxon tests with little difference. In this study, we showed half our participants text about a data breach from a company, and half text about the same breach from a news source. We then asked them to respond to 10 statements about the breach, and we expected that their responses would differ for all 10 statements depending on the source of the text. As we described above, for most statements this is what occurred. For others, however, we saw little difference. For statement 1 (victimhood of customers), it seems that both sources led participants to feel that the customers were indeed victims, whereas statement 2 shows that people who read the company text more strongly felt that the company was also a victim. However, statements 5 and 6 (who the attackers wanted to harm) presented no difference in responses, despite their similarity to statements 1 and 2. We speculate that participants perceived a difference between the concept of victimhood and the concept of intent to harm. Both customers and company might, or might not, be victims, but intent to harm was more difficult to distinguish. For statement 9 (accountability of company), those who read the company text were less strong in feeling the company was accountable for the breach, but for statement 10 (accountability of customers), there was agreement that customers themselves were not accountable. Both these seem understandable. For statement 9, both sources suggest that the company has some level of accountability, but the non-company text makes it seem more clear. For statement 10, neither source suggests that customers have any accountability. On reflection, the issue of accountability has a subtlety that needs to be addressed. It is clear the attackers have a key role in the breach, so they can be seen as having accountability. The company, however, has a duty of care, and any failing in that duty might also be seen as involving accountability. In the physical world, burglars might rob a bank, but if it emerged that the bank left their doors unlocked at night, we suggest any bank customer would regard them as also accountable. The primary goal of our work was to explore how organization communications about data breaches might affect user perception. To do this, we first studied the nature of the communication itself. Using Image Repair Theory, we analyzed press releases posted on official company websites. We found that Equifax press releases had characteristics consistent with tactics to reduce reputational damage and therefore financial loss. Recognizing that the way the news media frames a crisis might be different to the framing in an organization's press releases, we next explored that issue. We used techniques from narrative semiotics to examine the structure of the stories being told, and found that the agents were not positioned the same way. Considering the first narrative story studied, our comparison of the Equifax press releases with news and GAO reports shows important differences with respect to the positioning of Equifax (see Figure 4). In the press releases, there was emphasis on Equifax as a helper, presenting the company's protection actions. In the news and GAO reports there was emphasis on Equifax as an opponent, presenting the company's weak security protection of consumer data. Moreover, the new media had a focus on the company and its security failure, whereas the company appears to use scapegoating as its primary crisis response strategy and suggesting responsibility lay with a single unnamed IT staff member. The ethics of scapegoating is doubtful, suggesting a manipulative approach used deflect responsibility. The company's apparently lax attitude in crisis response was heavily criticized by the media. The news text suggests Equifax shares responsibility for this incident. However, Equifax positioned itself as a receiver to emphasize it is a victim, a strategy that consistent with an attempt to reduce its responsibility. Equifax appears to map all their actions to the helper category, in a manner consistent with Image Repair Theory. For example, a bolstering strategy places the company in a helper position, deflecting responsibility by shifting the blame, scapegoating puts the company in victim position, and compensation strategies stress the company acts as a helper. However, when the news media narrates the story, the mapping of the actions and agents goes to the opponent category, since the media is not concerned with Image Repair. Our second step was to explore how the strategies used in the company press releases might influence the public understanding of data breaches. We conducted a questionnaire study to see if data breach incident descriptions from different sources, the company's and the news media, in different perceptions of the incident. After reading the text extracted from the company's press release, participants tended to rate the company's after-breach action and security measures higher. They also thought that the company was helping their customers and did not put them at any risk. However, we got different from participants who read the news texts reporting the same incident; participants disagreed that the company took the security seriously and their after breach protective actions were not acceptable to help the customers. The company was regarded as a victim after reading the company's description; however, the news approach in narrating the data breach ed in a different perception of participants. This therefore confirmed our speculations based on our text analysis. It also confirms the effectiveness of IRT and its relevance in crisis communication. Of course, it is not surprising that companies tend to present themselves in a better light that the news media. Nor is it surprising that they used strategies that have been developed to help them to do this. However, our text study shows that their Image Repair strategies exhibit some important characteristics. In particular, they show differences in how agency is presented, which, in turn affects readers' understanding of what happened. Studies of HCI and security highlight the role of user mental models in understanding issues relating to security (e.g. ), and we know many users continue to use breached services (see ). Our study shows that organizations, by their communications strategies, may be contributing to users weak mental models. One design implication might be improving software design to support users ability to track access to their data, with enough detail to help users determine provenance and legitimacy, perhaps with alarms for especially sensitive data. This would enable user engagement and oversight of their own information. This would allow users to inform organizations of discrepancies, and knowledge of this new transparency would also promote increased diligence by organizations. Moreover, organizations could make protective measures more explicit to users, both to assure users, and emphasize their diligence. The potential effect of all such measures could be studied in future research. We should also acknowledge that some of the measures we suggest may require changes in underlying software architecture, but research could show that such changes are justified, and potentially imperative. The possibility of crisis communications influencing user perception of accountability should be considered by communications professionals and legal scholars, to better establish the line between promoting the organization and misleading users. There are some limitations to address in this research. First, the communication study was focused on one case study, and same analysis on several data breaches may reveal different . Second, the wording of each statement in the questionnaire study might cause the difference that participants perceived in the concept of victimhood and the concept of intent to harm. In this paper, we presented our study on communication about data breach events which exposed private consumer data. We first analysed Equifax press releases and notifications to identify their the strategies, and then analysed news stories and government reports on the same events; we studied 58 stories in all. We found that the company used crisis communication strategies to reduce its reputational damage and financial loss. Our analysis also showed that there are differences between press releases, major newspaper and technical news when reporting the same data breach incident. In our narrative-semiotic analysis, we found the company mapped their after-breach actions into helper category; but the narrator of news reports mapped them into the opponent category. These narrative changes affected reader perception about these data breaches. Our questionnaire study revealed that the dissimilar approach detected in document analysis when narrating the same story from a different point of view (companies and news) has a considerable influence on the general public's perception of a data breach incident. Large scale data breaches are a serious matter, not just for organizations, but for the thousands or millions of users who have private data exposed, making them vulnerable to a range of consequences. Despite this, it is unclear if users understand what exactly has happened, where accountability lies, and how to proceed. In work on human factors in computer security, it has often been found that users have only weak mental models of online threats and defences, e.g.. When user data is exposed by a large scale data breach, communication with the user may well be primarily from the organization itself. Our research suggests that communication from organizations may misrepresent the data breach events leading to misleading perceptions of the crisis and the company's accountability. Design of software that stores sensitive personal information should support users in maintaining better awareness of data breaches.

Represent this text to find a much much longer article about it with say hundreds-thousands of words!

"In this paper, we tested communication strategies' influence on users mental models of a data breach."

represent this text to find a much much longer article about it with say hundreds-thousands of words

Argumentation frameworks are used to represent causality of plans/models to be utilized for explanations.


Question: Represent the text to find a very short summary (1-2 sentences max)

Question: Goal recognition is the problem of inferring the correct goal towards which an agent executes a plan, given a set of goal hypotheses, a domain model, and a (possibly noisy) sample of the plan being executed. This is a key problem in both cooperative and competitive agent interactions and recent approaches have produced fast and accurate goal recognition algorithms. In this paper, we leverage advances in operator-counting heuristics computed using linear programs over constraints derived from classical planning problems to solve goal recognition problems. Our approach uses additional operator-counting constraints derived from the observations to efficiently infer the correct goal, and serves as basis for a number of further methods with additional constraints. Agents that act autonomously on behalf of a human user must choose goals independently of user input and generate plans to achieve such goals ). When such agents have complex sets goals and require interaction with multiple agents that are not under the user's control, the ing plans are likely to be equally complex and non-obvious for human users to interpret BID0. In such environments, the ability to accurately and quickly identify the goals and plans of all involved agents is key to provide meaningful explanation for the observed behavior. Goal recognition is the problem of inferring one or more goals from a set of hypotheses that best account for a sequence of observations, given a fixed initial state, a goal state, and a behavior model of the agent under observation. Recent approaches to goal recognition based on classical planning domains have leveraged data-structures and heuristic information used to improve planner efficiency to develop increasingly accurate and faster goal recognition algorithms BID1 BID2. Specifically, BID2 use heuristics based on planning landmarks BID1 ) to accurately and efficiently recognize goals in a wide range of domains with various degrees of observability and noise. This approach, however, does not deal with noise explicitly, relying on the implicit necessity of landmarks in valid plans for goal hypotheses to achieve com- petitive accuracy with other methods BID3 BID3, while increasing the number of recognized goals (spread).Thus, goal recognition under partial observability (i.e., missing observations) in the presence of noisy observation is a difficult problem to address while achieving both reasonable recognition time (i.e., a few seconds), high accuracy and low spread. In this paper, we address these limitations by leveraging recent advances on operator-counting heuristics (Pommerening et al. 2014; BID4). Operator-counting heuristics provide a unifying framework for a variety of sources of information from planning heuristics BID1 ) that provide both an estimate of the total cost of a goal from any given state and and indication of the actual operators likely to be in such plans. This information proves to be effective at differentiating between goal hypotheses in goal recognition. Our contributions are threefold. First, we develop three, increasingly more accurate goal recognition approaches using operator-counting heuristics. Second, we empirically show that these heuristics are very effective at goal recognition, overcoming existing approaches in almost all domains in terms of accuracy while diminishing the spread of recognized goals. Such approaches are substantially more effective for noisy settings. Third, we discuss a broad class of operator-counting heuristics for goal recognition that can use additional constraints to provide even finer handling of noise and missing observations. We review the key for the approaches we develop in this paper. First, the recognition settings we assume for our approach follows the standard formalization of goal recognition as planningSecond, while there is substantial literature on linear programming heuristic unified on the operator-counting framework, we focus on the specific types of operator-counting constraints we actually use in our experimentation. Definition 1 (Predicates and State). A predicate is denoted by an n-ary predicate symbol p applied to a sequence of zero or more terms (τ 1, τ 2, ..., τ n) -terms are either constants or variables. We refer to grounded predicates that represent logical values according to some interpretation as facts, which are divided into two types: positive and negated facts, as well as constants for truth and falsehood (⊥). A state S is a finite set of positive facts f that follows the closed world assumption so that if f ∈ S, then f is true in S. We assume a simple inference relation |= such that S |= f iff f ∈ S, S |= f iff f ∈ S, and S |= f 1 ∧... ∧ f n iff {f 1, ..., f n} ⊆ S.Definition 2 (Operator and Action). An operator a is represented by a triple name(a), pre(a), eff(a): name(a) represents the description or signature of a; pre(a) describes the preconditions of a, a set of predicates that must exist in the current state for a to be executed; eff(a) represents the effects of a. These effects are divided into eff(a) + (i.e., an add-list of positive predicates) and eff(a) − (i.e., a delete-list of negated predicates). An action is a ground operator instantiated over its free variables. Definition 3 (Planning Domain). A planning domain definition Ξ is represented by a pair Σ, A, which specifies the knowledge of the domain, and consists of a finite set of facts Σ (e.g., environment properties) and a finite set of actions A.Definition 4 (Planning Instance). A planning instance Π is represented by a triple Ξ, I, G, where Ξ = Σ, A is the domain definition; I ⊆ Σ is the initial state specification, which is defined by specifying values for all facts in the initial state; and G ⊆ Σ is the goal state specification, which represents a desired state to be achieved. Definition 5 (Plan). An s-plan π for a planning instance Π = Ξ, I, G is a sequence of actions a 1, a 2,..., a n that modifies a state s into a state S |= G in which the goal state G holds by the successive execution of actions in π starting from s. An I-plan is just called a plan. A plan π * with length |π * | is optimal if there exists no other plan π for Π such that |π | < |π * |.A goal recognition problem aims to select the correct goal of an agent among a set of possible goals using as evidence a sequence of observations. These observations might be actions executed by the agent or noise observation which are part of a valid plan but are not executed by the agent. Definition 6 (Observation Sequence). An observation sequence O = o 1, o 2,..., o n is said to be satisfied by a plan π = a 1, a 2,..., a m, if there is a monotonic function f that maps the observation indices j = 1,..., n into action indices i = 1,..., n, such that a f (j) = o j.Definition 7 (Goal Recognition Problem). A goal recognition problem is a tuple T GR = Ξ, I, G, O, where: Ξ = Σ, A is a planning domain definition; I is the initial state; G is the set of possible goals, which include a correct hidden goal G * (i.e., G * ∈ G); and O = o 1, o 2,..., o n is an observation sequence of executed actions, with each observation o i ∈ A, and the corresponding action being part of a valid plan π (from Definition 5) that transitions I into G * through the sequential execution of actions in π. Definition 8 (Solution to a Goal Recognition Problem). A solution to a goal recognition problem T GR = Ξ, I, G, O is a nonempty subset of the set of possible goals G ⊆ G such that ∀G ∈ G there exists a plan π G generated from a planning instance Ξ, I, G and π G is consistent with O. Recent work on linear programming (LP) based heuristics has generated a number of informative and efficient heuristics for optimal-cost planning BID4 Pommerening et al. 2014; BID0. These heuristics rely on constraints from different sources of information that every plan π (Definition 5) must satisfy. All operator-counting constraints contain variables of the form Y a for each operator a such that setting Y a to the number of occurrences of a in π satisfies the constraints. In this paper we adopt the formalism and definitions of Pommerening et al. for LP-based heuristics 1. for all a ∈ A. A constraint set for s is a set of operatorcounting constraints for s where the only common variables between constraints are the operator-counting constraints. While the framework from Pommerening et al. 2013 unifies many types of constraints for operator-counting heuristics, we rely on three types of constraints for our goal recognition approaches: landmarks, state-equations, and post-hoc optimization. Planning landmarks consist of actions (alternatively state-formulas) that must be executed (alternatively made true) in any valid plan for a particular goal BID1. Thus, landmarks are necessary conditions for all valid plans towards a goal, and, as such, provide the basis for a number of admissible heuristics (Karpas and Domshlak 2009) and as conditions to strengthen existing heuristics (Bonet 2013). Importantly, planning landmarks form the basis for the current state-of-the-art goal recognition algorithms BID2 BID2. Disjunctive action landmarks BID5 ) for a state s are sets of actions such that at least one action in the set must be true for any s-plan, and make for a natural operator-counting constraint. DISPLAYFORM0 Net change constraints generalize state equation heuristic, which itself relate the planning instance in question with Petri nets that represent the transitions of state variables induced by the actions, and such that tokens in this task represent net changes to the states of the problem. Finally, Post-hoc optimization constraints (Pommerening et al. 2013) use the fact that certain heuristics can rule out the necessity of certain operators from plans (and thus from the heuristic estimate). For example, Pattern Database (PDBs) heuristics (Culberson and Schaeffer 1998) create projections of the planning task into a subset of state variables (with this subset being the pattern), such that the heuristic can partition operators into two sets of each pattern, one that changes variables in the pattern (i.e., contributes towards transitions) and the other than does not (i.e., is non-contributing). Definition 13 (Post-hoc Optimization Constraint). Let Π be a planning task with operator set A, let h be an admissible heuristic for Π, and let N ⊆ A be a set of operators that are noncontributing in that h is still admissible in a modified planning task where the cost of all operators in N is set to 0.Then the post-hoc optimization constraint c P H s,h,N for h, N, and state s of Π consists of the inequality. DISPLAYFORM1 We now bring together the operator-counting constraints into three operator-counting heuristics suitable for goal recognition, ranging from the simplest way to employ operator counts to compute the overlap between counts and observed actions, to modifying the constraints used by the operator counts to enforce solutions that agree with such observations,and finally accounting for possible noise by comparing heuristic values. We start with a basic operator-counting heuristic h(s), which we define to be the LP-heuristic of Def. 11 where C comAlgorithm 1 Goal Recognition using the Operator Counts. Input: Ξ planning domain definition, I initial state, G set of candidate goals, and O observations. Output: Recognized goal(s). DISPLAYFORM0 Hits ← Initialize empty dictionary 3:for all G ∈ G do Compute overlap for G 4: DISPLAYFORM1 for all o ∈ O do 8:if Yo > 0 then 9:HitsG ← HitsG + 1 10: DISPLAYFORM2 prises the constraints generated by Landmarks (Def. 12), post-hoc optimization (Def. 13), and net change constraints as described by. This heuristic, computed following Def. 11, yields two important bits of information for our first technique, first, it generates the actual operator counts Y a for all a ∈ A from Def. 10, whose minimization comprises the objective function h(s).The heuristic values h(s) of each goal candidate G ∈ G tells us about the optimal distance between the initial state I and G, while the operator counts indicate possible operators in a valid plan from I to G. We can use these counts to account for the observations O by computing the overlap between operators with counts greater than one and operators observed for recognition. Algorithm 1 shows how we can use the operator counts directly in a goal recognition technique. In order to rank the goal hypotheses we keep a dictionary of Hits (Line 2) to store the overlap, or count the times operators counts hit observed actions. The algorithm then iterates over all goal hypotheses (Lines 3-10) computing the operator counts for each hypothesis G and comparing these counts with the actual observations (Lines 7-10). We recognize goals by choosing the hypotheses whose operator counts hit the most observations (Line 11). The technique of Algorithm 1 is conceptually similar to the Goal Completion heuristic of BID2 in that it tries to compare heuristically computed information with the observations. However, this initial approach has a number of shortcomings in relation to their technique. First, while the landmarks themselves are enforced by the LP used to compute the operator counts (and thus observations that correspond to landmarks count as hits), the overlap computation loses the ordering of the landmarks that the Goal Completion heuristic uses to account for missing observations. Second, a solution to a set of operator-constraints, i.e., a set of operators with non-negative counts may not be a feasible plan for a planning instance. Thus, these counts may not correspond to the plan that generated the observations. While operator-counting heuristics on their own are fast and informative enough to help guide search when dealing with millions of nodes, goal recognition problems often reAlgorithm 2 Goal Recognition using ObservationConstrained Operator Counts. Input: Ξ planning domain definition, I initial state, G set of candidate goals, and O observations. Output: Recognized goal(s). DISPLAYFORM0 quire the disambiguation of a dozen or less goal hypotheses. Such goal hypotheses are often very similar so that the operator-counting heuristic value (i.e., the objective function over the operator counts) for each goal hypothesis is very similar, especially if the goals are more or less equidistant from the initial state. Thus, we refine the technique of Observation Overlap by introducing additional constraints into the LP used to compute operator counts. Specifically, we force the operator counting heuristic to only consider operator counts that include every single observation o ∈ O. The ing LP heuristic (which we call h C) then minimizes the cost of the operator counts for plans that necessarily agree with all observations. We summarize this Observation Constraint Enforcement approach in Algorithm 2. This technique is similar to that of Algorithm 1 in that it iterates over all goals computing a heuristic value. However, instead of computing observation hits by looking at individual counts, it generates the constraints for the operator-counting heuristic (Line 3) and adds constraints to ensure that the count of the operators corresponding to each observation is greater than one (Lines 4-5). Finally, we choose the goal hypotheses that minimize the operator count heuristic distance from the initial state (Line 8). Although enforcing constraints to ensure that the LP heuristic computes only plans that do contain all observations helps us overcome the limitations of computing the overlap of the operator counts, this approach has a major shortcoming: it considers all observations as valid operators generated by the observed agent. Therefore, the heuristic ing from the minimization of the LP might overestimate the actual length of the plan for the goal hypothesis due to noise. This may happen for one of two reasons: either the noise is simply a sub-optimal operator in a valid plan, or it is an operator that is completely unrelated to the plan that generated the observations. In both cases, the ing heuristic value may prevent the algorithm from selecting the actual goal from among the goal hypotheses. This overestimation, however, has an important property in relation to the basic operator counting heuristic, which is that h C always dominates the operator counting heuristic h, in Proposition 1.Algorithm 3 Goal Recognition using Heuristic Difference of Operator Counts. Input: Ξ planning domain definition, I initial state, G set of candidate goals, and O observations. Output: Recognized goal(s). DISPLAYFORM0 HG ← a∈A Ya 6: DISPLAYFORM1 HC,G ← a∈A Ya 10: DISPLAYFORM2 Proposition 1 (h C dominates h). Let h be the operatorcounting heuristic from Defs. 10-11, h C be the overconstrained heuristic that accounts for all observations o ∈ O, and s a state of Π. Then h C (s) ≥ h(s).Proof. Let C h be set of constraints used in h(s), and C h C be set of constraint used to compute h C (s). Every feasible solution to C h C is a solution to C h. This is because to generate C h C we only add constraints to C h. Thus, a solution to C h C has to satisfy all constraints in C h. Therefore, since we are solving a minimization problem the value of the solution for C h cannot be larger than the solution to C h C.The intuition here is that the operator-counting heuristic h estimates the total cost of any optimal plan, regardless of the observations, while h C estimates a plan following all observations, including noise, if any. If there is no noise, the sum of the counts must agree (even if the counts are different), whereas if there is noise and assuming the noise is evenly distributed, there will be differences in all counts. Thus, our last approach consists of computing the difference between h C and h, and infer that the goal hypothesis for which these values are closer must be the correct goal. We call the ing heuristic h δ and formalize this approach in Algorithm 3. Here we compute the LP twice, once with only the basic operator-counting constraints (Line 4), and once with the constraints enforcing the observations in the operator counts (Line 8), using these two values to compute h δ (Line 10). The algorithm then returns goal hypotheses that minimize h δ (Line 11). To evaluate the effectiveness of our approaches, we implemented each of the algorithms described earlier and performed the goal recognition process over the large dataset introduced by BID2. This dataset contains thousands of problems for goal and plan recognition under varying levels of observability for a number of traditional IPC domains (Vallati et al. 2018 2 . We implemented our approaches using PYTHON 2.7 for the main recognition algorithms with external calls to a customized version of the FAST-DOWNWARD BID1) planning system to compute the operator counts. Our customized planner returns not only the operator counts and can also introduce additional constraints before running the CPLEX 128 optimization system. We ran experiments in a single core of a 24 core Intel R Xeon R CPU E5-2620 @2.00Ghz with 48GB of RAM, with a 2-minute time limit and a 2GB memory limit. Table 1 shows the for the partially observable, nonnoisy fragment of the dataset, whereas TAB3 shows the noisy fragment of the dataset 3. For the noisy experiments, each set of observations contained at least two spurious actions, which, while valid for the plan, were not actually executed by the agent being observed. These show that, while not nearly as fast as the h uniq approach from Pereira et al. with a θ = 0 recognition threshold, the accuracy (Acc %) of our h δ approach is either competitive or superior in virtually all domains (except for some levels of observability in IPC-GRID, DWR and KITCHEN), and, even for the domains where the accuracy is similar, or lower, the spread (SinG) of the ing goals is consistently lower, i.e., the returned goals are unique for most problems. The accuracy of our approach, thus, consistently matches or surpasses that of R&G 2009, with a computational cost that is also often smaller than FG 2015. Importantly, the cost of all of our approaches is basically the same within each domain, regardless of the level of observability and noise, since our technique relies on a single call to a planner that computes the operator counts for a single state and then stops the planner. We argue that this is attributable to our inefficient implementation rather than the technique, for the h δ approach, the overhead of the FAST-DOWNWARD pre-processing step is paid multiple times. Unlike R&G 2009, that uses a modified planning heuristic, and FG 2015, that builds a data structure and explores it at very high computational cost. We note that the for noisy observations show the greatest impact of h δ with an overall higher accuracy and lower spread across all domains but KITCHEN.Finally, for the KITCHEN domain stand out in our experiments in that our some of our approaches consistently show underwhelming performance both in noisy and nonnoisy domains. Counter-intuitively, for this particular do-main, the more observations we have available, the worse the performance. This seems to be a problem for all other approaches under noisy conditions, though not under incomplete observations. Moreover, since the loss of accuracy with fuller observability also occurs for the non-noisy setting, we surmise this to stem from the domain itself, rather than the algorithm's ability to handle noise, and defer investigation of this issue to future work. Our work follows the traditional of goal and plan recognition as planning algorithms as defined by BID2 BID3. The former work yields higher recognition accuracy in our settings (and hence we chose it as a baseline), whereas the latter models goal recognition as a problem of estimating the probability of a goal given the observations. Such work uses a Bayesian framework to compute the probability of goals given observations by computing the probability of generating a plan given a goal, which they accomplish by running a planner multiple times to estimate the probability of the plans that either comply or not with the observations. Recent research on goal recognition has yielded a number of approaches to deal with partial observability and noisy observations, of which we single out three key contributions. First, BID1 developed a goal recognition approach based on constructing a planning graph and propagating operator costs and the interaction among operators to provide an estimate of the probabilities of each goal hypothesis. While their approach provides probabilistic estimates for each goal, its precision in inferring the topmost goals is consistently lower than ours, often ranking multiple goals with equal probabilities (i.e., having a large spread). Second, BID3 developed an approach that also provides a probabilistic interpretation and explicitly deals with noisy observations. Their approach works through a compilation of the recognition problem into a planning problem that is processed by a planner that computes a number of approximately optimal plans to compute goal probabilities under R&G's Bayesian framework. Finally, BID2 develop heuristic goal recognition approaches using landmark information. This approach is conceptually closer to ours in that we also compute heuristics, but we aim to overcome the potential sparsity of landmarks in each domain by using operator-count information, as well as explicitly handle noise by introducing additional constraints in heuristic h C and comparing the distance to the unconstrained h heuristic. We developed a novel class goal recognition technique based on operator-counting heuristics from classical planning (Pommerening et al. 2014) which, themselves rely on ILP constraints to estimate which operators occur in valid optimal plans towards a goal. The ing approaches are competitive with the state of the art in terms of high accuracy and low false positive rate (i.e., the spread of returned goals), at a moderate computational cost. We show empirically that the overall accuracy of our best approach is sub- stantially superior to the state-of-the-art over a large dataset. Importantly, the values of the operator-counting constraints we compute for each of the heuristics can be used as explanations for recognized goals. The techniques described in this paper use a set of simple additional constraints in the ILP formulation to achieve substantial performance, so we expect substantial future work towards further goal recognition approaches and heuristics that explore more refined constraints to improve accuracy and reduce spread, as well as deriving a probabilistic approach using operator-counting information. Examples of such work include using the constraints to force the LP to generate the counterfactual operator-counts (i.e., non-compliant with the observations) used by the R&G approach, or, given an estimate of the noise, relax the observation constraints to allow a number of observations to not be included in the ing operator-counts. DISPLAYFORM0

Represent the short summary for finding the corresponding scientific article (long)!

A goal recognition approach based on operator counting heuristics used to account for noise in the dataset.

A backward model of previous (state, action) given the next state, i.e. P(s_t, a_t | s_{t+1}), can be used to simulate additional trajectories terminating at states of interest! Improves RL learning efficiency.


Question: Represent the scientific article to find a short TLDR

Question: It can be challenging to train multi-task neural networks that outperform or even match their single-task counterparts. To help address this, we propose using knowledge distillation where single-task models teach a multi-task model. We enhance this training with teacher annealing, a novel method that gradually transitions the model from distillation to supervised learning, helping the multi-task model surpass its single-task teachers. We evaluate our approach by multi-task fine-tuning BERT on the GLUE benchmark. Our method consistently improves over standard single-task and multi-task training. Building a single model that jointly learns to perform many tasks effectively has been a longstanding challenge in Natural Language Processing (NLP). However, applying multi-task NLP remains difficult for many applications, with multitask models often performing worse than their single-task counterparts BID30 BID1 BID25. Motivated by these , we propose a way of applying knowledge distillation BID3 BID0 BID14 so that single-task models effectively teach a multi-task model. Knowledge distillation transfers knowledge from a "teacher" model to a "student" model by training the student to imitate the teacher's outputs. In "born-again networks" BID10, the teacher and student have the same neural architecture and model size, but surprisingly the student is able to surpass the teacher's accuracy. Intuitively, distillation is effective because the teacher's output distribution over classes provides more training signal than a one-hot label; BID14 suggest that teacher outputs contain "dark knowledge" capturing additional information about training examples. Our work extends born-again networks to the multi-task setting. We compare Single→Multi 1 born-again distillation with several other variants (Single→Single and Multi→Multi), and also explore performing multiple rounds of distillation (Single→Multi→Single→Multi). Furthermore, we propose a simple teacher annealing method that helps the student model outperform its teachers. Teacher annealing gradually transitions the student from learning from the teacher to learning from the gold labels. This method ensures the student gets a rich training signal early in training but is not limited to only imitating the teacher. Our experiments build upon recent success in self-supervised pre-training BID7 BID28 and multi-task fine-tune BERT BID8 to perform the tasks from the GLUE natural language understanding benchmark BID41. Our training method, which we call Born-Again Multi-tasking (BAM) 2, consistently outperforms standard single-task and multi-task training. Further analysis shows the multi-task models benefit from both better regu- 1 We use Single→Multi to indicate distilling single-task "teacher" models into a multi-task "student" model. 2 Code is available at https://github.com/ google-research/google-research/tree/ master/bam larization and transfer between related tasks. Multi-task learning for neural networks in general BID4 and within NLP specifically BID6 BID24 has been widely studied. Much of the recent work for NLP has centered on neural architecture design: e.g., ensuring only beneficial information is shared across tasks BID21 or arranging tasks in linguistically-motivated hierarchies BID37 BID13 BID34. These contributions are orthogonal to ours because we instead focus on the multi-task training algorithm. Distilling large models into small models BID19 BID26 or ensembles of models into single models BID20 BID22 has been shown to improve for many NLP tasks. There has also been some work on using knowledge distillation to aide in multi-task learning. In reinforcement learning, knowledge distillation has been used to regularize multi-task agents BID27 BID39. In NLP, BID38 distill singlelanguage-pair machine translation systems into a many-language system. However, they focus on multilingual rather than multi-task learning, use a more complex training procedure, and only experiment with Single→Multi distillation. Concurrently with our work, several other recent works also explore fine-tuning BERT using multiple tasks BID29 BID23 BID18. However, they use only standard transfer or multi-task learning, instead focusing on finding beneficial task pairs or designing improved task-specific components on top of BERT. Model. All of our models are built on top of BERT BID8. This model passes byte-pairtokenized BID35 input sentences through a Transformer network BID40, producing a contextualized representation for each token. The vector corresponding to the first input token 3 c is passed into a task-specific classifier. For classification tasks, we use a standard softmax layer: softmax(W c). For regression tasks, we normalize the labels so they are between 0 and 1 and then use a size-1 NN layer with a sigmoid activation: sigmoid(w T c). In our multi-task models, all of the model parameters are shared across tasks except for these classifiers on top of BERT, which means less than 0.01% of the parameters are task-specific. Following BERT, the token embeddings and Transformer are initialized with weights from a self-supervised pre-training phase. Training. Single-task training is performed as in BID8. For multi-task training, examples of different tasks are shuffled together, even within minibatches. The summed loss across all tasks is minimized. We use DISPLAYFORM0 } to denote the training set for a task τ and f τ (x, θ) to denote the outputs for task τ produced by a neural network with parameters θ on the input x (for classification tasks this is a distribution over classes). Standard supervised learning trains θ to minimize the loss on the training set: DISPLAYFORM1 where for classification tasks is usually crossentropy. Knowledge distillation trains the model to instead match the predictions of a teacher model with parameters θ: DISPLAYFORM2 Note that our distilled networks are "born-again" in that the student has the same model architecture as the teacher, i.e., all of our models have the same prediction function f τ for each task. For regression tasks, we train the student to minimize the L2 distance between its prediction and the teacher's instead of using cross-entropy loss. Intuitively, knowledge distillation improves training because the full distribution over labels provided by the teacher provides a richer training signal than a one-hot label. See BID10 for a more thorough discussion. Multi-Task Distillation. Given a set of tasks T, we train a single-task model with parameters θ τ on each task τ. For most experiments, we use the single-task models to teach a multi-task model with parameters θ: DISPLAYFORM3 However, we experiment with other distillation strategies as well. Teacher Annealing. In knowledge distillation, the student is trained to imitate the teacher. This raises the concern that the student may be limited by the teacher's performance and not be able to substantially outperform the teacher. To address this, we propose teacher annealing, which mixes the teacher prediction with the gold label during training. Specifically, the term in the summation becomes DISPLAYFORM4 where λ is linearly increased from 0 to 1 throughout training. Early in training, the model is mostly distilling to get as useful of a training signal as possible. Towards the end of training, the model is mostly relying on the gold-standard labels so it can learn to surpass its teachers. Data. We use the General Language Understanding Evaluation (GLUE) benchmark BID41, which consists of 9 natural language understanding tasks on English data. Tasks cover textual entailment (RTE and MNLI) question-answer entailment (QNLI), paraphrase (MRPC), question paraphrase (QQP), textual similarity (STS), sentiment (SST-2), linguistic acceptability (CoLA), and Winograd Schema (WNLI).Training Details. Rather than simply shuffling the datasets for our multi-task models, we follow the task sampling procedure from, where the probability of training on an example for a particular task τ is proportional to |D τ | 0.75. This ensures that tasks with very large datasets don't overly dominate the training. We also use the layerwise-learning-rate trick from BID16. If layer 0 is the NN layer closest to the output, the learning rate for a particular layer d is set to BASE LR · α d (i.e., layers closest to the input get lower learning rates). The intuition is that pre-trained layers closer to the input learn more general features, so they shouldn't be altered much during training. Hyperparameters. For single-task models, we use the same hyperparameters as in the original BERT experiments except we pick a layerwiselearning-rate decay α of 1.0 or 0.9 on the dev set for each task. For multi-task models, we train the model for longer (6 epochs instead of 3) and with a larger batch size (128 instead of 32), using α = 0.9 and a learning rate of 1e-4. All models use the BERT-Large pre-trained weights. Reporting Results. Dev set report the average score (Spearman correlation for STS, Matthews correlation for CoLA, and accuracy for the other tasks) on all GLUE tasks except WNLI, for which methods can't outperform a majority baseline. Results show the median score of at least 20 trials with different random seeds. We find using a large number of trials is essential because can vary significantly for different runs. For example, standard deviations in score are over ±1 for CoLA, RTE, and MRPC for multi-task models. Single-task standard deviations are even larger. Main Results. We compare models trained with single-task learning, multi-task learning, and several varieties of distillation in TAB1. While standard multi-task training improves over single-task training for RTE (likely because it is closely related to MNLI), there is no improvement on the other tasks. In contrast, Single→Multi knowledge distillation improves or matches the performance of the other methods on all tasks except STS, the only regression task in GLUE. We believe distillation does not work well for regression tasks because there is no distribution over classes passed on by the teacher to aid learning. The gain for Single→Multi over Multi is larger than the gain for Single→Single over Single, suggesting that distillation works particularly well in combination with multi-task learning. Interestingly, Single→Multi works substantially better than Multi→Multi distillation. We speculate it may help that the student is exposed to a diverse set of teachers in the same way ensembles benefit from a diverse set of models, but future work is required to fully understand this phenomenon. In addition to the models reported in the table, we also trained Single→Multi→Single→Multi models. However, the difference with Single→Multi was not statistically significant, suggesting there is little value in multiple rounds of distillation. Avg. g constructed from SQuAD BID31 h BID11 BERT-Base BID8 78.5 BERT-Large BID8 80.5 BERT on STILTs BID29 82.0 MT-DNN BID23 82 Overall, a key benefit of our method is robustness: while standard multi-task learning produces mixed , Single→Multi distillation consistently outperforms standard single-task and multitask training. We also note that in some trials single-task training ed in models that score quite poorly (e.g., less than 91 for QQP or less than 70 for MRPC), while the multi-task models have more dependable performance. DISPLAYFORM0 Test Set Results. We compare against recent work by submitting to the GLUE leaderboard. We use Single→Multi distillation. Following the procedure used by BERT, we train multiple models and submit the one with the highest average dev set score to the test set. BERT trained 10 models for each task (80 total); we trained 20 multi-task models. Results are shown in TAB3.Our work outperforms or matches existing published that do not rely on ensembling. However, due to the variance between trials dis-cussed under "Reporting Results," we think these test set numbers should be taken with a grain of salt, as they only show the performance of individual training runs (which is further complicated by the use of tricks such as dev set model selection). We believe significance testing over multiple trials would be needed to have a definitive comparison. Single-Task Fine-Tuning. A crucial difference distinguishing our work from the STILTs, Snorkel MeTaL, and MT-DNN KD methods in TAB3 is that we do not single-task fine-tune our model. That is, we do not further train the model on individual tasks after the multi-task training finishes. While single-task fine-tuning improves , we think to some extent it defeats the purpose of multi-task learning: the of training is one model for each task instead of a model that can perform all of the tasks. Compared to having many single-task models, a multi-task model is simpler to deploy, faster to run, and arguably more scientifically interesting from the perspective of building general language-processing systems. We evaluate the benefits of single-task finetuning in TAB5. Single-task fine-tuning initializes models with multi-task-learned weights and then performs single-task training. Hyperparameters are the same as for our single-task models except we use a smaller learning rate of 1e-5. While single-task fine-tuning unsurprisingly improves , the gain on top of Single→Multi distillation is small, reinforcing the claim that distillation provides many of the benefits of singletask training while producing a single unified model instead of many task-specific models. Ablation Study. We show the importance of teacher annealing and the other training tricks in improve scores. Using pure distillation without teacher annealing (i.e., fixing λ = 0) performs no better than standard multi-task learning, demonstrating the importance of the proposed teacher annealing method. Comparing Combinations of Tasks. Training on a large number of tasks is known to help regularize multi-task models BID32. A related benefit of multi-task learning is the transfer of learned "knowledge" between closely related tasks. We investigate these two benefits by comparing several models on the RTE task, including one trained with a very closely related task (MNLI, a much large textual entailment dataset) and one trained with fairly unrelated tasks (QQP, CoLA, and SST). We use Single→Multi distillation (Single→Single in the case of the RTE-only model). Results are shown in TAB8. We find both sets of auxiliary tasks improve RTE performance, suggesting that both benefits are playing a role in improving multi-task models. Interestingly, RTE + MNLI alone slightly outperforms the model performing all tasks, perhaps because training on MNLI, which has a very large dataset, is already enough to sufficiently regularize the model. We have shown that Single→Multi distillation combined with teacher annealing produces consistently better than standard single-task or multi-task training. Achieving robust multi-task gains across many tasks has remained elusive in previous research, so we hope our work will make multi-task learning more broadly useful within NLP. However, with the exception of closely related tasks with small datasets (e.g., MNLI helping RTE), the overall size of the gains from our multi-task method are small compared to the gains provided by transfer learning from self-supervised tasks (i.e., BERT). It remains to be fully understood to what extent "self-supervised pre-training is all you need" and where transfer/multi-task learning from supervised tasks can provide the most value.

Represent this text to find a much much longer article about it with say hundreds-thousands of words

distilling single-task models into a multi-task model improves natural language understanding performance

Meta-learning methods used for vision, directly applied to NLP, perform worse than nearest neighbors on new classes; we can do better with distributional signatures.