FAA Analyzes Data on Past Safety Events to Prevent Their Recurrence and Plans to Use Data to Support a More Proactive Approach to Managing Risks

		FAA Is Shifting from a Reactive to a Proactive Approach to Using Data to Manage Risk

For decades, the aviation industry and federal regulators, including FAA,  have used data reactively to identify the causes of aviation accidents and  incidents and take actions to prevent their recurrence. In addition,  several experts we spoke with said that proactively identifying risks is  necessary to maintain the current level of safety and possibly achieve an  even higher level of safety in the future. FAA’s goal is for the Office of Aviation Safety to have initial operating  capabilities in place for SMS by the end of fiscal year 2010. Because SMS relies on data to identify emerging risks, FAA has an effort  under way to enhance its access to industry data and improve its analysis  capability. According to a senior FAA official, this automated analysis  process is unique and pathbreaking and will allow for more efficient safety  analyses. FAA Is Developing a Plan for Using Data under SMS, but the Plan Does Not Fully Address Data, Analysis, and Staffing Requirements

As part of its efforts to develop initial SMS capabilities, FAA expects to  address how data and analysis will help it identify emerging aviation safety  risks. This combination of promised  immunity for self-reporting and threat of enforcement and disciplinary  action for remaining silent creates an incentive for industry personnel to  participate in the voluntary reporting programs. However, FAA lacks carrier-specific  information on why air carriers are not participating in voluntary reporting  programs. FAA Lacks Data to Assess the Safety of Certain Industry Sectors

FAA’s ability to monitor and manage risk for certain industry sectors, such  as general aviation, air ambulance operators, and air cargo carriers, is  limited by incomplete data. FAA Has Various Processes in Place to Help Ensure Data Quality, but Weaknesses Still Exist

		FAA Has Taken a Number of Steps to Help Ensure Data Quality

FAA, along with the international aviation community, recognizes that  high-quality data—that is, reliable, valid data—are essential to the  effectiveness of a data-driven approach to safety, such as SMS. For example, agencies should have managers review data,  have procedures in place to verify that data are complete and accurate,  and correct erroneous data. While NextGen technologies and procedures are intended to increase the  safety, efficiency, and capacity of the national airspace system, their  introduction could have unintended effects on system safety if not done in  a comprehensive manner. Recommendations for Executive Action

To help improve and expand FAA’s capability to use data for aviation  safety oversight, we recommend that the Secretary of Transportation  direct the FAA Administrator to take the following four actions:    develop and implement a comprehensive plan that addresses how data fit  into FAA’s implementation of a proactive approach to safety oversight and  ensure that this plan fully describes the relevant data challenges (such as  ensuring data quality and continued access to voluntarily reported safety  data), analytical approaches, and staffing requirements and integrates  efforts to address them;    given the importance of high-quality data, extend standard quality  controls, as appropriate, to the databases that support aviation safety  oversight to ensure that the data are as reliable and valid as possible;    proceed with all deliberate speed to develop the capability to model the  impact of NextGen changes on the national airspace system and manage  any risks emerging from these changes; and    systematically identify the reasons that carriers are not participating in  voluntary reporting programs, such as through a survey, and identify and  implement further steps to encourage greater program participation,  especially by smaller carriers. We then  identified 13 databases available to FAA that contained data on these  safety events and reviewed these databases. To determine the extent to which FAA has access to data for monitoring  aviation safety and the safety performance of various aviation industry  sectors, we interviewed FAA data analysts, contractors, and other officials  responsible for data management.
Why GAO Did This Study

To improve aviation safety, the Federal Aviation Administration (FAA) plans to have in place the initial capabilities of a risk-based approach to safety oversight, known as a safety management system (SMS), by the end of fiscal year 2010. FAA is also implementing new procedures and technologies to enhance the safety, capacity, and efficiency of the national airspace system. Data are central to SMS and FAA's ability to test the impact of these changes on safety. This congressionally requested report addresses FAA's (1) current and planned use of data to oversee aviation safety, (2) access to data for monitoring aviation safety and the safety performance of various industry sectors, and (3) efforts to help ensure data quality. To perform this work, GAO reviewed 13 databases that contain data on key aviation safety events, assessed data quality controls for the databases, and interviewed agency and industry officials, as well as 10 experts in aviation safety and data.

What GAO Found

FAA analyzes data on past safety events, such as engine failures, to prevent their recurrence and plans to use data to support a more proactive approach to managing risk. For example, weather and air traffic control data helped identify factors associated with injuries from turbulence. As part of SMS, FAA plans to analyze data proactively to support a risk-based approach to safety oversight. For example, FAA plans to use data to model the impact of proposed changes in procedures and technologies on the safety of the national airspace system. Experts said that identifying risks is necessary to maintain the current level of safety and possibly achieve a higher level of safety in the future. Because SMS relies on data to identify emerging risks, FAA has an effort under way to enhance its access to industry data and improve its capability for automated analysis of multiple databases. According to FAA, this effort will allow for more efficient safety analyses. FAA is also developing a plan for managing data under SMS, but the plan does not fully address data, analysis, or staffing requirements. Without such requirements, the plan will not provide timely guidance for implementing SMS. FAA has access to some voluntarily reported data, which are important for SMS, but not all carriers and aviation personnel participate in FAA's voluntary reporting programs. Such data are gathered electronically by equipment on aircraft or reported by aviation personnel or carriers following noncriminal, unintentional violations or safety events. Industry personnel have some incentives to participate in voluntary programs, such as promised immunity from disciplinary action, but concerns about sanctions and the cost of equipment have deterred full participation, especially by smaller carriers. While FAA has some information on reasons for nonparticipation and has taken some steps to promote greater participation, it lacks carrier-specific information on why air carriers are not participating. FAA also lacks data to assess the safety performance of certain industry sectors, such as air cargo and air ambulance operators. GAO has previously made recommendations to address this lack of data. FAA concurred with GAO's prior recommendations and is taking actions to address them. To help ensure data quality--that is, data that are reliable (complete and accurate) and valid (measure what is intended)--FAA has implemented a number of data quality controls that are consistent with GAO's standards for data quality, but some weaknesses exit. For example, all the databases GAO reviewed had at least some controls in place to ensure that erroneous data are identified, reported, and corrected. However, about half the databases lack an important control--managers do not review the data prior to entry into the data system. FAA is taking steps to address its data weaknesses, but vulnerabilities remain, potentially limiting the usefulness of FAA's data for the safety analyses planned to support SMS.