TSA Has a Risk-Based Covert Testing Strategy to Identify Vulnerabilities and Measure the Performance of Selected Aviation Security Systems, but Could Strengthen Its Testing Efforts

TSA has designed and implemented risk-based national and local covert  testing programs to achieve its goals of identifying vulnerabilities in and  measuring the performance of passenger checkpoint and checked baggage  screening systems and airport perimeters and access controls, and has  begun to determine the extent to which covert testing will be used to  identify vulnerabilities and measure the effectiveness of security practices  related to non-aviation modes of transportation. OI used information on  terrorist threats to design and implement its national covert tests and  determine at which airports to conduct tests based on analyses of risks. Without systematically recording reasons for  test failures, such as failures caused by screening equipment not working  properly, as well as reasons for test passes, TSA is limited in its ability to  mitigate identified vulnerabilities. TSA recently redesigned its local covert  testing program to address limitations in its previous program. Furthermore, according to OI officials, identifying a single cause for a test  failure may be difficult since covert testing failures can be caused by  multiple factors. As a result, it is too  soon to determine whether ASAP will meet its goals of measuring the  performance of passenger and checked baggage screening systems and  identifying vulnerabilities. TSA Could More Fully Use the Results of Covert Tests to Mitigate Security Vulnerabilities Identified in the Commercial Aviation System

TSA’s national aviation covert testing program has identified  vulnerabilities in select aspects of the commercial aviation security system  at airports of all sizes; however, the agency is not fully using the results of  these tests to mitigate identified vulnerabilities. However, OSO  lacks a systematic process to ensure that OI’s recommendations are  considered, and does not systematically document its rationale for why it  did or did not implement OI’s recommendations. In the absence of a systematic process for  considering OI’s recommendations, documenting their decision-making  process, and evaluating whether corrective actions mitigated identified  vulnerabilities, TSA is limited in its ability to use covert testing results to  improve the security of the commercial aviation system. OSO senior  leadership stated that opportunities exist to improve the agency’s  processes in this area. While the  specific results of these tests and the vulnerabilities they identified are  classified, covert test failures can be caused by multiple factors, including  TSOs not properly following TSA procedures when screening passengers,  screening equipment that does not detect a threat item, or TSA screening  procedures that do not provide sufficient detail to enable TSOs to identify  the threat item. DHS and TSA concurred with the  findings and recommendations, and stated that the report will be useful in  strengthening TSA’s covert testing programs. In response to our recommendation that TSA develop a process to  systematically coordinate with domestic and foreign transportation  organizations as the agency explores the use of covert testing in non- aviation modes of transportation to learn from their experiences, DHS  stated that it is taking a number of actions. To identify TSA’s strategy for conducting covert testing of the  transportation system and the extent to which the agency has designed  and implemented its covert tests to achieve identified goals, we reviewed  applicable laws, regulations, policies, and procedures to determine the  requirements for conducting covert testing in the transportation sector. To determine the results of TSA’s national covert tests and the extent to  which TSA used the results of these tests to mitigate security  vulnerabilities in the aviation system, we obtained and analyzed a database  of the results of TSA’s national covert tests conducted from September  2002 to June 2007. We analyzed the test data according to airport category,  threat item, and type of test conducted between September 2002 and June  2007.
Why GAO Did This Study

The Transportation Security Administration (TSA) uses undercover, or covert, testing to approximate techniques that terrorists may use to identify vulnerabilities in and measure the performance of airport security systems. During these tests, undercover inspectors attempt to pass threat objects through passenger and baggage screening systems, and access secure airport areas. In response to a congressional request, GAO examined (1) TSA's strategy for conducting covert testing of the transportation system and the extent to which the agency has designed and implemented its covert tests to achieve identified goals; and (2) the results of TSA's national aviation covert tests conducted from September 2002 to June 2007, and the extent to which TSA uses the results of these tests to mitigate security vulnerabilities. To conduct this work, GAO analyzed covert testing documents and data and interviewed TSA and transportation industry officials.

What GAO Found

TSA has designed and implemented risk-based national and local covert testing programs to achieve its goals of identifying vulnerabilities in and measuring the performance the aviation security system, and has begun to determine the extent to which covert testing will be used in non-aviation modes of transportation. TSA's Office of Inspection (OI) used information on terrorist threats to design and implement its national covert tests and determine at which airports to conduct tests based on the likelihood of a terrorist attack. However, OI did not systematically record the causes of test failures or practices that resulted in higher pass rates for tests. Without systematically recording reasons for test failures, such as failures caused by screening equipment not working properly, as well as reasons for test passes, TSA is limited in its ability to mitigate identified vulnerabilities. OI officials stated that identifying a single cause for a test failure is difficult since failures can be caused by multiple factors. TSA recently redesigned its local covert testing program to more effectively measure the performance of passenger and baggage screening systems and identify vulnerabilities. However, it is too early to determine whether the program will meet its goals since it was only recently implemented and TSA is still analyzing the results of initial tests. While TSA has a well established covert testing program in commercial aviation, the agency does not regularly conduct covert tests in non-aviation modes of transportation. Furthermore, select domestic and foreign transportation organizations and DHS components use covert testing to identify security vulnerabilities in non-aviation settings. However, TSA lacks a systematic process for coordinating with these organizations. TSA covert tests conducted from September 2002 to June 2007 have identified vulnerabilities in the commercial aviation system at airports of all sizes, and the agency could more fully use the results of tests to mitigate identified vulnerabilities. While the specific results of these tests and the vulnerabilities they identified are classified, covert test failures can be caused by multiple factors, including screening equipment that does not detect a threat item, Transportation Security Officers (TSOs), formerly known as screeners, not properly following TSA procedures when screening passengers, or TSA screening procedures that do not provide sufficient detail to enable TSOs to identify the threat item. TSA's Administrator and senior officials are routinely briefed on covert test results and are provided with test reports that contain recommendations to address identified vulnerabilities. However, TSA lacks a systematic process to ensure that OI's recommendations are considered and that the rationale for implementing or not implementing OI's recommendations is documented. Without such a process, TSA is limited in its ability to use covert test results to strengthen aviation security. TSA officials stated that opportunities exist to improve the agency's processes in this area. In May 2008, GAO issued a classified report on TSA's covert testing program. That report contained information that was deemed either classified or sensitive. This version of the report summarizes our overall findings and recommendations while omitting classified or sensitive security information.