As shown in figure 2, TSOs then  review images of the property running through the X-ray machine and  look for signs of prohibited items. Screener Training Requirements

In accordance with ATSA, screeners must complete a minimum of 40  hours of classroom instruction, 60 hours of on-the-job training, and  successfully complete an on-the-job training examination before they are  certified as security screeners. TSA also requires remedial training for TSOs who fail an  annual proficiency review. TSA Uses a Variety of Programs to Train TSOs and Is Developing a Plan to Expand Evaluations of TSO Training

		TSA Uses New Hire, Recurrent, Remedial, and Return-to-Duty Training Programs for TSOs

TSO training is comprised of a compendium of courses that includes  basic training for initial hires, recurrent training, remedial training, and  return-to-duty training. TSOs  receive remedial training when they have failed an operational or  certification test, or if a supervisor identifies a need for further training,  among other things. In addition, TSA expects to approve a Management  Directive and Standard Operating Procedures for the training evaluation  program by May 2016 to define the roles and responsibilities for TSA  offices running the training evaluation program as well as lay out the  steps for analyzing and reporting data collected from the training  evaluations. TSA Uses a Variety of Methods to Measure TSO Performance, and Results Vary by Type of Task Tested and Airport Risk Category TSA Measures TSO Performance through Tests Conducted in a Non- Operational Setting and at Active Checkpoints

TSA uses a variety of methods to measure the performance of its TSOs,  including the Annual Proficiency Review (APR)—an annual certification  program to evaluate TSOs’ skill in performing the various screening  functions. TSA has other testing programs that take place during active operations  at the checkpoints to assess TSOs’ level of adherence to screening SOPs  and associated management directives. Results of APR and PACE Screener Performance Tests Varied by Specific Task Tested and Airport Risk Category

TSA data on the results of APR and PACE testing show that TSOs’ pass  rates on both of these tests varied by airport risk category over the time  periods we reviewed. The specific data reliability  concerns related to these two testing programs are discussed later in this  report. TSOs’ scores on PACE tests generally remained  above 80 percent from fiscal years 2009 through 2014. TSA Uses TSO Performance Data but Is Constrained by Incomplete and Unreliable Data and Lack of National Analysis and Recommendation Follow-Up

		In 2014, TSA Reviewed Selected Annual Proficiency Review Results to Develop Courses for the 2015 National Training Plan

As noted previously, TSA normally uses APR testing results primarily to  assess individual TSOs’ skills for performing screening functions in order  to annually re-certify them to continue participating in screening  operations. Officials stated that, due to software compatibility issues with the  new machines, TIP image capability was turned off for an extended  period of time, meaning that TIP testing was not occurring on these  machines and, therefore, TIP data were neither collected nor reported for  these airports. ASAP Test Results are Unreliable, and TSA Does Not Ensure that Recommendations from Nationwide Analysis of ASAP Scores are Implemented

			ASAP Pass Rate Results Are Unreliable

In an effort to assess the quality of ASAP testing conducted by TSA field  officials at commercial airports, TSA headquarters officials brought in a  contractor in fiscal year 2015 to independently perform ASAP covert  testing at 40 airports and thereby verify the validity of the testing results at  the airports. TSA has put an extensive program in place to train  its TSOs to perform these critical screening functions and responded to  recent covert test findings of the DHS OIG by implementing a retraining  program for all its screening officers to address issues identified in the  testing. Further, the agency could use these data on  a nationwide level to inform and potentially improve training of TSOs in  screening passenger carry-on baggage for prohibited items. Recommendations for Executive Action

To improve TSA’s ability to take full advantage of testing results to inform  and potentially improve screening operations, we recommend that the  Secretary of the Department of Homeland Security direct the  Administrator of TSA to take the following three actions:
Ensure that TSA officials at individual airports submit complete TIP  results to the TSA national database as required, including manually  submitting data when automated uploading is not available. Track implementation by airports of ASAP recommendations to  ensure that corrective actions identified through ASAP testing are  being applied. Agency Comments and Our Evaluation

We provided a draft of the sensitive version of this report to DHS for their  review and comment. How does the Transportation Security Administration (TSA) train  Transportation Security Officers (TSO), and to what extent does TSA  evaluate the training? 2. 3. To what extent does TSA use TSO performance data to enhance  TSO performance? For our second objective, to determine how TSA measures the  performance of TSOs and what the performance data show, we analyzed  data from four different performance evaluation programs and we  interviewed TSA officials responsible for collecting and analyzing the  data. Officials stated that  this was due to two reasons. TSA could not provide us with information on the extent of the  missing data and we were not able to determine based on the data  provided how many X-ray machines were unaccounted for between 2009  and 2014.
Why GAO Did This Study

TSA trains TSOs to screen passengers and baggage for items that could pose a threat at nearly 440 airports across the country. One way TSA and the Department of Homeland Security (DHS) Office of Inspector General (OIG) measure TSO performance is through covert testing of TSA screening operations. In response to the findings from recent DHS OIG covert testing, the Secretary of DHS directed TSA in June 2015 to conduct further training for all TSOs and supervisors. GAO was asked to review TSA's efforts to train and test TSOs. This report examines (1) how TSA trains TSOs and evaluates the training; (2) how TSA measures TSO performance and what the data show; and (3) to what extent TSA uses TSO performance data to enhance TSO performance. GAO analyzed TSO performance data from 2009 through 2015, reviewed documents regarding TSA training and testing, and interviewed TSA officials at headquarters and 10 airports. GAO selected these airports based on airport risk categories, among other things. Information from these airports was not generalizable, but provided insights into TSO training and testing. This is a public version of a sensitive report that GAO issued in May 2016.

What GAO Found

The Transportation Security Administration (TSA) uses a variety of programs to train and evaluate Transportation Security Officers (TSO) who are responsible for screening passengers and baggage for threats to aviation security. For example, by law, TSOs must complete 40 hours of classroom training, 60 hours of on-the-job training, and certification tests before performing screening. Once certified, TSA requires TSOs to complete annual training under the National Training Plan. Since 2013, TSA has been phasing in a program to evaluate its training to inform use of training resources. TSA expects that this evaluation program should help the agency determine how well training meets TSOs' needs, provides them with needed knowledge and skill, and has an impact on their performance.
TSA measures TSO performance in various ways, including (1) annual proficiency reviews, which certify TSOs by evaluating their ability to carry out screening standard operating procedures; (2) assessments of X-ray machine operators' ability to identify prohibited items by displaying fictional threat items, such as guns or explosives, onto X-ray images of actual baggage; and (3) covert testing programs that use role players to take prohibited items through screening checkpoints to test TSOs or determine how TSOs interact with the public, among other things. Over the time periods GAO reviewed, TSA data on the results of annual proficiency reviews and covert testing on how TSOs interact with the public show that TSOs' scores (pass rates) varied by airport security risk category. GAO is not providing TSOs' scores for annual proficiency reviews, X-ray machine operator assessments, or covert testing for prohibited items at checkpoints in this report due to the sensitive or classified nature of the data or the data reliability concerns discussed below.
TSA has made use of annual proficiency review data to enhance TSO training, but its use of other testing data is constrained by incomplete and unreliable data. Specifically, due to software compatibility issues and a lack of automatic uploading capability, airport reporting on assessments of X-ray machine operators was not complete, as required by TSA policy, for each year of data GAO examined (fiscal years 2009 through 2014), limiting their reliability and use to enhance TSO training. In addition, for the data it does collect on these assessments, TSA has not taken steps to analyze these data nationwide, which could help the agency identify potential trends or opportunities to improve TSO performance. Furthermore, in 2015, TSA determined that prior year results of one of its two covert testing programs to assess TSOs' ability to identify prohibited items at checkpoints were unreliable, resulting in pass rates that were likely higher than actual TSO performance. TSA has since taken steps to enhance reliability by hiring a contractor to perform independent validation testing, among other things. Finally, TSA does not require or track implementation by field personnel of national recommendations related to these covert tests, thereby limiting the agency's ability to take advantage of the corrective actions identified from the tests.

What GAO Recommends

GAO recommends that TSA (1) collect complete data on assessments of X-ray machine operators, (2) analyze these data nationally for opportunities to enhance TSO performance, and (3) track the implementation of covert testing recommendations. TSA concurred with the recommendations.