TSA Collects and Uses Data to Track Canine Program Performance, but Could Better Analyze These Data to Identify Program Trends

TSA collects and uses key canine program data in its Canine Website  System, a central management database, but it could better analyze  these data to identify program trends. Through CWS, TSA captures the  amount of time canine handlers spend on proficiency training as well as  screening for explosives, among other functions. However,  TSA has not fully analyzed the performance data it collects in CWS to  identify program trends and areas that are working well or in need of  corrective action. Such analyses could better position TSA to determine  canine teams’ proficiency, guide future deployment efforts, and help  ensure that taxpayer funds are being used effectively. Our analysis of CWS  utilization data for the period from May 2011 through April 2012 showed  that LEO teams consistently reported greater levels of monthly utilization  minutes than TSI teams, and that on average, from September 2011  through July 2012, TSI air cargo teams exceeded their monthly screening  requirement. Our analysis of TSA’s  cargo screening data from September 2011 through July 2012 showed  that TSI air cargo teams nationwide generally exceeded their monthly  requirement to screen air cargo placed on passenger aircraft. This  suggests that TSA could increase the percentage of air cargo it requires  canine teams to screen, or redeploy additional TSI air cargo teams. TSA Has Not Deployed Passenger Screening Canine Teams to the Highest- Risk Airports and Did Not Determine Their Effectiveness Prior to Deployment

TSA’s 2012 Strategic Framework calls for the deployment of PSC teams  based on risk; however, airport stakeholder concerns about the  appropriateness of TSA’s response resolution protocols for these teams  have resulted in PSC teams not being deployed to the highest-risk airport  terminals and concourses. TSA  officials agreed that the decision to defer to airport stakeholders’  willingness to have PSC teams deployed to their airports has resulted in  PSCs not being optimally utilized. TSA Plans to Complete PSC Effectiveness Assessments in 2013; Initial Results Indicate Performance Challenges

TSA began deploying PSC teams in April 2011 prior to determining the  teams’ operational effectiveness and before identifying where within the  airport environment these teams would be most effectively utilized. To assess the effectiveness of PSC teams in detecting  explosives odor, TSA and DHS S&T are assessing their capabilities  based on several factors. Assessing the effectiveness of PSCs in the  operational environment could help provide TSA with reasonable  assurance that PSCs are effective in identifying explosives odor on  passengers and provide an enhanced security benefit. Recommendations for Executive Action

To help ensure TSA analyzes canine team data to identify program  trends, and determines if PSC teams provide an added security benefit to  the civil aviation system, and if so, deploys PSC teams to the highest-risk  airports, we recommend that the Administrator of the Transportation  Security Administration direct the Manager of the NCP to take the three  following actions:
Regularly analyze available data to identify program trends and areas  that are working well and those in need of corrective action to guide  program resources and activities. Expand and complete testing, in conjunction with DHS S&T, to assess  the effectiveness of PSCs and conventional canines in all airport  areas deemed appropriate (i.e., in the sterile area, at the passenger  checkpoint, and on the public side of the airport) prior to making  additional PSC deployments to help (1) determine whether PSCs are  effective at screening passengers, and resource expenditures for PSC  training are warranted, and (2) inform decisions regarding the type of  canine team to deploy and where to optimally deploy such teams  within airports. We believe it is important for TSA to also review its  analysis of canine teams’ performance screening air cargo in order to  inform future decisions on the percentage of air cargo that it requires  canine teams to screen, as well as the number of TSI air cargo canine  teams needed to meet TSA’s air cargo screening requirement. DHS concurred with the third recommendation, to deploy PSC teams to  the highest-risk airports. (2) To what extent has TSA deployed passenger screening canine  teams using a risk-based approach and determined their  effectiveness prior to deployment? To determine what data TSA has on its canine program, what these data  show, and the extent to which TSA has analyzed these data to identify  program trends, we interviewed TSA’s Office of Security Operations  headquarters and field officials about the type of data they collect and  how they analyze the data to monitor canine team effectiveness  consistent with Standards for Internal Control in the Federal  Government.
Why GAO Did This Study

TSA has implemented a multilayered system composed of people, processes, and technology to protect the transportation system. One of TSA's security layers is NCP, composed of over 760 deployed explosives detection canine teams, including PSC teams trained to detect explosives on passengers. As requested, GAO examined (1) data TSA has on its canine program, what these data show, and to what extent TSA analyzed these data to identify program trends, and (2) the extent to which TSA deployed PSC teams using a risk-based approach and determined their effectiveness prior to deployment. To address these questions, GAO conducted visits to four geographic locations selected based on the number and type of canine teams deployed. The results of these site visits are not generalizable, but provided insights into NCP. GAO also analyzed TSA data from 2011 through 2012, such as utilization data; reviewed documents, including response protocols; and interviewed DHS officials.

What GAO Found

The Transportation Security Administration (TSA), the federal agency that administers the National Canine Program (NCP), is collecting and using key data on its canine program, but could better analyze these data to identify program trends. TSA collects canine team data using the Canine Website System (CWS), a central management database. TSA uses CWS to capture the amount of time canine teams conduct training as well as searching for explosives odor, among other functions. However, TSA has not fully analyzed the data it collects in CWS to identify program trends and areas that are working well or in need of corrective action. Such analyses could help TSA to determine canine teams' proficiency, inform future deployment efforts, and help ensure that taxpayer funds are used effectively. For example:
GAO analysis of canine team training data from May 2011 through April 2012 showed that some canine teams were repeatedly not in compliance with TSA's monthly training requirement, which is in place to ensure canine teams remain proficient in explosives detection.
GAO analysis of TSA's cargo-screening data from September 2011 through July 2012 showed that canine teams primarily responsible for screening air cargo placed on passenger aircraft exceeded their monthly screening requirement. This suggests that TSA could increase the percentage of air cargo it requires air cargo canine teams to screen or redeploy teams.
TSA has not deployed passenger screening canines (PSC)--trained to identify and track explosives odor on a person--consistent with its risk-based approach, and did not determine PSC teams' effectiveness prior to deployment. TSA's 2012 Strategic Framework calls for the deployment of PSC teams based on risk; however, GAO found that PSC teams have not been deployed to the highest-risk airport locations. TSA officials stated that the agency generally defers to airport officials on whether PSC teams will be deployed, and some airport operators have decided against the use of PSC teams at their airports because of concerns related to the composition and capabilities of PSC teams. As a result of these concerns, the PSC teams deployed to higher-risk airport locations are not being used for passenger screening as intended, but for other purposes, such as screening air cargo or training. TSA is coordinating with aviation stakeholders to resolve concerns related to PSC team deployment, but has been unable to resolve these concerns, as of September 2012. Furthermore, TSA began deploying PSC teams in April 2011 prior to determining the teams' operational effectiveness and before identifying where within the airport these teams would be most effectively utilized. TSA is in the process of assessing the effectiveness of PSC teams in the operational environment, but testing is not comprehensive since it does not include all areas at the airport or compare PSCs with already deployed conventional canines (trained to detect explosives in stationary objects). As a result, more comprehensive testing could provide TSA with greater assurance that PSC teams are effective in identifying explosives odor on passengers and provide an enhanced security benefit.
This is a public version of a sensitive report that GAO issued in December 2012. Information TSA deemed Sensitive Security Information has been redacted.

What GAO Recommends

GAO is recommending that TSA (1) regularly analyze data to identify program trends and areas working well or in need of corrective action, and (2) take actions to comprehensively assess the effectiveness of PSCs. If PSCs are determined to be effective, GAO is recommending that TSA coordinate with stakeholders to deploy PSC teams to the highest-risk airport locations and utilize them as intended. DHS concurred with GAO's recommendations.