The expected users of the TIM  system come from multiple offices under the Office of Intelligence and  Analysis, including the Security Threat Assessment Operations office,  which is responsible for conducting the security threat assessments, and  the Program Management office, which is responsible for managing  TSA’s maritime, surface, and aviation credentialing programs. In April 2016, the Under Secretary for Management approved the TIM  program’s new strategy and, in September 2016—almost 2 years after  the program was initially suspended—the program was rebaselined to  reflect the new strategy. Additionally, the program  provided initial Agile training for key program staff when it began  transitioning to Agile software development. TSA and DHS Have Not Fully Implemented Most Key Practices for Overseeing the TIM Program’s Cost, Schedule, and Performance

According to leading practices, to ensure effective program oversight of  cost, schedule, and performance, organizations should: ensure that corrective actions are identified and tracked until the  desired outcomes are achieved, document relevant governance and oversight policies and monitor program performance and progress, and  rely on complete and accurate data to review performance against  expectations. While TSA fully implemented the first practice, the remaining three  practices were not fully implemented by DHS and TSA. TSA and DHS have documented selected policies and procedures for  governance and oversight of the TIM program. However, TSA and DHS have not developed or finalized other key  oversight and governance documents. According to TSA officials in May 2017,  they planned to update the Systems Engineering Life Cycle Tailoring  Plan to reflect the revised governance framework, but they did not  have a specific time frame for completing the revision. The TIM program management office conducts frequent and regular  performance reviews and focuses on several important Agile release- level metrics. Department-level oversight bodies have focused on reviewing certain  program life-cycle metrics for the TIM program. This dashboard was to  measure, among other things, post-deployment user satisfaction. However, TIM’s reported performance data were not always complete  and accurate. As  such, program officials were reporting that they were testing every line of  code, even though they were unable to confirm that they were actually  doing so, thus calling into question the reliability and accuracy of the data  reported. An indication of concern is that the program is currently  experiencing a delay of at least 6 months in the rebaselined schedule for  delivering TSA Pre® capabilities. While the program has also taken certain steps to successfully make the  transition from a waterfall development approach to Agile software  development—a substantial and complex effort—TIM has not defined key  roles and responsibilities, prioritized features and user stories, or  implemented automated capabilities that are essential to ensuring  effective adoption of Agile. While selected  corrective actions have been taken, until the TIM program is implemented  in accordance with leading practices, the program will be putting at risk its  ability to deliver a quality system that strengthens and enhances the  sophistication of TSA’s security threat assessment and credentialing  programs. Moreover, until the Office of  the Chief Technology Officer completes guidance for Agile programs to  use for collecting and reporting on performance metrics, and DHS-level  oversight bodies require the TIM program to report on key Agile  performance and cost metrics and use them to inform management  oversight decisions, the department will also be limited in its ability to hold  the TIM program accountable and ensure that it is meeting its cost,  schedule, and performance targets. Appendix I: Objectives, Scope, and Methodology

Our objectives were to (1) describe the Transportation Security  Administration’s (TSA) past implementation efforts for the Technology  Infrastructure Modernization (TIM) program and its new implementation  strategy; (2) determine the extent to which TSA’s new strategy for the  program addresses the challenges encountered during earlier  implementation attempts; (3) determine the extent to which TSA has  implemented selected key practices for transitioning to an Agile software  development framework for the program; and (4) determine the extent to  which the TSA and the Department of Homeland Security (DHS) are  effectively overseeing and governing the TIM program to ensure that it is  meeting cost, schedule, and performance requirements. To determine the extent to which TSA and DHS are effectively overseeing  and governing the program to ensure that it is meeting cost, schedule,  and performance requirements, we identified leading practices and  guidance outlined in the following sources:
TSA Agile Scrum guidance
CMMI for Development, version 1.3
Software Engineering Institute, Agile Metrics  After reviewing the sources listed, we grouped practices related to  oversight and governance for programs using Agile software development  into four key practice areas and we discussed the practices with DHS and  TSA officials.
Why GAO Did This Study

TSA conducts security threat assessment screening and credentialing activities for millions of workers and travelers in the maritime, surface, and aviation transportation industries that are seeking access to transportation systems. In 2008, TSA initiated the TIM program to enhance the sophistication of its security threat assessments and to improve the capacity of its supporting systems. However, the program experienced significant cost and schedule overruns, and performance issues, and was suspended in January 2015 while TSA established a new strategy. The program was rebaselined in September 2016 and is estimated to cost approximately $1.27 billion and be fully operational by 2021 (about $639 million more and 6 years later than originally planned).
GAO was asked to review the TIM program's new strategy. This report determined, among other things, the extent to which (1) TSA implemented selected key practices for transitioning to Agile software development for the program; and (2) TSA and DHS are effectively overseeing the program's cost, schedule, and performance. GAO compared program documentation to key practices identified by the Software Engineering Institute and the Office of Management and Budget, as being critical to transitioning to Agile and for overseeing and governing programs.

What GAO Found

The Transportation Security Administration's (TSA) new strategy for the Technology Infrastructure Modernization (TIM) program includes using Agile software development, but the program only fully implemented two of six leading practices necessary to ensure successful Agile adoption. Specifically, the Department of Homeland Security (DHS) and TSA leadership fully committed to adopt Agile and TSA provided Agile training. Nonetheless, the program had not defined key roles and responsibilities, prioritized system requirements, or implemented automated capabilities that are essential to ensuring effective adoption of Agile. Until TSA adheres to all leading practices for Agile implementation, the program will be putting at risk its ability to deliver a quality system that strengthens and enhances the sophistication of TSA's security threat assessments and credentialing programs.
TSA and DHS fully implemented one of the key practices for overseeing the TIM program, by establishing a process for ensuring corrective actions are identified and tracked. However, TSA and DHS did not fully implement the remaining three key practices, which impede the effectiveness of their oversight. Specifically,
TSA and DHS documented selected policies and procedures for governance and oversight of the TIM program, but they did not develop or finalize other key oversight and governance documents. For example, TSA officials developed a risk management plan tailored for Agile; however, they did not update the TIM system life-cycle plan to reflect the Agile governance framework they were using.
The TIM program management office conducted frequent performance reviews, but did not establish thresholds or targets for oversight bodies to use to ensure that the program was meeting acceptable levels of performance. In addition, department-level oversight bodies have focused on reviewing selected program life-cycle metrics for the TIM program; however, they did not measure the program against the rebaselined cost, or important Agile release-level metrics.
TIM's reported performance data were not always complete and accurate. For example, program officials reported that they were testing every line of code, even though they were unable to confirm that they were actually doing so, thus calling into question the accuracy of the data reported.
These gaps in oversight and governance of the TIM program were due to, among other things, TSA officials not updating key program management documentation and DHS leadership not obtaining consensus on needed oversight and governance changes related to Agile programs. Given that TIM is a historically troubled program and is at least 6 months behind its rebaselined schedule, it is especially concerning that TSA and DHS have not fully implemented oversight and governance practices for this program. Until TSA and DHS fully implement these practices to ensure the TIM program meets its cost, schedule, and performance targets, the program is at risk of repeating past mistakes and not delivering the capabilities that were initiated 9 years ago to protect the nation's transportation infrastructure.

What GAO Recommends

GAO is making 14 recommendations, including that DHS should prioritize requirements and obtain leadership consensus on oversight and governance changes. DHS concurred with all 14 recommendations.