All Programs with a Critical Change Reported Required Elements, but Most Missed Deadlines and Obligations Were Not Monitored

According to statute, for programs that declare a critical change, the  report that is submitted to Congress must include a written certification  stating that:  the automated information system or IT investment to be acquired is  essential to the national security or to the efficient management of the  DOD;  there is no alternative to the system or IT investment which will  provide equal or greater capability at less cost;  the new estimates of the costs, schedule, and performance  parameters have been determined, with the concurrence of the  Director of Cost Assessment and Program Evaluation, to be  reasonable; and  the management structure for the program is adequate to manage  and control program costs. All 18 MAIS critical change reports in our review contained the required  elements. Specifically, 16 exceeded the 60-day reporting requirement and 10 of  those programs took over 100 days to report. CAC2S program was late because of the need to conduct an  independent assessment that was directed by DOD. According to a DOD AT&L official, 60 days is too short to perform a  program evaluation and achieve all the coordination necessary for an  important communication with Congress. Further, DOD does not have a mechanism to monitor and ensure that  MAIS programs with late reports were restricted as required by law from  obligating funds on major contracts prior to Congress receiving the report. Selected MAIS Programs Had Varying Cost and Schedule Estimate Changes, One Did Not Meet Its Technical Performance Targets

The extent to which the three selected MAIS programs in our study  experienced changes in their cost and schedule estimates and met  performance targets varied. All three selected programs implemented IT acquisition best practices for  risk management, but requirements management best practices were not  consistently implemented by the programs. The Army had implemented three requirements management best  practices for the program, but did not fully implement two: the practice of  managing requirements changes and ensuring that work products are in  alignment with requirements. The Air Force Program Implemented Risk Management Practices, but Did Not Apply One Requirements Management Practice

The Air Force implemented all seven risk management best practices for  the DEAMS program. However, the organization responsible for supervising MAIS acquisition  programs—AT&L—is not represented on the Dashboard. Nonetheless, since only the DOD CIO is represented on the  Dashboard, the public and other users may be unaware that AT&L has  overall oversight for the acquisition performance of MAIS programs,  minimizing the intended accountability the Dashboard is to provide. Managing  requirements effectively is especially necessary since MAIS programs are  intended to help the department sustain its key operations. Recommendations for Executive Action

To help improve the management of MAIS programs, we are making six  recommendations that:
The Secretary of Defense examine the MAIS critical change reporting  process to identify root causes for delays and implement corrective  actions for the timely delivery of critical change reports. The IT Dashboard publicly  shows DOD’s CIO as the responsible party, pursuant to OMB’s direction. Appendix I: Objectives, Scope, and Methodology

The National Defense Authorization Act for Fiscal Year 2012 includes a  provision that we select, assess, and report on selected Department of  Defense (DOD) major automated information system (MAIS) programs  annually through March 2018. Our objectives for this report were to (1)  evaluate DOD’s implementation of statutory reporting requirements for  MAIS programs experiencing a critical change; (2) describe the extent to  which selected MAIS programs have changed their planned cost and  schedule estimates, and met performance targets; (3) assess the extent  to which selected MAIS programs have used key IT acquisition best  practices, including risk management; and (4) determine the extent to  which MAIS programs are accurately represented on the Federal IT  Dashboard. For those programs that were found to not be reported on the  Dashboard, we met with agency officials from the Office of the Chief  Information Officer (OCIO) and the Under Secretary of Defense  (Acquisition, Technology, and Logistics) (AT&L) to determine the  reasons for not reporting. Did Not Fully Meet System Performance Targets  As of October 2015, DEAMS program officials reported that it did not  meet all of its nine key performance parameters.
Why GAO Did This Study

The National Defense Authorization Act for Fiscal Year 2012 includes a provision for GAO to select, assess, and report on DOD MAIS programs annually through March 2018. MAIS programs are intended to help the department sustain its key operations. This report: (1) evaluates DOD's implementation of statutory reporting requirements for MAIS programs experiencing a critical change; (2) describes the extent to which selected MAIS programs have changed their planned cost and schedule estimates, and met performance targets; (3) assesses the extent to which selected MAIS programs have used key IT acquisition best practices, including requirements and risk management; and (4) determines the extent to which MAIS programs are represented on the Dashboard. GAO compared information on programs with a critical change to the reporting requirements. GAO selected three programs based on factors, such as representation from each military service (Air Force, Army, and Navy), identified changes to cost, schedule, and performance, and assessed them against selected best practices. GAO traced the programs to the Dashboard and reviewed relevant processes.

What GAO Found

All 18 major automated information system (MAIS) programs that experienced a critical change to program cost, schedule, or system performance targets submitted complete reports to Congress that contained all four statutory elements, but 16 programs did not meet the requirement to report to Congress within 60 days of the program manager's submission to the senior Department of Defense (DOD) official that led to the critical change determination. Of the 16 critical change reports that exceeded the 60-days to report, 10 of the programs took over 100 days. Officials said that 60 days is too short to perform a program evaluation. Since the reports were not always timely, Congress may not have the necessary information when it is needed to make decisions. Finally, the DOD did not demonstrate that it had an internal control to ensure that MAIS programs not in compliance with reporting requirements were restricted from obligating funds on major contracts as required by law.
All three MAIS programs GAO selected to review experienced changes in their cost and schedule estimates, and one program did not fully meet its technical performance targets (see table).
Source: GAO analysis of data provided by DOD officials. | GAO-16-336
a Delay was attributed to a major change in project scope and restructuring of the program.
The three selected programs implemented all seven IT acquisition best practices for risk management, and most of the best practices were implemented for requirements management: the Army and Navy implemented three of five best practices and the Air Force implemented four of five best practices. For example, the Army program did not adequately manage requirements changes and ensure that deliverables were in alignment with requirements. Until the programs fully implement best practices for requirements management, management of development efforts will likely be impaired.
As of October 2015, all appropriate programs were represented on the Federal IT Dashboard (Dashboard) as required by the Office of Management and Budget (OMB); however, the organization responsible for performance of MAIS programs was not provided. Specifically, DOD's Chief Information Officer is shown as the responsible party because OMB requires this, but the Under Secretary of Defense for Acquisition, Technology, and Logistics (AT&L) has overall responsibility for the MAIS programs. Therefore, users of the Dashboard are unaware that AT&L is the responsible organization and, thus, public accountability of the MAIS programs is decreased.

What GAO Recommends

GAO recommends, among others, that DOD examine the critical change reporting process and implement corrections for the reports' timeliness, and address weaknesses with requirements management, and add AT&L as a responsible organization for MAIS programs to the Dashboard. DOD concurred with all recommendations. OMB did not concur but GAO continues to believe that improved transparency is needed.