DOD Does Not Have Reliable Data on ACAT II and III Programs and Component Actions to Improve Data Do Not Fully Address Limitations

DOD components could not provide sufficiently reliable data for us to  accurately determine the number, total cost, or performance of DOD’s  current ACAT II and III programs. For example, the components have not established  systematic processes to perform data quality tests on PEO-submitted  data and assess the results to help identify problems, such as basic math  errors or missing data, for further review. These types of tests and  assessments can be an important step in determining whether data can  be used for its intended purposes. Cost and schedule data for these reports  are pulled from DOD’s web-based DAMIR system. ACAT II and III Program Performance Most Frequently Affected by Requirements Changes and Testing Issues

Thirteen of the 15 ACAT II or III programs we reviewed in-depth had  exceeded the cost or schedule targets in their original APBs. These  programs cited changing requirements, testing issues, quantity changes,  and flaws in original cost estimates, among other factors, as the reasons  for cost and schedule growth. We have previously reported that similar factors  affect the performance of DOD’s MDAPs. While tailoring documentation and reporting requirements for “smaller”  programs can be a reasonable approach to help prioritize limited  oversight resources, if DOD and its components are to effectively manage  their investment dollars, they must be able to account for how they are  spending their money and how well they are spending it on the full range  of acquisition programs. Having timely and reliable data on smaller  acquisition programs is also critical for providing effective oversight and  bringing the right oversight resources to bear, when needed, to make  sure troubled smaller programs do not grow into major ones. DOD components have taken and continue to take  steps to improve the reliability of ACAT II and III program data, but they  do not fully address the limitations we identified—missing data,  widespread data entry issues and inconsistent reporting—or the causes  of these issues, including: the lack of a common definition of a current  acquisition program; insufficient data reliability testing; and inconsistent  compliance with requirements for acquisition program baselines and  reporting on ACAT II and III programs that may become major programs  due to cost growth. Specifically, we recommend  that the Secretary of Defense direct the Under Secretary of Defense for  Acquisition, Technology, and Logistics, in consultation with DOD  components, to take the following actions: establish guidelines on what constitutes a “current” ACAT II or III  program for reporting purposes; the types of programs, if any, that do  not require ACAT designations; and whether the rules for identifying  current MDAPs would be appropriate for ACAT II and III programs;  and determine what metrics should be used and what data should be  collected on ACAT II and III programs to measure cost and schedule  performance; and whether the use of DAMIR and the MDAP selected  acquisition report format may be appropriate for collecting data on  ACAT II and III programs. However, as discussed  below, it is unclear whether the actions that DOD plans to take will fully  address the issues we raised in this report. Appendix I: Objectives, Scope, and Methodology

Our objectives were to assess (1) the extent to which information is  available on the number of the Department of Defense’s (DOD) current  acquisition category (ACAT) II and III programs, their total estimated  acquisition cost, and cost and schedule performance; (2) the factors  affecting the cost and schedule performance of selected ACAT II and III  programs; and (3) the number of current ACAT II and III programs that  are likely to become major defense acquisition programs (MDAP). To address our second objective, we selected a non-generalizable  sample of 15 programs from the data provided by DOD in response to our  DCI.
Why GAO Did This Study

DOD requested $168 billion in fiscal year 2014 to develop, test, and acquire weapon systems and other products and equipment. About 40 percent of that total is for major defense acquisition programs or ACAT I programs. DOD also invests in other, non-major ACAT II and III programs that are generally less costly at the individual program level. These programs typically have fewer reporting requirements and are overseen at lower organizational levels than ACAT I programs, although they may have annual funding needs that are just as significant.
GAO was asked to examine ACAT II and III programs. This report addresses, among other issues, (1) the extent to which information is available on the number, cost, and performance of ACAT II and III programs and (2) factors that affected the performance of selected ACAT II and III programs. GAO collected program and cost data on current ACAT II and III programs from five DOD components. GAO also selected a non-generalizable sample of 15 programs based on program cost and other criteria and reviewed documentation and interviewed officials about program performance.

What GAO Found

The Department of Defense (DOD) could not provide sufficiently reliable data for GAO to determine the number, total cost, or performance of DOD's current acquisition category (ACAT) II and III programs. These non-major programs range from a multibillion dollar aircraft radar modernization program to soldier clothing and protective equipment programs in the tens of millions of dollars. GAO found that the accuracy, completeness, and consistency of DOD's data on these programs were undermined by widespread data entry issues, missing data, and inconsistent identification of current ACAT II and III programs. See the figure below for selected data reliability issues GAO identified.
DOD components are taking steps to improve ACAT II and III data, but these steps do not fully address the problems GAO identified. For example, the components have not established systematic processes to perform data quality tests and assess the results to help identify problems for further review. These types of tests and assessments can be an important step in determining whether data can be used for its intended purposes. Additionally, DOD lacks metrics to assess ACAT II and III cost and schedule performance trends across programs and in some cases was missing baseline cost and schedule data to measure performance. Having timely and reliable cost, schedule, and performance data on smaller acquisition programs is critical to ensuring that DOD and its components can account for how they are spending their money and how well they are spending it. Reliable data are also essential for effective oversight and bringing the right oversight resources to bear when programs approach the cost threshold to become a major defense acquisition program due to cost growth.
Thirteen of the 15 ACAT II or III programs GAO reviewed in-depth had exceeded their original cost or schedule targets. Program officials from ACAT II and III programs GAO reviewed cited changing performance requirements, testing issues, quantity changes, and flaws in original cost estimates, among other factors, as the reasons for cost and schedule growth. GAO has previously found that similar factors affect the performance of major acquisition programs.

What GAO Recommends

GAO recommends that DOD establish guidelines on what constitutes a current ACAT II and III program, take steps to improve data reliability, and determine how to measure cost and schedule performance. DOD partially concurred with the recommendations and described actions it plans to take. However, as discussed in the report, DOD's planned actions may not fully address the issues that GAO identified.