GAO has documented these problems for  decades. Those changes include numerous executive branch initiatives  and legislative actions as well as roughly 11 revisions to DOD’s acquisition  policy between 1971 and 2005. Despite these efforts, defense acquisition  programs in the past 3 decades continued to routinely experience cost  overruns, schedule slips, and performance shortfalls. Figure 1 illustrates the continued problem of development cost overruns. If properly  implemented and enforced this approach could help DOD’s decision  makers gain the confidence they need to make significant and sound  investment decisions for major weapon systems. Of the 23 major programs we assessed, 10 have already reported estimated  development cost growth greater than 30 percent or expected delays of at  least 1 year in delivery of an initial operational capability to the warfighter. The programs listed in table 2 will not achieve the return on  investment that DOD anticipated when they began development. DOD Is Not Effectively Implementing the Policy’s Knowledge- Based, Evolutionary Approach

DOD is not effectively implementing the knowledge-based process and  evolutionary approach emphasized in its acquisition policy. While the  policy outlines a specific knowledge-based process of concept refinement  and technology development to help ensure a sound business case is  developed before committing to a new development program, almost   80 percent of the programs we reviewed were permitted to bypass this  process. Programs Continue Past Design Reviews before Design Maturity is Demonstrated

In addition to emphasizing the importance of capturing technology  knowledge before starting system development, DOD’s policy also  highlights the importance of demonstrating design maturity before moving  from the integration phase of system development into system  demonstration and initial manufacturing. Evolutionary Acquisition Is Not Being Used

Despite the revised policy’s guidance that capabilities should be developed  and delivered in individually defined and separately managed increments,  a majority of major weapon acquisition programs we assessed continue to  be structured to achieve revolutionary increases in capability within one  development program. Conclusions

Failing to consistently implement the knowledge-based process and  evolutionary principles emphasized in the revised acquisition policy— coupled with a lack of specific criteria for making key investment  decisions—are keeping DOD on its historical path of poor cost and  schedule outcomes. Most programs are incurring the same scope of cost  overruns and schedule delays as programs managed under prior DOD  policies. However, in  order for this initiative to be effective DOD must establish and enforce  specific criteria at key decision points. However, without enforceable criteria,  defense officials are challenged to determine whether adequate knowledge  has been obtained for investing taxpayer dollars. At a  minimum those controls should require program officials to demonstrate  that they have achieved a level of knowledge that meets or exceeds the  following criteria at each respective decision point:    Program start (Milestone B): Start of product development    Demonstrate technologies to high readiness levels    Ensure that requirements for the product are informed by the systems-   Establish cost and schedule estimates for product on the basis of  knowledge from preliminary design using system engineering tools    Conduct decision review for program start   Design readiness review: Beginning of system demonstration    Complete 90 percent of design drawings    Complete subsystem and system design reviews    Demonstrate with prototype that design meets requirements    Obtain stakeholders’ concurrence that drawings are complete and    Complete the failure modes and effects analysis    Identify key system characteristics   Identify critical manufacturing processes     Establish reliability targets and growth plan on the basis of  demonstrated reliability rates of components and subsystems    Conduct decision review to enter system demonstration   Production commitment (Milestone C): Initiation of low-rate  production    Demonstrate manufacturing processes    Build production-representative prototypes    Test production-representative prototypes to achieve reliability goal    Test production-representative prototypes to demonstrate product in    Collect statistical process control data    Demonstrate that critical processes are capable and in statistical    Conduct decision review to begin production  Furthermore, to ensure that major decisions are transparent and that program  officials and decision makers are held accountable, we recommend that the  Secretary of Defense require decision makers to include written rationale for  each major decision in acquisition decision documentation. Best Practices: Successful Application to Weapon Acquisition Requires  Changes in DOD’s Environment.
Why GAO Did This Study

The Department of Defense (DOD) is planning to invest $1.3 trillion between 2005 and 2009 in researching, developing, and procuring major weapon systems. How DOD manages this investment has been a matter of congressional concern for years. Numerous programs have been marked by cost overruns, schedule delays, and reduced performance. Over the past 3 decades, DOD's acquisition environment has undergone many changes aimed at curbing cost, schedule, and other problems. In order to determine if the policy DOD put in place is achieving its intended goals, we assessed the outcomes of major weapons development programs initiated under the revised policy. Additionally, we assessed whether the policy's knowledge-based, evolutionary principles are being effectively implemented, and whether effective controls and specific criteria are in place and being used to make sound investment decisions.

What GAO Found

Changes made in DOD's acquisition policy over the past 5 years have not eliminated cost and schedule problems for major weapons development programs. Of the 23 major programs we assessed, 10 are already expecting development cost overruns greater than 30 percent or have delayed the delivery of initial operational capability to the warfighter by at least 1 year. The overall impact of these costly conditions is a reduction in the value of DOD's defense dollars and a lower return on investment. Poor execution of the revised acquisition policy is a major cause of DOD's continued problems. DOD frequently bypasses key steps of the knowledge-based process outlined in the policy, falls short of attaining key knowledge, and continues to pursue revolutionary--rather than evolutionary or incremental--advances in capability. Nearly 80 percent of the programs GAO reviewed did not fully follow the knowledge-based process to develop a sound business case before committing to system development. Most of the programs we reviewed started system development with immature technologies, and half of the programs that have held design reviews did so before achieving a high level of design maturity. These practices increase the likelihood that problems will be discovered late in development when they are more costly to address. Furthermore, DOD's continued pursuit of revolutionary leaps in capability also runs counter to the policy's guidance. DOD has not closed all of the gaps in the policy that GAO identified nearly 3 years ago, particularly with regard to adding controls and criteria. Effective controls require decision makers to measure progress against specific criteria and ensure that managers capture key knowledge before moving to the next acquisition phase. However, DOD's policy continues to allow managers to approach major investment decisions with many unknowns. Without effective controls that require program officials to satisfy specific criteria, it is difficult to hold decision makers or program managers accountable to cost and schedule targets. In this environment, decision-making transparency is crucial, but DOD is lacking in this area as well.