On the 11 weapon systems GAO reviewed, these  problems have resulted in billions of dollars in cost overruns, years of  schedule delays, and reduced weapon system availability. Prime  contractors’ poor systems engineering practices related to requirements  analysis, design, and testing were key contributors to these quality  problems. Quality problems occurred  despite the fact that each of the prime contractors for these programs is  certified to commercial quality standards and most provided us with  quality plans that address systems engineering activities such as design, as  well as manufacturing, and supplier quality. The contractor also experienced  major problems with the aircraft canopy. This problem was  discovered in March 2000 and temporarily grounded the flight test aircraft. Two examples are the Wideband Global  SATCOM and Patriot Advanced Capability-3 programs. Cummins officials noted  that these activities allow them to catch problems earlier in development,  when problems are less costly to fix. Leading commercial companies we met with set high expectations for  supplier quality. Boeing Commercial Airplanes categorizes its suppliers by  rates of defective parts per million. Officials with Space Systems/Loral discussed how they adopted a more  disciplined product development process following quality problems in the  1990s with some of its satellites. During systems development, DOD usually pays for a  contractor’s best efforts, which can include efforts to achieve overly  optimistic requirements. This environment provides little incentive for contractors to utilize the  best systems engineering, manufacturing, and supplier quality practices  discussed earlier in this report to ensure manageable requirements, stable  designs, and controlled manufacturing processes to hold costs down. For example, DOD as the  customer for the Expeditionary Fighting Vehicle signed a cost  reimbursement contract with the prime contractor, General Dynamics, to  develop a new weapon system that would meet performance and  reliability requirements that had not yet been adequately informed by  systems engineering analysis. The F-22A program illustrates this point. Recommendations for Executive Action

To ensure that the department is taking steps to improve the quality of  weapon systems, we recommend that the Secretary of Defense take the  following actions related to recent initiatives highlighted in DOD’s Defense  Acquisition Transformation and Program Manager Empowerment and  Accountability reports to Congress to improve its focus on setting  achievable requirements and oversight:
As a part of the concept decision review initiative, have   contractors perform more detailed systems engineering analysis to  develop sound requirements before DOD selects a prime  contractor for the systems development contract, which would  help ensure that weapon system requirements, including those for  reliability, are achievable with given resources. Specifically, we (1) determined the impact of quality problems on  selected DOD weapon systems and defense contractors’ practices that  contributed to the problems, (2) identified practices used by leading  commercial companies that can be used to improve the quality of DOD weapon  systems, (3) identified problems DOD faces in terms of improving quality, and  (4) identified recent DOD initiatives that could improve quality. To determine the impact of quality problems on selected DOD weapon  systems and defense contractors’ practices that contribute to the problems,  we selected and reviewed 11 DOD weapon systems with known deficiencies  from each of the military services and identified the quality problems  associated with each deficiency. To identify practices used by leading commercial companies that can be  used to improve the quality of DOD weapon systems, we selected and  visited five companies based on several criteria: companies that make  products similar to DOD weapon systems in terms of complexity;  companies that have been recognized in quality management literature or  by quality-related associations/research centers for their high-quality  products; companies that have won quality-related awards; and/or  companies that have close relationships with customers when developing  and producing products. To identify recent DOD initiatives that could improve weapon system  quality, we reviewed DOD’s formal response to Sections 804 and 853 of the  John Warner National Defense Authorization Act for Fiscal Year 2007. Most of the  programs had problems in more than one of these categories. DOD Acquisition Outcomes: A Case for Change.
Why GAO Did This Study

A Senate report related to the National Defense Authorization Act for Fiscal Year 2007 asked GAO to compare quality management practices used by the Department of Defense (DOD) and its contractors to those used by leading commercial companies and make suggestions for improvement. To do this, GAO (1) determined the impact of quality problems on selected weapon systems and prime contractor practices that contributed to the problems; (2) identified commercial practices that can be used to improve DOD weapon systems; (3) identified problems that DOD must overcome; and (4) identified recent DOD initiatives that could improve quality. GAO examined 11 DOD weapon systems with known quality problems and met with quality officials from DOD, defense prime contractors, and five leading commercial companies that produce complex products and/or are recognized for quality products.

What GAO Found

Problems related to quality have resulted in major impacts to the 11 DOD weapon systems GAO reviewed--billions in cost overruns, years-long delays, and decreased capabilities for the warfighter. For example, quality problems with the Expeditionary Fighting Vehicle program were so significant that DOD extended development 4 years at a cost of $750 million. The F-22A fighter aircraft experienced cracks in the plane's canopy that grounded the flight test aircraft, and initial operating capability for the Wideband Global SATCOM satellite was delayed 18 months because a supplier installed some fasteners incorrectly. GAO's analysis of 11 DOD weapon systems illustrates that defense contractors' poor practices for systems engineering activities as well as manufacturing and supplier quality problems contributed to these outcomes. Reliance on immature designs, inadequate testing, defective parts, and inadequate manufacturing controls are some of the quality problems that GAO found. Senior prime contractor officials GAO met with generally agreed with GAO's assessment of the causes of the quality problems. In contrast, leading commercial companies GAO contacted use more disciplined systems engineering, manufacturing, and supplier quality practices. For example, rather than wait to discover defects after the fact, Boeing Commercial Airplanes tries to design parts that can be assembled only one way. Effective use of many systems engineering practices has helped Space Systems/Loral, a satellite producer, improve overall quality, for example, by allowing the company to operate its satellites for more than 80 million consecutive hours in orbit with just one failure. Companies also put significant effort into validating product design and production processes to catch problems early on, when problems are less costly to fix. They conduct regular audits of their suppliers and hold them accountable for quality problems. DOD faces its own set of challenges--setting achievable requirements for systems development and providing effective oversight during the development process. In conducting systems development, DOD generally pays the allowable costs incurred for the contractor's best efforts. These conditions contribute to an acquisition environment that is not conducive for incentivizing contractors to build high-quality weapon systems and DOD, which typically uses cost-reimbursement contracts to develop weapon systems, assumes most of the risks and pays contractors to fix most of the problems. DOD has taken steps to improve its acquisition practices by experimenting with a new concept decision review practice, selecting different acquisition approaches according to expected fielding times, and establishing panels to review weapon system configuration changes that could adversely affect program cost and schedule. None of these initiatives focus exclusively on quality issues, and none specifically address problems with defense contractors' practices.