During the material solutions phase, prior to milestone A, the 2008 policy  instruction requires the analysis of alternatives to assess “manufacturing  feasibility.” During the technology development phase, prior to milestone  B, the instruction states the following:    Prototype systems or appropriate component-level prototyping shall  be employed to “evaluate manufacturing processes.”
A successful preliminary design review will “identify remaining design,  integration, and manufacturing risks.”
A program may exit the technology development phase when “the  technology and manufacturing processes for that program or  increment have been assessed and demonstrated in a relevant  environment” and “manufacturing risks have been identified.”
After milestone B, one of the purposes of the engineering and  manufacturing development phase is to “develop an affordable and  executable manufacturing process.” The instruction says that: “the  maturity of critical manufacturing processes” is to be described in a post- critical design review assessment; system capability and manufacturing  process demonstration shall show “that system production can be  supported by demonstrated manufacturing processes;” and the system  capability and manufacturing process demonstration effort shall end,  among other things, when “manufacturing processes have been effectively  demonstrated in a pilot line environment, prior to milestone C.”
Finally, at milestone C, the instruction establishes two entrance criteria for  the production and deployment phase, which include “no significant  manufacturing risks” and “manufacturing processes  under control (if  Milestone C is full-rate production).” Low-rate initial production follows in  order to ensure an “adequate and efficient manufacturing capability.” In  order to receive full-rate production approval, the following must be  shown:  1. As a result, systems cost far more and take far longer to  produce than estimated. These include the inattention to  manufacturing during planning and design, poor supplier management,  and lack of a knowledgeable manufacturing workforce. Manufacturing Contributed to Growth in Cost and Delays in Schedule

Defense acquisition programs continue to be troubled by unstable  requirements, immature technology, and a lack of manufacturing  knowledge early in design, resulting in more costly products that take  longer to produce. These issues illustrate the major problems we  discussed with defense and contractor officials, but do not encompass all  the manufacturing problems experienced by the programs. MRLs Have Been Proposed to Improve the Way DOD Identifies and Manages Manufacturing Risk and Readiness

The Joint Defense Manufacturing Technology Panel working group has  proposed MRLs as new manufacturing readiness criteria that could  improve weapon system outcomes by standardizing the way programs  identify and manage manufacturing risks associated with developing and  fielding advanced weapon systems. In addition, the Army and Air Force report that their use of  MRLs on pilot programs contributed to substantial cost benefits on a  variety of programs, including major acquisition programs. Analysis Shows MRLs Address Manufacturing Gaps in DOD’s Technical Reviews

An analysis conducted by the working group shows that MRLs address  many of the manufacturing gaps identified in several of DOD’s technical  reviews that provide program oversight and determine how well  programs are meeting expected goals, particularly the reviews conducted  in the early acquisition phases. The best practices they employed were focused on gathering a sufficient  amount of knowledge about their products’ producibility in order to lower  manufacturing risks and included stringent manufacturing readiness  criteria—to measure whether the product was mature enough to move  forward in its development. Best Practice: Commercial Companies Emphasize Manufacturing Criteria Early and at Every Stage of the Product-Development Life Cycle

Each commercial firm we visited developed a disciplined framework for  product development that assessed producibility at each gate using clearly  defined manufacturing-maturity criteria that are similar in many respects  to DOD’s MRLs. This starts with a supplier self-assessment. Commercial Firms Require That Manufacturing Processes Be in Control Earlier Than DOD’s MRLs

Although the firms we visited used manufacturing readiness criteria  similar to DOD's proposed MRLs, one important difference we observed is  that the commercial best practice is to have manufacturing processes in  control prior to the production decision, while DOD's MRLs require  manufacturing processes and procedures to be established and controlled  during MRL 9, which occurs after the milestone C production decision,  which authorizes a program to enter low-rate initial production, or  equivalent. The product is not moved into production until the firm is  satisfied that these processes are in control. For example, obtaining agreement on a policy that  would institutionalize MRLs defensewide has proven difficult. Concerns  raised by the military-service policymakers have centered on when and  how the MRL assessments would be used. Table 4  shows the decrease in the manufacturing career field across DOD from  2001 to 2007. Yet, they have not been adopted DOD-wide. Specifically, we assessed (1) the manufacturing  problems experienced by DOD, (2) how manufacturing readiness levels  (MRLs) can address DOD’s manufacturing problems, (3) how proposed  MRLs compare to manufacturing best practices of leading commercial  companies, and (4) the challenges and barriers to implementing MRLs at  DOD. Each of the military services and other DOD components has  been actively planning and deploying initiatives that support the DOD acquisition  workforce growth strategy. Related GAO Products

Defense Acquisitions: Assessments of Selected Weapon Programs.
Why GAO Did This Study

Cost growth and schedule delays are prevalent problems in acquiring defense weapon systems. Manufacturing systems has proven difficult, particularly as programs transition to production. In December 2008, the Department of Defense (DOD) issued an updated version of its acquisition policy that reflects earlier consideration of manufacturing risks. A joint defense and industry group developed manufacturing readiness levels (MRL) to support assessments of manufacturing risks. Use of MRLs on all weapon acquisition programs has been proposed. In response to a congressional request, this report assesses the manufacturing problems faced by DOD, how MRLs can address manufacturing problems, how MRLs compare to manufacturing best practices of leading commercial firms, and challenges and barriers to implementing MRLs at DOD. In conducting our work, we contacted DOD, military services, and contractors; held interviews with leading commercial firms; reviewed program documents and policy proposals; and spoke with manufacturing experts.

What GAO Found

DOD faces problems in manufacturing weapon systems--systems cost far more and take much longer to build than estimated. Billions of dollars in cost growth occur as programs transition from development to production, and unit-cost increases are common after production begins. Several factors contribute to these problems including inattention to manufacturing during planning and design, poor supplier management, and a deficit in manufacturing knowledge among the acquisition workforce. Essentially, programs did not identify and resolve manufacturing risks early in development, but carried risks into production where they emerged as significant problems. MRLs have been proposed as new criteria for improving the way DOD identifies and manages manufacturing risks and readiness. Introduced to the defense community in 2005, MRLs were developed from an extensive body of manufacturing knowledge that includes defense, industry, and academic sources. An analysis of DOD's technical reviews that assesses how programs are progressing show that MRLs address many gaps in core manufacturing-related areas, particularly during the early acquisition phases. Several Army and Air Force centers that piloted MRLs report these metrics contributed to substantial cost benefits on a variety of technologies and major defense acquisition programs. To develop and manufacture products, the commercial firms we visited use a disciplined, gated process that emphasizes manufacturing criteria early in development. The practices they employ focus on gathering sufficient knowledge about the producibility of their products to lower risks, and include stringent manufacturing readiness criteria to measure whether the product is sufficiently mature to move forward in development. These criteria are similar to DOD's proposed MRLs in that commercial firms (1) assess producibility at each gate using clearly defined manufacturing criteria to gain knowledge about manufacturing early, (2) demonstrate manufacturing processes in a production-relevant environment, and (3) emphasize relationships with critical suppliers. However, a key difference is that commercial firms, prior to starting production, require their manufacturing processes to be in control--that is, critical processes are repeatable, sustainable, and consistently producing parts within the quality standards. DOD's proposed MRL criteria do not require that processes be in control until later. Acceptance of MRLs has grown among some industry and DOD components. Yet, DOD has been slow to adopt a policy that would require MRLs across DOD. Concerns raised by the military services have centered on when and how the MRL assessments would be used. While a joint DOD and industry group has sought to address concerns and disseminate information on benefits, a consensus has not been reached. If adopted, DOD will need to address gaps in workforce knowledge, given the decrease in the number of staff in the production and manufacturing career fields.