Knowledge   point 2 occurs when a developer determines that a product’s design is  stable—that is, it will meet customer requirements and cost and  schedule targets. NASA’s Revised Policy Does Not Fully Support a Knowledge- Based Approach to Acquisitions

NASA’s revised acquisition policy for developing flight and ground support  systems incorporates some of the elements of a knowledge-based  acquisition approach. NASA’s policy  defines a phased life cycle approach and requires a major decision review  to move from project formulation to implementation. NASA’s policy emphasizes many  elements needed at the NAR (or KP1) to match needs to resources, such as  validating requirements, developing realistic cost and schedule estimates  and human capital plans, and establishing a preliminary design. The policy  does not, however, require projects to demonstrate technologies at high  levels of maturity before launching a project. While NASA requires projects to develop plans that describe how  technologies will be matured and to provide alternative development  strategies for technologies that do not mature as expected, it does not  establish a minimum threshold for technology maturity. NASA’s policy also does not require a major decision review before  beginning manufacturing. Furthermore, lacking a major decision  review to ensure that projects have gained the appropriate levels of  knowledge at KP2 and KP3, NASA decision makers cannot be provided a  high level of certainty that the project will meet cost, schedule, and  performance requirements and have no assurance that the provisions of  the key management documents required by NPR 7120.5C are being  executed after the NAR. As a result, each  center reports a different level and type of knowledge about a project at  key decision points. These situations make it  difficult for NASA decision makers to evaluate center projects on a  common foundation of knowledge and make sound investment decisions  and tradeoffs based on those evaluations. A standardized, knowledge- based approach would prepare NASA to face competing budgetary  priorities and make difficult decisions regarding the investment in and  termination of projects. Some centers have developed  center-specific policies and criteria for implementing NASA’s project  management policies and system engineering guidance, while others have  not. Some NASA centers have also developed criteria in their  policies that are similar to the criteria used to ensure a knowledge-based  approach is followed; other centers lack such criteria. In addition to individual policy requirements, centers may rely on project  managers and systems engineers to employ good project management and  systems engineering practices. Reliance on project  managers to implement good practices, however, could be problematic  given the diminishing number of experienced project managers available  to lead projects and the decline of in-house systems engineering and  technical capabilities agencywide caused by increasing retirements and  outsourcing. NASA estimates that it will  cost approximately $104 billion over the next 13 years to accomplish these  initial goals. The complex  technical requirements associated with fulfilling the President’s Vision,  the fiscal constraints under which NASA will be required to operate, the  diminished number of experienced project managers and systems  engineers, and the potential for increased production make following a  knowledge-based approach for flight systems and ground support projects  all the more critical. Since NASA is  currently in the process of revising its program and project management  policies and developing an agencywide systems engineering policy, we  believe this presents a unique opportunity for the agency to correct some  of the problems identified during our review. Key contributors to this report are acknowledged in  appendix IV. Defense Acquisitions: Despite Restructuring, SBIRS High Program  Remains at Risk of Cost and Schedule Overruns. Defense Acquisitions: DOD Faces Challenges in Implementing Best  Practices.
Why GAO Did This Study

The National Aeronautics and Space Administration (NASA) plans to spend over $100 billion on capabilities and technologies to achieve the initial goals of the President's 2004 Vision for Space Exploration. In the past, NASA has had difficulty meeting cost, schedule, and performance objectives for some of its projects because it failed to adequately define project requirements and quantify resources. NASA will be further challenged by a constrained federal budget and a shrinking experienced NASA workforce. To help face these challenges and manage projects with greater efficiency and accountability, NASA recently updated its program and project management policy and is developing an agencywide systems engineering policy. GAO has issued a series of reports on the importance of obtaining critical information and knowledge at key junctures in major system acquisitions to help meet cost and schedule objectives. This report (1) evaluates whether NASA's policy supports a knowledge-based acquisition approach and (2) describes how NASA centers are implementing the agency's acquisition policies and guidance.

What GAO Found

While NASA's revised policy for developing flight systems and ground support projects incorporates some of the best practices used by successful developers, it lacks certain key criteria and major decision reviews that support a knowledge-based acquisition framework. For example, NASA's policy requires projects to conduct a major decision review before moving from formulation to implementation. Further, before moving from formulation to implementation, projects must validate requirements and develop realistic cost and schedule estimates, human capital plans, a preliminary design, and a technology plan--key elements for matching needs to resources. However, NASA's policies do not require projects to demonstrate technologies at high levels of maturity before program start. By not establishing a minimum threshold for technology maturity, NASA increases the risk that design changes will be required later in development, when such changes are typically more costly to make. In addition, although NASA's policy does require project managers to establish a continuum of technical and management reviews, it does not specify what these reviews should be, nor does it require major decision reviews at other key points in a product's development. Acquiring knowledge at key junctures will become increasingly important as NASA proceeds to implement elements of the Vision. Without a major decision review at key milestones to ensure that the appropriate level of knowledge has been achieved to proceed to the next phase, the risk of cost and schedule overruns, as well as performance shortfalls, increases. NASA centers have varying approaches for implementing the agency's policies and guidance. Some centers have established product development criteria that are similar to the criteria used in a knowledge-based acquisition, while other centers have not. As a result, each center reports a different level and type of knowledge about a project at key decision points. Centers also rely on project managers and systems engineers to employ good project management and systems engineering practices. However, given the loss of experienced project managers and the decline of in-house systems engineering and technical capabilities, that reliance could be problematic. These situations make it difficult for decision makers to evaluate projects on the same basis and make sound investment decisions and tradeoffs based on those evaluations. A standardized, knowledge-based approach would prepare NASA to face competing budgetary priorities and better position the agency to make difficult decisions regarding the investment in and termination of projects.