Since 1990, we have  identified NASA’s contract management as a high-risk area. This  assessment has been based in part on our repeated finding that NASA does  not have good cost-estimating processes or the financial information  needed to develop good cost estimates for its programs, making it difficult  for NASA to oversee its contracts and control costs. More than half of the programs’ development cost estimates  increased, and for some programs, this increase was significant. While 8 of the 27 programs experienced  slight decreases in their development cost estimates, 17 experienced  cost growth—as much as almost 94 percent. The development cost estimates for each of the 10 programs that we  reviewed in detail have been rebaselined—for some programs, as many as  four times—and for 7 of the 10 programs, the cost estimate increased each  time it was rebaselined (see fig. For the 10 programs we reviewed in detail, NASA cited specific reasons for  changes in the baseline development cost estimates and the recalculated  baselines—many of which were related to technical problems and  subsequent delays in the development or delivery of key system  components, and insufficient funding and reserves, as illustrated in the  following examples:  Technical problems in the MERs program required a significant redesign  of components and the development of a new landing system. Further, NASA lacks a process for  systematically identifying and assessing programs that are not achieving  their cost, schedule, and performance goals. Many of these criteria are included in NASA’s  cost-estimating guidance. Further, none of the programs fully met certain key  criteria. Failing to meet these criteria puts programs at certain risk. For example,  underestimating a program’s full life-cycle costs creates the risk that a  program could be underfunded and subject to major cost overruns, which  would ultimately result in the program being reduced in scope or additional  funding being requested and appropriated to ensure the program meets its  objectives. Without this knowledge, NASA may have little specific basis to  determine adequate financial reserves, schedule margins, and technical  performance margins to provide managers the flexibility needed to  address programmatic, technical, cost, and schedule risks, as required by  NASA policy. Utility of Cost-Estimating Tools Depends on the Reliability of NASA’s Financial and Performance Data

According to NASA officials, state-of-the-art cost-estimating tools have  been funded and implemented. For more than a decade, we  have reported that NASA has failed to develop a system to capture reliable  financial and performance information, posing significant challenges to  NASA’s ability to estimate and control program costs. In addition,  NASA implemented the system before it had the capability to capture the  full costs of its programs and projects. Efforts Under Way to Remove Some Barriers and Improve Cost Estimating

NASA has several initiatives under way to improve the agency’s  cost-estimating processes. Without a process that prevents programs  from proceeding before they have sufficiently demonstrated that key  cost-estimating criteria have been met, NASA programs will continue to  be at risk of cost and schedule overruns. Further, because much of NASA’s work is  performed through grants and contracts, NASA’s revised procedures will  emphasize how risk and technical complexity affect contractor  performance. To assess NASA’s cost-estimating processes and methodologies, we used  cost-estimating criteria developed by Carnegie Mellon University’s  Software Engineering Institute (SEI) designed to assess the reliability of  project cost and schedule estimates. Assessments of 10 Programs Reviewed in Detail

This appendix provides a program by program assessment of the 10 NASA  programs we reviewed in detail. Each assessment provides  a brief description of the program’s mission;  the status of the program—that is, whether it is in development,  operational, or terminated;  the year the program was initiated; the fiscal year in which the Congress approved the program—that is,  when full-scale design and development funds were appropriated;  a comparison of the initial and current (as of April 2003) baseline   an assessment of the program’s cost-estimating processes,  methodologies, and practices to determine the extent they met  the 14 cost-estimating criteria that we used to measure program  performance.
Why GAO Did This Study

For more than a decade, GAO has identified the National Aeronautics and Space Administration's (NASA) contract management as a high-risk area--in part because of NASA's inability to collect, maintain, and report the full cost of its programs and projects. Lacking this information, NASA has been challenged to manage its programs and control program costs. The scientific and technical expectations inherent in NASA's mission create even greater challenges--especially if meeting those expectations requires NASA to reallocate funding from existing programs to support proposed new efforts. Because cost growth has been a persistent problem in a number of NASA programs, GAO was asked to examine NASA's cost estimating for selected programs, assess NASA's cost-estimating processes and methodologies, and describe any barriers to improving NASA's cost-estimating processes. To conduct GAO's work, GAO analyzed a total of 27 NASA programs--10 of which GAO reviewed in detail.

What GAO Found

Considerable change in NASA's program cost estimates--both increases and decreases--indicates that NASA lacks a clear understanding of how much its programs will cost and how long they will take to achieve their objectives. For example, the development cost estimates for more than half of the 27 programs that GAO reviewed have increased and for some programs this increase was significant--as much as 94 percent. Cost estimates changed for each of 10 programs that GAO reviewed in detail. For 8 of the 10 programs, the estimates increased. Although NASA cited specific reasons for the changes, such as technical problems and funding shortages, the variability in the cost estimates indicates that the programs lacked the sufficient knowledge needed to establish priorities, quantify risks, and make informed investment decisions, and thus predict costs. Most notably, NASA's basic cost-estimating processes--an important tool for managing programs--lack the discipline needed to ensure that program estimates are reasonable. Specifically, GAO found that none of the 10 NASA programs that GAO reviewed in detail met all of GAO's cost-estimating criteria, which are based on criteria developed by Carnegie Mellon University's Software Engineering Institute. Moreover, none of the 10 programs fully met certain key criteria--including clearly defining the program's life cycle to establish program commitment and manage program costs, as required by NASA. In addition, only three programs provided a breakdown of the work to be performed. Without this knowledge, the programs' estimated costs could be understated and thereby subject to underfunding and cost overruns, putting programs at risk of being reduced in scope or requiring additional funding to meet their objectives. Finally, only two programs have a process in place for measuring cost and performance to identify risks. NASA has limited ability to collect the program cost and schedule data needed to meet basic cost-estimating criteria. For example, as GAO has previously reported, NASA does not have a system to capture reliable financial and performance data--key to using effectively the cost-estimating tools that NASA officials state that programs employ. Further, without adequate financial and nonfinancial data, programs cannot easily track an acquisition's progress and assess whether the program can meet its cost and schedule goals before it incurs significant cost and schedule overruns. NASA identified other barriers, including limited cost-estimating staff. According to NASA officials, several initiatives are under way to remove such obstacles and improve the agency's cost-estimating practices.