Specifically,  an estimate is  comprehensive when it accounts for all possible costs associated with  a program, is structured in sufficient detail to ensure that costs are  neither omitted nor double counted, and documents all cost- influencing assumptions; well-documented when supporting documentation explains the  process, sources, and methods used to create the estimate, contains  the underlying data used to develop the estimate, and is adequately  reviewed and approved by management; accurate when it is not overly conservative or optimistic, is based on  an assessment of the costs most likely to be incurred, and is regularly  updated so that it always reflects the current status of the program;  and credible when any limitations of the analysis because of uncertainty or  sensitivity surrounding data or assumptions are discussed, the  estimate’s results are cross-checked, and an independent cost  estimate is conducted by a group outside the acquiring organization to  determine whether other estimating methods produce similar results. While the eight agencies varied in the extent to which their cost- estimating policies and procedures addressed best practices, most did  not address several key components of an effective policy. Clear requirement for cost estimating: Six of the eight agencies fully  addressed this policy component by establishing a clear requirement  for all programs to perform life-cycle cost estimates, and in certain  cases specified more stringent requirements for programs designated  as major investments. Agencies that do not establish a central and  independent cost-estimating team may lack the ability to improve the  implementation of cost-estimating policies, support cost-estimating  training, and validate the reliability of program cost estimates at the  department or agency level. Without establishing a standard  structure for defining work products, agencies will not be positioned to  ensure that they can effectively compare programs and collect and  share data among programs. Until the selected agencies address the identified weaknesses in their  cost-estimating policies, it will be difficult for them to make effective use of  program cost estimates for informed decision making, realistic budget  formation, and meaningful progress measurement. Selected Agencies’ Program Cost Estimates Do Not Provide a Fully Reliable Basis for Program and Budget Decisions

A reliable cost estimate is critical to the success of any government  acquisition program, as it provides the basis for informed investment  decision making, realistic budget formulation and program resourcing,  and meaningful progress measurement. These practices can be organized into four characteristics— comprehensive, well-documented, accurate, and credible. While all 16 major acquisition programs we reviewed had developed cost  estimates and were using them to inform decision making, all but one of  the estimates were not fully reliable and did not provide a sound basis for  informed program and budget decisions. However, nearly all of these programs had estimates that did not fully  reflect important cost-estimating practices. Inadequate Implementation Was Largely Due to Weaknesses in Policy

The lack of reliable cost estimates across the investments exists in part  because of the weaknesses previously identified in the eight agencies’  cost-estimating policies. Recommendations for Executive Action

To address weaknesses identified in agencies’ policies and practices for  cost estimating, we are making the following recommendations:  We recommend that the Secretaries of Agriculture, Commerce,  Homeland Security, Labor, and Veterans Affairs, the Attorney General,  and the Administrator of the Environmental Protection Agency direct  responsible officials to modify policies governing cost estimating to  ensure that they address the weaknesses that we identified. These  agencies generally agreed with our results and recommendations,  although EPA disagreed with our assessment of the cost-estimating  practices used for one of its programs. Appendix I: Objectives, Scope, and Methodology

Our objectives were to (1) assess the extent to which selected  departments and agencies have appropriately implemented cost- estimating policies and procedures, and (2) evaluate whether selected  information technology (IT) investments at these departments and  agencies have reliable cost estimates to support budget and program  decisions. For this review, we assessed eight federal agencies and 16  investments. These agencies were the Departments of Agriculture, Commerce,  Defense, Homeland Security, Justice, Labor, and Veterans Affairs, and  the Environmental Protection Agency. According to  program documentation, these increases were due, in part, to  incorporating costs for the operation and maintenance of the system.
Why GAO Did This Study

The federal government plans to spend at least $75 billion on information technology (IT) investments in fiscal year 2012. The size of this investment highlights the importance of reliably estimating the costs of IT acquisitions. A reliable cost estimate is critical to the success of any IT program, providing the basis for informed decision making and realistic budget formation. Without the ability to generate such estimates, programs risk missing their cost, schedule, and performance targets.
GAO was asked to (1) assess selected federal agencies implementation of cost-estimating policies and procedures, and (2) evaluate whether selected IT investments at these agencies have reliable cost estimates to support budget and program decisions. To do so, GAO compared policies and procedures to best practices at eight agencies. GAO also reviewed documentation supporting cost estimates for 16 major investments at these eight agenciesrepresenting about $51.5 billion of the planned IT spending for fiscal year 2012.

What GAO Found

While the eight agencies GAO reviewedthe Departments of Agriculture, Commerce, Defense, Homeland Security, Justice, Labor, and Veterans Affairs, and the Environmental Protection Agencyvaried in the extent to which their cost-estimating policies and procedures addressed best practices, most had significant weaknesses. For example, six of the eight agencies had established a clear requirement for programs to develop life-cycle cost estimates. However, most of the eight agencies policies lacked requirements for cost-estimating training, a standard structure for defining work products, and a central, independent cost-estimating team, among other things. The weaknesses in agencies policies were due, in part, to the lack of a priority for establishing or enhancing department or agency-level cost-estimating functions. Until agencies address weaknesses in their policies, it will be difficult for them to make effective use of program cost estimates for informed decision making, realistic budget formation, and meaningful progress measurement.
The 16 major acquisition programs had developed cost estimates and were using them, in part, to support program and budget decisions. However, all but 1 of the estimates were not fully reliablemeaning that they did not fully reflect all four characteristics of a reliable cost estimate identified in the GAO cost-estimating guide: comprehensive, well-documented, accurate, and credible. For example, the estimates for many of these investments did not include all life-cycle costs, such as costs for operating and maintaining the system; did not adequately document the source data and methodologies used to develop the estimate; were not regularly updated so that they accurately reflected current status; and lacked credibility because they were not properly adjusted to account for risks and uncertainty. The inadequate implementation of cost-estimating best practices was largely due to weaknesses in agencies policies. Until cost-estimating best practices are fully implemented, these programs face an increased risk that managers will not be able to effectively use their cost estimates as a sound basis for informed program and budget decision making.

What GAO Recommends

GAO is recommending that the selected agencies modify cost-estimating policies to be consistent with best practices and update future cost estimates of the selected acquisition programs to address identified weaknesses. The seven agencies that commented on a draft of this report generally agreed with GAOs results and recommendations, although the Environmental Protection Agency disagreed with the assessment of one of its investments. However, GAO stands by its assessment.