DOE and NNSA Cost Estimating Requirements and Guidance for Programs

DOE and NNSA cost estimating requirements and guidance for programs  are primarily defined in four documents. DOE and NNSA Cost Estimating Requirements and Guidance Generally Do Not Reflect Best Practices for Developing Cost Estimates

DOE and NNSA cost estimating requirements and guidance for projects  and programs generally do not reflect best practices for developing cost  estimates. For projects, DOE requires one best practice, and DOE’s cost  estimating guide does not fully reflect cost estimating best practices, and  is not mandatory. DOE’s 2010 project management order requires the use of only 1 of the  12 cost estimating best practice steps. For example, the order  does not require any of the other 11 best practice steps such as  conducting a risk and uncertainty analysis, identifying ground rules and  assumptions, documenting the estimate, developing a point estimate, or  determining the estimating structure. Appendix I  provides a comparison of the extent to which DOE’s cost guide contains  information on the 12 best practice steps. However, because the cost guide was issued in 2011, a year  after DOE’s latest version of the order was issued, the order contains no  reference to this guide that a preparer of a cost estimate could use in  developing an estimate, or that a reviewer could use in reviewing one,  and users of the order may not be aware of the guide’s availability and  may not benefit from its usefulness. In the absence of a DOE or NNSA requirement for programs to follow  cost estimating best practices, we found that managers for the programs  we reviewed—the B61 LEP and the Plutonium Disposition Program— used different processes in developing cost estimates for these programs. In February 2014, we found that  NNSA’s life-cycle cost estimate for the Plutonium Disposition Program  did not follow all key steps for developing high-quality cost estimates  in part because it did not have a requirement to develop a life-cycle  cost estimate. Project Reviews Indicate Cost Estimating Weaknesses and Program Reviews Are Not Required and Therefore the Extent of Weaknesses Is Largely Unknown

NNSA’s project and program reviews for fiscal years 2009 to 2014  identified cost estimating weaknesses that can be attributed to not  following best practices. DOE and NNSA require reviews of projects,  including reviews of cost estimates at various CD points and at the  discretion of project managers; however, because DOE and NNSA do not  require reviews of program cost estimates, the extent of weaknesses in  program cost estimates is largely unknown. Project Reviews Identified Cost Estimating Weaknesses That Can Be Attributed to Not Following Best Practices

Of the 50 NNSA project reviews conducted from February 2009 through  February 2014, 39 identified a total of 113 cost estimating weaknesses. We determined that 71 of the 113 weaknesses—or about 63 percent— can be attributed to not following 4 of the 12 best practice steps. These identified several weaknesses in the cost  estimates for this program that can be attributed to not following three  best practice steps: (1) obtaining data, (2) defining program  characteristics, and (3) determining the estimating structure. However,  we also found that NNSA did not follow other key steps such as  conducting an independent cost estimate and, as a result, the estimate  was not reliable. Without a requirement for conducting program reviews, NNSA does not  have the appropriate internal controls necessary for assessing program  performance. Because DOE does not require the use of most of  the 12 cost estimating best practices for its projects and programs, it is  unlikely that NNSA and its contractors will consistently develop reliable  cost estimates. Finally, without a requirement for conducting reviews  of programs with project-like characteristics, including the life-cycle cost  estimates of these programs, neither DOE nor NNSA have appropriate  internal controls to assess the quality of program performance over time. In regard to the report’s fourth recommendation—revise DOE directives  that apply to programs to require that DOE and NNSA and its contractors  develop cost estimates in accordance with the 12 cost estimating best  practices, including developing lifecycle cost estimates for programs— DOE stated in its written comments that it is in the process of  substantially revising the existing 1995 DOE Order 130.1, Budget  Formulation, and that as part of this effort, DOE will assess the  requirement for program cost estimates and will revise the order to  provide more specificity on the cost estimating requirements.
Why GAO Did This Study

NNSA is responsible for the nation's nuclear security programs. These programs often include the design and construction of large projects to meet program needs. NNSA has a history of struggling to complete these and other projects and programs within cost estimates.
Senate Report 112-73 mandated that GAO report on NNSA's cost estimating practices. This report examines: (1) the extent to which DOE and NNSA cost estimating requirements and guidance for projects and programs reflect best practices and (2) the extent to which recent NNSA project and program reviews identified cost estimating weaknesses, and the extent to which the weaknesses can be attributed to not following best practices. GAO reviewed DOE and NNSA cost estimating practices and compared them to best practices, NNSA project and program reviews, and two programs selected among the largest and most complex NNSA programs. GAO also interviewed DOE and NNSA officials about requirements and guidance for cost estimates.

What GAO Found

The Department of Energy's (DOE) and its National Nuclear Security Administration's (NNSA) cost estimating requirements and guidance for projects and programs do not fully reflect best practices for developing cost estimates. In regard to cost estimating requirements for projects, DOE's 2010 project management order requires 1 of the 12 cost estimating best practices—conducting an independent cost estimate—for larger projects at certain stages of development. In contrast, DOE's 2011 cost estimating guide describes recommended approaches for using 10 of 12 best practices and partially contains information about the other 2. Furthermore, because DOE's cost estimating guide was issued in 2011—after DOE's 2010 project order was issued—it is not referenced in the order. As a result, users of the order may not be aware of the guide's availability and may not benefit from its usefulness. In addition, although NNSA programs are required to follow DOE's budget formulation order and NNSA's budget process, both of which require the development of cost estimates, neither the order nor the process requires the use of best practices in developing the estimates. In February 2014, for example, GAO found that NNSA's lifecycle cost estimate for the Plutonium Disposition Program did not follow all key steps for developing high-quality cost estimates, in part because the agency did not have a requirement to develop a life-cycle cost estimate. In the absence of a requirement for using best practices, it is unlikely that DOE, NNSA, and their contractors will consistently develop reliable cost estimates.
NNSA's project and program reviews issued during fiscal year 2009 through March 2014 identified cost estimating weaknesses that can be attributed to not following best practices. DOE and NNSA require independent project reviews, including reviews of cost estimates at certain stages of development and at the discretion of project managers. Of the 50 reviews GAO analyzed, 39 identified a total of 113 cost estimating weaknesses. GAO determined that 71 of the 113 weaknesses—or about 63 percent—can be attributed to not following four best practices: (1) determining the estimating structure, (2) identifying ground rules and assumptions, (3) conducting risk and uncertainty analysis, and (4) documenting the estimate. Neither DOE nor NNSA, however, requires reviews of program cost estimates. Of the three program reviews conducted during fiscal years 2009 to 2013, two were of the B61 Life Extension Program, which is to extend the operational life of this nuclear weapon. Both reviews identified weaknesses in the cost estimates that can be attributed to not following three best practices: (1) determining the estimating structure, (2) defining program characteristics, and (3) obtaining data. In addition, a February 2014 GAO report on NNSA's program to dispose of weapons-grade plutonium found that NNSA did not follow several cost estimating best practices, such as conducting an independent cost estimate and as a result, the program cost estimate was not reliable. While the program reviews and GAO's February 2014 report indicate weaknesses in a few program cost estimates, the extent of program cost estimate weaknesses is largely unknown because neither DOE nor NNSA requires reviews of program cost estimates. Without a requirement for conducting independent program reviews, NNSA does not have the internal control necessary for assessing program performance over time.

What GAO Recommends

GAO recommends that DOE, among other things, revise DOE requirements and guidance that apply to projects and programs to ensure: (1) the guidance is referenced in the project management order, (2) DOE and NNSA and its contractors develop cost estimates in accordance with the 12 best practices, (3) its cost estimating guide fully reflects best practices, and (4) independent reviews of programs are conducted periodically. DOE agreed with the report and its recommendations.