DOE’s Requirements and Guidance for Analysis of Alternatives

DOE’s mandatory requirements and optional guidance for identifying,  analyzing, and selecting alternatives when conducting AOAs also apply to  NNSA. DOE’s guides suggest summarizing a planned approach to  conduct an analysis of alternatives; considering at least three  viable alternatives for analysis, including one that represents the  status quo; developing cost estimates that are explicit ranges  instead of point estimates; including life-cycle cost estimates of  the alternatives being considered; quantifying the benefits of  alternatives over their life cycle; adjusting life-cycle cost and  benefit estimates for risk; considering various selection criteria for  the alternatives; weighting the selection criteria for relative  importance; and comparing alternatives using net present value.The guidance suggests that, at the end of the analysis, the AOA  team present the recommended alternative based on the  preceding analysis in an integrated form, summarizing why an  alternative is preferred and supporting the recommendation of the  preferred alternative with facts from the analysis. NNSA program offices. We identified these best practices by (1) compiling  and reviewing commonly mentioned AOA policies and guidance used by  different government and private-sector entities and (2) incorporating  experts’ comments on our draft set of practices to develop a final set of  practices. DOE’s AOA Requirements and Guidance Do Not Conform to Best Practices

Neither DOE’s AOA requirements nor its guidance conform to best  practices and, therefore, DOE does not have assurance that applying  these requirements and guidance may lead to reliable AOAs. DOE’s requirements do  not fully or substantially meet best practices in any of the four AOA  categories, and they conform to only 1 of the 24 best practices we  identified—the practice of having the customer define functional  requirements based on the mission need. DOE’s requirements combined with guidance  conform to 9 of 24 best practices. Selecting a preferred alternative. DOE and NNSA officials told us that unreliable AOAs are a risk factor for  major cost increases and schedule delays for NNSA projects. As we previously  reported, federal standards for internal control related to risk assessment  call for agency management to assess the risks faced entity-wide and at  the activity level, and that once risks have been identified, management  should decide what actions should be taken to mitigate them.developing a reliable AOA process, DOE may not be successful in  mitigating the risk it has identified related to this process. NNSA’s AOAs for Three Recent Projects Did Not Conform to Best Practices and Did Not Consistently Follow Certain DOE Guidance

For three recent projects, our analysis found that NNSA did not conform  to most of the best practices in conducting the AOAs,concerns about the reliability of these AOAs. NNSA conformed to 11 of 24 best practices when  conducting this AOA. As noted above, DOE has one requirement that conforms to best  practices, the best practice of defining functional requirements based on  the mission need. NNSA followed this requirement for all three AOAs we  reviewed. DOE and  NNSA officials acknowledge that unreliable AOAs are a risk factor for  major cost increases and schedule delays for NNSA projects. Developing reliable AOAs is  particularly important at this time because NNSA has recently reassessed  or is in the process of reassessing alternatives for all its major  construction projects, after spending billions of dollars on designing and  partially constructing these projects. Without a process to develop reliable  AOAs, NNSA may continue on this path and continue to have limited  assurance that it is selecting alternatives that best meet its mission needs  and will not result in major cost increases and schedule delays in the  future. Appendix I: Scope and Methodology

To identify and describe best practices for the analysis of alternatives  (AOA) process, we first searched for a source of generally accepted best  practices that we could use as criteria for comparison with the  Department of Energy’s (DOE) AOA process, but we could not identify a  single source that was broadly recognized by government and private- sector entities and that could serve as the definitive source of best  practices for the AOA process. We also sent  a request to subject-matter experts, including experts from DOE, the  National Nuclear Security Administration (NNSA), and various other  government or private-sector entities, to help us identify further relevant  information. the Association for the  To determine the extent to which DOE’s requirements and guidance for  conducting an AOA conform to AOA best practices, we first identified the  process NNSA is required to follow when conducting an AOA. In addition, as part of this  objective, we determined the extent to which NNSA conformed to certain  DOE requirements and guidance—those DOE requirements and  guidance that conformed to best practices—in conducting the AOAs for  these projects. These practices can be applied to a wide range of activities  in which an alternative must be selected from a set of possible options,  and to a broad range of capability areas, projects, and programs.
Why GAO Did This Study

NNSA manages numerous programs that require the design and construction of one-of-a-kind facilities. NNSA's selection of preferred alternatives for these projects is governed by DOE requirements and guidance related to conducting an AOA. In recent years, NNSA has incurred substantial cost increases and schedule delays for such projects.
GAO was mandated to review the AOA process applied by NNSA. This report (1) identifies and describes AOA best practices, (2) determines the extent to which DOE requirements and guidance for conducting an AOA conform to AOA best practices, and (3) determines the extent to which NNSA conformed to best practices and followed certain DOE requirements and guidance in conducting the AOAs for recent NNSA projects.
To do this work, GAO examined relevant AOA guidance from the public and private sectors and DOE's AOA requirements and guidance, sought input from AOA experts, and interviewed agency officials. GAO reviewed three AOAs that NNSA completed or was scheduled to complete between November 2010 and September 2014.

What GAO Found

GAO identified 24 best practices for analysis of alternatives (AOA)—a process that is a key first step in capital asset acquisition. The process entails identifying, analyzing, and selecting a preferred alternative to best meet the mission need by comparing the operational effectiveness, costs, and risks of potential alternatives. Because no single set of best practices for AOAs was broadly recognized by government and private-sector entities, GAO developed a set of practices by reviewing AOA policies and guidance used by seven public and private-sector entities with experience in the AOA process, and verified these practices with subject matter experts. These best practices include, among other things, defining functional requirements based on mission need, conducting the AOA without a predetermined solution, including the status-quo alternative, and conducting an independent review of the entire AOA process. These practices can be applied to a wide range of activities, projects, and programs.
The Department of Energy's (DOE) requirements and guidance—found in the agency's orders and associated guides—for conducting an AOA do not conform to the 24 best practices GAO identified. Therefore, DOE does not have assurance that applying these requirements and guidance may lead to reliable AOAs. GAO's review of DOE's requirements for AOAs found that they conform to only 1 of the 24 best practices: the practice of defining functional requirements based on mission need. GAO's review of DOE's requirements combined with associated guidance—which includes nonmandatory approaches for meeting requirements—found that they conform to 9 of the 24 best practices. For example, DOE's guidance suggests identifying and considering at least three viable alternatives, including the status quo—a best practice that is not included in the requirements. Federal standards for internal control related to risk assessment call for agency management to decide on actions to mitigate identified risks. Without developing a reliable AOA process, DOE may not be successful in mitigating the risk it has identified related to this process.
DOE's requirements and guidance for AOAs also apply to the National Nuclear Security Administration (NNSA), a separately organized agency in DOE. For three recent NNSA projects that GAO reviewed, NNSA did not conform to most of the best practices GAO identified for conducting AOAs, therefore, raising concerns about the reliability of these AOAs. Specifically, for the three projects' AOAs, NNSA conformed to 6, 8, and 11 of the 24 identified best practices, respectively. For all three projects, NNSA consistently followed the one DOE requirement that conformed to a best practice—to define functional requirements based on mission need. DOE and NNSA officials acknowledge that unreliable AOAs are a risk factor for major cost increases and schedule delays for NNSA projects. As GAO has previously reported, NNSA has spent billions of dollars designing and partially constructing projects with an estimated cost of $750 million or more, only to later reassess alternatives. Without a process to develop reliable AOAs, NNSA may continue on this path and continue to have limited assurance that it is selecting alternatives that best meet its mission needs and will not result in major cost increases and schedule delays in the future.

What GAO Recommends

GAO recommends that DOE incorporate best practices into its AOA requirements. DOE agreed with this recommendation.