Computer Models Provide Critical Information to EM’s Environmental Cleanup Decision- Making Process

EM uses computer models to provide critical information for its decision- making process. Following these actions, EM used  computer models to simulate the movement of the remaining  contamination through the soil and groundwater over the next 1,000 years. EM Uses Computer Models to Assess the Performance of Selected Cleanup Activities

After a particular cleanup alternative is selected, EM also uses computer  modeling to demonstrate that the cleanup activity will result in reduced  future contamination levels that meet regulatory requirements. Under the order, this performance assessment is to document that the  disposal facility is designed, constructed, and operated in a manner that  protects workers, the public, and the environment. For example, at Hanford, a computer model known as the Hanford Tank  Waste Operations Simulator is designed to track the retrieval and  treatment of over 55 million gallons of radioactive waste held in  underground storage tanks. EM Has General Quality Policies for Its Computer Models, but It Has Not Regularly Assessed Contractors’ Implementation of Quality Assurance Procedures

Although EM uses general departmental quality assurance policies and  standards that apply to computer models and relies on contractors to  implement specific procedures that reflect these policies and standards,  these policies and standards do not specifically provide guidance on  ensuring the quality of the computer models used in cleanup decisions. Moreover, EM officials have not regularly performed periodic quality  assurance assessments, as required by DOE policy to oversee contractors’  development and use of cleanup models and the models’ associated  software. For example, the state of Washington has cited flaws in a model  EM uses to analyze soil and groundwater contamination and has told EM  that it will no longer accept the use of this model for chemical exposure  analysis at Hanford. In our review of eight cleanup  decisions at Hanford and SRS, we found EM had conducted only three  quality assurance assessments that addressed quality standards for the  models used in those decisions. EM Does Not Have an Overall Strategy and Guidance for Managing Its Cleanup Models

Although EM has recently begun some efforts to promote consistency in  the use of models across its various sites, these efforts are still in early  stages and, to date, some have had limited involvement of modeling  officials at the sites and federal, state, and local stakeholders who are  affected by decisions made using the output of computer models. For example, EM, in fiscal year 2010, began developing a  set of state-of-the-art computer models to support soil and groundwater  cleanup across the nuclear weapons complex. Similar comprehensive, coordinated efforts are lacking within  EM and, as a result, EM may be losing opportunities to improve the quality  of its models, reduce duplication, keep abreast of emerging computer  modeling and cleanup technologies, and share lessons learned across EM’s  sites. Although the importance of comprehensive guidelines for managing  computer models is well established, according to its officials, EM does  not have such overarching guidance. For a number of years, other  federal agencies and offices within DOE have recognized the importance  of a comprehensive guidance for managing computer models. Recommendations for Executive Action

To help EM increase confidence in the quality of information provided to  the public and its stakeholders resulting from the use of computer  modeling, we recommend the Secretary of Energy take the following three  actions:    Clarify specific quality assurance requirements for computer models used  in environmental cleanup decisions, including to analyze the potential  effectiveness of cleanup alternatives, assess the performance of selected  cleanup activities, and assist in planning and budgeting cleanup activities. Ensure that the models are assessed for compliance with these  requirements. Appendix I: Scope and Methodology

To determine how the Department of Energy’s (DOE) Office of  Environmental Management (EM) uses computer modeling in cleanup  decisions, we focused on cleanup decisions EM has made at its Hanford  Site in Washington state and Savannah River Site (SRS) in South Carolina  because together these two sites account for more than one-half of EM’s  annual cleanup spending and approximately 60 percent of the total  estimated cost of approximately $275 billion to $329 billion to clean up the  entire nuclear weapons complex. To address EM’s overall strategy for managing computer models that are  used in cleanup decisions, we interviewed DOE officials from  headquarters and from each site.
Why GAO Did This Study

The Department of Energy's (DOE) Office of Environmental Management (EM) is responsible for one of the world's largest cleanup programs: treatment and disposal of radioactive and hazardous waste created as a by-product of nuclear weapons production and energy research at sites across the country, such as EM's Hanford Site in Washington State and the Savannah River Site (SRS) in South Carolina. Computer models--which represent physical and biogeochemical processes as mathematical formulas--are one tool EM uses in the cleanups. GAO was asked to (1) describe how EM uses computer models in cleanup decisions; (2) evaluate how EM ensures the quality of its computer models; and (3) assess EM's overall strategy for managing its computer models. GAO analyzed the use of selected models in decisions at Hanford and SRS, reviewed numerous quality assurance documents, and interviewed DOE officials as well as contractors and regulators.

What GAO Found

EM uses computer models to support key cleanup decisions. Because the results of these decisions can cost billions of dollars to implement and take decades to complete, it is crucial that the models are of the highest quality. Computer models provide critical information to EM's cleanup decision- making process, specifically to: (1) Analyze the potential effectiveness of cleanup alternatives. For example, computer models at SRS simulate the movement of contaminants through soil and groundwater and provide information used to predict the effectiveness of various cleanup strategies in reducing radioactive and hazardous material contamination. (2) Assess the likely performance of selected cleanup activities. After a particular cleanup strategy is selected, EM uses computer modeling to demonstrate that the selected strategy will be designed, constructed, and operated in a manner that protects workers, the public, and the environment. (3) Assist in planning and budgeting cleanups. EM also uses computer models to support lifecycle planning, scheduling, and budgeting for its cleanup activities. For example, a Hanford computer model simulates the retrieval and treatment of radioactive waste held in underground tanks and provides information used to project costs and schedules. EM uses general departmental policies and industry standards for ensuring quality, but they are not specific to computer models used in cleanup decisions. EM has not regularly performed periodic quality assurance assessments, as required by DOE policy, to oversee contractors' development and use of cleanup models and the models' associated software. In our review of eight cleanup decisions at Hanford and SRS that used computer modeling as a critical source of information, GAO found EM conducted required assessments of the quality of computer models in only three cases. In addition, citing flaws in a model EM uses to analyze soil and groundwater contamination, regulators from Washington state have told EM that it will no longer accept the use of this model for chemical exposure analysis at Hanford. EM does not have an overall strategy for managing its computer models. EM has recently begun some efforts to promote consistency in the use of models. For example, it is developing a set of state-of-the-art computer models to support soil and groundwater cleanup decisions across its sites. However, these efforts are still in early stages and are not part of a comprehensive, coordinated effort. Furthermore, although other federal agencies and DOE offices have recognized the importance of comprehensive guidance on the appropriate procedures for managing computer models, EM does not have such overarching guidance. As a result, EM may miss opportunities to improve the quality of computer models, reduce duplication between DOE sites, and share lessons learned across the nuclear weapons complex.

What GAO Recommends

GAO recommends that DOE (1) clarify specific quality assurance requirements for computer models used in environmental cleanup decision making; (2) ensure that the models are assessed for compliance with these requirements; and (3) develop a comprehensive strategy and guidance for managing its models. DOE agreed with our recommendations.