EPA Applied, at Least in Part, about Two-thirds of the Recommendations to Its Particulate Matter Health Benefit Analysis in the Proposed Rule

The January 2006 regulatory impact analysis on particulate matter  represents a snapshot of an ongoing EPA effort to respond to the National  Academies’ recommendations on developing estimates of health benefits  for air pollution regulations. Specifically, the agency applied, at least in  part, approximately two-thirds of the recommendations—8 were applied  and 14 were partially applied—by taking steps toward conducting a more  rigorous assessment of uncertainty for proposed air pollution regulations  by, for example, evaluating the different assumptions about the link  between human exposure to particulate matter and health effects and  discussing sources of uncertainty not included in the benefit estimates. In response, EPA devoted significant resources to applying an  alternative technique called expert elicitation in a multiphased pilot  project. In  addition to the uncertainty underlying the causal link between exposure  and premature death that EPA analyzed, other key uncertainties can  influence the estimates. EPA Plans to Address Some of the Remaining Recommendations in the Final Rule and Has Research and Development Under Way to Address Others

EPA did not apply the remaining 12 recommendations to the analysis for  various reasons. EPA  officials viewed most of these recommendations as relevant to its health  benefit analyses and, citing the need for additional research and  development, emphasized the agency’s commitment to continue to respond  to the recommendations. According to a senior EPA official, insufficient  resources impeded the agency’s progress in applying the  recommendations. This official cited limited availability of skilled staff,  time, and other resources to conduct the required analyses and research  and development. EPA did not believe the state of scientific knowledge on relative toxicity  was sufficiently developed at the time it prepared the draft regulatory impact analysis to include this kind of analysis. However, EPA is  sponsoring research on this issue. The officials also said  that EPA intends to include in the executive summary of the regulatory  impact analysis supporting the final rule a summary table that describes  key analytical information. EPA has not directly addressed this recommendation. In particular, the  agency will need to ensure that it allocates resources to needed research on  emerging issues, such as the relative toxicity of particulate matter  components; assessing which sources of uncertainty have the greatest  influence on benefit estimates; and estimating other benefits, such as  environmental improvements. The agency’s draft regulatory impact  analysis illustrates that estimates of health benefits can be highly  uncertain. In fact, EPA officials viewed these estimates as so uncertain that  they chose to not present them in the executive summary of the regulatory  impact analysis. While EPA officials said they expect to reduce the  uncertainties associated with the health benefit estimates in the final  particulate matter analysis, robust uncertainty analysis will nonetheless be  important for decision makers and the public to understand the likelihood  of attaining the estimated health benefits. To the extent EPA continues to make progress  addressing the Academies’ recommendations, decision makers and the  public will be able to better evaluate the basis for EPA’s air regulations. Officials from  the Office of Policy Analysis and Review within EPA’s Office of Air and  Radiation noted in their technical comments that the report provides a fair  and balanced representation of EPA’s efforts to apply the National  Academies’ recommendations to the draft particulate matter regulatory  impact analysis. We note  that this report does identify, as appropriate, EPA’s research and  development efforts for recommendations EPA did not apply to the draft  particulate matter analysis, its plans to apply some additional  recommendations to the final particulate matter regulatory impact  analysis, and the agency’s responses to recommendations in prior rule- making analyses of air programs. Scope and Methodology

We were asked to determine whether and how the Environmental  Protection Agency (EPA) applied the National Academies (Academies)  recommendations in its estimates of the health benefits expected from the  January 2006 proposed revisions to the particulate matter national ambient  air quality standards.
Why GAO Did This Study

A large body of scientific evidence links exposure to particulate matter--a widespread form of air pollution--to serious health problems, including asthma and premature death. Under the Clean Air Act, the Environmental Protection Agency (EPA) periodically reviews the appropriate air quality level at which to set national standards to protect the public against the health effects of particulate matter. EPA proposed revisions to these standards in January 2006 and issued a draft regulatory impact analysis of the revisions' expected costs and benefits. The estimated benefits of air pollution regulations have been controversial in the past. A 2002 National Academies report generally supported EPA's approach but made 34 recommendations to improve how EPA implements its approach. GAO was asked to determine whether and how EPA applied the Academies' recommendations in its estimates of the health benefits expected from the January 2006 proposed revisions to the particulate matter standards. GAO examined the draft analysis, met with EPA officials, and interviewed members of the National Academies' committee. In providing technical comments on the report, EPA officials said it was fair and balanced and noted the agency's progress in addressing recommendations via research and development and other analyses.

What GAO Found

EPA has begun to change the way it conducts and presents its analyses of health benefits in response to recommendations from the National Academies. Specifically, EPA applied, at least in part, 22--or about two-thirds--of the Academies' recommendations to its health benefit analysis of proposed revisions to particulate matter standards. For example, in response to some of the recommendations, EPA took steps toward conducting a more rigorous assessment of uncertainty by, for instance, evaluating how benefits could change under different assumptions and discussing sources of uncertainty not included in the benefit estimates. In one case, EPA applied an alternative technique, called expert elicitation, for evaluating uncertainty by systematically gathering expert opinion about the uncertainty underlying the causal link between exposure to particulate matter and premature death. Consistent with the National Academies' recommendation to assess uncertainty by developing ranges of estimates and specifying the likelihood of attaining them, EPA used expert elicitation to develop ranges of reductions in premature death expected from the proposed revisions. EPA officials said that ongoing research and development efforts will allow the agency to gradually achieve more progress in applying the recommendations. We note that robust uncertainty analysis is important because estimates of health benefits can be highly uncertain, as the draft regulatory impact analysis for particulate matter illustrates. EPA viewed the estimates in this analysis as so uncertain that it chose not to present them in the executive summary. For various reasons, EPA has not applied the remaining 12 recommendations to the analysis, such as the recommendation to evaluate the impact of using the simplifying assumption that each component of particulate matter is equally toxic. EPA officials viewed most of these recommendations as relevant to its health benefit analyses and, citing the need for additional research and development, emphasized the agency's commitment to continue to respond to the recommendations. For example, EPA did not believe that the state of scientific knowledge on the relative toxicity of particulate matter components was sufficiently developed to include in the January 2006 regulatory impact analysis, and the agency is currently sponsoring research on this issue. In addition, a senior EPA official said that insufficient resources impeded the agency's progress in applying the recommendations, citing, in particular, the limited availability of skilled staff, time, and other resources to conduct the required analyses and research and development. EPA officials also said that some of the recommendations the agency did not apply to the draft analysis, such as one calling for a summary table describing key analytical information to enhance transparency, will be applied to the analysis supporting the final rule. To the extent that EPA continues to make progress addressing the Academies' recommendations, decision makers and the public will be better able to evaluate the basis for EPA's air regulations.