Respondents Reported Basic Information Needed to Assess Value Available for About Half of Selected Interventions

About half of the respondents to our questionnaire reported basic  information on the effect of their intervention on both quality of care  and costs—the two types of data needed to determine whether or  to what extent a particular intervention enhanced the value of  health care. Overall, the vast majority of our respondents reported  at least some information on the observed effect of their  intervention on quality of care. Relatively fewer—though still over  half—of our respondents reported at least some information on the  effect of their intervention on costs. The ability of policymakers to identify interventions that  substantially improve quality and reduce costs depends on the  availability of basic information on the size of the effect of an  intervention on both quality of care and costs. The Strength of Evidence on the Effect of Interventions Can Be Assessed along Three Dimensions

Policymakers and others can assess the strength and limitations of  available evidence from studies on the effect of health care  interventions on quality of care and costs along three dimensions. One, the credibility of evidence on the effect of health care  interventions on quality of care and costs depends primarily on  whether those studies apply rigorous study designs. Two, the  applicability of the results of studies to a broader population  depends on the extent to which the study population is  representative of that larger population. Finally, the capacity of  health care interventions for widespread replication can be  examined in terms of the consistency of results obtained by each  intervention across diverse organizations. The methodological experts we consulted uniformly  emphasized the primacy of study design in determining the  credibility of evidence on the effect of health care interventions on  quality of care and costs. However, for a  substantially larger number of the 127 interventions, the studies we  identified employed the types of study designs that do not isolate  the effect of the intervention from other factors. Leadership Support and Other Factors Reported As Important for Both Implementation and Replication of Interventions

Respondents to our questionnaire reported, generally by large  margins, that leadership support as well as other factors, such as  organizational culture and staff resources, significantly facilitated  implementation. However, respondents were more divided when  asked about the reported effect that health IT had on  implementation, and most respondents reported that financial  incentives were not a factor in the implementation of their  intervention. A majority of respondents reported that each of these  factors, with the exception of financial incentives, was expected to  be either very or somewhat important if one were to attempt to  replicate their intervention as widely as possible. Concluding Observations

Our work suggests that progress in achieving greater value in  health care in the U.S. will depend, in part, on the availability of  information regarding the effect of different interventions on quality  of care and costs and on how policymakers and others assess and  use that information. Such information can guide the choices of  policymakers among multiple interventions vying for support, but  those decisions will have a sounder basis if the information meets  certain criteria regarding its content and strength of evidence. Agency Comments

We requested comments from the Department of Health and  Human Services, but none were provided. GAO staff who made  major contributions to this report are listed in appendix V.

Appendix I: Scope and Methodology

To examine the availability of information on the effect of selected  health care interventions on quality of care and costs as well as  factors that can facilitate the implementation and replication of  these interventions, we studied a diverse set of specific  interventions that seek to enhance the value of health care through  making changes in the way care is delivered.   The intervention targeted a population or problem that was  relevant to the U.S. health care system.   The intervention was implemented in at least one health care  setting.
Why GAO Did This Study

The U.S. has devoted an increasing proportion of its economy and federal budget to the provision of health care services, but high levels of spending do not guarantee good care. Policymakers, health practitioners, and others have implemented numerous health care interventions that make discrete changes in the organization of health care services in order to enhance the value of health care--that is, improve the quality of care while reducing costs. Examples include programs to reduce bloodstream infections and to coordinate patient care following hospital discharges. This report (1) examines the availability of evidence on the effect of selected interventions on quality of care and costs; (2) identifies key dimensions for assessing the strength of such evidence; and (3) examines factors that can facilitate the implementation and replication of health care interventions. GAO identified a broad and diverse set of health care interventions using published and unpublished sources. For 127 of those interventions, GAO analyzed responses to a questionnaire that it sent to persons knowledgeable about available information on the effect of that particular intervention on quality of care and costs. GAO's questionnaire also asked respondents to assess the relative importance of seven factors in the implementation and potential replication of the health care intervention. In addition, GAO consulted the methodological literature and experts on assessing evidence on the effects of health care interventions.

What GAO Found

About half of the respondents to our questionnaire reported some information on the effect of an intervention on both quality of care and costs--the two types of data needed to determine whether or to what extent a particular intervention enhanced the value of health care. Overall, the vast majority of our respondents reported at least some information on the observed effect of the intervention on quality of care. Relatively fewer--though still over half--of our respondents reported at least some information on the effect of the intervention on costs. Whether or not policymakers can rely on information that indicates an intervention enhances value depends on the strength of the underlying evidence about quality and cost effects. From studies on the effect of health care interventions on quality of care and costs, policymakers and others can assess the strength and limitations of available evidence along three dimensions. One, the credibility of evidence on the effect of health care interventions on quality of care and costs depends primarily on whether those studies apply rigorous study designs. Two, the applicability of the results of studies to a broader population depends on the extent to which the study population is representative of that larger population. Finally, the capacity of health care interventions for widespread replication can be examined in terms of the consistency of results obtained by each intervention across diverse health care organizational contexts. Respondents reported, generally by large margins, that leadership support as well as other factors, such as organizational culture and staff resources, significantly facilitated implementation. However, respondents were more divided when asked about the reported effect that health IT had on implementation, and most respondents reported that financial incentives were not a factor in the implementation of the intervention. A majority of respondents reported that each of these factors, with the exception of financial incentives, would be either very or somewhat important if one were to attempt to replicate the intervention as widely as possible. Progress in achieving greater value in the U.S. health care system will depend, in part, on the availability of information regarding the effect of interventions on quality of care and costs and on how policymakers and others assess and use that information. Information can guide the choices of policymakers among multiple interventions vying for support, but those decisions will have a sounder basis if the information meets certain criteria regarding its content and strength of evidence. At least some information on both cost and quality effects was available for about half of the interventions GAO examined. However, for many interventions the credibility of this information was put into question by widespread reliance on studies that did not incorporate rigorous designs that could isolate the effect of an intervention from other factors. We requested comments from the Department of Health and Human Services, but none were provided.