AHRQ Has Taken Some Steps to Disseminate Comparative Clinical Effectiveness Research, but Has Not Taken Other Actions to Help It Fully Address Its Dissemination Requirements

AHRQ has taken some steps to disseminate CER as required under  PPACA, including the creation of systematic reviews to develop CER  findings, tools to disseminate CER, plans for a website to list and provide  links to research databases that include CER, and plans for receiving  feedback from stakeholders to whom information is disseminated. For each systematic review AHRQ synthesizes CER findings from  existing research, and the agency disseminates these findings to various  targeted stakeholder groups. Tools to organize and disseminate CER. AHRQ’s marketing plans  include various informational tools to disseminate CER. According to GAO’s Standards for Internal Control in the Federal  Government, significant events need to be clearly documented to ensure  management goals are carried out.which together describe the key activities of its dissemination process,  including the steps the agency takes to identify key CER findings from  systematic reviews, draft and finalize its marketing plans, and distribute  its informational tools to the public. As of  November 2014, AHRQ officials also have not developed and  documented a specific implementation plan to create a publicly available  database for CER. Additionally, AHRQ officials told  us they have not determined how to address potential limitations with this  new approach. ASPE Has Coordinated and Funded Projects to Build Data Capacity for Research, but Its Approach Lacks Key Elements Needed to Ensure Its Effectiveness

ASPE has coordinated among various agencies to fund projects intended  to build data capacity for CER. However, its approach to building data  capacity for CER lacks key elements, such as defined objectives,  milestones, and time frames, that are necessary to ensure effectiveness. Beginning in fiscal year 2013, ASPE officials worked with the Office of the  National Coordinator for Health Information Technology (ONC) to develop  a strategic road map to guide both the identification and selection of  ASPE’s PCORTF projects beginning in fiscal year 2014 through fiscal  year 2019. Standard  practices for project management call for agencies to conceptualize,  define, and document specific goals and objectives in the planning  process, along with the appropriate steps, milestones, time frames, and  resources needed to achieve those results. Specifically, ASPE identified several guiding principles, such as ensuring  that data infrastructure projects are “non-duplicative of other related  federal and non-federal investments” and “achieve synergy with PCORI  and AHRQ.” It also included priority objectives, such as further enabling  the collection of standardized clinical data, but many of the objectives  were broad and not clearly defined—and did not specify milestones or  time frames—as would be consistent with effective project management. Although ASPE identified and considered related, ongoing federal and  non-federal data infrastructure investments in an attempt to identify needs  or gaps, opportunities where contributions could be made, and ways to  avoid duplication, its strategic road map was unclear on the timing and  level of coordination necessary for its investments to work together with  existing projects—such as PCORI’s PCORnet initiative—to improve data  capacity. While HHS has a strategic  road map with information on projects that it is funding to build the  capacity for CER data, the road map does not include key elements, such  as clearly defined objectives, milestones, and time frames needed to  assess the agency’s progress toward the goal of building data capacity  for CER, as would be consistent with practices for effective project  management. Recommendations for Executive Action

To help ensure that HHS fully addresses its dissemination requirements  under PPACA, we recommend that the Secretary of Health and Human  Services direct AHRQ to take the following four actions:  1.  identify and document time frames for the implementation and  distribution of marketing plans and informational tools;  2.  expand dissemination efforts to federal and private health plans and  vendors of health information technology focused on clinical decision  support;  3.  document and complete plans to develop a publicly available  database, including plans to meet the needs of various potential users  in the general public; and  4.  develop specific plans on how it will collaborate with NIH on its  dissemination activities. In addition, to ensure that HHS fully addresses the PPACA requirements  to build data capacity for CER, the Secretary should direct ASPE to  include clearly defined objectives, milestones, and time frames, or other  indicators of performance, in its strategic road map that is used to identify  its PCORTF projects. As noted in our  findings, these plans include creating a web page to list and provide  users with links to existing publicly available databases that could be  used to search for these studies.
Why GAO Did This Study

PPACA imposed new requirements on HHS related to CER—research that evaluates and compares health outcomes and the clinical effectiveness, risks, and benefits of two or more medical treatments or services. Among other things, PPACA required AHRQ to broadly disseminate findings from federally funded CER and the Secretary of HHS (who, by delegation, charged ASPE) to coordinate federal programs to build data capacity for CER. PPACA also mandated that GAO review HHS's CER activities. This report examines (1) AHRQ's activities to disseminate the results of federally funded CER and (2) ASPE's activities to coordinate federal programs to support CER by building the capacity to collect, link, and analyze data, among other objectives. GAO reviewed relevant legal requirements and HHS documentation; interviewed HHS officials; and obtained information from five stakeholder groups that AHRQ targeted to receive disseminated information or were otherwise involved in AHRQ's dissemination efforts.

What GAO Found

The Agency for Healthcare Research and Quality (AHRQ), an agency within the Department of Health and Human Services (HHS), has taken some steps to disseminate comparative clinical effectiveness research (CER), as required under the Patient Protection and Affordable Care Act (PPACA), but has not taken other actions to help it fully address its dissemination requirements. The steps it has taken include the creation of tools that organize and disseminate research findings to certain targeted stakeholder groups and the development of plans for a publicly available database that includes CER. For example, AHRQ's marketing plans—customized plans to help convey key messages about AHRQ's research—include various informational tools to disseminate CER, such as research summaries that communicate research findings to clinicians, consumers, caregivers, and policymakers. However, the agency has not clearly defined how to disseminate information to certain stakeholder groups specified in the law, nor has it identified and documented time frames to implement the marketing plans and distribute the associated informational tools, as would be consistent with federal internal control standards, which state that significant events need to be clearly documented to ensure management goals are carried out. Additionally, in order to implement PPACA's requirement for developing a publicly available database that contains CER evidence, AHRQ officials told GAO that they plan to create a web page to list and provide users with links to existing publicly available databases that could be used to search for CER, but they have not documented a specific implementation plan that includes time frames and strategies to address known potential limitations, such as difficulties that certain users may face in searching the databases for CER results.
HHS's Assistant Secretary for Planning and Evaluation (ASPE) has coordinated among various agencies to fund projects intended to build data capacity for CER, but its approach lacks key elements needed to ensure its effectiveness. For example, these projects include an effort to better standardize data that could be used in multiple research projects. However, HHS's approach to building data capacity for CER lacks key elements, such as defined objectives, milestones, and time frames, that are necessary to ensure effectiveness. ASPE officials worked with the Office of the National Coordinator for Health Information Technology to develop a strategic road map to guide both the identification and selection of ASPE's projects beginning in fiscal year 2014 through fiscal year 2019. Although the February 2014 strategic framework for the road map highlighted several priority objectives, such as enabling the collection of standardized clinical data, these objectives were broad and not clearly defined. For example, although ASPE identified and considered related, ongoing federal and non-federal data infrastructure projects in an attempt to identify needs or gaps, among other things, its strategic road map is unclear on the timing and level of coordination that would be necessary for its projects to work together with these related projects to improve data capacity. Standard practices for project management call for agencies to conceptualize, define, and document specific goals and objectives in the planning process, along with the appropriate steps, milestones, time frames, and resources needed to achieve those results.

What GAO Recommends

GAO recommends that HHS direct (1) AHRQ to take several actions related to its dissemination efforts, including identifying and documenting time frames for the implementation and distribution of marketing plans and informational tools, and (2) ASPE to include clearly defined objectives, milestones, and time frames, or other indicators of performance, in its strategic road map used to identify its CER-funded projects. HHS concurred with the recommendations.