The NIH Revitalization Act of 1993 and NIH Inclusion Policy

The Revitalization Act required NIH to ensure the appropriate inclusion of  women in NIH-funded clinical research, including clinical trials. More Women Than Men Were Enrolled in NIH Research Overall, but NIH Does Not Examine Detailed Data to Identify Potential Challenges to Enrolling Women

NIH data show that over the last decade more women than men have  been enrolled across all NIH-funded clinical research, including phase III  clinical trials. As of fiscal year 2015, these data are reported to NIH  through its Inclusion Monitoring System—one part of NIH’s awardee data  system. Specifically, in fiscal year 2014, among all NIH-funded clinical   research studies, 57 percent of enrollees (16.4 million) were women,   and 39 percent (11 million) were men. NIH Does Not Make IC- Level Enrollment Data Readily Available or Examine Detailed Enrollment Data to Identify Potential Challenges to Enrolling Women in Specific Research and Disease Areas

NIH collects and reviews aggregated enrollment data from the ICs;  however, NIH officials do not make these IC-level enrollment data readily  available to the public or other interested parties. Information is needed throughout the agency to achieve all of its  objectives, and effective communication should occur in a broad sense,  with information flowing down, across, and up the organization. In addition, by not routinely examining more  detailed enrollment data that is aggregated by sex—such as data at the  disease and condition level—NIH is limited in its ability to identify whether  women are sufficiently represented in studies in specific areas that cross  ICs—such as obesity. Further, NIH does not have information of sufficient  detail to monitor and determine if the aggregate enrollment data from  across NIH inadvertently mask low enrollment for particular research  areas or diseases or conditions. NIH Does Not Record Whether Clinical Trials Will Analyze Sex Differences to Allow Summary Analysis and Reporting for Oversight of Its Inclusion Policy

NIH’s Inclusion Policy requires that individual awardees conducting phase  III clinical trials consider whether analysis of potential differences in study  outcomes between women and men is needed in their studies, consistent  with the Revitalization Act’s provisions regarding the design of certain  clinical trials. However, the agency does not maintain, analyze, or report  summary data to oversee whether analysis of outcomes by sex are  planned or conducted, when applicable, across all NIH-funded clinical  trials. To ensure awardees’ compliance with this  requirement, NIH officials told us they rely on the agency’s peer review  process for reviewing applications, and after awards are made, on IC  program officers’ monitoring of individual awardees. NIH program officers monitor individual awardees’ compliance with the  analysis requirement of the Inclusion Policy; however, the agency lacks  summary data on awardees’ analysis plans, including the percentage of  awardees in a given year with trials designed to identify potential sex  differences, when applicable. NIH’s lack of summary data and reporting regarding the analysis  requirement of the Inclusion Policy conflicts with federal internal control  standards. First, federal internal control standards require that federal  agencies have control activities in place to ensure that management’s  directives are carried out and that these controls are monitored. By not  examining more detailed enrollment data—such as data aggregated by  research area or specific to various diseases and conditions—NIH cannot  know whether it is adequately including women across all of the research  it supports. Without  summary data, such as the proportion of trials being conducted that  intend to analyze differences in outcomes for men and women, and  reporting on that data, NIH and Congress cannot know whether or to what  extent current efforts are helping to ensure that differences in clinical  outcomes by sex are identified and that NIH is supporting research that  can be used to shape improved medical practices for both women and  men. Recommendations for Executive Action

To ensure effective implementation of the Inclusion Policy in a manner  consistent with the Revitalization Act’s provisions regarding the design of  certain clinical trials, the NIH Director should take the following five  actions: make IC-level enrollment data readily available through public means,  such as NIH’s regular biennial report to Congress on the inclusion of  women in research, or through NIH’s website; examine approaches for aggregating more detailed enrollment data at  the disease and condition level, and report on the status of this  examination to key stakeholders and through its regular biennial  report to Congress on the inclusion of women in research; ensure that program officers have a means for recording information  obtained from monitoring awardees’ plans for and progress in  conducting analyses of potential differences in outcomes by sex; on a regular basis, systematically collect and analyze summary data  regarding awardees’ plans to conduct analyses of potential sex  differences, such as the proportion of trials being conducted that  intend to analyze differences in outcomes for men and women; and  report on this summary data and the results of this analysis in NIH’s  regular biennial report to Congress on the inclusion of women in  research. We also spoke with or received written responses  from officials and program officers from 3 institutes and centers (ICs) of  the National Institutes of Health (NIH) about the factors affecting women’s  participation in clinical research reported by NIH awardees.
Why GAO Did This Study

Women make up over half the U.S. population, but historically have been underrepresented in clinical research supported by NIH and others. As a result, differences in the manifestation of certain diseases and reactions to treatment in women compared with men were not identified. For example, there have been instances of women having adverse effects that differed from those of men related to medications and other treatments. NIH's Inclusion Policy established requirements governing women's inclusion in its clinical research.
GAO was asked to provide information on women's participation in NIH research. Among other reporting objectives, GAO examined (1) women's enrollment and NIH's efforts to monitor this enrollment in NIH-funded clinical research; and (2) NIH's efforts to ensure that NIH-funded clinical trials are designed and conducted to analyze potential sex differences, when applicable. To do this, GAO reviewed relevant laws and policies, including the Inclusion Policy, and federal standards for internal control; reviewed and analyzed NIH enrollment data from fiscal years 2005-2014; and interviewed NIH and IC officials and other experts.

What GAO Found

Data from the National Institutes of Health (NIH) show that more women than men were enrolled in NIH-funded clinical research for fiscal years 2005-2014, but NIH does not make certain enrollment data readily available to interested parties or examine other detailed data to identify potential challenges to enrolling women in specific research and disease or condition areas. In fiscal year 2014, for example, NIH reported that across all of the clinical research studies it funded—including phase III clinical trials, the largest studies involving human subjects— 57 percent of enrollees (16.4 million) were women. NIH collects enrollment data from individual awardees through its Institutes and Centers (IC)—which generally fund studies in different research areas—and publicly reports data on aggregate enrollment as part of its implementation of the Inclusion Policy developed to implement provisions of the NIH Revitalization Act of 1993. However, NIH does not make the IC-level enrollment data from each of the 25 ICs that report data readily available to interested parties, so that interested parties must make an effort to seek out this data. In addition, NIH does not routinely examine more detailed enrollment data, such as enrollment data organized by the disease and condition being studied. As a result, NIH is limited in its ability to identify whether women are sufficiently represented in studies in specific areas—such as cardiovascular disease—or if the agency-wide data inadvertently mask enrollment challenges. By not examining more detailed data on enrollment below the aggregate level, NIH cannot know whether it is adequately including women in all of the research it supports, in a manner consistent with its Inclusion Policy. Further, NIH's reporting and monitoring in this area is inconsistent with federal internal control standards, which call for agencies to have controls to help ensure effective information flow and effective monitoring of agency activities.
NIH requires that phase III clinical trial awardees consider whether analysis of potential differences in outcomes between women and men is needed in their studies—one of the key requirements of its Inclusion Policy; however, the agency does not maintain, analyze, or report summary data to oversee whether analysis of outcomes by sex are planned or conducted. NIH officials told GAO that they rely on peer review and program officer monitoring to ensure awardee compliance with the analysis requirement. However, NIH program officers do not have a required field in a reporting system or other means to record the information they collect to monitor awardees' analysis plans and compliance with the Inclusion Policy requirement. In addition, there is no data element in NIH's data system to indicate whether an awardee's study should or does include plans for an analysis of potential differences in research outcomes by sex. As a result, NIH lacks summary data, such as the percentage of awardees in a given year with trials designed to identify potential differences in clinical outcomes by sex. Without this summary information, NIH cannot report this information in the agency's biennial reports to Congress and other stakeholders. The lack of summary data and reporting compromises NIH's monitoring of its implementation of the Inclusion Policy and conflicts with federal internal control standards, which call for agencies to ensure the flow of information about agency activities, provide for internal and external communication, and conduct periodic monitoring. Further, it limits NIH's assurance that it is supporting research that can be used to shape improved medical practice for both women and men.

What GAO Recommends

GAO recommends that NIH examine and report more detailed data on women's enrollment in NIH-funded studies, and collect, examine, and report data on the extent to which these studies include analyses of potential differences between women and men. NIH agreed with GAO's recommendations and plans to take action to implement them.