reporting on PEPFAR results. Data Indicate Progress in Achieving Treatment Program Results, but Some Indicators Have Limitations

From fiscal year 2010 through 2012, OGAC reported PEPFAR results in  terms of three primary indicators: (1) the number of people currently on  treatment directly supported by PEPFAR (PEPFAR direct number of  people on treatment), (2) the percentages of eligible people receiving  treatment in partner countries (national treatment coverage rates), and (3)  the percentage of adults and children known to be alive and on treatment  12 months after starting treatment (PEPFAR direct treatment retention  rates). Regarding the first indicator, although the number of  people on treatment directly supported by PEPFAR has increased  significantly, this indicator alone does not provide complete information  needed for assessing PEPFAR’s contributions to partner countries’  treatment programs. 1). The data are not always complete. Second, PEPFAR implementing agencies  conducted data quality assessments in three PEPFAR countries. However, OGAC has not established a common set of indicators to  assess results of these activities. PEPFAR country  teams fulfill many M&E functions at facilities where PEPFAR supports  direct treatment services, and they assist partner countries in carrying out  their M&E responsibilities by providing staff, training, and technical  assistance and other support. Partner countries’ M&E systems often are unable to produce complete  and timely data, thus limiting their usefulness for patient, clinic, or  program management. OGAC’s 2010 Next Generation Indicators Reference Guide recommends  several indicators for tracking partner-country outcomes related to  strengthening health systems, such as the existence of M&E plans and  the percentage of health facilities with record-keeping systems for   In addition, OGAC’s technical guidance  monitoring HIV/AIDS programs.to country teams for developing partner countries’ M&E systems identifies  a number of key efforts, such as developing M&E leadership and  organizations, improving the policy environment, and ensuring the  advancement and sustainability of technical capacity in PEPFAR partner  countries. However, OGAC has not issued guidance  identifying minimum standards that data generated by partner countries’  M&E systems should meet—such as standards related to completeness  and timeliness—in order for PEPFAR country teams to assess, together  with partner countries and other donors, whether the systems are ready  for use in PEPFAR program management and results reporting. This limits the  usefulness of PEPFAR’s direct treatment indicator for assessing progress  toward expanding partner-country treatment programs. Furthermore,  OGAC has not yet established a common set of indicators to measure the  results of PEPFAR technical assistance and other support intended to  improve the quality of treatment programs. Global AIDS Coordinator to take the following three actions in  collaboration with PEPFAR implementing agencies: develop a method that better accounts for PEPFAR’s contributions to establish a common set of indicators to measure the results of  treatment program quality improvement efforts; and establish a set of minimum standards for data generated by partner  countries’ M&E systems, to enable PEPFAR country teams to assess  those systems’ readiness for use in treatment program management  and reporting. Responding jointly with CDC and USAID, State provided written  comments (see app. In its written comments, State generally agreed with our three  recommendations. GAO staff who made major contributions to this report are  listed in appendix V.

Appendix I: Objectives, Scope, and Methodology

In this report, we examine the President’s Emergency Plan for AIDS  Relief’s (PEPFAR) (1) treatment program results and how the Department  of State’s (State) Office of the U.S. Global AIDS Coordinator (OGAC)  measures them and (2) assistance to improve partner countries’  monitoring and evaluation (M&E) systems. We interviewed officials from OGAC, the Department of Health and  Human Services’ (HHS) Centers for Disease Control and Prevention  (CDC), and the U.S. Agency for International Development (USAID) in  Washington, D.C., and Atlanta, Georgia. GAO-12-673. GAO-10-836.
Why GAO Did This Study

PEPFAR, first authorized in 2003, has supported significant advances in HIV/AIDS prevention, treatment, and care in more than 30 countries. In reauthorizing the program in 2008, Congress directed OGAC to continue to expand the number of people receiving care and treatment through PEPFAR while also making it a major policy goal to help partner countries develop independent, sustainable HIV programs. As a result, PEPFAR began shifting efforts from directly providing treatment services toward support for treatment programs managed by partner countries. GAO was asked to review PEPFAR treatment programs. GAO examined (1) PEPFAR treatment program results and how OGAC measures them and (2) PEPFAR assistance to improve partner countries' M&E systems. GAO reviewed PEPFAR plans, performance reports, and guidance and interviewed officials from OGAC, the Centers for Disease Control and Prevention (CDC), and the U.S. Agency for International Development (USAID). GAO also synthesized findings of treatment program studies and conducted fieldwork in three countries.

What GAO Found

The Department of State's (State) Office of the U.S. Global AIDS Coordinator (OGAC) has reported on President's Emergency Plan for AIDS Relief (PEPFAR) treatment program results primarily in terms of (1) numbers of people on treatment directly supported by PEPFAR, (2) percentages of eligible people receiving treatment, and (3) percentages of people alive and on treatment 12 months after starting treatment. However, these indicators do not reflect some key PEPFAR results. First, although the number of people on treatment directly supported by PEPFAR grew from about 1.7 million to 5.1 million in fiscal years 2008 through 2012, this indicator alone does not provide complete information needed for assessing PEPFAR's contributions to partner countries' treatment programs. Second, although 10 PEPFAR country teams reported that percentages of people alive and on treatment after 12 months exceeded 80 percent, data for this indicator are not always complete and have other limitations. To improve these data, according to OGAC officials, OGAC clarified its guidance and conducted data quality assessments. However, OGAC has not yet established a common set of indicators to monitor the results of PEPFAR's efforts to improve the quality of treatment programs.
As PEPFAR partner countries assume greater responsibility for managing their treatment programs, fully functioning monitoring and evaluation (M&E) systems are critical for tracking results and ensuring treatment program effectiveness. PEPFAR country teams assist partner countries in carrying out their M&E responsibilities by providing staff, training, technical assistance, and other support. With this assistance, partner countries have made some progress in expanding and upgrading these M&E systems. Nevertheless, partner countries' M&E systems often are unable to produce complete and timely data, thus limiting the usefulness of these data for patient, clinic, or program management. OGAC has not yet established minimum standards for partner countries' M&E systems, particularly relating to data completeness and timeliness, in order for PEPFAR country teams to assess those systems' readiness for use in treatment program management and results reporting.

What GAO Recommends

The Secretary of State should direct OGAC to (1) develop a method that better accounts for PEPFAR's contributions to partner-country treatment programs, (2) establish a common set of indicators to measure the results of treatment program quality improvement efforts, and (3) establish a set of minimum standards for data generated by partner countries' M&E systems. Commenting jointly with CDC and USAID, State generally agreed with the report's recommendations.