(See app. Evaluation questions are aligned with program goals. Thus, evaluations were  designed to provide useful information about program results. Limitations we identified revealed that conducting  evaluations overseas can pose challenges for evaluators. For example,  travel to remote areas with safety and security concerns may limit an  evaluator’s ability to conduct appropriate sampling and collect primary  data for the study. About 40 percent of evaluations had limitations in, or provided insufficient  information about, their sampling methodology. Evaluation Costs Ranged Widely and Varied by Type and Agency, but Most Cost Less Than $200,000

Costs for the majority of the foreign aid evaluations whose costs we  reviewed were less than $200,000, but the costs ranged widely and  varied by type of evaluation and agency. These  costs greatly exceeded the median costs for all evaluations. High-Quality Evaluations Tend to Cost More, but Some Lower-Cost Evaluations also Met All Quality Criteria

Our analysis found that high-quality evaluations tend to be more  expensive, but well-designed lower-cost evaluations also met the criteria  we identified for a high-quality evaluation. Selected Agencies’ Evaluations Are Generally Available Online, but Some Agencies Can Improve Dissemination

We assessed DOD’s, HHS’s, MCC’s, State’s, USAID’s, and USDA’s use  of six dissemination practices that federal, AEA, and other guidance  indicate agencies should generally use to ensure effective dissemination  of evaluations. In addition to publicly posting  the report, all of the agencies used other means to actively disseminate  evaluation findings. We examined the agencies’ dissemination of 193 evaluations. While USDA does not  currently publicly post its evaluations, USDA reported that it makes these  evaluations internally available through its grant management system. Conclusions

Foreign assistance evaluations can be challenging to implement, but they  are an essential tool for guiding agency decision making and allocation of  resources. Global AIDS Coordinator and Health Diplomacy) each  develop a plan for improving the quality of evaluations for the programs  included in our review, focusing on areas where our analysis has shown  the largest areas for potential improvement. To better ensure that the evaluation findings reach their intended  audiences and are available to facilitate incorporating lessons learned  into future program design or budget decisions, we recommend that  the Secretary of Health and Human Services direct the Centers for  Disease Control and Prevention to update its guidance and practices  on the posting of evaluations to require PEPFAR evaluations to be  posted within the timeframe required by PEPFAR guidance;  the Chief Executive Officer of MCC adjust MCC evaluation practices  to make evaluation reports available within the timeframe required by  MCC guidance;  the Secretary of State amend State’s evaluation policy to require the  completion of dissemination plans for all agency evaluations; and  the Secretary of Agriculture implement guidance and procedures for  making FAS evaluations available online and searchable on a single  website that can be accessed by the general public. HHS  did not comment on our recommendation that it develop a plan for  improving the quality of evaluations. The six agencies we identified are USAID, the Department of State  (State), the Millennium Challenge Corporation (MCC), the Department of  Health and Human Services (HHS), the U.S. Department of Agriculture  (USDA) and the Department of Defense (DOD). To assess the extent to which the results of foreign assistance program  evaluations are supported by their evidence and whether they assess if  programs have met their goals, we assessed the sample of agency fiscal  year 2015 evaluation reports against quality criteria we identified. The evaluation  quality criteria were judged on a four-part scale for most of the judgmental  questions: generally addressed: the evaluation mostly addressed the key  element(s) of the criterion but did not have to completely address all  elements in the subquestions; partially addressed: the evaluation had one or more clear area(s) for  improvement on the criterion; not at all addressed: the evaluation did not show that steps were  taken to address the criterion; and  insufficient information: reviewers could not make a determination due  to a lack of information in the evaluation and any other associated  materials. Tables 9 through 26 below  provide further detail on the characteristics and quality of the design,  implementation, and conclusions of fiscal year 2015 evaluations we  reviewed summarized for all six agencies and then individually  for (1) the  President’s Emergency Plan for AIDS Relief (PEPFAR) programs  implemented by the Centers for Disease Control and Prevention (CDC) of  the Department of Health and Human Services (HHS), (2) the Millennium  Challenge Corporation (MCC), (3) the Department of State (State), (4) the  U.S. Agency for International  Development (USAID) , and (5) the U.S.  Department of Agriculture’s (USDA) Foreign Agricultural Service’s food  aid programs.
Why GAO Did This Study

The U.S. government plans to spend approximately $35 billion on foreign assistance in 2017. Evaluation is an essential tool for U.S. agencies to assess and improve the results of their programs. Government-wide guidance emphasizes the importance of evaluation, and the Foreign Aid Transparency and Accountability Act of 2016 requires the President to establish guidelines for conducting evaluations. However, evaluations can be challenging to conduct. GAO has previously reported on challenges in the design, implementation, and dissemination of the evaluations of individual foreign assistance programs.
GAO was asked to review foreign aid evaluations across multiple agencies. This report examines the (1) quality, (2) cost, and (3) dissemination of foreign aid program evaluations. GAO assessed a representative sample of 173 fiscal year 2015 evaluations for programs at the six agencies providing the largest amounts of U.S. foreign aid —USAID, State, MCC, HHS's Centers for Disease Control and Prevention under the President's Emergency Plan for AIDS Relief, USDA's Foreign Agricultural Service, and DOD's Global Train and Equip program—against leading evaluation quality criteria; analyzed cost and contract documents; and reviewed agency websites and dissemination procedures.

What GAO Found

An estimated 73 percent of evaluations completed in fiscal year 2015 by the six U.S. agencies GAO reviewed generally or partially addressed all of the quality criteria GAO identified for evaluation design, implementation, and conclusions (see fig.). Agencies met some elements of the criteria more often than others. For example, approximately 90 percent of all evaluations addressed questions that are generally aligned with program goals and were thus able to provide useful information about program results. About 40 percent of evaluations did not use generally appropriate sampling, data collection, or analysis methods. Although implementing evaluations overseas poses significant methodological challenges, GAO identified opportunities for each agency to improve evaluation quality and thereby strengthen its ability to manage aid funds more effectively based on results.
Evaluation costs ranged widely and were sometimes difficult to determine, but the majority of evaluations GAO examined cost less than $200,000. Millennium Challenge Corporation (MCC) evaluations had a median cost of about $269,000, while median costs for the U.S. Agency for International Development (USAID), the U.S. Department of Agriculture (USDA), and the Department of State (State) ranged from about $88,000 to about $178,000. GAO was unable to identify the specific costs for the Department of Defense (DOD) and Department of Health and Human Services (HHS) evaluations. High-quality evaluations tend to be more costly, but some well-designed lower-cost evaluations also met all quality criteria. Other factors related to evaluation costs include the evaluation's choice of methodology, its duration, and its location.
Agencies generally posted and distributed evaluations for the use of internal and external stakeholders. However, shortfalls in some agency efforts may limit the evaluations' usefulness.
Public posting . USDA has not developed procedures for reviewing and preparing its evaluations for public posting, but the other agencies posted nonsensitive reports on a public website.
Timeliness . Some HHS reports and more than half of MCC reports were posted a year or more after completion.
Dissemination planning. State does not currently have a policy requiring a plan that identifies potential users and the means of dissemination.

What GAO Recommends

GAO recommends that each of the six agencies develop a plan to improve the quality of its evaluations and that HHS, MCC, State, and USDA improve their procedures and planning for disseminating evaluation reports.
The agencies concurred with our recommendations.