USAID Missions in Five Selected Countries Are Implementing, Monitoring, and Evaluating Reading Programs Consistent with USAID Guidance

In the five countries where we conducted fieldwork, the USAID missions  had implemented primary grade reading programs that included  intervention elements recommended in USAID’s guidance related to its  education strategy. In several of the countries, the need to develop new  reading materials in mother-tongue languages and train classroom  teachers in the use of these materials (see fig. As a result,  portions of targeted student populations in four countries are not expected  to benefit from the programs in time to contribute to USAID’s strategic  goal of 100 million students with improved reading skills by December  2015. Agency officials attributed  the incomplete data and information to a lack of timely guidance, among  other factors. Fourth, the education strategy did  not set interim targets for measuring progress toward USAID’s reading  goal in comparison to planned performance, as suggested by leading  performance management practices. Without a methodology and  interim targets, USAID officials cannot determine USAID’s progress  toward its goal or identify a realistic goal for a future education strategy. About Two-Thirds of Missions with Primary Grade Reading Programs Are Expected to Have Data for Assessing Overall Progress toward USAID’s Reading Goal by 2015

Because it took some missions longer to implement primary grade  reading programs than USAID estimated in its implementation guidance  to the education strategy, USAID expects only about two-thirds of  missions with funding for such programs to provide baseline and midline  or endline assessment data in time for the data to be used in estimating   As figure 8  the agency’s progress toward its primary grade reading goal.shows, 26 missions (62 percent) of the 42 with funding for such programs  had implemented them by the end of fiscal year 2012, the target date  established in the education strategy’s implementation guidance to allow  at least 3 years of implementation. USAID Lacks Complete Data and Information Needed to Aggregate and Analyze Reading Assessment Results

Although the USAID contractor responsible for aggregating and analyzing  missions’ reading assessment results has begun to collect available data,  as of April 2015, only a small number of the missions with completed  reading assessments had provided both complete data and the  supporting information that the contractor needs. USAID Has Not Selected a Methodology for Estimating Total Numbers of Children with Improved Reading, Limiting Its Ability to Measure Progress toward Its Reading Goal

As of April 2015, USAID had not yet selected and documented a  methodology for using missions’ assessment data and supporting  information to estimate progress toward its goal of improved reading skills  for 100 million children by 2015. Gain score methodology. To help ensure that USAID reports a reliable estimate of the results of its  efforts to achieve the goal of improving reading skills of 100 million  children, we recommend that the Acting Administrator  select a methodology for estimating the total numbers of children with  improved reading skills as a result of exposure to primary grade  reading programs and document a description of the selected methodology as well as the  information necessary to evaluate the estimate (i.e., sampling  designs, sample sizes, and modes of data collection) when reporting  progress toward the reading goal. This report (1) examines five USAID  missions’ implementation, performance monitoring, and evaluation of  primary grade reading programs and (2) assesses USAID’s efforts to  estimate progress toward the 2011-2015 education strategy’s primary  grade reading goal. In addition, we  conducted fieldwork in five countries—Ethiopia, Malawi, Peru, the  Philippines, and Uganda—which we selected because they are  geographically diverse and received relatively large levels of U.S. funding  for basic education in fiscal year 2013. To assess USAID’s efforts to estimate progress toward the education  strategy’s primary grade reading goal—our second objective—we  reviewed the USAID education strategy, guidance for implementing the  strategy, technical notes to the strategy, and the strategy’s reporting  guidance to identify the agency’s policies and guidelines for completing  reading assessments at USAID missions and for analyzing the data and  supporting information.
Why GAO Did This Study

While many developing countries have achieved important gains in primary school enrollment, students' reading skills remain very low. In response, USAID's 2011-2015 Education Strategy set a goal of improved reading skills for 100 million primary grade children by the end of 2015 (Goal 1). The 42 missions with funding for primary grade reading programs were to start programs aligned with the strategy by the end of fiscal year 2012 to allow time to assess results by 2015.
GAO was asked to review USAID's efforts to implement primary grade reading programs—Goal 1 of its education strategy. This report (1) examines five USAID missions' implementation, monitoring, and evaluation of reading programs and (2) assesses USAID's efforts to estimate progress toward its reading goal. GAO analyzed USAID's education strategy and other guidance, reading assessment data and methodologies, performance monitoring plans and reports, and program evaluations, and conducted fieldwork in five countries selected based on geographic diversity, funding for basic education, and availability of reading assessment data.

What GAO Found

U.S. Agency for International Development (USAID) missions in all five countries where GAO conducted fieldwork were implementing primary grade reading interventions recommended in USAID's education strategy guidance. A key intervention in four of these countries is developing new reading instruction materials in mother-tongue languages—in some instances, multiple languages. In Ethiopia, for example, USAID's reading program developed new student textbooks and teacher's guides for each of eight grades and seven languages and involved teams of local teachers, language experts, and story writers, among others. The complexity of these interventions in Ethiopia, Malawi, the Philippines, and Uganda increased the time before they could be introduced in classrooms. As a result, some targeted student populations are not expected to benefit from improved reading instruction until late 2015 or 2016, too late to be measured to contribute to USAID's strategic goal of 100 million students with improved reading skills by the end of 2015. In addition, the missions were monitoring their reading programs and planned to conduct program evaluations consistent with USAID guidance.
USAID is currently unable to estimate progress toward the education strategy's reading goal because of four factors. First, because it took some missions longer to implement reading programs than USAID estimated, only about two-thirds of missions are expected to have data to estimate progress toward the goal by the end of 2015. Second, only a small number of missions provided USAID the complete data and supporting information that it needs to aggregate and analyze reading assessment results. Agency officials attributed the lack of complete data and information to an absence of timely guidance. In 2014, USAID issued updated reporting guidance. Third, USAID has not selected a methodology for calculating percentages of assessed children demonstrating improved reading skills, and extrapolating the results to estimate the total numbers of children with improved reading skills. Along with incomplete data and information, other factors contributed to a delay in identifying a methodology—for example, USAID did not ask its contractor to examine the methodology proposed in 2012 and explore alternatives until 2014. Fourth, the education strategy did not set interim targets for assessing progress toward the reading goal as suggested by leading performance management practices. Without a methodology and interim targets, USAID officials and others lack aggregate information about progress toward the goal to assess the current strategy and plan a realistic goal for a future strategy.

What GAO Recommends

USAID should select a methodology for estimating total numbers of children with improved reading skills, document a description of the methodology when reporting results, and set interim targets to assess progress toward a reading goal in any future education strategy. USAID concurred with GAO's recommendations.