Transportation Infrastructure Projects

The seven MCC compacts completed in 2010 and 2011 each included a  transportation infrastructure project that—except in the case of Armenia— received 50 percent or more of the compact’s total funding (see fig.1). 2.) Average annual daily traffic: A measure of the volume and type of  traffic using a road. Beneficiary Assessment

MCC defines beneficiaries as people who realize income gains or  expenditure savings as a result of its investment. MCC Data Show that Some Key Performance Targets Were Met, but Data Quality Problems Call into Question Reported Results

MCC reduced the scope of early road and port projects and reports mixed  success in meeting key performance targets. In addition, MCC did not consistently account for third-party  funding. Five of six compacts met their reduced targets for kilometers  completed. We identified problems with the measurement of road roughness for  some of the six compacts with road projects, including the use of  inconsistent application of methodologies and calculation errors. Therefore, MCC reports that it did not meet the compact’s final  kilometers-completed target, even though data suggest that the  kilometers MCC funded were complete at the end of the compact
The government of Cape Verde secured funding from the government  of Portugal to rehabilitate 22.9 kilometers of secondary roads that  MCC removed from the scope of its compact. However, MCC reported all 57.2 kilometers as a part of its final  results, even though it did not fund all of the work. MCC Reduced the Cape Verde Port Project Scope and Did Not Meet Many Targets for the Benin Port Project

For the Cape Verde compact, MCC reduced the scope of the port project,  deferred the measurement of key indicators, and therefore did not  conduct any data quality reviews of those indicators. We identified problems with MCC’s measurement of the volume-of- merchandise indicators, and MCA-Benin’s data quality reviews identified  additional problems with all three key indicators. MCC subsequently revised its  beneficiary estimates for ongoing compacts, but the implementation of the  new approach was not always consistent and suffered from varying  degrees of quality control problems. MCC’s Updated Beneficiary Analysis Guidelines Do Not Include a Formal Quality Review Process

In our previous work, we found that MCC did not have a consistent  methodology for estimating the number of beneficiaries across  compacts. Third, the beneficiary figures in  MCC’s public documents are sometimes inaccurate. Conclusions

MCC recognizes the importance of a disciplined, transparent, and  accountable approach to tracking compact results to make well-informed  decisions about U.S. investments. Because MCC’s new approach to estimating beneficiary  numbers does not include a formal quality review process, the beneficiary  estimates contain incorrect numbers and formulas and inconsistent  calculations. To improve the reliability of results measurement, MCC should  improve guidance for common indicators by requiring that they are  included in data quality reviews, measured uniformly across  compacts, and reported in a standardized manner—including when  third-party funding is used to complete the original scope of a project;  improve guidance for baseline measurements by requiring MCAs to  document the date, source, and methodology for establishing the  baseline; and enforce current monitoring and evaluation policy requiring MCAs to  conduct data quality reviews and to provide written comments in  response to identified issues, discussing how recommendations will  be implemented or explaining why changes may not be made. Agency Comments and Our Evaluation

In written comments on a draft of this report, MCC stated that it agrees  with our four recommendations and outlined steps it will take or has taken  to address them. This report examines the extent to which  MCC has, for transportation infrastructure projects in compacts ending in  2010 and 2011, (1) achieved expected performance targets and (2) used  a consistent methodology in estimating numbers of beneficiaries. To assess the extent to which MCC has achieved its performance targets  for transportation infrastructure projects in compacts ending in 2010 and  2011, we reviewed MCC guidance and policy documents and analyzed  compacts, monitoring and evaluation plans, and indicator tracking tables  for the compacts with Honduras, Cape Verde, Nicaragua, Georgia,  Vanuatu, Armenia, and Benin. Appendix II: Compact Timeframes

The Millennium Challenge Corporation (MCC) completed seven compacts  in 2010 and 2011—Honduras, Cape Verde, Nicaragua, Georgia, Benin,  Vanuatu, and Armenia (see fig. 3).
Why GAO Did This Study

To help developing countries reduce poverty and stimulate economic growth, MCC has approved 26 bilateral compact agreements totaling about $9.3 billion. In the seven compacts that ended in 2010 and 2011--Honduras, Cape Verde, Nicaragua, Vanuatu, Georgia, Armenia, and Benin--transportation infrastructure projects generally received about 50 percent of the compact's total funding. To measure the results of its compacts, MCC sets targets for various performance indicators--such as number of kilometers paved or volume of merchandise passing through a port--and estimates the number of beneficiaries. This report, responding to a congressional mandate, examines the extent to which MCC has, for transportation infrastructure projects, (1) achieved expected performance targets and (2) consistently estimated numbers of beneficiaries. GAO analyzed MCC documents, interviewed MCC officials, and drew on fieldwork completed for related work in four of the seven countries.

What GAO Found

The Millennium Challenge Corporation (MCC)--a U.S. government corporation-- recognizes the importance of a disciplined, transparent, and accountable approach to tracking compact results. However, it reduced the scopes of its early transportation infrastructure projects and reports mixed success in meeting key performance targets. In addition, problems with data quality call into question the reliability of those reported results. GAO found the following for the seven compacts ending in 2010 and 2011, each with a road project or a port project. Road Projects: (1) MCC reduced kilometers to be paved under six compacts--Honduras, Cape Verde, Nicaragua, Vanuatu, Georgia, and Armenia--by a combined 63 percent (from about 1,800 to 600 km) because of increased construction costs and political problems in partner governments. MCC reported meeting reduced targets for five compacts. However, for three compacts, MCC did not consistently account for kilometers completed with funding from third parties. (2) MCC reported meeting revised targets for road roughness--a measure of pavement quality--for five of the compacts. However, reported data have quality issues, including the inconsistent application of measurement methodologies and calculation errors that resulted in overstated results. (3) MCC reported meeting targets for annual average daily traffic--a measure of the volume of traffic using the road--for three of the compacts. However, weaknesses in traffic baseline estimates may have affected the establishment of targets and therefore MCC's ability to measure results. Port Projects: (1) In Cape Verde, MCC funding ($53.7 million) was insufficient to construct all planned port elements. As a result, MCC reduced the project's scope and deferred measuring the results of key indicators. (2) In Benin, MCC completed most of the envisioned scope of the port project. MCC's data show that the compact met the original target for one of three key performance indicators, volume of merchandise. (The other two indicators relate to the measurement of shipping costs.) However, GAO found that MCC's estimation of this indicator's baseline may lead to overstated results. In addition, data quality reviews identified problems with the data used, which MCA-Benin did not formally address. In 2009, MCC improved its methodology for estimating beneficiaries--people who realize income gains or expenditure savings as a result of its investment--by standardizing its approach. MCC subsequently revised its beneficiary numbers for all compacts. However, the new approach did not include a formal quality review process. As a result, implementation of the new approach suffered from varying degrees of quality problems. For example, (1) MCC did not implement its beneficiary estimation methodology consistently across early transportation infrastructure projects; (2) beneficiary calculations contained incorrect formulas and numbers, and differed from supporting documents; and (3) beneficiary figures in MCC's public documents were sometimes inaccurate.

What GAO Recommends

MCC should strengthen existing policies and practices regarding measuring and evaluating results data and formalize a quality review process to improve its beneficiary calculations. MCC agreed with all of our recommendations and outlined some steps the agency will take or has already taken to address them.