In 1995, HRSA exercised that option, and the contract now requires that the evaluation cover all 5 years of the originally planned demonstration. The national evaluation has two major components: an impact evaluation and a process evaluation. The impact evaluation is used to determine whether the infant mortality rates in Healthy Start communities have declined and whether related outcomes have improved. The process evaluation describes how the program actually operates. In its final evaluation report, MPR intends to synthesize these two components, linking outcomes with processes to determine why Healthy Start has or has not succeeded in communities and which strategies are likely to be successful elsewhere. MPR’s Preliminary Evaluation Results Were Not Conclusive

In the fall of 1997, MPR reported some preliminary evaluation results, including a draft interim report on its impact evaluation, which led to press accounts suggesting a variety of interpretations about the success of the Healthy Start program. Although the impact evaluation suggested that Healthy Start has not reduced infant mortality, such conclusions about the program would be premature because the impact evaluation does not include data from all the program sites or data from all the years of the program. Two detailed reports on specific interventions are available only in draft form. MPR’s Final Report Will Be Based on Fewer Years of Impact Data and Program Operation Than Originally Planned

The final report as currently planned will not be the final evaluation of Healthy Start. Conclusions

Since the national evaluation of the Healthy Start program has yet to be completed, preliminary results should not be used to conclude that the program has or has not achieved its goals. HRSA and MPR plan for the “final” report of the national evaluation to include an extensive description of the program, indicate whether it has reduced infant mortality rates at Healthy Start sites, and provide an analysis of how program characteristics have influenced outcomes. However, the final report will analyze infant mortality data from only 4 years of the demonstration. Primarily because implementation of the projects was slower than anticipated, data from the first 4 years of the demonstration may be insufficient for judging the success of Healthy Start in lowering infant mortality. Thus, even the final report will be inconclusive. Thus, at a relatively modest cost, MPR’s evaluation would be further strengthened by including data from the sixth and final year of the demonstration.
Why GAO Did This Study

Pursuant to a congressional request, GAO reviewed the preliminary results of a national evaluation of the Healthy Start program, focusing on: (1) the plan for national evaluation; (2) what Mathematica Policy Research, Inc.'s (MPR) preliminary evaluation results indicate; and (3) what is expected from the final evaluation.

What GAO Found

GAO noted that: (1) MPR's preliminary reports from the national evaluation of Healthy Start do not provide a complete assessment of the program and, therefore, should not be used to judge the program's success; (2) even the final report will not contain all the data expected to be analyzed for the national evaluation; (3) if the evaluation plan were expanded to include data from the sixth and final year of the demonstration, conclusions about whether the program has met its goals of reducing infant mortality could be strengthened; (4) the national evaluation of the Healthy Start program had two major components: (a) an impact evaluation, to determine whether the infant mortality rates in Healthy Start communities have declined; and (b) a process evaluation to describe how the program actually operates; (5) once these evaluations are completed, MPR plans to link outcomes with processes in its final report to determine why Healthy Start has or has not succeeded and what would be required for a similar intervention elsewhere; (6) while MPR's draft report on its impact evaluation suggests that Healthy Start has little effect in reducing infant morality in targeted communities, drawing such a conclusion at this time would be premature for several reasons; (7) the process evaluation is also incomplete; (8) only some of the reports that it comprises are available; (9) eventually, MPR plans to cover program implementation at all sites, the characteristics of program participants, and details about some of the most important strategies used by the program; (10) with these two major components of the evaluation in preliminary stages or incomplete, MPR cannot yet relate process to impact; (11) the final evaluation is expected to include an analysis of infant morality data from the original 5 years of the demonstration for all 15 sites; (12) however, the final report on the evaluation, now planned for early 1999, will include data from only the first 4 years; and (13) further, because implementation of the program was slower than anticipated, and the program was mature for fewer years of the original demonstration period than planned for in the evaluation, even results from the final report are likely to be inconclusive and should be considered preliminary.