ACF monitors the success of local agencies that receive Head Start grants  in meeting Head Start program goals and complying with program  requirements by conducting on-site monitoring reviews of grantee  programs every 3 years; administering an extensive, annual PIR survey of  grantees; and reviewing required financial reports and annual audit  reports. We made a number of recommendations to address the  problems that we identified, including:  1.  producing a comprehensive risk assessment of the Head Start  2.  strengthening on-site reviewer training and certification procedures;  3.  developing a more consistent approach to conducting on-site reviews  4. implementing a quality assurance process that ensures on-site reviews  are conducted within established guidelines and ACF managers are  held accountable for the quality of on-site reviews;  5.  ensuring the accuracy of PIR survey data by independently verifying  key data submitted by grantees, or ensuring that grantees have systems  in place to collect and report accurate, verifiable data;  6.  making greater use of available information on the status and use of  7.  taking steps to obtain competition for grants that are being refunded if  it is determined that current grantees have failed to meet program,  financial management, or other requirements. 1.) ACF Has Not Undertaken a Comprehensive Risk Assessment, and Data Reliability and Other Challenges Remain

ACF has not undertaken a comprehensive assessment of risks to the  federal Head Start program, despite our 2005 recommendation, and little  progress has been made in ensuring that the data from its annual PIR  survey of grantees, which could facilitate such an assessment, are reliable. Further, both systems depend  to some extent on unreliable data from the annual PIR survey of grantees. New Systems Could Facilitate Comprehensive Risk Assessment but Limitations Exist

ACF is in the process of developing two systems that may help it assess  programwide risks; however, significant limitations or uncertainty exist  with respect to each that could constrain ACF’s ability to use them to  conduct a meaningful risk assessment. Although the refunding analysis system allows grantees that are  considered high risk to be brought to the attention of Head Start program  managers, it does not allow for a broader assessment of other sources of  risk, such as those we previously identified. ACF Improved On-site Monitoring by Strengthening Review Processes and Is Working to Establish a System to Evaluate Reviews

ACF has implemented several changes aimed at improving the quality and  consistency of its on-site reviews of Head Start grantees, in response to  our 2005 recommendations. Specifically, ACF has implemented a more rigorous process for  certifying reviewers and new processes to improve the consistency of  reviews, and is working to establish a system for evaluating reviews on an  ongoing basis. The draft report is then forwarded to the  grantee’s home region and to the review team leader’s region for review  and comment. ACF is currently working to  establish an ongoing system for evaluating its on-site review process that  would address this new requirement. ACF Uses Data to Identify Grantees in Need of Assistance, but Weaknesses Have Hindered Efforts to Improve Grantee Performance

To improve Head Start grantee performance, ACF uses data from multiple  sources to identify underperforming Head Start grantees. For example, ACF does not have clear  criteria for determining which grantees need additional oversight. These  decisions are typically made on an ad-hoc basis, which may result in  grantees with similar problems receiving inconsistent levels of oversight. This will give ACF the  ability to open competition for Head Start grants to additional grantees,  thereby enhancing ACF’s ability to remove severely underperforming  grantees from the program, on a regular basis. To obtain grantees’ opinions on ACF’s oversight of the Head Start  program, we administered a Web-based survey to a nationally- representative sample of Head Start and Early Head Start program  directors. We reviewed ACF studies from 1995 and 2007 on the validity and accuracy  of PIR data, the results of which are discussed in this report. Managing for Results: Efforts to Strengthen the Link between Resources  and Results at the Administration for Children and Families.
Why GAO Did This Study

In February 2005, GAO issued a report that raised concerns about the effectiveness of the Department of Health and Human Services (HHS) Administration for Children and Families' (ACF) oversight of about 1,600 local organizations that receive nearly $7 billion in Head Start grants. GAO was asked to report on (1) ACF's progress in conducting a risk assessment of the Head Start program and ensuring the accuracy and reliability of data from its annual Program Information Report (PIR) survey of grantees, (2) efforts to improve on-site monitoring of grantees, and (3) how data are used to improve oversight and help grantees meet program standards. For this report, GAO surveyed a nationally representative sample of Head Start program directors and interviewed ACF officials. GAO also reviewed ACF studies on the validity of PIR data and conducted tests of data from the 2006 PIR database.

What GAO Found

ACF has not undertaken a comprehensive assessment of risks that may limit Head Start's ability to meet federal program objectives, despite GAO's 2005 recommendation, and little progress has been made to ensure that the data from its annual PIR survey of grantees, which could facilitate such an assessment, are reliable. To conduct a comprehensive risk assessment, ACF needs to identify external and internal risks, estimate their significance, and decide how to best manage them. While ACF says it is working to establish two systems to address programwide risk, our analysis suggests that these systems fall short of that goal. The first system, by which ACF assesses grantees before providing new funds each year, only assesses risk posed to the program by poorly performing grantees and does not allow for a broader assessment of other sources of risk, such as improper payments to contractors. The second system, a new, integrated management information system, has been in development for over 4 years and it is not clear how it will facilitate a more comprehensive risk assessment for the Head Start program. Both initiatives depend, in part, on data from the annual PIR survey of grantees, which have been found to be unreliable. ACF has taken steps to improve oversight of Head Start grantees by implementing a more rigorous process for certifying reviewers who conduct on-site monitoring visits, implementing new processes to improve the consistency of reviews, and working to establish a system for evaluating reviews on an ongoing basis. Now, ACF verifies reviewers' qualifications and requires them to pass online tests in writing and computer literacy. Reviewers must also complete ongoing training and are evaluated by their peers at the end of each review. ACF has also taken a number of steps to improve the consistency and objectivity of reviews, including developing a Web-based data collection tool to facilitate information gathering, assigning review team leaders from outside the grantee's home region to increase independence, and centralizing the review and preparation of monitoring reports. ACF is also working to establish an ongoing system for evaluating its on-site review process. ACF uses data to track grantee performance and target assistance to underperforming grantees, but weaknesses may have hindered these efforts to improve grantee performance. For example, ACF does not have clear criteria for determining which grantees need additional oversight as part of its refunding analysis. Such decisions are made on an ad-hoc basis, which may result in grantees with similar problems receiving different levels of oversight. Moreover, prior to the December 2007 reauthorization of Head Start, ACF was limited in its ability to increase competition for grants to replace underperforming grantees. Under the new law, ACF will have more flexibility to open competition for Head Start grants to other prospective grantees when current grantees fail to deliver high-quality, comprehensive Head Start programs.