Head Start is administered by HSB within ACF. Figure 1 provides information on the numbers of Head Start  grantees, delegate agencies, centers and classrooms. Since the inception of Head Start, questions have been raised about the  effectiveness of the program. The Framework covers a broad range of child  skill and development areas and incorporates each of the legislatively  mandated goals, such as that children “use and understand an increasingly  complex and varied vocabulary” and “identify at least 10 letters of the  alphabet.”
Since 2000, HSB has required every Head Start grantee to include each of  the areas in the Framework in the child assessments that each grantee  adopts and implements. Grantees are permitted to determine how to  assess children’s progress in these areas. The results of the assessments are used for the purposes of  individual program improvement and instructional support and are not  aggregated across grantees or systematically shared with federal officials. The NRS, prompted by the April 2002 announcement of President Bush’s  Good Start, Grow Smart initiative, builds on the 1998 legislation by  requiring all Head Start programs to implement the same assessment,  twice a year, to all 4- and 5-year-old Head Start participants who will  attend kindergarten the following year. The  contractor, working closely with HSB, was responsible for the design and  field testing of the NRS, including developing training materials to support  national implementation of the reporting system by grantees. Among the most important concepts in test development are  validity and reliability. This process generally takes more than 1 year, especially  if the test is designed to measure changes in performance. In the remainder of the report, we will discuss how the focus of the NRS  was determined and the assessment was developed, HSB’s response to  problems in initial implementation as well as some implementation issues  that remain unaddressed, and the extent to which the assessment meets  the professional and technical standards to support specific purposes  identified by HSB. NRS Assesses Selected Skills Using Adaptations of Other Assessments

The NRS assesses vocabulary, letter recognition, simple math skills, and  screens for understanding of spoken English. As initially conceived by  HSB, the NRS was to gauge the progress of Head Start children in 13  congressionally mandated indicators of learning. However, time  constraints and technical matters precluded HSB from assessing children  on all of the indicators and led HSB to consider, and eventually adopt,  portions of other assessments for use in the NRS. The field test also included a Spanish  version of the NRS. Figures 3 and 4 are examples from the letter naming  and early math skills components of the NRS.
Why GAO Did This Study

In September 2003, the Head Start Bureau, in the Department of Health and Human Services (HHS) Administration for Children and Families (ACF), implemented the National Reporting System (NRS), the first nationwide skills test of over 400,000 4- and 5-year-old children. The NRS is intended to provide information on how well Head Start grantees are helping children progress. Given the importance of the NRS, this report examines: what information the NRS is designed to provide; how the Head Start Bureau has responded to concerns raised by grantees and experts during the first year of implementation; and whether the NRS provides the Head Start Bureau with quality information.

What GAO Found

The Head Start Bureau developed the NRS to gauge the extent to which Head Start grantees help children progress in specific skill areas, including understanding spoken English, recognizing letters, vocabulary, and early math. Due to time constraints and technical matters, the Head Start Bureau adapted portions of other assessments for use in the NRS. Head Start Bureau officials have responded to some concerns raised during the first year of NRS implementation, but other issues remain. For example, the Head Start Bureau has modified training materials and is exploring the feasibility of sampling. However, it is not monitoring whether grantees are inappropriately changing instruction to emphasize areas covered in the NRS. Head Start Bureau officials have said NRS results will eventually be used for program improvement, targeting training and technical assistance, and program accountability; however, the Head Start Bureau has not stated how NRS results will be used to realize these purposes. Currently, results from the first year of the NRS are of limited value for accountability purposes because the Head Start Bureau has not shown that the NRS meets professional standards for such uses, namely that (1) the NRS provides reliable information on children's progress during the Head Start program year, especially for Spanish-speaking children, and (2) its results are valid measures of the learning that takes place. The NRS also may not provide sufficient information to target technical assistance to the Head Start centers and classrooms that need it most.