Education also provides states with technical  assistance in meeting the academic assessment requirements. States Reported That Assessment Spending Has Increased Since NCLBA Was Enacted and Test Development Has Been the Largest Assessment Cost in Most States

		Assessment Expenditures Have Grown in Nearly Every State since 2002, and Most States Reported Spending More for Vendors than State Staff

State ESEA assessment expenditures have increased in nearly every state  since the enactment of NCLBA in 2002, and the majority of these states  reported that adding assessments was a major reason for the increased  expenditures. Forty-eight of 49 states that responded to our survey said  their states’ overall annual expenditures for ESEA assessments have  increased, and over half of these 48 states indicated that adding  assessments to their state assessment systems was a major reason for  increased expenditures. Similarly, states reported that test and  item development was the largest assessment cost for alternate  assessments, followed by scoring. Rhode Island  state officials also reported that alternate assessments cost much more  than general assessments. As of  January 2009, 19 states said their state’s total ESEA assessment budget  had been reduced as a result of state fiscal cutbacks. States Have Considered Cost and Time in Making Decisions about Assessment Item Type and Content

		States Used Primarily Multiple Choice Items in Their ESEA Assessments Because They Are Cost- Effective and Can Be Scored within Tight Time Frames for Reporting Results

States have most often chosen multiple choice items over other item types  on assessments. State assessment  officials also told us that they used multiple choice items because they can  be scored quickly, and assessment vendors reported that states were  under pressure to release assessment results to the public before the  beginning of the next school year in accordance with NCLBA  requirements. State  officials and their technical advisors told us that they have faced  significant trade-offs between their efforts to assess highly cognitively  complex content and their efforts to accommodate cost and time  pressures. Moreover, open/constructed  response items do not always assess highly complex content, according to  an alignment expert. States Faced Several Challenges in Their Efforts to Ensure Valid and Reliable ESEA Assessments, including Staff Capacity, Alternate Assessments, and Assessment Security

		States Varied in Their Capacity to Guide and Oversee Vendors

State officials are responsible for guiding the development of the state  assessment program and overseeing vendors, but states varied in their  capacity to fulfill these roles. Officials in Texas and  other states said that having high assessment staff capacity—both in terms  of number of staff and measurement-related expertise—allows them to  research and implement practices that improve student assessment. In our work, we identified several gaps in state assessment security  policies. Education Has Provided Assistance to States, but the Peer Review Process Did Not Allow for Sufficient Communication

		Education Provided Technical Assistance with Assessments, including Those for Students with Disabilities and LEP Students

Education provided technical assistance to states in a variety of ways. In our survey, states  reported they most often used written guidance and Education-sponsored  meetings and found these helpful. The Education official who manages the standards and assessments peer  review process said that in some cases the differences between decision  letters and peer reviewers’ written comments led to state officials being  unclear about whether they were required to address the issues in  Education’s decision letters, comments from peer reviewers, or both. However, Education  has not made assessment security a focus of its peer review process and  has not incorporated best practices in assessment security into its peer  review protocols. Education agreed with our recommendations to develop methods to  improve communication during the review process and to identify for  states why its peer review decisions in some cases differed from peer  reviewers’ written comments. Appendix I: Objectives, Scope, and Methodology

The objectives of this study were to answer the following questions: (1)  How have state expenditures on assessments required by the Elementary  and Secondary Education Act of 1965 (ESEA) changed since the No Child  Left Behind Act of 2001 (NCLBA) was enacted in 2002, and how have  states spent funds? (3) What challenges, if any, have states faced in ensuring the  validity and reliability of their ESEA assessments? Describing the Extent to Which Education Has Supported State Efforts to Comply with ESEA Assessment Requirements

To address the extent of Education’s support of ESEA assessment  implementation, we reviewed Education guidance, summaries of  Education assistance, peer review training documents, and previous GAO  work on peer review processes. 2. 4.
Why GAO Did This Study

The No Child Left Behind Act of 2001 (NCLBA) requires states to develop high-quality academic assessments aligned with state academic standards. Education has provided states with about $400 million for NCLBA assessment implementation every year since 2002. GAO examined (1) changes in reported state expenditures on assessments, and how states have spent funds; (2) factors states have considered in making decisions about question (item) type and assessment content; (3) challenges states have faced in ensuring that their assessments are valid and reliable; and (4) the extent to which Education has supported state efforts to comply with assessment requirements. GAO surveyed state and District of Columbia assessment directors, analyzed Education and state documents, and interviewed assessment officials from Maryland, Rhode Island, South Dakota, and Texas and eight school districts in addition to assessment vendors and experts.

What GAO Found

States reported their overall annual expenditures for assessments have increased since passage of the No Child Left Behind Act of 2001 (NCLBA), which amended the Elementary and Secondary Education Act of 1965 (ESEA), and assessment development was the largest expense for most states. Forty-eight of 49 states that responded to our survey said that annual expenditures for ESEA assessments have increased since NCLBA was enacted. Over half of the states reported that overall expenditures grew due to development of new assessments. Test and question--also referred to as item--development was most frequently reported by states to be the largest ESEA assessment expense, followed by scoring. State officials in selected states reported that alternate assessments for students with disabilities were more costly than general population assessments. In addition, 19 states reported that assessment budgets had been reduced by state fiscal cutbacks. Cost and time pressures have influenced state decisions about assessment item type--such as multiple choice or open/constructed response--and content. States most often chose multiple choice items because they can be scored inexpensively within tight time frames resulting from the NCLBA requirement to release results before the next school year. State officials also reported facing trade-offs between efforts to assess highly complex content and to accommodate cost and time pressures. As an alternative to using mostly multiple choice, some states have developed practices, such as pooling resources from multiple states to take advantage of economies of scale, that let them reduce cost and use more open/constructed response items. Challenges facing states in their efforts to ensure valid and reliable assessments involved staff capacity, alternate assessments, and assessment security. State capacity to provide vendor oversight varied, both in terms of number of state staff and measurement-related expertise. Also, states have been challenged to ensure validity and reliability for alternate assessments. In addition, GAO identified several gaps in assessment security policies that were not addressed in Education's review process for overseeing state assessments that could affect validity and reliability. An Education official said that assessment security was not a focus of its review. The review process was developed before recent efforts to identify assessment security best practices. Education has provided assistance to states, but issues remain with communication during the review process. Education provided assistance in a variety of ways, and states reported that they most often used written guidance and Education-sponsored meetings and found these helpful. However, Education's review process did not allow states to communicate with reviewers during the process to clarify issues, which led to miscommunication. In addition, state officials were in some cases unclear about what review issues they were required to address because Education did not identify for states why its decisions differed from the reviewers' written comments.