States Are Implementing Some Performance Measures Using Different Approaches and Report That the Greatest Challenge Is Collecting Data on Technical Skill Attainment and Student Placement

		Flexibility in Law and Guidance Allows for Differences in How States Implement Some Performance Measures and Results in Variation in the Student Outcome Data Education Will Collect

A key feature of Perkins IV—to enhance state and local flexibility in  developing, implementing, and improving career and technical  education—allows for considerable variation in how states implement  some performance measures. 1). Consistent with Perkins IV, Education’s guidance to states also allows for  flexibility. 2). States Face the Most Challenge Collecting Data on Student Technical Skill Attainment and Placement Measures because of Cost and Data Concerns

States reported in our surveys that they face the most difficulty in  collecting student data for two of the performance measures: technical  skill attainment and student placement (see fig. 4). In addition to challenges  due to cost, states are limited in their ability to access accurate and  complete data. According to Education’s inventory of open monitoring findings, as of May  2009, 9 of the 28 open findings were related to accountability and states  failing to submit complete or reliable data. Most states reported that the assistance provided by Education has helped  them implement the performance measures, but that more assistance in  the area of technical skill attainment would be helpful. We found that Education officials were aware of states’ need for additional  assistance and that Education has taken some actions to address these  needs, particularly in the area of technical assessments. Education Relies on the Performance Measures to Gauge the Success of State CTE Programs

State performance measures are the primary source of data available to  Education for determining the effectiveness of CTE programs, and  Education relies on student outcomes reported through these measures to  gauge the success of states’ programs. While Perkins IV requires states to  evaluate their programs supported with Perkins funds, it only requires  states to report to Education—through their state plans—how they intend  to evaluate the effectiveness of their CTE programs. Because only 2 of 11 measures have been implemented and reported on  thus far, Education has little information to date on program outcomes. In our surveys of state CTE directors, nearly half of states (23  states at the secondary level and 21 states at the postsecondary level)  responded that they have conducted or sponsored a study, in the past 5  years, to examine the effectiveness of their CTE programs. Agency Comments

We provided a draft of this report and the electronic supplement to the  Department of Education for review and comment. Education provided  technical comments on the report, which we incorporated as appropriate. Appendix I: Scope and Methodology

To obtain national-level information on states’ implementation of Perkins  IV, we designed and administered two Web-based surveys, at the  secondary and postsecondary levels, to state directors of career and  technical education (CTE) in the 50 states and the District of Columbia. We also interviewed officials  from local recipients of Perkins funds—that is, school districts and  postsecondary institutions. Through our interviews with state and local  officials, we collected information on efforts to implement the Perkins  performance measures and uses of Perkins funding, experiences with  Education’s monitoring and technical assistance, and methods for CTE  program evaluation.
Why GAO Did This Study

The Carl D. Perkins Career and Technical Education Act of 2006 (Perkins IV) supports career and technical education (CTE) in high schools and postsecondary institutions, such as community colleges. Perkins IV established student performance measures at the secondary and postsecondary levels for state agencies, such as state educational agencies, and local recipients, such as school districts, eligible to receive funds. GAO examined (1) how states have implemented the Perkins IV performance measures and what, if any, challenges they have faced in implementing the measures; (2) to what extent the Department of Education (Education) has ensured that states are implementing the new performance measures and supported states in their efforts; and (3) what Education knows about the effectiveness of CTE programs. To collect national-level data, GAO surveyed state CTE directors in the 50 states and District of Columbia between January and April 2009, and received responses from all states and the District of Columbia. To view survey results, click on http://www.gao.gov/special.pubs/gao-09-737sp/index.html . We provided a draft copy of this report to Education for comment. We received technical comments, which we incorporated into the draft where appropriate.

What GAO Found

States are implementing some of the Perkins IV performance measures using different approaches and report that the greatest challenge is collecting data on technical skill attainment and student placement. Flexibility in Perkins IV and Education's guidance permits differences in how states implement the measures. According to our surveys, 34 states at the secondary level and 29 at the postsecondary level intend to adopt Education's recommended use of assessments--such as those for industry certifications--to measure technical skills. States reported that they face the most challenge collecting data on the technical skill attainment and student placement measures because of cost and concerns with their ability to access complete and accurate data. Education ensures states are implementing the Perkins IV accountability requirements through on-site monitoring and off-site document reviews, and supports states through technical assistance and guidance. Monitoring findings were most often related to states failing to submit complete or reliable data, and Education uses its findings to guide the technical assistance it provides to states. States reported that Education's assistance has helped them implement the performance measures, but that more assistance with technical skill attainment would be helpful. Education is aware of states' need for additional assistance and has taken actions to address this, including facilitating a state-led committee looking at technical assessment approaches. State performance measures are the primary source of data available to Education for determining the effectiveness of CTE programs, and Education relies on student outcomes reported through these measures to gauge the success of states' programs. Because only 2 of 11 measures (secondary and postsecondary have 3 measures in common) have been implemented and reported on thus far, Education has little information to date on program outcomes. In addition, Perkins IV does not require states to report to Education the findings of their program evaluations. In our surveys of state CTE directors, nearly half of states responded that they have conducted or sponsored a study to examine the effectiveness of their CTE programs. We reviewed 7 of these studies and found that only 4 were outcome evaluations.