Several Key Issues Have Affected States’ Efforts to Ensure the Quality of WIA Performance Data

Three key issues—flexibility in federal guidance, major changes to states’  information technology (IT) systems and limited monitoring efforts—have  compromised states’ early efforts to collect and report WIA performance  data. More than three-fourths of the states told us that they  had made major modifications to their WIA IT systems since  implementation. In a  recent review we found considerable variation in exit practices at the state  and local levels. States Have Taken Steps to Improve the Quality of WIA Performance Data

States have made efforts to address data quality concerns and improve the  quality of WIA performance data. Most states have taken actions to clarify  Labor’s guidance to help local areas determine who should be tracked in  the performance measures. Almost all states reported on our survey that  they have controls for IT systems, such as edit checks or reports to help  screen for errors or missing data. In addition, most states reported to us  that they monitor local areas to ensure data quality and consistency by  assessing local procedures and policies. Over 40 states reported to us that they  provide guidance to help local areas determine which jobseekers should  be tracked—or registered—for WIA and when participants leave—or  exited—services, and therefore get counted in the performance measures. 5). Labor Has Taken Steps to Improve WIA Data Quality, but Some Issues Remain

Labor recently began addressing data quality issues, however, some data  quality issues remain. However, Labor does not currently have methods in place to review states’  data validation efforts and hold states accountable to the data validation  requirements. Labor’s guidance requiring states to implement common  performance measures on July 1, 2005, clarified some key data elements  that had been problematic with regard to the WIA performance measures,  but it does not address all the issues. Further, Labor has some federal  monitoring processes in place but lacks a standard monitoring guide to  address data quality. Labor’s Data Validation Requirements May Be Having Some Positive Effects on States and Local Areas

It is too soon to fully assess whether Labor’s efforts have improved data  quality, however, at least 46 states reported on our survey that Labor’s new  requirements have helped increase awareness of data accuracy and  reliability at the state and local level (see fig. In addition, over 30 states said that the new  requirements have helped them in their monitoring of outcomes and  eligibility. The magnitude of  changes required considerable retooling of states’ IT systems, which had a  negative effect on the integrity of WIA performance data during the initial  years of implementation. Appendix I: Objectives, Scope, and Methodology

We examined (1) the data quality issues that have affected states’ efforts to  collect and report Workforce Investment Act (WIA) performance data; (2)  states’ actions to address them; and (3) the actions the Department of  Labor (Labor) is taking to address data quality issues, and the issues that  remain.
Why GAO Did This Study

Federal programs carried out in partnership with states and localities continually balance the competing objectives of collecting uniform performance data with giving program implementers the flexibility they need. Our previous work identified limitations in the quality of performance data for the key employment and training program--the Workforce Investment Act (WIA). WIA relies on states and localities to work together to track and report on participant outcomes, and it changed the way outcomes are measured. Given the magnitude of changes and the impact such changes can have on data quality, we examined (1) the data quality issues that affected states' efforts to collect and report WIA performance data; (2) states' actions to address them; and (3) the actions the Department of Labor (Labor) is taking to address data quality issues, and the issues that remain.

What GAO Found

Three key issues--flexibility in federal guidance, major changes to states' information technology (IT) systems, and limited monitoring--compromised states' early efforts to collect and report WIA performance data. Labor's initial guidance allowed states and local areas flexibility in deciding which jobseekers to track and when jobseekers leave services and get counted in the measures. As a result, states and local areas have differed on whom they track and for how long. States took various approaches to implement IT systems for meeting WIA reporting requirements. Thirty-nine states reported to us that they made major modifications to their IT systems since WIA was first implemented in 2000. Thirteen of them said the changes resulted in problems affecting data quality, and 5 states are still trying to resolve these problems. In addition, oversight of WIA performance data was insufficient at all levels during early implementation. Almost all states have made efforts to improve the quality of WIA performance data--at least 40 states have controls in their IT systems that capture WIA performance data, such as edit checks or exception reports to help screen for errors or missing data. Forty-three states have taken actions to clarify Labor's guidance and help local areas determine who should be tracked in the performance measures. In addition, most states said they monitor local areas by assessing local procedures and policies. Labor recently began addressing data quality issues, however, some issues remain. In 2004, Labor addressed some data quality concerns by requiring states to validate their data and ensure the accuracy of their performance outcomes. Most states told us that Labor's requirements have increased awareness of data quality at the state and local level. However, Labor does not have methods in place to review states' validation efforts or hold states accountable for complying with its requirements. Labor issued guidance requiring states to implement common performance measures on July 1, 2005, which clarified some key data elements, but does not address all the issues. Labor has some federal monitoring processes in place but lacks a standard monitoring guide to address data quality.