DOL’s Guidance and Limitations in State Information Systems Have Resulted in Inconsistent and Incomplete National Data on WIA Participants

		Flexibility in DOL’s Guidance Continues to Affect the Consistency of Data

DOL’s guidance to states, in the form of Training and Employment  Guidance Letters and Training and Employment Notices, details how  states should collect and report data on participants in the WIA Adult and  Dislocated Worker Programs. The flexibility in the guidance  stems from the flexibility inherent in WIA, which allows states and local  areas to tailor service delivery to their needs. This flexibility, however, results in variations in how  states and local areas report participant data, which makes it challenging  for DOL to aggregate WIA data at the national level. In addition,  DOL’s guidance is open to interpretation, allowing states to define and  report some variables differently, further contributing to inconsistencies in  the data that states report to DOL on these programs. For example,  state officials that we interviewed told us that they funded core services at  their American Job Centers exclusively through WIA, exclusively through  a partner program, or through a blend of both WIA and partner program  funds. DOL Has Increased Its Oversight Efforts and Taken Steps to Improve Data Consistency, but Data Quality Issues Have Not Been Resolved DOL Performed Various Oversight Activities, but It Did Not Consistently Use the Results to Improve the Data

To improve the quality of data on participants in the WIA Adult or  Dislocated Worker Programs, DOL has enhanced its oversight efforts and  introduced new initiatives including having its data contractor produce  quarterly reports on data issues, requiring states to validate their WIA  data on an annual basis, and engaging its regional offices in periodic  reviews of case files from states. DOL requires each state to validate the data it collects and report on  participants in the WIA Adult and Dislocated Worker Programs on an  annual basis, but the findings from these validation efforts have not been  strategically used to identify systemic issues with or to improve the quality  of the data on WIA participants. DOL, however, does not know what effect state data validation efforts  have had on the quality of participant data for the WIA Adult and  Dislocated Worker Programs. Officials from DOL’s national office said that they were  surprised by this, and that data element validation results are available to  states through DOL’s reporting system. DOL’s regional offices review a sample of case files from states as part of  their oversight of the quality of data for the WIA Adult and Dislocated  Worker Programs, but they have not used the results of these reviews to  identify systemic issues with the quality of the data on WIA participants. For example, DOL has provided general technical assistance on data  reporting for the WIA Adult and Dislocated Worker Programs to states,  and officials from three of the eight states said that the assistance  provided by DOL’s regional offices was useful in helping them address  some of the challenges related to data reporting. National and regional DOL data specialists also  said they hold biweekly meetings to discuss data issues. This effort would entail upgrades to state  information systems that may resolve some data reporting issues  currently attributed to limited technological capacities. Conclusions

Collecting and reporting consistent and complete data is important for  program oversight and management and to evaluate the effectiveness of  program activities and services, but it can be difficult when federal  programs are carried out in partnership with states and local areas. DOL  has taken steps to improve the quality of the data on WIA’s Adult and  Dislocated Worker Programs. Moreover, since 2007, two states have been piloting a new  information system that tracks program participants across several of  DOL’s employment and training programs, but DOL does not plan to  evaluate its effects on the quality of the data collected on participants in  the WIA Adult or Dislocated Worker or other programs before it expands  the system to other states. DOL officials did not state  whether they agreed or disagreed with our recommendations. Appendix I: Scope and Methodology

Our objectives were to determine: (1) what factors have affected the  ability to report consistent and complete data on participants in the  Workforce Investment Act (WIA) Adult and Dislocated Worker Programs,  and (2) what actions has the Department of Labor (DOL) taken to improve  the quality of participant data. Within each state, we visited or  contacted at least one American Job Center—formerly known as a one- stop center—or a local workforce board. Appendix II: Data on Participants in the WIA Adult and Dislocated Worker Programs for Program Year 2011

We reviewed the data collected by the Department of Labor (DOL) in the  Workforce Investment Act Standardized Record Data (WIASRD) system  on the number of, characteristics of, and services provided to participants  in the WIA Adult and Dislocated Worker Programs in program year 2011.
Why GAO Did This Study

Having reliable program data is important in effectively managing a program. However, there have been longstanding concerns about the quality of data on job seekers enrolled in the WIA Adult and Dislocated Worker Programs, which rely on states and local areas to track participants and the services they receive. Given these concerns and WIA's anticipated reauthorization, GAO was asked to examine the data on these WIA participants. This report addresses: (1) the factors that have affected the ability to report consistent and complete data on participants in the WIA Adult and Dislocated Worker Programs, and (2) actions that DOL has taken to improve the quality of these data. To conduct this work, GAO reviewed relevant federal laws, regulations, guidance, and documentation from DOL. GAO interviewed officials from DOL's national and regional offices and state and local workforce officials from a nongeneralizable sample of eight states. GAO also analyzed WIA data from program year 2011 to determine the number of, characteristics of, and services provided to WIA participants.

What GAO Found

Flexibility in the Department of Labor's (DOL) data reporting guidance and limitations in some state information systems continue to impair the quality of the data on participants in the Workforce Investment Act (WIA) Adult and Dislocated Worker Programs. The flexibility in the guidance stems from the inherent nature of WIA, which allows states and local areas to tailor program design and service delivery to their needs. As a result, DOL's guidance on collecting and reporting the data allows variation in how some WIA data elements are defined, collected, and reported. Specifically, an American Job Center--formerly known as a one-stop center--can choose to provide certain basic services exclusively through WIA programs, exclusively through a partner program, or through a blend of both WIA and partner programs. However, this flexibility involves variations in data reporting that have contributed to inconsistencies among states regarding when job seekers are counted as WIA participants. Moreover, some aspects of DOL's guidance are open to interpretation, leaving it to states to define variables such as type of training service received, further contributing to data inconsistencies. In addition, some state information systems used to collect and report WIA participant data have limitations that hamper the affected states' ability to report uniform and complete data. For example, data are incomplete to the extent that states may not have information systems that can track participants who access services online without significant staff assistance. Having inconsistent and incomplete data makes it difficult for DOL to compare data on program participants across states or to aggregate the data at a national level.
DOL engages in various oversight activities designed to ensure the accuracy of states' data on participants in the WIA Adult and Dislocated Worker Programs and has taken steps to improve data consistency across states. However, DOL does not consistently use the results of its oversight to identify and resolve systemic data issues nor has it evaluated the effect of oversight on the quality of WIA participant data. Specifically, DOL requires states to validate the data they collect and report on participants in the WIA Adult and Dislocated Worker Programs on an annual basis, but it does not strategically use the findings from this effort to identify systemic data issues or improve the quality of the data. Similarly, although DOL's regional offices review a sample of each state's WIA participant files every few years to assess states' compliance with data reporting and validation requirements, DOL officials said they have not analyzed the findings from the most recent reviews to identify nation-wide reporting issues. DOL has taken steps to improve the consistency of the data by providing general technical assistance to states and local areas and through standardizing the way DOL collects WIA data. For example, since 2007, two states have been piloting a unified reporting system developed by DOL that uses standardized data definitions and is integrated across certain American Job Center programs administered by DOL. However, DOL officials said they have no plans to evaluate the system before expanding it to other states. Without an evaluation, DOL does not know what impact the pilot has had on the quality of WIA participant data.

What GAO Recommends

GAO recommends that DOL take steps to improve the consistency and completeness of data reported across states and to promote a continuous process for improving the data's quality. DOL officials did not agree or disagree with GAO's overall recommendations and detailed how data quality is being addressed primarily through existing efforts. However, GAO believes that the recommendations remain valid as discussed in the report.