Some States Are Not Using All Available Data Sources to Determine Employment Outcomes for TAA Participants

Some states are not using all available data sources to determine TAA  participants’ employment outcomes. Limited IT system capabilities. Labor Makes Some TAA Performance Data Available, but Its Usefulness Is Limited

Labor reports data on TAA petition and certification activity, program  participation, and key performance measures, but this information may  not be useful for gauging current program performance. The information  may be helpful in providing a long-term national picture of program  outcomes, but it represents past, rather than current, performance. Most of  the outcome data reported in a given program year actually reflect  participants who left the program up to 2 years earlier. However, the information Labor makes publicly available may not provide  a clear picture of current TAA performance because, in addition to being  incomplete and perhaps inaccurate, the data represent past performance  and are not consistently reported by type of service, state, or industry. Most States Find TAA Performance Data Useful, but Many Would Like Additional Information

While approximately one-third of the states found TAA performance  information they currently receive from Labor to be greatly useful, some  would like Labor to provide them with additional information to help  manage their program. Nearly half of the 46 states we surveyed told us that  they find the performance information they receive from Labor to be  moderately useful (see fig. Labor Has Taken Steps to Improve the Quality of Performance Data, but Issues Remain

While it has limited authority to hold states accountable, Labor has taken  steps to improve the quality of TAA data states submit, but these steps do  not fully address all issues. It is too  soon to fully assess whether Labor’s efforts have improved data quality,  but most states reported on our survey that Labor’s new requirements  have increased awareness of data quality at the state and local levels. Many states reported that the  changes are burdensome, and some states are experiencing delays in  implementing the changes. Coordinating exit dates improves data quality by avoiding the  problem of counting a participant as unemployed in the program’s  performance measures when, in fact, the participant is still  receiving services in another program and is not ready to be  counted in the performance measures. To help ensure that TAA participant data reported by states are consistent,  complete, and accurate, Labor should    clarify through guidance and other communications with states  that all participants who exit the program should be included in the  TAPR and  the documentation needed to verify the training completion date;    ensure that the core monitoring guide currently under development for  regional office site visits includes guidance for assessing whether  states’ data collection processes for performance reporting capture all  participants; and    provide states with opportunities to share lessons learned with other  states on issues that may affect data quality. GAO staff who made major contributions to  this report are listed in appendix V.

Appendix I: Objectives, Scope, and Methodology

We examined (1) whether the Trade Adjustment Assistance (TAA)  performance data provide a credible picture of the program’s  performance, (2) what TAA performance data Labor makes available to  the public and states and the usefulness of the data for managing the  program, and (3) what Labor is doing to address issues with the quality of  TAA data submitted by states.
Why GAO Did This Study

In the current tight budgetary environment, program performance is likely to be an increasingly significant factor used to help policymakers assess programs and determine funding levels. Given concerns over the quality of performance data for the Trade Adjustment Assistance (TAA) program and the importance of having meaningful information to assess program performance, we examined (1) whether the TAA performance data provide a credible picture of the program's performance, (2) what TAA performance data the Department of Labor (Labor) makes available to the public and states and the usefulness of the data for managing the program, and (3) what Labor is doing to address issues with the quality of TAA data submitted by states.

What GAO Found

The performance information that Labor makes available on the TAA program does not provide a complete and credible picture of the program's performance. Only half the states are including all participants, as required, in the performance data they submit to Labor--states were more likely to report participants who received training than those who received other benefits and services but not training. In addition, many states are not using all available data sources to determine participants' employment outcomes. This may result in lower reported outcomes because states may be inaccurately recording some workers as unemployed who actually have jobs. To compile their performance data, some states are using manual processes or automated systems that lack key capabilities to help minimizes errors, but many states have plans to improve their systems' capabilities. Labor reports data on TAA activity levels, services provided to TAA participants, and key performance measures. The performance data may be useful for providing a long-term national picture of program outcomes, but it represents participants who left the program up to 30 months earlier and, thus, is not useful for gauging the TAA program's current performance. Also, the performance information is not displayed using categories that would be informative to policymakers, such as type of service received and industry of dislocation. Most states find the performance information they receive from Labor to be at least moderately useful, but many want more information. Labor has taken steps to improve the quality of TAA performance data, but issues remain. In 2003, Labor began requiring states to validate their data, and most states reported that this increased awareness of data quality at the state and local level. However, the validation process does not address the problem of participants being excluded from the performance data. In fiscal year 2006, Labor instituted a set of common measures, and many states reported they are experiencing delays in implementing all required changes. States also expressed interest in receiving more opportunities to share lessons learned on topics relevant to TAA data quality.