Over Two-Thirds of the 424 Grantees We Analyzed Did Not Consistently Meet More Federal Requirements From Fiscal Years 2000 through 2008, But Practices Used by Some Grantees Could Improve Other Grantees’ Performance

Our analysis of OTRAK data shows that 51 percent of 424 grantees had  mixed results in terms of meeting more triennial review requirements from  fiscal years 2000 through 2008, as demonstrated by both increases and  decreases in their numbers of findings over three consecutive reviews. One management practice included linking the job  descriptions of pertinent individuals to the triennial review—a practice  that grantee executives said contributed to a culture of accountability. FTA officials stated that during the  workshops, they plan to emphasize ways to identify systemic deficiencies. However, we found a few areas where FTA could improve  the triennial review process by ensuring and documenting grantees receive  a complete review at least once every three years and by coordinating the  schedules of multiple oversight reviews in the same fiscal year. For example, contractors recorded in the triennial review reports  of ten grantees that the drug and alcohol program area was “not reviewed”  during the 2008 triennial review because these grantees had received  special drug and alcohol program reviews in fiscal year 2006. FTA headquarters officials indicated that regional officials and contractors  discuss all 23 areas during the triennial review process and identify  concerns about the grantees. Further,  FTA is aware of the burden oversight reviews place on grantees and works  to limit this burden. FTA’s Two Triennial Review Performance Measures Meet Some, But Not All Key Attributes of Successful Performance Measures

FTA established two performance measures to assess timeliness of steps  in the triennial review program: (1) 80 percent of triennial review findings  are to be closed within 30 days of their due date (“Close Findings Timely”);  and (2) 95 percent of the final triennial review reports are to be issued  within 30 days of the completion of the review (“Issue Reports Timely”). Measurable target. Both program performance measures have unique  names, definitions, and do not overlap. Objectivity. Core program activities. While FTA’s  existing performance measure of Close Findings Timely may increase  adherence to federal requirements by encouraging grantees to address  findings, it does not provide an overall program level assessment of  grantees’ performance over time. However, the measures do not evaluate other governmentwide  priorities, including the outcome of the reviews—the extent to which  grantees are meeting federal requirements, or the quality of the triennial  reviews. FTA has no performance measures for assessing the quality of  the triennial review program. Balance. Although one of the goals of the triennial review program is to assist  grantees in meeting more federal requirements over time, our analysis of  OTRAK data shows many grantees with mixed success in reducing their  numbers of findings over three triennial reviews from fiscal years 2000  through 2008; and 17 percent of the grantees we reviewed met fewer  federal requirements in their latest triennial review than they did in their  two previous reviews. Second, until the Oversight Review Council  performs its role as oversight review coordinator or delegates this role to  another FTA body, uncoordinated, multiple oversight reviews may not  effectively increase FTA’s coverage of grantees and may place undue  burden on a few grantees. 2. To help ensure that a complete performance review and evaluation of  each grantee is conducted at least once every three years, in  accordance with the statutory requirement, revise the contractors’  guide to remove the “not reviewed” category, require an updated  evaluation of areas reviewed in the prior two years of the triennial  review, and describe how contractors should document the triennial  review reports with updates to other reviews, such as special oversight  reviews conducted in the prior two years. 3. 5. To determine FTA’s performance measures for the triennial review and to  what extent FTA’s performance measures meet nine key attributes, we  obtained, reviewed, and analyzed, among other documents: GAO reports  describing the importance of establishing effective performance measures,  GAO guidance on assessing the strengths and weaknesses of federal  agencies’ performance measures and identifying the key attributes of  successful performance measures, FTA triennial review end-of-year  reports, and FTA performance appraisal templates. Programs must have a drug and alcohol testing program in place for all safety-sensitive  employees.
Why GAO Did This Study

The Federal Transit Administration (FTA) oversees about $5.5 billion in federal funds each year to transit agencies serving urban areas (grantee), in part through its triennial review program, which evaluates grantee adherence to federal requirements at least once every 3 years. GAO recommended in a 1998 oversight report that FTA improve the program. The subcommittee requested that GAO review this program. GAO identified (1) the extent to which triennial reviews indicate that grantees met applicable federal requirements from fiscal years 2000 through 2008; (2) the strengths and weaknesses of the triennial review process; and (3) FTA's performance measures for the triennial review and the extent to which they meet key attributes of successful performance measures. GAO addressed these objectives by analyzing oversight data on 424 grantees that had three triennial reviews, reviewing triennial review reports and guidance, assessing FTA's performance measures; and interviewing FTA headquarters and regional officials, contractors who conduct the reviews, and grantees.

What GAO Found

GAO's analysis of FTA's triennial review oversight data found that over two-thirds of the 424 grantees analyzed have not consistently improved overall performance in terms of meeting more federal requirements from fiscal years 2000 through 2008. Fifty-one percent of grantees had mixed results in meeting requirements and 17 percent consistently met fewer requirements; while 31 percent consistently met more requirements--one of the goals of the triennial review program. Executives from three grantees that met most requirements attributed their performance to, among other things, having job descriptions that link employee responsibilities to the triennial review--a practice they said contributed to a culture of accountability. During the same time, grantees had the greatest number of findings in 5 of 23 triennial review areas, including the procurement and drug and alcohol testing areas. While FTA helps grantees address findings, additional efforts to identify the underlying causes and the severity of findings could further benefit grantees. FTA's triennial review program uses some strong management practices--having a well-defined process, using an information system to monitor grantees, and issuing reports timely. Still, two areas could be strengthened. First, while FTA is legislatively required to conduct a complete review of grantees' adherence to federal requirements at least once every 3 years, GAO identified a few instances where documentation does not clearly show that FTA reviewed all requirement areas. For example, 10 triennial review reports for 2008 showed that the drug and alcohol program area was "not reviewed." FTA's practice is to review all areas, regardless of documentation, but because FTA's guidance is not clear about how to document the review of areas where FTA has conducted a related special review in the prior two years, a few grantees may not be reviewed for 5 years. FTA plans to revise its guidance to avoid ambiguity. Second, FTA is aware of the burden oversight reviews place on grantees and works to limit this burden. However, in a limited number of cases, FTA did not coordinate its special oversight reviews with the triennial review schedule, which may place undue burden on a few grantees receiving multiple oversight reviews in the same fiscal year. FTA's two timeliness performance measures for assessing the triennial review program--(1) closing 80 percent of grantees' deficient findings within 30 days of their due date and (2) issuing 95 percent of the final triennial review reports within 30 days of completing a review--meet some, but not all key attributes of successful measures. Although both measures link throughout the organization, have measurable targets, are clearly stated, and do not overlap, the "close findings" measure does not meet the objectivity and reliability attribute. For example, data inaccuracies in past "close findings" data raised questions about the reliability of the measure. Also, both measures do not assess the core program activity to evaluate grantees' performance or governmentwide priorities, such as the quality of the triennial review program, and thus, as a whole, are not balanced, making it difficult for managers to not overemphasize one priority at the expense of others.