FTA’s Project Evaluation Measures Include a Range of Information, but Not All Project Benefits Are Fully Captured

FTA primarily uses the cost-effectiveness and land use criteria to evaluate  New Starts projects, but concerns have been raised about the extent to  which the measures for these criteria capture total project benefits. For example, some  experts and project sponsors we spoke to said that the TSUB measure  does not account for benefits for nontransit users or capture any  economic development benefits that are not directly correlated to mobility  improvements. In these  cases, it is unclear the extent to which FTA’s current approach to  estimating benefits impacts how projects are ranked in FTA’s evaluation  and ratings process. FTA officials acknowledged these limitations, but  noted that improvements in local travel models are needed to resolve  some of these issues. Experts and Other Program Stakeholders Expressed Concerns That FTA’s Current Evaluation Measures Could Be Underestimating Total Project Benefits

Project sponsors and experts we interviewed raised concerns about how  FTA uses and measures different New Starts project justification criteria  in the evaluation framework, which could potentially result in certain  project benefits being underestimated. FTA Faces Several Systemic Challenges to Improving the New Starts Program

FTA faces several systemic challenges to improving the New Starts  program, including addressing multiple program goals, limitations of local  travel models, the need to maintain the rigor while minimizing the  complexity of the evaluation process, and developing clear and consistent  guidance for incorporating qualitative information into the evaluation  process. Additionally, models used to generate local travel demand forecasts have  limited capabilities and may not provide all of the information needed to  properly evaluate transit projects. The presence of  multiple program goals within the statute, as articulated by the evaluation  criteria, has led to different interpretations by FTA and project sponsors  about what project benefits should be emphasized in the New Starts  evaluation process. However, FTA and Office of the Secretary officials told us that a  significant investment of resources by all levels of government will likely  be required to overcome current modeling limitations. Different Options for Evaluating Proposed New Starts Projects Exist, but All Have Limitations

Different options for evaluating proposed transit projects exist. One option is to revise the current evaluation  process as proposed by FTA in the August 2007 NPRM and proposed  policy guidance. They generally agreed with FTA’s assumption that  societal benefits from transit projects generally result from user benefits— that is, reductions in the real and perceived cost of travel. In contrast, benefit-cost  analysis would attempt to monetize all benefits and costs, which experts  told us would be a more comprehensive approach to evaluating projects. Recommendations for Executive Action

To improve the New Starts evaluation process and the measures of project  benefits, which could change the relative ranking of projects, we  recommend that the Secretary of Transportation take the following five  actions:  (1) Seek additional resources to improve local travel models in the next  authorizing legislation;  (2) Seek a legislative change to allow FTA to consider the dollar value of  mobility improvements in evaluating projects, developing regulations, or  carrying out any other duties;  (3) Direct the Administrator of FTA to establish a timeline for issuing,  awarding, and implementing the result of its request for proposals on  short- and long-term approaches to measuring highway user benefits from  transit improvements;  (4) Direct the Administrator of FTA to establish a timeline for initiating  and completing its longer-term effort to develop more robust measures of  transit projects’ environmental benefits that are practically useful in  distinguishing among proposed projects, including consultation with the  transit community, and;  (5) Direct the Administrators of FTA and FHWA to collaborate in efforts to  improve the consistency and reliability of local travel models, including  the aforementioned request for proposals on approaches to measuring  highway user benefits. Appendix II: Scope and Methodology

To address our objectives, we reviewed previous GAO reports, FTA’s  existing and proposed New Starts policy guidance, FTA’s August 2007  Notice of Proposed Rulemaking (NPRM) for New Starts, and the  provisions of SAFETEA-LU that address the New Starts program to  identify the information captured by the current and proposed New Starts  project justification criteria.
Why GAO Did This Study

Through the New Starts program, the Federal Transit Administration (FTA) evaluates and recommends new fixed guideway transit projects for funding using the evaluation criteria identified in law. In August 2007, FTA issued a Notice of Proposed Rulemaking (NPRM), in part, to incorporate certain provisions within the Safe, Accountable, Flexible, Efficient, Transportation Equity Act: A Legacy for Users (SAFETEA-LU) into the evaluation process. SAFETEA-LU requires GAO to annually review FTA's New Starts process. This report discusses (1) the information captured by New Starts project justification criteria, (2) challenges FTA faces as it works to improve the New Starts program, and (3) options for evaluating New Starts projects. To address these objectives, GAO reviewed statutes, FTA guidance and regulations governing the New Starts program, and interviewed experts, project sponsors, and Department of Transportation (DOT) officials.

What GAO Found

FTA primarily uses cost-effectiveness and land use criteria to evaluate New Starts projects, but concerns have been raised about the extent to which the measures for these criteria capture total project benefits. FTA's current transportation system user benefits measure, which assesses a project's cost effectiveness, focuses on how proposed projects will improve mobility by reducing the real and perceived cost of travel. FTA told GAO that such mobility improvements are a critical goal of all transit projects. While the literature and most experts that GAO consulted with generally agree with this assertion, they also raised concerns that certain benefits are not captured. As a result, FTA may be underestimating transit projects' total benefits, but it is unclear the extent to which this impacts FTA's evaluation and rating process. FTA officials acknowledged many of these limitations but noted that resolving these issues would be difficult without a substantial investment of resources by all levels of government to improve and update local travel models. FTA faces several systemic challenges to improving the New Starts program, including addressing multiple program goals, limitations in local travel models, the need to maintain the rigor while minimizing the complexity of the evaluation process, and developing clear and consistent guidance for incorporating qualitative information. The evaluation criteria identified in the law reflect multiple goals for the program, which has led to varying expectations between FTA and project sponsors about what types of projects should be funded. Also, models that generate local travel demand forecasts are limited and may not provide all of the information needed to properly evaluate transit projects. FTA has taken steps to mitigate the modeling limitations, such as incorporating proxy measures to account for certain project impacts and developing a request for proposals to improve local travel models so that they can better predict changes in highway user benefits. However, according to FTA officials, the request for proposals is only a first step in improving local travel models, and additional resources are needed. Experts and project sponsors GAO interviewed discussed different options for evaluating proposed transit projects but identified significant limitations of each option. One option is to revise the current New Starts evaluation process as proposed by FTA in the August 2007 NPRM. While some experts GAO spoke to appreciated the rigor of the current evaluation process, others noted that the NPRM may still underestimate total project benefits. For example, FTA's measure of mobility improvements does not account for benefits accruing to highway users, and its measures of environmental benefits may not properly distinguish among projects. Experts also discussed other options for evaluating proposed transit projects, including benefit-cost analysis. Unlike FTA's current evaluation process, benefit-cost analysis would attempt to monetize all benefits and costs, which experts told GAO would be a more comprehensive approach to evaluating projects. FTA is currently prohibited by statute from considering the dollar value of mobility improvements in evaluating projects.