The goal of PART is to evaluate programs systematically, consistently, and  transparently. OMB went to great lengths to encourage consistent  application of PART in the evaluation of government programs, including  pilot testing the instrument, issuing detailed guidance, and conducting  consistency reviews. Although there is undoubtedly room for continued  improvement, any tool is inherently limited in providing a single  performance answer or judgment on complex federal programs with  multiple goals. Performance measurement challenges in evaluating complex federal  programs make it difficult to meaningfully interpret a bottom-line rating. OMB published both a single, bottom-line rating for PART results and  individual section scores. In a case like this, the individual  section ratings provided a better understanding of areas needing  improvement than the overall rating alone. Any tool that is sophisticated enough to take into account the complexity  of the U.S. government will always require some interpretation and  judgment. Therefore it is not surprising that OMB staff were not fully  consistent in interpreting complex questions about agency goals and  results. The lack of program performance information also creates challenges in  effectively assessing program performance. As a result, the goals and measures used  in PART must meet OMB’s needs. Federal programs cannot be assessed  in isolation. Targeting PART assessments based  on such factors as the relative priorities, costs, and risks associated with  related clusters of programs and activities addressing common strategic  and performance goals not only could help ration scarce analytic  resources but also could focus decision makers’ attention on the most  pressing policy and program issues. The relationship between PART and its process and the broader GPRA  strategic planning process is still evolving. As part of the executive branch  budget formulation process, PART must clearly serve the President’s  interests. PART offers the potential to build on the  infrastructure of performance plans and information ushered in by GPRA  and the law’s intent to promote the use of these plans in resource  allocation decision making. Continue to improve the PART guidance by (1) expanding the discussion  of how the unit of analysis is to be determined to include trade-offs made  when defining a unit of analysis, implications of how the unit of analysis is  defined, or both; (2) clarifying when output versus outcome measures are  acceptable; and (3) better defining an “independent, quality evaluation.”
Clarify OMB’s expectations to agencies regarding the allocation of scarce  evaluation resources among programs, the timing of such evaluations, as  well as the evaluation strategies it wants for PART, and consider using  internal agency evaluations as evidence on a case-by-case basis—whether  conducted by agencies, contractors, or other parties. Attempt to generate, early in the PART process, an ongoing, meaningful  dialogue with congressional appropriations, authorization, and oversight  committees about what they consider to be the most important  performance issues and program areas that warrant review. GPRA’s strategic planning goals  could be used to anchor the selection and review of programs by providing  a foundation to assess the relative contribution of related programs and  tools to broader performance goals and outcomes.
Why GAO Did This Study

The Office of Management and Budget's (OMB) Performance Assessment Rating Tool (PART) is meant to provide a consistent approach to evaluating federal programs during budget formulation. The subcommittee asked GAO to discuss its overall findings and recommendations concerning PART, based on a recent report, Performance Budgeting: Observations on the Use of OMB's Program Assessment Rating Tool for the Fiscal Year 2004 Budget (GAO-04-174).

What GAO Found

PART helped structure OMB's use of performance information for internal program and budget analysis and stimulated agency interest in budget and performance integration. Moreover, it illustrated the potential to build on GPRA's foundation to more actively promote the use of performance information in budget decisions. OMB deserves credit for inviting scrutiny of its federal program performance reviews and sharing them on its Web site. The goal of PART is to evaluate programs systematically, consistently, and transparently. OMB went to great lengths to encourage consistent application of PART in the evaluation of government programs, including pilot testing the instrument, issuing detailed guidance, and conducting consistency reviews. Although there is undoubtedly room for continued improvement, any tool is inherently limited in providing a single performance answer or judgment on complex federal programs with multiple goals. Performance measurement challenges in evaluating complex federal programs make it difficult to meaningfully interpret a single bottom-line rating. The individual section ratings for each PART review provided a better understanding of areas needing improvement than the overall rating alone. Moreover, any tool that is sophisticated enough to take into account the complexity of the U.S. government will always require some interpretation and judgment. Therefore it is not surprising that OMB staff were not fully consistent in interpreting complex questions about agency goals and results. The lack of program performance information at the agency level also creates challenges in effectively measuring program performance. PART provides an opportunity to consider strategically targeting the assessments on groups of related programs contributing to common outcomes to more efficiently use scarce analytic resources and focus decision makers' attention on the most pressing performance issues cutting across individual programs and agencies. The relationship between PART and the broader GPRA strategic planning process is still evolving and highlights the critical importance of defining the unit of analysis for program evaluation. Although PART can stimulate discussion on program-specific performance measurement issues, it is not a substitute for GPRA's strategic, longer-term focus on thematic goals, and department- and governmentwide crosscutting comparisons. PART clearly serves OMB's needs, but questions remain about whether it serves the various needs of other key stakeholders. If PART results are to be considered in the congressional debate, it will be important for OMB to (1) involve congressional stakeholders early in providing input on the focus of the assessments; (2) clarify any significant limitations in the assessments and underlying performance information; and (3) initiate discussions with key congressional committees about how they can best leverage PART information in congressional authorization, appropriations, and oversight processes.