While there were references to efficiency in several  different sections of the 2007 and 2008 PART guidance, two PART  questions focused specifically on development of program-level efficiency  measures with annual targets for improvement:  “Does the program have procedures (e.g., competitive sourcing/cost  comparisons, information technology (IT) improvements, appropriate  incentives) to measure and achieve efficiencies and cost effectiveness in  program execution?”
In order to receive a “yes” response for this question, a program was to  have regular procedures in place to achieve efficiencies and cost  effectiveness, and had to have at least one efficiency measure with  baseline and targets. (In some cases, the available information on  the measure was insufficient for us to place it into one of the three  categories, so we labeled these measures as “unclear.”)
As figures 1 and 2 below illustrate, our analysis of the 36 efficiency  measures from our selected programs and a random sample of the  remaining efficiency measures indicates that about half of the efficiency  measures contained typical elements by including both an input and an  output or outcome. Programs Showed Mixed Results in Terms of Improvements in Efficiency and Use of Efficiency Measures for Decision Making

The selected programs that had measures with both elements of a typical  efficiency measure reported mixed results under PART in terms of gains  and losses in efficiency. Forest Service officials said  they reached agreement with OMB to develop an outcome-oriented  efficiency measure based on the cost of improving or maintaining the  condition of watershed acres. GPRA Could Provide a Framework for Structuring a More Strategic Approach to Improving Government Efficiency

The administration has not clearly indicated whether it will continue to  emphasize measuring efficiency at the program level as it did under PART. This provision has not  been fully implemented, however. Further, as the focal point for overall management in the executive  branch, OMB could provide guidance and management and reporting tools  to increase federal agencies’ focus on efficiency improvements. OMB could also support mechanisms to share information and encourage  agency efforts to improve efficiency. OMB has sponsored  various management councils, such as the President’s Management  Council and the Performance Improvement Council, which include  representatives of agencies and serve as forums for information sharing  among agencies and with OMB. Nevertheless, other forms of  measures intended to improve efficiency, such as those focused on  reducing costly error rates, could still provide useful information. Officials for some selected programs we reviewed indicated that the  efficiency measures reported for PART were useful and described ways in  which they used data for efficiency measures, such as to evaluate  proposals from field units, lower the cost of a contract, or make decisions  to shift production. Other officials we interviewed did not find the  measures useful for decision making. Officials for all of the programs  described challenges to developing and using efficiency measures that  were similar to challenges we previously reported on in prior work on  PART and performance measures in general. GAO staff who made major contributions to  this report are listed in appendix V.

Appendix I: Objectives, Scope, and Methodology

The objectives of our review were to examine: (1) the types of efficiency  measures reported through the Program Assessment Rating Tool (PART)  for agency programs overall, and particularly for selected programs in five  selected agencies, focusing on the extent to which they included typical  elements of an efficiency measure; (2) for selected programs, the extent to  which programs reporting efficiency measures through PART have shown  efficiency gains and how programs have used efficiency measures for  decision making; (3) for selected programs, the types of challenges to  developing and using efficiency measures they have faced; and (4) other  strategies that can be used to improve efficiency. Using this information, we determined  whether each of the measures included the program’s inputs (such as cost  or hours worked by employees) as well as its outputs or outcomes.
Why GAO Did This Study

Given record budget deficits and continuing fiscal pressures, the federal government must seek to deliver results more efficiently. The prior Administration sought to improve efficiency under the Program Assessment Rating Tool (PART) by requiring programs to have at least one efficiency measure and procedures for improving efficiency, and show annual efficiency gains. The current administration has also emphasized efficiency in some initiatives. GAO was asked to examine (1) the types of PART efficiency measures and the extent to which they included typical elements of an efficiency measure; (2) the extent to which selected programs showed gains and how they used efficiency measures for decision making; (3) the challenges selected programs faced in developing and using efficiency measures; and (4) other strategies that can be used to improve efficiency. GAO analyzed the 36 efficiency measures in 21 selected programs in 5 agencies and a generalizable sample from the other 1,355 measures governmentwide, reviewed documents and interviewed officials from selected programs, reviewed literature on efficiency, and interviewed experts.

What GAO Found

Under PART, most programs developed an efficiency measure. However, according to GAO's analysis, 26 percent did not include both typical efficiency measure elements--an input (e.g., labor hours or costs) as well as an output or outcome (e.g., the product, service, or result produced). Most frequently missing was the input (69 percent). For example, a measure developed by the National Nuclear Safety Security Administration considered the number of information assets reviewed for certification without considering costs of review. This could result in measures that do not capture efficiency. GAO has previously recommended agencies improve cost information for decision making, but they are in various stages of implementation. However, alternative forms of measurement, such as reducing costly error rates, could still be useful. Of the efficiency measures GAO reviewed that had both typical elements, a similar number reported gains and losses. Officials for some programs stated that the efficiency measures reported for PART were useful, and described ways in which they used the data, such as to evaluate proposals from field units, lower the cost of a contract, or make decisions to shift production. Others did not find the efficiency measures useful because, for example, the program lacked control over key cost drivers, such as contractually required staffing levels, or because of concern that raising output could lower quality. Officials for all of the programs reviewed described challenges to developing and using program-level efficiency measures and performance measures in general. Challenges included interpreting outcome-level efficiency information, such as the cost of improving or maintaining the condition of watershed acres, when factors other than program funding, such as past impacts from mining, affected conditions as well; achieving required annual efficiency gains in cases where a program intervention takes years to implement; and inconsistent or limited guidance and technical assistance from the Office of Management and Budget (OMB) to agencies on how to measure efficiency. A variety of approaches have been used to improve efficiency, including governmentwide reviews, agency restructurings, process and technology improvements, and strategic spending approaches. The Administration has some initiatives along these lines, such as information technology and procurement reforms. The Government Performance and Results Act (GPRA) provides a framework for planning future efficiency gains while maintaining or improving effectiveness and quality of outputs or outcomes. OMB, as the focal point for management in the executive branch, provides guidance and supports information-sharing mechanisms, such as the Performance Improvement Council, which could also be used to create a more strategic and crosscutting focus on agency efforts to improve efficiency. OMB has not clearly indicated whether programs should continue measuring efficiency nor has it emphasized efficiency in its GPRA guidance to agencies.