In  November 2001, DOD identified PBL as the preferred weapons system  support strategy. Most Remaining Programs Used Business Case Analyses That Were Not Comprehensive

Service program officials could provide documentation of the business  case analyses conducted for 14 of the PBL arrangements we reviewed, but  all but 1 of the 14 analyses were missing one or more of the elements  recommended in DOD’s economic analysis instruction. Further, with the  exception of the Army, the services’ have not established effective internal  controls to ensure that the analyses are prepared in accordance with  service policies and guidance. Various other factors, such as  the lack of systems that are supported by both PBL and non-PBL support  arrangements, the lack of sound program baseline information, and  changing operational and materiel conditions, also limited our ability to  assess the impact of PBL implementation on support costs. As a result, DOD cannot be  assured that PBL arrangements will reduce support costs and provide  cost-effective support for DOD systems. However, program officials could not provide the data on which  they based their decision. A Few Indicators Showed That PBL Arrangements Reduced Support Costs for Some Programs

Only a few of programs we reviewed were able to provide some indicators  of reduced weapon system support costs that could be attributed to the  use of a PBL arrangement. Availability of Cost Metrics and Effective Incentives

Finally, many of DOD’s PBL arrangements do not contain cost metrics or  offer specific incentives to encourage reduced costs. DOD Did Not Emphasize the Potential for PBL Arrangements to Reduce Costs or Require the Collection and Reporting of Detailed Contractor Support Cost Data

Although PBL arrangements were included in a DOD pilot program  intended to demonstrate the ability of various initiatives to reduce support  costs, DOD did not emphasize this goal in its guidance or requirements as  it established the concept as the department’s preferred weapon system  support strategy. In general, improved performance was given greater  emphasis, and we found only a few references to cost reduction in DOD’s  guidance on implementing PBL arrangements. Use and Characteristics of Availability Contracts by the United Kingdom’s Ministry of Defence

The United Kingdom’s Ministry of Defence also uses performance-based  arrangements to support its weapon systems. In general, the availability  contracts used by the ministry are significantly longer than those used by  DOD, and the ministry uses an “open book accounting” arrangement to  gain visibility into the contractors’ costs to provide support. Nonetheless, in the future they intend to pursue transfer of inventory  ownership as much as possible. This access is not without  limits. Although DOD’s  guidance recommends using business case analyses to guide decisions  about using PBL arrangements for weapon system support, the DOD  guidance does not require these analyses and almost half of the programs  we reviewed either did not perform a business case analysis or did not  retain documentation of their analysis. Recommendations for Executive Action

To ensure that PBL arrangements are the most cost-effective option for  weapon system support, we recommend that the Secretary of Defense  direct the Under Secretary of Defense (Acquisition, Technology and  Logistics) to take the following five actions:  revise DOD’s Acquisition Directive to require development of a  business case analysis to support the decision-making process  regarding weapon system support alternatives, including PBL;  revise PBL business case analysis guidance to more clearly define  what should be included in a business case analysis and to establish  specific criteria and methods for evaluating PBL support  arrangements, including evaluation at the subsystem and component  levels;  revise PBL business case analysis guidance to more clearly define  when business case analyses should be updated during the weapon  system life cycle;  require that each service revise guidance to implement internal  controls to ensure that program offices prepare and update business  case analyses that are comprehensive and sound; and  require program offices to collect and report cost data for PBL  arrangements in a consistent, standardized format with sufficient  detail to support traditional cost analysis and effective program  management. Thus, reducing costs was a central focus of the adoption of  PBL as DOD’s preferred support strategy. Appendix I: Scope and Methodology

To evaluate (1) the extent to which the Department of Defense (DOD)  used business case analyses to guide decisions regarding performance  based logistics (PBL) arrangements and (2) the impact PBL arrangements  have had on weapon system support costs, we selected a nonprobability  sample of 29 PBL arrangements for weapon system support initiated from  1996 through 2007. In addition, we  asked program officials to identify support cost reductions that occurred  as a result of PBL implementation. We also interviewed officials from programs  identified by the Ministry of Defence as using availability contracts for  weapon system support to identify the characteristics of the specific  arrangements and the impact that the use of these contracts had on  support costs.
Why GAO Did This Study

In 2001, the Department of Defense (DOD) identified performance based logistics (PBL) as the preferred weapon system support strategy. Within DOD, PBL is the purchase of performance outcomes, such as system availability, rather than the purchase of individual elements of logistics support--such as parts, repairs, and engineering support. Although PBL initially arose from efforts to reduce support costs, questions have arisen about whether PBL has reduced support costs as originally intended. GAO was asked to evaluate the extent to which DOD has used business case analyses to guide decisions related to PBL arrangements and the impact PBL arrangements have had on weapon system support costs. In conducting the review, GAO analyzed the implementation of PBL arrangements for 29 weapon system programs. GAO also looked at the use and characteristics of performance-based contracting in the United Kingdom's Ministry of Defence.

What GAO Found

Although DOD's guidance recommends that business case analyses be used to guide decision making regarding the implementation of PBL to provide weapon system support, the services are not consistent in their use of such analyses. About half of the DOD program offices responsible for the 29 PBL arrangements GAO reviewed either did not use a business case analysis or could not provide documentation for significant parts of their analyses. Almost all of the remaining analyses were missing one or more of the recommended elements in DOD's instruction for economic analysis. Finally, business case analyses were often not updated in accordance with service policies and guidance. Program office use of these analyses is inconsistent because DOD only recommends, but does not require, that they be prepared and because DOD's guidance on preparing a business case analysis is not comprehensive and does not adequately specify the criteria to be included. Also, most of the services have not established effective internal controls to ensure that the analyses are prepared or that they provide a consistent and comprehensive assessment. As a result, DOD has implemented PBL arrangements without the benefit of sound analyses that ensure that the chosen approach will provide the most cost-effective support option. While one of DOD's goals in moving toward the use of PBL arrangements was to reduce weapon system support costs, the ability of these arrangements to reduce costs remains unclear 7 years after DOD first identified PBL as the preferred weapon system support strategy. Many DOD program offices that implemented PBL arrangements have limited cost data, and various other factors--such as the lack of business case analyses--further limit an evaluation of the costs of this support strategy. Available data from the programs GAO reviewed indicated mixed results. Although a few programs in GAO's sample provided evidence of some cost reductions, GAO's analysis of the only two systems in its sample that are managed using both a PBL arrangement and a more traditional, non-PBL arrangement indicated that in both cases the PBL arrangement had higher costs. Also, GAO found that certain characteristics of DOD's PBL arrangements--contract length, funding stability, ownership of inventory, and the lack of cost metrics and effective incentives--could limit the ability of and incentive for contractors to reduce support costs. Neither DOD nor the services require detailed cost reporting for PBL arrangements and the lack of detailed cost data hinders DOD's ability to determine whether PBL has reduced support costs as intended. GAO describes the use of performance-based arrangements for weapon system support in the United Kingdom's Ministry of Defence, which the ministry refers to as contracting for availability. The Ministry of Defence began awarding availability contracts as an approach to reduce weapon system support costs, and officials believe that support cost reductions have been achieved as a result of using availability contracts. In general, the availability contracts used are significantly longer than those used by DOD, and the ministry uses an "open book accounting" arrangement to gain visibility into the contractors' costs to provide support.