In 1997, the then-Commander of the Air Force’s Air  Combat Command proposed an innovative approach of buying training as  a service, under which the contractors would own, operate, and maintain  the simulator hardware and software. The military services are relying on industry to capitalize the required up- front investment needed to acquire simulator hardware and software, with  the understanding that the contractors will amortize this investment by  selling training services by the hour. The decision to buy simulator training  as a service allowed use of O&M funds, which would alleviate the need to  compete for procurement funds. However, the decision to embark on a services approach was  not supported by a thorough analysis of the costs and benefits, despite a  DOD directive providing that the acquisition of simulators is to be based  on an evaluation of the benefits and trade-offs of potential alternative  training solutions. While Training Has Improved, the Expected Number of Simulator Sites Has Not Been Activated

Air Force and Army officials told us the new simulators are big  improvements over what they had previously. Later, the company cited  Air Force funding problems and schedule slips as the basis for claims  against the Air Force and notified the Air Force that its financial situation  under the contract was no longer viable. The Air Force will let the current  F-16 simulator training contract expire in June 2007 and is in the process  of re-competing the contract, which will likely result in a training gap for  pilots and additional costs to the Air Force. Army  officials told us that, to date, O&M funding for the Flight School XXI  program has not been reduced. Army’s Simulator Activation Schedule Was Also Delayed

The Army has twice rebaselined the activation schedules for the Flight  School XXI simulators—the TH-67 and the advanced aircraft virtual  simulators (AAVS)—as shown in figure 1. Return on the Expenditure of Taxpayer Dollars Is Not Effectively Tracked

The Air Force and the Army are not effectively tracking the return on their  expenditure of taxpayer dollars to acquire simulator training services. The government is paying for activities  conducted during the simulator development period but lacks insight into  what it is actually paying for. Finally, award-term evaluations that were  established to encourage excellent contractor performance do not always  measure key acquisition outcomes such as simulator availability and  concurrency, and can result in additional contract years being awarded for  only “satisfactory” performance. Program  officials told us that, because the Army is contracting for simulator  training to be available, there was no need to track the extent to which the  government is using this availability. Although  the invoices reflect only the discrete tasks, such as training capabilities  assessments, the wide range of invoice amounts—from $91,000 to more  than $6.5 million for similar tasks—and our discussions with contractor  officials suggest that the government is actually making milestone  payments to the contractors for a portion of their up-front costs to acquire  and develop the simulators. Figure 3 shows the development period before the start of simulator  services and the original hourly rate structure under the F-15C contract. Conclusion

Because simulator training had lost out in the internal competition for  procurement funds, the Air Force and Army turned to service contracts,  expecting that O&M funds would be made available to meet requirements. Recommendations for Executive Action

To help ensure that the best approach is used to provide the war-fighter  with needed training, we recommend that the Secretary of Defense direct  the Secretaries of the Air Force and Army to conduct a thorough analysis  of the costs and benefits of using service contracts for simulator training  to determine if it is indeed the best approach. Agency Comments and Our Evaluation

In written comments on a draft of this report, DOD concurred with all but  one of our recommendations. To determine if the Air Force and Army are effectively tracking the return  on their expenditure of taxpayer dollars, we analyzed simulator utilization  data and military service guidance on utilization rates; analyzed contractor  performance measurements, annual evaluations, and award-term plans for  the simulator training contracts; and compared preparatory service costs  charged to the government under each of the four Air Force contracts and  the Army contract.
Why GAO Did This Study

The Air Force has turned to service contracts for the F-15C, F-16, Airborne Warning and Control System, and F-15E, and the Army has done the same for helicopter simulator training at its Flight School XXI. The contractors own, operate, and maintain the simulator hardware and software. The military services rely on industry to capitalize the required up-front investment, with the understanding that the contractors will amortize this investment by selling training services by the hour. GAO was asked to address (1) the factors that led the Air Force and Army to acquire simulator training as a service and whether the decision to use this approach was adequately supported; (2) whether implementation of the approach has resulted in the planned number of simulator training sites being activated; and (3) whether the Air Force and Army are effectively tracking the return on their expenditure of taxpayer dollars. GAO makes recommendations to the Secretary of Defense intended to improve management and oversight of these service contracts to help ensure that the best approach is used to provide the war-fighter with needed training. In written comments on a draft of this report, DOD concurred with all but one of the recommendations, only partially concurring with one pertaining to the Army's simulator utilization rates. GAO continues to believe that the Army needs to track the extent to which it is using simulator availability.

What GAO Found

The Air Force and Army turned to service contracts for simulator training primarily because efforts to modernize existing simulator hardware and software had lost out in the competition for procurement funds. As a result, the simulators were becoming increasingly obsolete. Buying training as a service meant that operation and maintenance (O&M) funds could be used instead of procurement funds. Shifting the responsibility for simulator ownership, operation, and maintenance from the government to the contractor was thought to more quickly enable simulator upgrades to match the changing configurations of aircraft. However, the decision to take a service contract approach was not supported by a thorough analysis of the costs and benefits as compared to other alternatives, despite a Department of Defense directive that provided for such an analysis. While Air Force and Army officials told GAO the new simulators are significant improvements over the previous ones, the expected number of Air Force training sites have not been activated. For the Air Force, O&M funds have not been allocated at the anticipated levels, leading to schedule slippages. The F-16 simulator contractor cited the funding problems and subsequent schedule slippages as the basis for notifying the Air Force that its situation under the contract was no longer financially viable. The Air Force is in the process of re-competing the F-16 training contract, which will likely result in a training gap for pilots--possibly over 2 years--and additional costs to the Air Force. The start date of the Army's flight simulator training was rebaselined twice, but Army officials told us that adequate training was in place for the flight school participants. The return on expenditure of taxpayer dollars is not being effectively tracked in three key ways: Air Force utilization of simulator training frequently falls well below the hours for which the government is paying. The Army is not collecting data on utilization rates at all. The government has little insight into what it is paying for during the development period before training is activated, which can take more than a year. While invoices for preparatory efforts reflect only discrete tasks such as training capabilities assessments, the wide range of invoice amounts and GAO's discussions with contractor representatives suggest that the government is actually making milestone payments to the contractors for a portion of their up-front costs to acquire and develop the simulators. Most of the contracts contain award-term provisions, where the contractors can earn an extension of the contract period for good performance. GAO found that the award-term evaluation factors do not always measure key acquisition outcomes such as simulator availability and concurrency with aircraft upgrades.