Testing of Critical Software Poses a Significant Challenge to the Delivery of F- 35 Warfighting Capabilities

Delays in the testing of critical mission systems software have put the  delivery of expected warfighting capabilities to the Marine Corps at risk,  and could affect the delivery of capabilities to the Air Force and Navy as  well. F-35 developmental flight testing is separated into two key areas:  mission systems and flight sciences. Mission systems testing is done to  verify that the software and systems that provide critical warfighting  capabilities function properly and meet requirements, while flight science  testing is done to verify the aircraft’s basic flying capabilities. In a March  2013 report we found that development and testing of mission systems  software was behind schedule, due largely to delayed software deliveries,  limited capability in the software when delivered, and the need to fix  problems and retest multiple software versions. The Director of Operational Test and Evaluation  (DOT&E) predicts that the delivery of expected warfighting capabilities to  the Marine Corps could be delayed by as much as 13 months. Delays of  this magnitude could also increase the already significant concurrency  between testing and aircraft procurement and result in additional cost  growth. Although mission systems testing is behind, the F-35 program  has been able to accomplish nearly all of its planned flight science  testing. At this time it is not clear exactly which of the expected  capabilities will be available as testing is still ongoing. However, given the uncertainty in mission systems software  testing and the significance of the F-35 to future force structure plans, it is  important that the military services have a clear understanding of the  specific capabilities that they can realistically expect to receive, and those  capabilities that are not likely to be delivered by their initial operational  capability dates—the first of which is scheduled for July 2015. In addition,  because the F-35 is DOD’s most costly and ambitious acquisition  program, it is important that DOD keep Congress informed about the  status of the program. Without a clear understanding of the specific  capabilities that will initially be delivered, Congress and the military  services may not be able to make fully informed resource allocation  decisions. The F-35 program is also scheduled to begin operational  testing in June 2015, to determine that the aircraft variants can effectively  perform their intended missions in a realistic threat environment. Program Progress in Addressing Key Technical Risks Has Been Mixed

While the F-35 program made progress addressing some key technical  risks in 2013, it continued to encounter slower than expected progress in  developing the Autonomic Logistics Information System (ALIS). The program is reducing unit costs to meet targets, but a significant  amount of additional cost reduction is needed if it expects to meet those  targets before the beginning of full rate production—currently scheduled  for 2019. Additionally, the most recent life-cycle sustainment cost  estimate for the F-35 fleet is more than $1 trillion, which DOD officials  have deemed unaffordable. The program’s long term sustainment  estimates reflect assumptions about key cost drivers that the program  does not control, including fuel costs, labor costs, and inflation rates. From fiscal year 2014 through fiscal  year 2037, the program projects that it will require, on average,  development and procurement funding of $12.6 billion per year, with  several peak years at around $15 billion. Estimated Sustainment Costs Remain High but Emphasis on Improving Reliability Could Reduce Costs

In addition to the concerns about the affordability of the F-35 acquisition  program, there are also significant concerns about the cost of operating  and supporting the F-35 fleet over the coming decades. As the number of aircraft in production has  increased, learning has taken place and manufacturing efficiency has  improved. For example, the prime contractor has seen reductions in  overall labor hours needed to manufacture the aircraft. In 2013, the contractor delivered 35 aircraft to the  government, 5 more than it delivered in 2012 and 26 more than it  delivered in 2011. The prime contractor has put in place a supplier  management system to oversee key supplier performance allowing them  to identify poor performers and take appropriate action to address issues  such as part shortages and poor quality. Eventually,  DOD will have to make contingency plans for these and other issues. The results of this  assessment should be shared with Congress and the military services as  soon as possible but no later than July 2015. DOD concurred with our recommendation. DOD agreed with GAO’s  observations. Appendix Il: Scope and Methodology

To assess the program’s ongoing development and testing we reviewed  the status of software development and integration and contractor  management improvement initiatives. We obtained and  analyzed data on flights and test points, both planned and accomplished  during 2013. Joint Strike Fighter: Progress Made and Challenges Remain.
Why GAO Did This Study

The F-35 Lightning II, also known as the Joint Strike Fighter, is DOD's most costly and ambitious acquisition program. The program seeks to develop and field three aircraft variants for the Air Force, Navy, and Marine Corps and eight international partners. The F-35 is integral to U.S. and international plans to replace existing fighter aircraft and support future combat operations. Total U.S. planned investment in the F-35 program is approaching $400 billion to develop and acquire 2,457 aircraft through 2037, plus hundreds of billions of dollars in long-term spending to operate and maintain the aircraft.
The National Defense Authorization Act for Fiscal Year 2010 mandated that GAO review the F-35 acquisition program annually for 6 years. In this, GAO's fifth annual report on the F-35, GAO assesses the program's (1) ongoing development and testing, (2) long-term affordability, and (3) manufacturing progress.
GAO reviewed and analyzed manufacturing data through December 2013, program test plans, and internal DOD analyses, and spoke with DOD, program, and contractor officials.

What GAO Found

Delays in developmental flight testing of the F-35's critical software may hinder delivery of the warfighting capabilities the military services expect. F-35 developmental flight testing comprises two key areas: mission systems and flight sciences. Mission systems testing verifies that the software-intensive systems that provide critical warfighting capabilities function properly and meet requirements, while flight sciences testing verifies the aircraft's basic flying capabilities. Challenges in development and testing of mission systems software continued through 2013, due largely to delays in software delivery, limited capability in the software when delivered, and the need to fix problems and retest multiple software versions. The Director of Operational Test and Evaluation (DOT&E) predicts delivery of warfighting capabilities could be delayed by as much as 13 months. Delays of this magnitude will likely limit the warfighting capabilities that are delivered to support the military services' initial operational capabilities—the first of which is scheduled for July 2015—and at this time it is not clear what those specific capabilities will be because testing is still ongoing. In addition, delays could increase the already significant concurrency between testing and aircraft procurement and result in additional cost growth. Without a clear understanding of the specific capabilities that will initially be delivered, Congress and the military services may not be able to make fully informed resource allocation decisions. Flight sciences testing has seen better progress, as the F-35 program has been able to accomplish nearly all of its planned test flights and test points. Testing of the aircraft's operational capabilities in a realistic threat environment is scheduled to begin in 2015. The program has continued to make progress in addressing some key technical risks.
To execute the program as planned, the Department of Defense (DOD) will have to increase funds steeply over the next 5 years and sustain an average of $12.6 billion per year through 2037; for several years, funding requirements will peak at around $15 billion. Annual funding of this magnitude clearly poses long-term affordability risks given the current fiscal environment. The program has been directed to reduce unit costs to meet established affordability targets before full-rate production begins in 2019, but meeting those targets will be challenging as significant cost reductions are needed. Additionally, the most recent cost estimate for operating and supporting the F-35 fleet is more than $1 trillion, which DOD officials have deemed unaffordable. This estimate reflects assumptions about key cost drivers the program can control, like aircraft reliability, and those it cannot control, including fuel costs, labor costs, and inflation rates. Reliability is lower than expected for two variants, and DOT&E reports that the F-35 program has limited additional opportunities to improve reliability.
Aircraft manufacturing continued to improve in 2013, and management of the supply chain is evolving. As the number of aircraft in production has increased, critical learning has taken place and manufacturing efficiency has improved. For example, the prime contractor has seen reductions in overall labor hours needed to manufacture the aircraft, as expected. In 2013, the contractor delivered 35 aircraft to the government, 5 more than it delivered in 2012 and 26 more than it delivered in 2011. The prime contractor has put in place a supplier management system to oversee key supplier performance.

What GAO Recommends

GAO recommends that DOD assess and identify the specific capabilities that realistically can be delivered to the military services to support their respective initial operational capabilities, and share its findings with the Congress and military services prior to July 2015. DOD concurred with this recommendation.