1). 2. 3. The F-35 program’s original  requirements state that it must include a fully functional and effective  logistics system to ensure operational readiness and availability. DOD Is Aware of Risks to ALIS Functionality That Users Have Identified, but Does Not Have a Plan to Prioritize and Address Them as Key Program Milestones Approach

DOD is aware of risks that could affect ALIS but does not have a plan to  prioritize and address them in a holistic manner to ensure that ALIS is  fully functional as the F-35 program approaches key milestones— including Air Force and Navy initial operational capability declarations in  2016 and 2018, respectively, and the start of the program’s full-rate  production in 2019. In addition, pilots and  maintainers at three sites expressed confidence in ALIS’s future  capabilities as the system continues to improve. However, during our focus-group sessions, ALIS users also identified  several issues, which, if not addressed, could result in operational and  schedule risks. DOD is aware of these risks and, as  discussed later, is addressing risks on a case-by-case basis. The Marine  Corps, which often deploys to austere locations, did not conduct  deployability tests prior to declaring initial operational capability in July  2015. Although  the more deployable version of ALIS was fielded in summer of 2015,  DOD has yet to complete comprehensive deployability testing. Currently, ALIS information, including data from all U.S.   F-35 sites, flows from the Standard Operating Units (SOU) to a single  national Central Point of Entry, and then to the lone Autonomic  Logistics Operating Unit (ALOU). The F-35 program office has taken some actions  in an attempt to address smaller ALIS functionality issues between major  software upgrades, and is considering the procurement of additional ALIS  servers to add redundancy to the system. Although key milestones—such as Air  Force and Navy initial operational capability declarations and the start of  full-rate production—are quickly approaching, DOD does not have a plan  that prioritizes ALIS risks to ensure that the most important are  expediently addressed and that ALIS is fully functional by these  milestones. By continuing to  respond to issues on a case-by-case basis rather than in a holistic  manner, there is no guarantee that DOD will address the highest risks,  and as a result, DOD may encounter further schedule and development  delays, including system upgrades, which could affect operations and  potentially lead to cost increases. DOD Has Estimated the Costs of ALIS, but Including Additional Analyses and Information Would Increase the Credibility and Accuracy of Its Estimate

DOD has estimated total ALIS costs to be approximately $16.7 billion  over its 56-year life cycle. In addition, credible cost estimates should  include sensitivity analyses to examine how changes to individual  assumptions and inputs affect the estimate as a whole. In  lieu of the analyses, program officials stated that they assume that if ALIS  does not perform as planned, aircraft could not be flown as frequently as  intended, and this lower-than-expected utilization rate would therefore  decrease sustainment costs. DOD stated  that the department considers the sensitivity analyses that the F-35  program office performs to be a form of uncertainty analysis, as described  in DOD’s Cost Assessment and Program Evaluation Operating & Support  Cost Estimating Guide; however, the DOD cost estimating guidance does  not require DOD to conduct a sensitivity or uncertainty analysis on ALIS  since DOD does not consider ALIS a major cost driver of the F-35  program. As our report states, according to GAO’s Cost Estimating and  Assessment Guide, cost estimates should include uncertainty analyses to  determine the level of uncertainty associated with the estimate in order to  be credible. For example, a 2013  DOD-commissioned plan found that any functionality problems or  schedule slippage with ALIS will have a significant impact on costs—with  downstream additional costs due to performance and schedule delays  potentially reaching up to $20-100 billion. DOD partially concurred with the recommendation that the Secretary of  Defense direct the F-35 Program Executive Officer to ensure that future  estimates of ALIS costs use historical data as available and reflect  significant program changes consistent with cost-estimating best  practices identified in GAO’s Cost Estimating and Assessment Guide. Appendix I: Scope and Methodology

To determine the extent to which the Department of Defense (DOD) has a  plan to ensure the Autonomic Logistics Information System (ALIS) is fully  functional as the key F-35 program milestones approach, we reviewed  documentation of program plans with relevant sustainment elements  including the F-35 Global Sustainment Plan, the Weapon System  Planning Document, the F-35 Autonomic Logistics Global Sustainment  Concept of Operations, and the F-35 Operational Requirements  Document. To determine the extent to which DOD has credibly and accurately  estimated ALIS costs, we evaluated the reliability of DOD’s estimate of  ALIS costs contained in the F-35 program office’s 2014 estimate of F-35  operating and support (O&S) costs, the most up-to-date estimate  completed at the time of our review.
Why GAO Did This Study

The F-35 is the most ambitious and expensive weapon system in DOD's history, with sustainment costs comprising the vast majority of DOD's $1.3 trillion cost estimate. Central to F-35 sustainment is ALIS—a complex system supporting operations, mission planning, supply-chain management, maintenance, and other processes. The F-35 program is approaching several key milestones: the Air Force and Navy are to declare the ability to operate and deploy the F-35 in 2016 and 2018 respectively, and full-rate production of the aircraft is to begin in 2019. However, ALIS has experienced developmental issues and schedule delays that have put aircraft availability and flying missions at risk. The National Defense Authorization Act for Fiscal Year 2016 included a provision that GAO review the F-35's ALIS. This report assesses, among other things, the extent to which DOD has (1) a plan to ensure that ALIS is fully functional as key program milestones approach and (2) credibly and accurately estimated ALIS costs. GAO reviewed F-35 program documentation, interviewed officials, and conducted focus groups with ALIS users.

What GAO Found

The Department of Defense (DOD) is aware of risks that could affect the F-35's Autonomic Logistics Information System (ALIS), but does not have a plan to ensure that ALIS is fully functional as key program milestones approach. ALIS users, including pilots and maintainers, in GAO's focus groups identified benefits of the system, such as the incorporation of multiple functions into a single system. However, users also identified several issues that could result in operational and schedule risks. These include the following:
ALIS may not be deployable : ALIS requires server connectivity and the necessary infrastructure to provide power to the system. The Marine Corps, which often deploys to austere locations, declared in July 2015 its ability to operate and deploy the F-35 without conducting deployability tests of ALIS. A newer version of ALIS was put into operation in the summer of 2015, but DOD has not yet completed comprehensive deployability tests.
ALIS does not have redundant infrastructure : ALIS's current design results in all F-35 data produced across the U.S. fleet to be routed to a Central Point of Entry and then to ALIS's main operating unit with no backup system or redundancy. If either of these fail, it could take the entire F-35 fleet offline.
DOD is taking some steps to address these and other risks such as resolving smaller ALIS functionality issues between major software upgrades and considering the procurement of additional ALIS infrastructure but the department is attending to issues on a case-by-case basis. DOD does not have a plan that prioritizes ALIS risks to ensure that the most important are expediently addressed and that DOD has a fully functional ALIS as program milestones draw close. By continuing to respond to issues on a case-by-case basis rather than in a holistic manner, there is no guarantee that DOD will address the highest risks by the start of full-rate production in 2019, and as a result, DOD may encounter further schedule and development delays, which could affect operations and potentially lead to cost increases.
DOD has estimated total ALIS costs to be about $16.7 billion over the F-35's 56-year life cycle, but performing additional analyses and including historical cost data would increase the credibility and accuracy of DOD's estimate. GAO's cost estimating best practices state that cost estimates should include uncertainty analyses to determine the level of uncertainty associated with the estimate in order to be credible. In addition, credible cost estimates should include sensitivity analyses to examine how changes to individual assumptions and inputs affect the estimate as a whole. DOD's guidance does not require the department to perform these analyses for ALIS, and DOD officials stated that they have not done so in part because ALIS constitutes less than 2 percent of the F-35's estimated total sustainment costs. Program officials said that if ALIS is not fully functional, the F-35 could not be operated as frequently as intended, but a DOD-commissioned plan found that schedule slippage and functionality problems with ALIS could lead to $20-100 billion in additional costs. Without uncertainty and sensitivity analyses, it is unclear how ALIS can affect costs. GAO also found that using historical cost data would make DOD's cost estimate more accurate.

What GAO Recommends

GAO is making four recommendations including that DOD develop a plan to address ALIS risks, and conduct certain analyses and include historical data to improve its ALIS cost estimate. DOD concurred with developing a plan and partially concurred with the cost estimating recommendations, stating that it follows its own guidance. GAO continues to believe the recommendations are valid, as discussed in the report.