We reviewed MDA’s use  of a particular type of contract action that authorizes a contractor to begin  work before contract terms, specifications, or price have been agreed  upon. MDA Made Some Progress, but Did Not Meet Many of Its Acquisition Goals, and Has Inconsistently Applied Its Capability Delivery Processes

In fiscal year 2017, MDA made some progress delivering assets,  including BMDS-level capabilities and conducting tests. However, MDA  did not meet many of its goals as expressed in the Ballistic Missile  Defense System Accountability Report for fiscal year 2017, its integrated  master test plan, and master integration plan. Specifically, MDA  continued to deliver interceptors for three elements and successfully  conducted its first test against an intercontinental ballistic missile target. MDA and the  Army have been directed by the Deputy Secretary of Defense to develop  a memorandum of agreement that would guide the transfer of the THAAD  and AN/TPY-2 programs to the Army, and the National Defense  Authorization Act for fiscal year 2018 requires the Secretary of Defense to  transfer the acquisition authority of all missile defense programs that have  received full-rate production authority, which includes THAAD, to the  military departments not later than the date the President’s fiscal year  2021 budget is submitted. However, program  documentation indicates that MDA encountered further challenges in  fiscal year 2017 that required an additional delay to March 2017. A summary of these tests can be found in table 3. Over the past 5 years, the average length of the undefinitized period and  not-to-exceed price for MDA’s undefinitized contract actions have  increased. This undefinitized contract action will allow MDA to continue with the RKV  program despite significant cost, schedule, and performance risks, some  of which the determination and findings document for the RKV  undefinitized contract action acknowledged. As a result, decision makers in Congress have limited insight into how  MDA is handling the risks that come with undefinitized contract actions, or  how the programs enacting these contracts are performing. Specifically, MDA continues to encounter challenges with ensuring that its  models and simulations are accredited for operational testing when they  are used to test BMDS capabilities, resulting in uncertain performance  outcomes in assessments supporting BMDS deliveries. MDA Has Taken Steps to Improve Its Modeling Capabilities, but Most Delivered BMDS Capabilities Were Tested Using Unaccredited Models

Since MDA cannot conduct enough system-level flight testing of the entire  BMDS to completely assess BMDS performance, BMDS decision makers  within MDA, DOD, Congress, and the executive branch use information  from model-based ground tests to evaluate the operational effectiveness  of the BMDS. Figure 7 shows the percentage of accredited models  that were used in the operational assessment of each BMDS capability  delivery in 2015 through 2017. However, we have found that MDA has encountered  similar challenges since 2009. Information about the Accreditation Status and Limitations of Models Used in Operational Assessments Is Not Communicated to Decision Makers

Although the warfighter and other decision makers inside DOD,  Congress, and the executive branch rely on models to provide information  about BMDS effectiveness, MDA’s capability delivery documentation  does not include information about the quality of modeling data. Specifically, MDA’s TCD memos and OCB change packages, which  describe technical capabilities delivered to the warfighter and their  limitations, do not discuss the extent to which the models used to assess  the new capability are verified, validated, and accredited for assessment,  or how ground test results were affected by model limitations. Moreover, its processes for communicating the  extent and limitations of these capabilities can be improved. MDA has recently taken  steps to document in policy its processes for communicating the extent  and limitations of deliveries of integrated capabilities. In its response, DOD concurred with five out of six of our  recommendations, and partially concurred with one. For  additional information on Aegis Weapons System, see appendix II. Missile Defense: Mixed Progress in Achieving Acquisition Goals and  Improving Accountability. 2012.
Why GAO Did This Study

Since 2002, MDA has been developing a Ballistic Missile Defense System that can identify and intercept enemy threats. MDA has received approximately $132 billion and is planning to spend an additional $47.8 billion through fiscal year 2022 to continue its efforts.
The National Defense Authorization Act for Fiscal Year 2012 included a provision that GAO annually assess and report on the extent to which MDA has achieved its acquisition goals and objectives. This report addresses (1) the progress MDA made in achieving fiscal year 2017 goals; (2) the extent to which MDA uses contracting vehicles known as undefinitized contract actions; and (3) the extent to which models provide credible information about the system's operational performance. To do this work, GAO reviewed planned fiscal year 2017 baselines and other documentation and assessed them against baseline reviews and GAO's acquisition best practices guides. In addition, GAO interviewed relevant officials.

What GAO Found

In fiscal year 2017, the Missile Defense Agency (MDA) made mixed progress in achieving its delivery and testing goals.
MDA continued to deliver assets to the military services. However, system-level integrated capabilities, such as some discrimination and integrated cyber defense improvements, were delayed and delivered with performance limitations.
Several programs achieved notable firsts, including the first intercept of an Intercontinental Ballistic Missile. However, one program experienced a failure, and other tests were delayed or deleted.
Moreover, GAO found challenges in MDA's processes for communicating the extent and limitations of integrated capabilities when they are delivered. As a result, warfighters do not have full insight into the capabilities MDA delivers.
GAO found that the average length of the undefinitized period and the not-to-exceed price of MDA's undefinitized contract actions, which authorize contractors to begin work before an agreement on terms, specifications, or price have been agreed upon, have increased over the past 5 years. While MDA policy permits use of undefinitized contracts on a limited basis, GAO and others have found that they can place unnecessary cost risks on the government.
MDA does not completely assess BMDS performance using traditional flight tests. Instead, MDA relies on models, some of which produce data with limited credibility. According to Department of Defense and MDA policy, models used to operationally assess weapons systems must be accredited to ensure they reflect the real-world system. In addition, using unaccredited models increases the risk that test results could be distorted, and leaves decision makers without key information on how the system will perform. While MDA has taken steps to improve its models, it has used many models in system operational ground tests that were not certified for that use (see figure). Additionally, MDA does not communicate model limitations to some decision makers.
Percentage of Accredited Models Used in Operational Assessments of Ballistic Missile Defense System Capability Deliveries, 2015 through 2017

What GAO Recommends

GAO is making six recommendations to, among other things, improve the way MDA communicates capability deliveries; better report information about MDA's use of undefinitized contract actions; and address the challenges MDA has encountered with certifying its test models and communicating limitations of those models. DOD partially concurred with the first recommendation and concurred with the other five. GAO continues to believe the recommendations are valid as discussed in the report.