EVM Data for the GMD and Targets and Countermeasures Programs Are Not Sufficiently Reliable

For GMD and Targets and Countermeasures, we determined that the EVM  data were not sufficiently reliable to analyze these contracts’ cost and  schedule performance because of instability in these programs. Without  reliable EVM data, we are unable to identify significant performance  drivers or forecast future cost and schedule performance. Further, when  the baseline on which the work is performed and measured against is no  longer representative of the program of record, program managers and  other decision makers lose the ability to develop constructive corrective  action plans to get the program on track. Further, he intends to report EVM information  to Congress annually. For  example, although the GMD program experienced a $1.3 billion dollar  restructure in 2007, another major restructure beginning in fiscal year 2008  for over $500 million that was completed in fiscal year 2009, and a third  in fiscal year 2010 for over $380 million, the GMD program has not  conducted an IBR since December 2006.DOD’s acquisition policy states  that an IBR is to be conducted within 6 months after contract award,  exercise of contract options, or major modifications to a contract.DCMA  officials told us that the GMD program had an IBR underway following the  restructure that began in fiscal year 2008 and completed in fiscal year  2009, but in May 2009 the program was again redirected and the baseline  review was cancelled. Based on discussions  with and reports issued by DCMA, the Targets and Countermeasures  contractor was unable to update its baseline because of numerous  program changes. In addition, during fiscal year 2009 DCMA identified several issues with the  stability of the Targets and Countermeasures program baseline. For  example, program changes since fiscal year 2008 on one delivery order  included over 20 contract changes to the scope of work or corrective  actions to quality issues. In addition, the schedule and quantity of planned  flight tests changed significantly. Because of the instability in  the baseline and the contractor’s inability to update the baseline with  these frequent changes, we determined the cost performance reports for  2009 do not reflect an appropriate baseline against which to measure cost  and schedule progress. During the  course of our audit, we interviewed DCMA representatives at each of the  contractor sites to understand the basis for the noncompliance  determination and to gain information to help us assess the reliability of  the data. BMDS Prime Contractors Aggregate Analysis Not Appropriate Due to Data Reliability Issues

We are unable in this year’s report to aggregate total projected underruns  or overruns in our analysis of the remaining 12 prime contracts because  we had to exclude the GMD and Targets and Countermeasures programs  due to data reliability issues. Nine of the remaining 12 contracts experienced cost overruns for fiscal  year 2009. Most of the overruns were because of issues with maturing  technologies, immature designs, or other technical issues. For example,  the ABL contractor experienced a failure in some of the system’s optics  which required it to develop and procure new high power optics, delaying  the test schedule and increasing program cost. Recommendation for Executive Action

We recommend the Secretary of Defense direct MDA to resolve prime  contractor data reliability issues by the beginning of fiscal year 2011 and, if  MDA has not resolved the data reliability problems, determine the barriers  preventing resolution and provide a report to Congress on:  the steps MDA is taking to make its contractor data sufficiently  reliable,    how the data reliability issues affect MDA’s ability to provide  oversight of its contractors, and  the effect these issues have on MDA’s ability to report contractor  progress to others, including Congress. DOD concurred with our recommendation to resolve prime contractor  EVM data reliability issues by 2011; however, DOD stated that MDA  considers its fiscal year 2009 prime contractor performance data to be  reliable. The contractor tracks earned value  management (EVM) by making comparisons that inform the program as to  whether the contractor is completing work at the cost budgeted and  whether the work scheduled is being completed on time and then reports  this information on Contract Performance Reports. Appendix III: Scope and Methodology

To examine the progress Missile Defense Agency (MDA) prime  contractors made in fiscal year 2009 in cost and schedule performance, we  examined contractor performance on 14 Ballistic Missile Defense System  (BMDS) element contracts. We  received this documentation through the Defense Contract Management  Agency (DCMA), which performs independent EVM surveillance of MDA  contractors.
Why GAO Did This Study

By law, GAO is directed to assess the annual progress the Missile Defense Agency (MDA) made in developing and fielding the Ballistic Missile Defense System (BMDS). GAO issued its latest assessment of MDA's progress covering fiscal year 2009 in February 2010. This report supplements that assessment to provide further insight into MDA's prime contractor performance for fiscal year 2009. Prime contractors track earned value management (EVM) by making comparisons that inform the program as to whether the contractor is completing work at the cost budgeted and whether the work scheduled is being completed on time. Our analysis of contractor EVM data included examining contract performance reports for 14 BMDS contracts, reviewing the latest integrated baseline reviews, performing extensive analysis of data anomalies, and conducting interviews with Defense Contract Management Agency (DCMA) officials--the independent reviewers of MDA contractor EVM systems.

What GAO Found

Unlike GAO's reports in previous years, GAO was unable to analyze the EVM data for all MDA contracts. GAO determined that the data for the Ground-based Midcourse Defense (GMD) and Targets and Countermeasures programs were not sufficiently reliable to include in our report because of instability in these programs' baselines. When the baseline on which the work is performed and measured against is no longer representative of the program of record, program managers and other decision makers lose the ability to develop constructive corrective action plans to get the program on track. Specifically, without reliable EVM data, GAO was unable to identify significant performance drivers or forecast future cost and schedule performance. Because the two contracts associated with these programs represent half of the budgeted cost at completion for the 14 contracts GAO reviewed, GAO also determined it was not appropriate in this report to aggregate total projected underruns or overruns of the remaining 12 prime contracts as GAO has in prior reports. The GMD prime contractor performance data was not sufficiently reliable to use as the basis for analysis because the contractor was unable to update its baseline to include numerous changes to the program and modifications to the contract. Despite three large restructures since 2007 totaling over $2 billion, the GMD program has not conducted an integrated baseline review since December 2006. DOD acquisition policy states that an integrated baseline review is to be conducted within 6 months after contract award, exercise of contract options, or a major modification to an existing contract. The Director, MDA has taken extra steps to gain insight into the contractor's performance. Further, he intends to report EVM information to Congress annually. Similarly, the EVM data for the Targets and Countermeasures contractor is also not sufficiently reliable to use in our analysis. DCMA identified several issues with the stability of the Targets and Countermeasures program baseline including a large amount of schedule and quantity changes to planned flight tests and over 20 contract changes to the scope of work or corrective actions to quality issues for one of the delivery orders over the course of a year. Because the contractor has not been able to update the established budget in the baseline, the cost performance reports do not reflect an appropriate baseline against which to measure cost and schedule progress. Nine of the remaining twelve contracts experienced cost overruns for fiscal year 2009 mostly because of issues with maturing technologies, immature designs, or other technical issues. For example, the Airborne Laser contractor experienced a failure in some of the system's optics which required it to develop and procure new high power optics, delaying the test schedule and increasing program cost.

What GAO Recommends

GAO recommends that MDA resolve prime contractor EVM data reliability issues by the beginning of fiscal year 2011. If, by this time, MDA has not resolved these issues, the Secretary of Defense should provide a report to Congress on the steps MDA is taking to resolve them. DOD concurred with our recommendation.