A cost risk  analysis determines the reliability of a program’s cost estimate by  determining a program’s cost drivers and the risk of cost overruns through  an analysis that links historical schedule information along with technical  issues and uncertainties in schedule and cost. JWST Is Meeting Schedule Commitment with Majority of Testing and Challenging Integration Work Still to Come

The JWST project is currently on schedule with 8.75 months of schedule  reserve remaining. This is a tenuous position  for the project given that it must complete five integration and test  periods, three of which have not yet started. To achieve mission success, the project will  have to address over 100 technical risks and ensure that the project’s  potential areas for mission failure are fully tested and understood before  project launch in October 2018. However, as shown in figure 4 below, the use of  schedule reserve on any element or major subsystem—two of which have  entered integration and testing phases—may reduce the overall project  schedule reserve. While some use of schedule reserve is expected, the  proximity of each element and major subsystem schedule to the critical  path means that the project must prioritize the mitigations when problems  occur. Our prior work has shown that it is in integration and testing  where problems are most likely to be found and as a result, schedules  tend to slip. Additionally, as the project moves further into integration and testing,  events become more serial so flexibility will be diminished. Project Continues to Meet Cost Commitments but Contractor Workforce Increases and Unreliable Contractor Performance Data Pose Management Challenges

The JWST project continued to meet its cost commitments throughout  fiscal year 2015 despite cryocooler delays that used a disproportionate  amount of cost reserves. To help manage the  project and account for new risks since the 2011 replan, JWST project  officials conducted a cost risk analysis of the Northrop Grumman contract. We found that while the cost risk analysis substantially met best  practices, these officials do not plan to periodically update it. Instead, the  project is using risk-adjusted analyses to update and inform its cost  position. Finally, we  also found the project lacks an independent surveillance mechanism for  the data to ensure anomalies are corrected by the contractor before being  incorporated into larger analyses. As a result, the project is relying  partially on unreliable information to inform its cost and schedule  decision-making. However, we found that the risk-adjusted analyses do not serve as an  adequate substitute for an updated cost risk analysis because they are a  simplified version of a cost risk analysis that does not allow the project to  prioritize risks or assign confidence levels to meet key milestones in the  schedule consistent with best practices for cost risk analyses. NASA implemented all of these  recommendations. Because the  project is not going to conduct another cost risk analysis, putting  independent surveillance in place to improve the accuracy of its risk- adjusted analysis—despite its weaknesses relative to the information a  cost risk analysis provides—will provide better information to inform its  decision making. Recommendation for Executive Action

To resolve contractor data reliability issues and ensure that the project  obtains reliable data to inform its analyses and overall cost position, we  recommend that the NASA Administrator direct JWST project officials to  require the contractors to identify, explain, and document all anomalies in  contractor-delivered monthly earned value management reports. GAO staff who made  major contributions to this report are listed in appendix V.

Appendix I: Objectives, Scope, and Methodology

Our objectives were to assess (1) the extent to which technical  challenges have impacted the James Webb Space Telescope (JWST)  project’s ability to meet its schedule commitments, (2) the current cost  status of the JWST project and the primary challenges that may influence  the project’s ability to meet its future cost commitments, and (3) the extent  to which independent oversight provides insight about project risks to  management.
Why GAO Did This Study

JWST is one of NASA's most complex and expensive projects, at an anticipated cost of $8.8 billion. With significant integration and testing scheduled in the 3 remaining years until the planned launch date, the JWST project will need to continue to address many challenges and identify problems, many likely to be revealed during its rigorous testing to come. The continued success of JWST hinges on NASA's ability to anticipate, identify, and respond to these challenges in a timely and cost-effective manner to meet its commitments.
Conference Report 112-284 included a provision for GAO to assess the project annually and report on its progress. This is the fourth such report. This report assesses (1) the extent to which JWST is meeting its schedule commitments and (2) the current cost status of the project, among other issues. To conduct this work, GAO reviewed monthly JWST reports, reviewed relevant policies, conducted independent analysis of NASA and contractor data, and interviewed NASA and contractor officials.

What GAO Found

The National Aeronautics and Space Administration's (NASA) James Webb Space Telescope (JWST) project is meeting its schedule commitments, but it will soon face some of its most challenging integration and testing. JWST currently has almost 9 months of schedule reserve—down more than 2 months since GAO's last report in December 2014—but still above its schedule plan and the Goddard Space Flight Center requirement. However, as GAO also found in December 2014, all JWST elements and major subsystems continue to remain within weeks of becoming the critical path—the schedule with the least amount of schedule reserve—for the overall project. Given their proximity to the critical path, the use of additional reserve on any element or major subsystem may reduce the overall project schedule reserve.
Before the planned launch in October 2018, the project must complete five major integration and test events, three of which have not yet begun. Integration and testing is when problems are often identified and schedules tend to slip. At the same time, the project must also address over 100 technical risks and ensure that potential areas for mission failure are fully tested and understood.
JWST continues to meet its cost commitments, but unreliable contractor performance data may pose a risk to project management. To help manage the project and account for new risks, project officials conducted a cost risk analysis of the prime contract. A cost risk analysis uses information about cost drivers, technical issues, and schedule to determine the reliability of a program's cost estimates. GAO found that while NASA's cost risk analysis substantially met best practices for cost estimating, officials do not plan to periodically update it. Instead, the project is using a risk-adjusted analysis to update and inform its cost position, but this analysis is a simplified version of a cost risk analysis—and not a replacement—and is based on contractor-provided performance data that contains anomalies that render the data unreliable. Further, the project does not have an independent surveillance mechanism, such as the Defense Contract Management Agency, to help ensure data anomalies are corrected by the contractor before being incorporated into larger cost analyses, as GAO recommended in 2012. As a result, the project is relying partially on unreliable information to inform its decision making and overall cost status.

What GAO Recommends

GAO recommends that the JWST project require contractors to identify, explain, and document anomalies in contractor-delivered monthly earned value management reports. GAO continues to believe that its 2012 recommendation to implement formal surveillance to help improve the reliability of contractor-provided data has merit and should be implemented. NASA concurred with the recommendation made in this report.