Table 1 provides examples of COIs and KPPs for  the NSC. Two of those key  test events are:  Initial Operational Test and Evaluation (IOT&E): This event is meant  to gather the data necessary to resolve COIs, determine an asset’s  operational effectiveness and suitability, and, according to Coast  Guard acquisition guidance, occur prior to a full-rate production  decision. While Initial Operational Testing Revealed Some Major Deficiencies, the NSC Met Most of Its Key Performance Parameters

As we reported in January 2016, IOT&E took place about 2 years later  than planned and after 7 of the 8 planned NSCs were under contract, with  3 operational. Five of the 10 major deficiencies pertained to  the NSC’s weapon systems and cutter boats. In January 2016 we recommended, and the  Coast Guard concurred, that the NSC’s KPPs for the operation of the  cutter boats should be clarified. Following IOT&E, DHS held an acquisition review board (ARB) to  discuss the outcome of IOT&E, which resulted in DHS approving the NSC  program for full rate production in October 2014. The resulting acquisition decision  memorandum (ADM) from October 2014 directed the Coast Guard to  conduct FOT&E and complete three action items: (1) complete testing of  the cybersecurity COI; (2) verify the correction of all major deficiencies,  including the unmet KPPs; and (3) assess the NSC’s cyber-security  capabilities. Table 4 shows the Coast Guard’s plans, as we reported in  January 2016, for resolving the major deficiencies. As we also found in January 2016, while the Coast Guard has plans to  conduct FOT&E for the NSC, it will have accepted the delivery of at least  the 6th NSC before the testing is complete, meaning that the Coast  Guard will be operating 6 NSCs before it has resolved issues from IOT&E  and knows the cutter’s full capabilities. Further, DHS’s directive on test and  evaluation does not include any direction or guidance on FOT&E. Without  updated guidance that establishes timeframes and responsibilities for  completing all testing, the Coast Guard risks encountering the same  scenario with the OPC—and other future DHS assets—that it has  experienced with the NSC. That is, the Coast Guard could continue to  buy assets without having demonstrated their full capabilities in testing. Problems Discovered Outside of IOT&E Are Preventing the Coast Guard from Operating Fully Capable NSCs

As we reported in January 2016, by the time of the spring 2014 IOT&E  event, the Coast Guard had nearly four years of experience operating the  NSCs. The Coast Guard has encountered several issues that require  major retrofits and design changes on the NSC to correct problems  encountered during operations and discovered during test events outside  of IOT&E. In order to minimize cost increases for some  of these changes, the Coast Guard plans to maintain the original  equipment design for the production of the remaining NSCs and plans to  conduct retrofits after accepting delivery of the cutters. In some instances,  replacement equipment is still in the prototype phase. Thus, we recommended, and DHS  agreed, to provide oversight and specify any further actions the NSC  program should take at the conclusion of the studies related to the  propulsion systems. Observations on the FRC and OPC Acquisitions

As the Coast Guard has progressed in its acquisition of cutters, it has  matured its acquisition processes, which has been demonstrated in its  approach with the FRC and OPC programs. The process to date reflects  some lessons learned from the NSC acquisition, for example in the areas  of competition and the schedule for IOT&E. Furthermore, as the $12  billion OPC program moves forward, it may have opportunities to further  incorporate some best practices that we have highlighted in our past work  on shipbuilding. In May 2009, we reported on best  practices that commercial shipbuilders use to ensure that ships are  delivered on time and within budget. We found that before a contract is  signed, a full understanding of the effort needed to design and construct  the ship is reached, enabling commercial buyers and shipbuilders to sign  a contract that fixes the price, delivery date, and ship performance  parameters.
Why GAO Did This Study

The Coast Guard's flagship, 418-foot NSC was first commissioned in 2008. It completed initial testing, an event designed to test all critical systems that are necessary for successful operations, in the spring of 2014, after 7 of the 8 planned cutters were already under contract and 3 ships were operational. GAO has been reviewing the NSC as part of its broader Coast Guard acquisition reviews since 2001.
This statement is primarily based on GAO's January 2016 report on the NSC's initial testing event ( GAO-16-148 ), and addresses issues related to (1) the results of the NSC's initial test event, (2) the Coast Guard's plans for follow-on testing, and (3) the performance of the NSC during operations. The statement also includes GAO's observations on the Coast Guard's acquisition approach for its Fast Response Cutter and Offshore Patrol Cutter programs. GAO reviewed these two programs in June 2014 ( GAO-14-450 ) and April 2015 ( GAO-15-171SP ) and also conducted selected updates on their acquisition status in January 2016. The statement also draws on GAO's prior work on commercial shipbuilding best practices.

What GAO Found

In January 2016, GAO reported that the Navy's Commander, Operational Test and Evaluation Force conducted the initial testing on the National Security Cutter (NSC) in spring 2014, when three of the cutters were already operational. The Navy deemed the NSC operationally effective and suitable. At the same time however, the testing revealed some major deficiencies. Two metrics used to assess an asset in testing are key performance parameters (KPP) and critical operational issues (COI). The NSC met 18 of 19 COIs and 12 of its 19 KPPs. Navy testers found 10 major deficiencies that varied in terms of their effect on the NSC program, including 4 deficiencies related to the NSC's weapon systems and 1 for its cutter boats. The Coast Guard plans to correct most of the NSC's major deficiencies.
Also, as GAO reported, following initial testing, a Department of Homeland Security (DHS) acquisition review board approved the NSC program for full rate production in October 2014. The Coast Guard plans to begin follow-on testing in fall 2016. DHS acquisition guidance does not specify the timing of follow-on testing for its programs or any actions program offices should take in response to the findings of follow-on testing. As a result, future DHS acquisitions risk fielding assets without knowing the full capabilities, as was the case with the NSC.
GAO also found that problems discovered outside of testing are preventing the Coast Guard from operating fully capable NSCs. By the time of initial testing, the Coast Guard had nearly 4 years' experience operating NSCs and has encountered issues that require retrofits. In order to minimize cost increases for some changes, the Coast Guard plans to maintain the original equipment for the production of the remaining NSCs and conduct retrofits after accepting delivery. In some instances, replacement equipment is still in the prototype phase. The identified problems will continue to affect the NSC until retrofits are implemented.
GAO has observed, based on prior work reviewing the Coast Guard's ongoing Fast Response Cutter program and plans for its upcoming Offshore Patrol Cutter program, that the Coast Guard has matured its acquisition process. The process to date reflects some lessons learned from the NSC acquisition, for example in the areas of competition and the schedule for initial testing. Furthermore, as the $12 billion Offshore Patrol Cutter program moves forward, it may have opportunities to further incorporate some best practices that GAO has highlighted in May 2009 ( GAO-09-322 ) and March 2013 ( GAO-13-325 ) on other shipbuilding work. For example, before a contract is signed, best practices call for a full understanding of the effort needed to design and construct the ship to be reached, enabling commercial buyers and shipbuilders to sign a contract that fixes the price, delivery date, and ship performance parameters.

What GAO Recommends

In January 2016 ( GAO-16-148 ), GAO recommended that DHS take several actions to strengthen oversight of test and evaluation of major assets. GAO also recommended that the Coast Guard direct the NSC program to clarify the key performance parameters for cutter boat operations. DHS and the Coast Guard concurred with all of these recommendations.