During 2016, More than Half of 26 Programs Were on Track to Meet Their Schedules and Cost Goals

For the first time since we began our annual assessments of DHS’s major  acquisition programs, all of the programs included in our review had a  department-approved baseline. The  remaining 9 of the 26 programs experienced schedule slips, including 4  that also experienced cost growth. The overall  schedule and cost changes were largely driven by increases experienced  by a few programs. Programs Generally Did Not Meet All KPPs before Deploying Capabilities and Late Requirements Definition May Affect Program Execution

Fourteen of the 26 programs we reviewed deployed capabilities prior to  meeting all of their department-approved KPPs—the most important  requirements that a system must meet to fulfill its purpose. As a result,  DHS faces increased risk of fielding capabilities that do not work as  intended. Program  officials identified multiple reasons that KPPs have not been met, such as  programs had not yet tested the KPPs or KPPs were poorly defined. This timing is counter to  acquisition best practices, and may potentially cause programs to  experience cost growth, schedule slips, and inconsistent performance if  requirements are not firmly established at the time the baseline is set. DHS officials identified several reasons why programs have deployed  capabilities, but not met all of their department-approved KPPs. The policy  requires programs to obtain department-level approval for initial APBs— including KPPs, schedules, and cost goals—at ADE 2A, that is, prior to  gaining full knowledge about the program’s technical requirements. For example, DHS established new  processes for assessing programs’ staffing needs and monitoring major  acquisition program progress. Further, we also found that no programs in our  review had reported performance breaches and that DHS’s policy does  not clearly define at what point not meeting KPPs constitutes a  performance breach. Senior DHS officials acknowledged that the department could better  document these decisions and leadership’s rationale in acquisition  decision memorandums. DHS leadership’s decisions to approve  programs to proceed through the acquisition process without meeting all  acquisition policy instruction requirements may be reasonable in any  given case. However, unless the  rationale for these decisions is documented and communicated through  acquisition decision memorandums, effective oversight and insight into  approval decisions for internal and external stakeholders is limited. Updated DHS Breach Policy Not Clear on Timing for Reporting Performance Breaches

DHS’s March 2016 revised acquisition policy instruction also includes  changes to the department’s breach policy, which applies to programs  that fail to meet any cost, schedule, or performance threshold in a  program’s approved APB. As a result, DHS may be missing opportunities for  oversight and correction of performance issues, and is at risk of fielding  systems that may not work as intended. Recommendations

To mitigate the risk of poor acquisition outcomes and strengthen the  department’s investment decisions, we recommend the Secretary of  Homeland Security direct the Undersecretary for Management to take the  following three actions:  Update the acquisition policy to:
Require that major acquisition programs’ technical requirements are  well defined and key technical reviews are conducted prior to  approving programs to initiate product development and establishing  APBs, in accordance with acquisition best practices. However, in August 2016,  NPPD officials told GAO that they were revising the program’s  operational requirements document as a part of the program’s  re-baseline effort. As of January  2017, DHS’s DOT&E had not independently validated these  results. We assessed the extent to which (1) DHS’s  major acquisition programs are on track to meet their schedule and cost  goals, (2) major acquisition programs are making progress in meeting key  performance parameters (KPP), and (3) DHS has taken actions to  strengthen implementation of its acquisition policy and to improve major  acquisition program outcomes. To determine the extent to which DHS has taken actions to improve major  acquisition program outcomes and to strengthen implementation of its  acquisition policy, we reviewed DHS’s acquisition policy and guidance,  including current and prior versions of the Acquisition Management  Directive Instruction 102-01-001; acquisition decision memorandums  issued in calendar year 2016; and key acquisition documentation for  major acquisition programs, such as APBs, LCCEs, operational  requirements documents, as well as breach notifications and remediation  plans. We assessed DHS’s acquisition management policies, guidance, and  practices against the Standards for Internal Control in the Federal  Government. Coast Guard Acquisitions: Better Information on Performance and  Funding Needed to Address Shortfalls.
Why GAO Did This Study

In fiscal year 2016, DHS planned to invest about $7 billion in major acquisitions. DHS's acquisition activities are on GAO's High Risk List, in part due to program management, requirements, and funding issues.
The Explanatory Statement accompanying the DHS Appropriations Act, 2015 included a provision for GAO to review DHS's major acquisitions. This report, GAO's third annual review, addresses the extent to which (1) DHS's major acquisition programs are on track to meet schedule and cost goals, (2) these programs are meeting KPPs, and (3) DHS has strengthened implementation of its acquisition policy.
GAO assessed DHS's 15 largest acquisition programs that were in the process of obtaining new capabilities as of May 2016, and 11 additional programs that GAO or DHS identified were at risk of poor outcomes. For all 26 programs, GAO reviewed key documentation, assessed performance against baselines established since DHS's 2008 acquisition policy, and met with program officials. GAO also met with DHS acquisition officials and assessed DHS's policies and practices against GAO acquisition best practices and federal internal control standards.

What GAO Found

For the first time since GAO began its annual assessments of the Department of Homeland Security's (DHS) major acquisitions, all 26 programs that were reviewed had a department-approved baseline. During 2016, over half of the programs reviewed (17 of the 26) were on track to meet their initial or revised schedule and cost goals. However, 7 of these 17 programs only recently established baselines, 6 of which operated for several years and deployed capabilities without approved baselines. The remaining 9 programs experienced schedule slips, including 4 that also experienced cost growth. The table shows the schedule and cost changes across all 26 programs reviewed, much of which was driven by changes in a few programs.
As of January 2017, 14 of the 26 programs deployed capabilities before meeting all key performance parameters (KPP)—the most important requirements that a system must meet. As a result, DHS may be deploying much-needed capabilities—such as border surveillance equipment and Coast Guard cutters—that do not work as intended. Programs did not meet KPPs for a variety of reasons, such as KPPs were not yet ready to be tested, systems failed to meet KPPs during testing, or KPPs were poorly defined. Contrary to acquisition best practices, DHS policy requires programs to establish schedule, cost, and performance baselines prior to gaining full knowledge about the program's technical requirements. As a result, DHS programs do not match their needs with available resources before starting product development, which increases programs' risk for cost growth, schedule slips, and inconsistent performance.
In 2016, DHS strengthened implementation of its acquisition policy by, for example, focusing on program staffing needs, requiring programs to obtain department-approval for key acquisition documents, and revising the process for when programs breach their cost goals, schedules, or KPPs. However, DHS could better document leadership's acquisition decisions to improve insight into cases that diverge from policy. For example, DHS approved six programs to proceed through the acquisition life cycle even though required documentation was not comprehensive or had not been approved, as required by DHS's policy. Senior DHS officials told GAO these decisions were also based on discussions held at the programs' formal acquisition reviews, but these considerations were not documented. Federal internal control standards require clear documentation of significant events. DHS leadership's decisions may be reasonable, but unless these decisions are documented, insight for internal and external stakeholders is limited. Furthermore, no programs reported a performance breach, even though some programs had not met KPPs. DHS's policy is not clear on how to determine whether a performance breach has occurred. As a result, DHS lacks insight into potential causes of performance issues that may contribute to poor outcomes.

What GAO Recommends

DHS should ensure that programs define technical requirements before setting baselines; document rationale for key acquisition decisions; and clarify when not meeting KPPs constitutes a breach. DHS concurred with GAO's recommendations.