CBP Does Not Have the Information Needed to Fully Support and Implement Its Plan

CBP does not have the information needed to fully support and  implement its Plan in accordance with DHS and OMB guidance. However, CBP has not    documented the analysis justifying the specific types, quantities, and  deployment locations of border surveillance technologies proposed in  the Plan;    defined the mission benefits or developed performance metrics to  assess its implementation of the Plan; or    developed a plan to assess and address operational issues with the  continuing use of SBInet systems along the highest risk section of the  border that could affect the new Plan’s implementation across the  remainder of Arizona. Without documentation of the analysis justifying the specific types,  quantities, and deployment locations of border surveillance technologies  proposed in the Plan, an independent party cannot verify the process  followed, identify how the AOA was used, determine whether CBP’s use  of the AOA considered the limitations identified by HSI, assess the validity  of the decisions made, or justify the funding requested. Without defining the expected  benefit or quantifying metrics, it will be difficult for CBP to assess the  effectiveness of the Plan as it is implemented. We have previously reported that a solid business case  providing an understanding of the potential return of large investments  can be helpful to decision makers for determining whether continued  investment is warranted. Effective use of existing SBInet systems is  essential for a comprehensive and integrated approach for surveillance  technology along the entire Arizona border. DHS guidance further states that the Post-Implementation Review should  occur when a system has been in operation at least 6 months or  immediately following investment termination, and the Operational  Analysis should be performed annually for information technology  investments in the steady-state or operations and maintenance phase like  SBInet. However, the officials did not determine a level of confidence  around their rough order of magnitude (ROM) estimate, inconsistent with  best practices. CBP’s Cost Estimate Is Substantially Comprehensive and Accurate but Partially Documented and Minimally Credible

Our analysis of CBP’s 10-year life-cycle cost estimate (LCCE) for the  Arizona Border Surveillance Technology Plan (the Plan) found that CBP  did not fully follow best practices for developing a reliable LCCE, which is  at the core of successfully managing a project within cost and affordability  guidelines. CBP’s estimate for the Plan is $1.5 billion. Our guide and OMB guidance emphasize that reliable cost estimates are  important for program approval and continued receipt of annual funding. As a result, our analysis  concluded that CBP’s cost estimate substantially, but not fully, reflected  best practices for comprehensiveness. However, CBP used the ROM estimate to  support its $242-million budget request for fiscal year 2012 because it  lacked the time needed to develop a more robust estimate. It could also help CBP in making decisions for future  technology deployments along the southwest border and provide a sound  basis for assessing and deploying alternative technologies. Because  CBP officials did not conduct a sensitivity analysis and a cost-risk and  uncertainty analysis to determine a level of confidence in the $1.5-billion  life-cycle cost estimate for the Plan, it will be difficult for decision makers  to determine what levels of contingency funding may be needed to cover  risks associated with implementing new technologies along the remaining  Arizona border. Until CBP officials accurately quantify the impacts of the  risks, the budget requests for fiscal year 2012 and beyond may not be  realistic and sufficient to achieve program aims. Recommendations

To increase the likelihood of successful implementation of the Arizona  Border Surveillance Technology Plan and maximize the effectiveness of  technology already deployed, we recommend that the Commissioner of  CBP take the following three steps in planning the agency’s new  technology approach:    ensure that the underlying analyses of the Plan are documented in  accordance with DHS guidance and internal controls standards;    determine the mission benefits to be derived from implementation of  the plan and develop and apply key attributes for metrics to assess  program implementation; and  conduct a post-implementation review and operational assessment of  SBInet, including consideration of the ATEC test results, and assess  the costs and benefits of addressing the issues identified to help  ensure the security of the 53 miles already covered by SBInet and  enhance security on the Arizona border. DHS did concur with the  recommendation that CBP conduct a post-implementation review and  operational assessment of SBInet. Key contributors to this report are listed in  appendix V.

Appendix I: Objectives, Scope, and Methodology

Our objectives were to determine the extent to which (1) U.S. Customs  and Border Protection (CBP) has the information needed to fully support  and implement its Arizona Border Surveillance Technology Plan in  according with Department of Homeland Security (DHS) and Office of  Management and Budget (OMB) guidance, and (2) CBP’s life-cycle cost  estimate for the Arizona Border Surveillance Technology Plan reflects  best practices. Secure Border Initiative Fence Construction Costs.
Why GAO Did This Study

In recent years, nearly half of all annual apprehensions of illegal aliens along the entire Southwest border with Mexico have occurred along the Arizona border. Keeping illegal flows of people and drugs under control remains a top priority for the Department of Homeland Security's (DHS) U.S. Customs and Border Protection (CBP). In 2005, the Secure Border Initiative Network (SBInet) was conceived as a surveillance technology to create a "virtual fence" along the border. After spending nearly $1 billion, DHS deployed SBInet systems along 53 miles of Arizona's border that represent the highest risk for illegal entry. In January 2011, in response to concerns regarding SBInet's performance, cost, and schedule, DHS cancelled future procurements. CBP developed the Arizona Border Surveillance Technology Plan (Plan) for the remainder of the Arizona border. Funding for this Plan for fiscal year 2012 is $242 million. GAO was requested to assess the extent to which CBP (1) has the information needed to support and implement the Plan and (2) estimated life-cycle costs for future investments in accordance with best practices. GAO analyzed Plan documents and cost estimates, compared those estimates with best practices, and interviewed CBP officials.

What GAO Found

CBP does not have the information needed to fully support and implement its Arizona Border Surveillance Technology Plan in accordance with DHS and Office of Management and Budget (OMB) guidance. In developing the Plan, CBP conducted an analysis of alternatives and outreach to potential vendors. However, CBP has not documented the analysis justifying the specific types, quantities, and deployment locations of border surveillance technologies proposed in the Plan. Best practices for developing and managing costs indicate that a business case analysis should be rigorous enough that independent parties can review it and clearly understand why a particular alternative was chosen to support mission requirements. Without documentation of the analysis, there is no way to verify the process CBP followed, identify how the underlying analyses were used, assess the validity of the decisions made, or justify the funding requested for the Plan. CBP officials also have not yet defined the mission benefits expected from implementing the new Plan. GAO has previously reported that a solid business case providing an understanding of the potential return of large investments can be helpful to decision makers for determining whether continued investment is warranted after deployment. Defining the expected benefit could help improve CBP's ability to assess the effectiveness of the Plan as it is implemented. CBP does not intend to assess and address operational issues regarding the effectiveness and suitability of SBInet, steps that could provide CBP with information to help make decisions regarding alternatives for implementing the Plan. OMB guidance suggests that a post-implementation review occur when a system has been in operation for 6 months or immediately following investment termination. Such a review could help CBP make the most effective use of existing SBInet systems that, in connection with the Plan, could build a comprehensive and integrated approach for surveillance technology along the entire Arizona border. CBP's 10-year life-cycle cost estimate for the Plan of $1.5 billion is based on a rough order of magnitude analysis, and agency officials were unable to determine a level of confidence in their estimate as best practices suggest. Specifically, GAO's review of the estimate concluded that the estimate reflected substantial features of best practices, being both comprehensive and accurate, but it did not sufficiently meet other characteristics of a high-quality cost estimate, such as credibility, because it did not identify a level of confidence or quantify the impact of risks. GAO and OMB guidance emphasize that reliable cost estimates are important for program approval and continued receipt of annual funding. In addition, because CBP was unable to determine a level of confidence in its estimate, it will be difficult for CBP to determine what levels of contingency funding may be needed to cover risks associated with implementing new technologies along the remaining Arizona border. Thus, it will be difficult for CBP to provide reasonable assurance that its cost estimate is reliable and that its budget request for fiscal year 2012 and beyond is realistic and sufficient. A robust cost estimate--one that includes a level of confidence and quantifies the impact of risk--would help ensure that CBP's future technology deployments have sufficient funding levels related to the relative risks.

What GAO Recommends

GAO recommends that CBP document the analysis justifying the technologies proposed in the Plan, determine its mission benefits, conduct a post-implementation review of SBInet and determine a more robust life-cycle cost estimate for the Plan. DHS concurred with the recommendations.