	CBP’s Program Schedules and Life- cycle Cost Estimates Reflect Some but Not All Best Practices

In our March 2014 report, we assessed OTIA’s schedules as of March  2013 for the IFT, RVSS, and MSC programs and found that these  program schedules addressed some, but not all, best practices for  scheduling. The Schedule Assessment Guide identifies 10 best practices  associated with effective scheduling, which are summarized into four  characteristics of a reliable schedule—comprehensive, well constructed,  credible, and controlled. According to our overall analysis, OTIA at least  partially met the four characteristics of reliable schedules for the IFT and  RVSS schedules (i.e., satisfied about half of the criterion), and partially or  minimally met the four characteristics for the MSC schedule, as shown in  table 1. For example, we reported that the schedule for the IFT program  partially met the characteristic of being credible in that CBP had  performed a schedule risk analysis for the program, but the risk analysis  was not based on any connection between risks and specific activities. DHS concurred with the  recommendation and stated that OTIA plans to ensure that scheduling  best practices are applied as far as practical when updating the three  programs’ schedules. Further, in March 2014 we reported that CBP has not developed an  Integrated Master Schedule for the Plan in accordance with best  practices. Rather, OTIA has used the separate schedules for each  individual program (or “project”) to manage implementation of the Plan. OTIA officials stated that an Integrated Master Schedule for the  overarching Plan is not needed because the Plan contains individual  acquisition programs as opposed to a plan consisting of seven integrated  programs. However, collectively these programs are intended to provide  CBP with a combination of surveillance capabilities to be used along the  Arizona border with Mexico. Moreover, while the programs themselves  may be independent of one another, the Plan’s resources are being  shared among the programs. According to schedule best  practices, an Integrated Master Schedule that allows managers to monitor  all work activities, how long the activities will take, and how the activities  are related to one another is a critical management tool for complex  systems that involve the incorporation of a number of different projects,  such as the Plan. DHS did not concur with this recommendation. In particular, DHS stated  that maintaining an Integrated Master Schedule for the Plan undermines  the DHS-approved implementation strategy for the individual programs  making up the Plan and that a key element of the Plan has been the  disaggregation of technology procurements. Cost-estimating best practices are summarized into four  characteristics—well documented, comprehensive, accurate, and  credible.completed at the time of our review for the IFT and RVSS programs  showed that these estimates at least partially met three of these  characteristics—well documented, comprehensive, and accurate. In  terms of being credible, these estimates had not been verified with  independent cost estimates in accordance with best practices. We  recommended that CBP verify the Life-cycle Cost Estimates for the IFT   Our analysis of CBP’s estimate for the Plan and estimates  and RVSS programs with independent cost estimates and reconcile any  differences. Rather, the Test and Evaluation Master Plan describes CBP’s  plans to conduct a limited user test of the IFT. However,  this approach is not consistent with DHS’s acquisition guidance, which  states that even for commercial off-the-shelf systems, operational test  and evaluation should occur in the environmental conditions in which a  system will be used before a full production decision for the system is  made and the system is subsequently deployed. We recommended that CBP revise the IFT Test and Evaluation Master  Plan to more fully test the IFT program, before beginning full production,  in the various environmental conditions in which IFTs will be used to  determine operational effectiveness and suitability. CBP Has Identified Mission Benefits, but Does Not Capture Complete Data on the Contributions of Its Surveillance Technologies

We reported in March 2014 that CBP has identified the mission benefits  of its surveillance technologies, but does not capture complete data on  the contributions of these technologies, which in combination with other  relevant performance metrics or indicators, could be used to better  determine the contributions of CBP’s surveillance technologies and inform  resource allocation decisions. CBP has identified mission benefits of  surveillance technologies to be deployed under the Plan, such as  improved situational awareness and agent safety. Although CBP has a field within its Enforcement Integrated Database  (EID) for maintaining data on whether technological assets, such as  SBInet surveillance towers, and non-technological assets, such as canine  teams, assisted or contributed to the apprehension of illegal entrants, and  seizure of drugs and other contraband, according to CBP officials, Border  Patrol agents are not required to record these data. Our analysis of apprehension events included instances in which an event had at  least one deportable individual. We recommended that CBP require data on asset assists to be recorded  and tracked within EID and that once these data are required to recorded  and tracked, analyze available data on apprehensions and technological  assists, in combination with other relevant performance metrics or  indicators, as appropriate, to determine the contribution of surveillance  technologies to CBP’s border security efforts. This is a work of the U.S. government and is not subject to copyright protection in the  United States.
Why GAO Did This Study

This testimony summarizes the information contained in GAO's March 3, 2014 report, entitled Arizona Border Surveillance Technology Plan: Additional Actions Needed to Strengthen Management and Assess Effectiveness , GAO-14-368 .

What GAO Found

The Department of Homeland Security's (DHS) U.S. Customs and Border Protection's (CBP) schedules and Life-cycle Cost Estimates for the Arizona Border Surveillance Technology Plan (the Plan) reflect some, but not all, best practices. Scheduling best practices are summarized into four characteristics of reliable schedules—comprehensive, well constructed, credible, and controlled (i.e., schedules are periodically updated and progress is monitored). GAO assessed CBP's schedules as of March 2013 for the three highest-cost programs that represent 97 percent of the Plan's estimated cost. GAO found that schedules for two of the programs at least partially met each characteristic (i.e., satisfied about half of the criterion), and the schedule for the other program at least minimally met each characteristic (i.e., satisfied a small portion of the criterion), as shown in the table below. For example, the schedule for one of the Plan's programs partially met the characteristic of being credible in that CBP had performed a schedule risk analysis for the program, but the risk analysis was not based on any connection between risks and specific activities. For another program, the schedule minimally met the characteristic of being controlled in that it did not have valid baseline dates for activities or milestones by which CBP could track progress.
Source: GAO analysis of CBP data.
Note: Not met—CBP provided no evidence that satisfies any of the criterion. Minimally met—CBP provided evidence that satisfies a small portion of the criterion. Partially met—CBP provided evidence that satisfies about half of the criterion. Substantially met—CBP provided evidence that satisfies a large portion of the criterion. Met—CBP provided complete evidence that satisfies the entire criterion.
Further, CBP has not developed an Integrated Master Schedule for the Plan in accordance with best practices. Rather, CBP has used the separate schedules for each program to manage implementation of the Plan, as CBP officials stated that the Plan contains individual acquisition programs rather than integrated programs. However, collectively these programs are intended to provide CBP with a combination of surveillance capabilities to be used along the Arizona border with Mexico, and resources are shared among the programs. According to scheduling best practices, an Integrated Master Schedule is a critical management tool for complex systems that involve a number of different projects, such as the Plan, to allow managers to monitor all work activities, how long activities will take, and how the activities are related to one another. Developing and maintaining an Integrated Master Schedule for the Plan could help provide CBP a comprehensive view of the Plan and help CBP better understand how schedule changes in each individual program could affect implementation of the overall Plan.
Moreover, cost-estimating best practices are summarized into four characteristics—well documented, comprehensive, accurate, and credible. GAO's analysis of CBP's estimate for the Plan and estimates completed at the time of GAO's review for the two highest-cost programs showed that these estimates at least partially met three of these characteristics: well documented, comprehensive, and accurate. In terms of being credible, these estimates had not been verified with independent cost estimates in accordance with best practices. Ensuring that scheduling best practices are applied to the three programs' schedules and verifying Life-cycle Cost Estimates with independent estimates could help better ensure the reliability of the schedules and estimates.
CBP did not fully follow key aspects of DHS's acquisition management guidance for the Plan's three highest-cost programs. For example, CBP plans to conduct limited testing of the highest-cost program—the Integrated Fixed Tower (IFT: towers with cameras and radars)—to determine its mission contributions, but not its effectiveness and suitability for the various environmental conditions, such as weather, in which it will be deployed. This testing, as outlined in CBP's test plan, is not consistent with DHS's guidance, which states that testing should occur to determine effectiveness and suitability in the environmental conditions in which a system will be used. Revising the test plan to more fully test the program in the conditions in which it will be used could help provide CBP with more complete information on how the towers will operate once they are fully deployed.
CBP has identified mission benefits for technologies under the Plan, but has not yet developed performance metrics. CBP has identified such mission benefits as improved situational awareness and agent safety. Further, a DHS database enables CBP to collect data on asset assists, defined as instances in which a technology, such as a camera, or other asset, such as a canine team, contributed to an apprehension or seizure, that in combination with other relevant performance metrics or indicators, could be used to better determine the contributions of CBP's surveillance technologies and inform resource allocation decisions. However, CBP is not capturing complete data on asset assists, as Border Patrol agents are not required to record and track such data. For example, from fiscal year 2010 through June 2013, Border Patrol did not record whether an asset assist contributed to an apprehension event for 69 percent of such events in the Tucson sector. Requiring the reporting and tracking of asset assist data could help CBP determine the extent to which its surveillance technologies are contributing to CBP's border security efforts.
This is a public version of a For Official Use Only—Law Enforcement Sensitive report that GAO issued in February 2014. Information DHS deemed as For Official Use Only—Law Enforcement Sensitive has been redacted.