In brief, Sentinel is to meet FBI’s  pressing need for a modern, automated capability for investigative case  management and information sharing to help field agents and intelligence  analysts perform their jobs more effectively and efficiently. Important Steps in Soliciting Sentinel Contractor Proposals and Awarding the Prime Contract Were Conducted

The FBI took a number of important steps when soliciting offers from  contractors to lead the development of Sentinel and in evaluating the  offers and making a contract award decision. An Effective Risk Management Process Has Been Defined and Is Being Followed for Sentinel

The FBI has established and is following effective processes for  proactively identifying and mitigating program risks before they have a  chance to become actual cost, schedule, or performance problems. Most Key Tracking and Oversight Activities Being Performed for Sentinel Contractors

The FBI is performing a range of activities to effectively define  expectations for its prime contractor and to measure performance against  and hold the contractor accountable for meeting these expectations. As a result, the FBI cannot adequately ensure that these  support contractors are performing required program management  functions effectively and efficiently. The FBI has taken a number of actions to satisfy these best practices with  respect to the Sentinel prime contractor; however, the bureau has not  done the same in tracking and overseeing the many support contractors  that are performing program management functions. However, it has not, for example, established and applied measurable  performance standards in its support contractors’ statements of work. FBI Policies and Procedures Governing Sentinel Schedule and Cost Estimates Do Not Reflect Important Best Practices

The FBI’s policies and procedures that form the basis for Sentinel’s  schedule and cost estimates are not fully consistent with reliable  estimating practices. While the FBI has issued an IT program management  handbook, related guidance, and tools that define how IT program  schedules and costs are to be estimated, this handbook and related  material do not, for example, address having a historical database of  program schedule and cost estimates to inform future estimates. In  particular, the estimates to date did not include all relevant costs and  could not be verified by supporting documentation. Without well-defined  policies, procedures, and supporting tools for estimating IT programs’  schedules and costs, the reliability of these programs’ respective estimates  is questionable and, in the case of Sentinel, a key basis of informed  investment management is missing. FBI Office of the CIO officials agreed that these practices are not included  in the bureau’s policies and procedures that form the basis for IT program  cost estimates and that they need to be. For example, according to  program officials, none of the estimates was derived using a historical  database reflecting actual and estimated costs on similar programs. Recommendations

To strengthen the FBI’s management of its Sentinel program, we are  recommending that the FBI Director instruct the bureau’s CIO to    work with Sentinel support contractors, where feasible, to establish  and implement performance standards in statements of work relative  to the quality and timeliness of products and the performance of  services and  revise the IT handbook and related guidance to address schedule and  cost estimating best practices that are identified in this report as not  being addressed in FBI policies and procedures and ensure that these  best practices are fully employed on all major IT programs, including  Sentinel. Key contributors to this  report are listed in appendix V.

Appendix I: Objectives, Scope, and Methodology

Our objectives were to determine the FBI’s (1) use of effective practices  for acquiring Sentinel and (2) basis for reliably estimating Sentinel’s  schedule and costs. Best Practices Relevant to Any IT Business Systems Acquisition

		1.
Why GAO Did This Study

The Sentinel program is intended to replace and expand on the Federal Bureau of Investigation's (FBI) failed Virtual Case File (VCF) project and thereby meet the bureau's pressing need for a modern, automated capability to support its field agents and intelligence analysts' investigative case management and information sharing requirements. Because of the FBI's experience with VCF and the importance of Sentinel to the bureau's mission operations, GAO was asked to conduct a series of reviews on the FBI's management of Sentinel. This review focuses on the FBI's (1) use of effective practices for acquiring Sentinel and (2) basis for reliably estimating Sentinel's schedule and costs. To address its objectives, GAO researched relevant best practices, reviewed FBI policies and procedures, program plans and other program documents, and interviewed appropriate program officials.

What GAO Found

The FBI is managing its Sentinel program according to a number of key systems acquisition best practices. For example, the FBI has followed best practices when soliciting offers from contractors to lead the development of Sentinel; it has also followed the practices in evaluating the offers and making a contract award decision. In addition, it has established and is following effective processes to proactively identify and mitigate program risks before they have a chance to become actual cost, schedule, or performance problems. Further, it has taken a range of steps to effectively define expectations for its prime contractor and to measure performance against these expectations and related incentives and hold the contractor accountable for results. However, the bureau has not done the same for one key aspect of tracking and overseeing its program management support contractors. In particular, it has not established performance and product quality standards for these support contractors. According to FBI officials, such standards are not necessary because they monitor their support contractors on a daily basis, including the review and approval of all work products. By not implementing this practice, GAO believes that the FBI's monitoring does not adequately ensure that Sentinel support contractors are performing important program management functions effectively and efficiently. The FBI's policies, procedures, and supporting tools that form the basis for Sentinel's schedule and cost estimates do not adequately reflect key best practices. While the FBI has issued an information technology (IT) program management handbook, related guidance, and tools that define how IT program schedules and costs are to be estimated, this handbook and related material do not, for example, address such key practices as having a historical database of program schedule and cost estimates to inform future estimates. As a result, the reliability of Sentinel's schedule and cost estimates is questionable. GAO's analyses of the Sentinel cost estimates and program officials' statements confirm this. For example, the analyses show that the estimates do not include all relevant costs, such as a technology refresh, and are not grounded in fully documented methodologies or a corporate history of experiences on other IT programs. FBI officials agreed that they need to update their IT program management handbook and related materials to incorporate schedule and cost estimating best practices and to establish a historical database of its estimating experiences on IT programs. Until FBI takes these steps, IT programs, such as Sentinel, are unlikely to have reliable schedule and cost estimates to support informed investment decision making, and their actual progress is unlikely to track closely to estimates.