In addition, we found that DHS guidance did not fully or  substantially incorporate five of GAO’s identified best practices for  conducting an AOA process. Based on these AAs, DHS  and the TRIO components selected the federal SSP alternative as their  preferred choice and subsequently selected IBC as their federal SSP. However, because Coast Guard’s and TSA’s AAs did not fully or  substantially meet all four characteristics of a reliable, high-quality AOA  process, they are at increased risk regarding their decision on the solution  that represents the best alternative for meeting their mission needs. DNDO Substantially, and Coast Guard and TSA Partially, Met Best Practices for Conducting AOAs

Based on the extent to which the DHS TRIO components followed the  GAO-identified 22 best practices for conducting an AOA process, we  found that DNDO’s AA process substantially met the four characteristics  of a reliable, high-quality AOA process while the Coast Guard and TSA  AA processes both substantially met one and partially met three of these  four characteristics. For example, although the guidance  addressed risk management in general terms, it did not detail the need to  document risk mitigation strategies for each alternative. Because of these limitations in guidance, and because Coast Guard and  TSA did not fully adhere to the GAO-identified best practices, Coast  Guard’s and TSA’s AAs did not fully or substantially reflect all four  characteristics of a reliable, high-quality AOA process. In addition,  DNDO determined that migrating to a federal SSP was its best alternative  in May 2013. For  example, a risk associated with IBC’s capacity and experience for  migrating large agencies the size of Coast Guard and TSA was  identified in July 2014. Rather, a contingency plan  working group (CPWG) to address this and other concerns was  established in January 2017, over 2 years after the risk was initially  identified. Key Factors and Challenges Impacting the TRIO Project and DHS’s Path Forward

Although DHS has taken various actions to manage the risks of using IBC  for the TRIO project, including some that were consistent with best  practices, the TRIO project has experienced challenges raising concerns  regarding the extent to which its objectives will be achieved. DHS ranked the  remaining four options using 13 OMB risk factors as selection criteria and  determined that migrating the solution to a DHS data center represented  the best option going forward. Conclusions

The TRIO project represents a key element of DHS’s efforts to address  long-standing deficiencies in its financial management systems and  further improve financial management. Further, DHS did not always follow best practices for  managing the risks of using IBC for the TRIO project. Plans for DHS’s path forward on  the TRIO project, as of May 2017, involve significant changes, such as  transitioning away from IBC and a 2-year delay in completing Coast  Guard’s and TSA’s migration to a modernized solution. (Recommendation 1)
The DHS Under Secretary for Management should improve the Risk  Management Planning Handbook and other relevant guidance for  managing risks associated with financial management system  modernization projects to fully incorporate risk management best  practices, including defining thresholds to facilitate review of performance metrics to  determine when risks become unacceptable;  identifying and analyzing risks to include periodically reconsidering  risk sources, documenting risks specifically related to the lack of  sufficient, reliable cost and schedule information needed to help  properly manage and oversee the project, and timely disposition of  IV&V contractor-identified risks; developing risk mitigation plans with specific risk-handling activities,  the costs and benefits of implementing them, and contingency plans  for selected critical risks; and  implementing risk mitigation plans to include establishing periods of  performance for risk-handling activities and defining time intervals for  updating and certifying the accuracy and completeness of information  on risks in DHS’s risk register. However, the documentation provided by DHS does not fully address our  recommendation. Key contributors to this report are listed in appendix V.

Appendix I: Scope and Methodology

To determine the extent to which the Department of Homeland Security  (DHS) followed best practices in analyzing the alternatives used in  choosing the preferred alternative for modernizing TRIO components’  financial management systems, we reviewed information that the TRIO  components provided as part of their alternatives analysis (AA) process,  referred to as the AA body of work, which includes the AA and other  supporting documentation that is not specifically included in the AA. The best practices were not used to determine whether DHS  made the correct decision in selecting Department of the Interior’s Interior  Business Center (IBC) to implement the financial management systems  modernization solution or whether the TRIO project would have arrived at  a different conclusion had it more fully conformed to these best practices. In particular, the key best practices for preparing for risk management are determine risk sources and categories, define risk parameters, and establish a risk management strategy. To determine the key factors or challenges that have impacted the TRIO  project and DHS’s plans for completing remaining key priorities, we met  with DHS, IBC, Office of Financial Innovation and Transformation, and  Unified Shared Services Management office officials and Office of  Management and Budget staff to obtain their perspectives. 3. Appendix III: GAO Assessment of TRIO Components’ Alternatives Analyses

The Department of Homeland Security’s TRIO components—the U.S.  Coast Guard (Coast Guard), Transportation Security Administration  (TSA), and Domestic Nuclear Detection Office (DNDO)—conducted  alternatives analyses (AA) during 2012 and 2013 to determine the best  alternative for transitioning to a modernized financial management system  solution.
Why GAO Did This Study

To help address long-standing financial management system deficiencies, DHS initiated its TRIO project, which has focused on migrating three of its components to a modernized financial management system provided by IBC, an OMB-designated, federal SSP. House Report Number 3128 included a provision for GAO to assess the risks of DHS using IBC in connection with its modernization efforts.
This report examines (1) the extent to which DHS and the TRIO components followed best practices in analyzing alternatives, and the key factors, metrics, and processes used in their choice of a modernized financial management system; (2) the extent to which DHS managed the risks of using IBC for its TRIO project consistent with risk management best practices; and (3) the key factors and challenges that have impacted the TRIO project and DHS's plans for completing remaining key priorities. GAO interviewed key officials, reviewed relevant documents, and determined whether DHS followed best practices identified by GAO as necessary characteristics of a reliable, high-quality AOA process and other risk management best practices.

What GAO Found

The Department of Homeland Security's (DHS) TRIO project represents a key effort to address long-standing financial management system deficiencies. During 2012 and 2013, the TRIO components—U.S. Coast Guard (Coast Guard), Transportation Security Administration (TSA), and Domestic Nuclear Detection Office (DNDO)—each completed an alternatives analysis (AA) to determine a preferred alternative for modernizing its financial management system. GAO found that DNDO's AA substantially met the four characteristics—well-documented, comprehensive, unbiased, and credible—that GAO previously identified for a reliable, high-quality analysis of alternatives (AOA) process. However, Coast Guard's and TSA's AAs did not fully or substantially meet three of these characteristics, and DHS guidance for conducting AAs did not substantially incorporate certain best practices, such as identifying significant risks and mitigation strategies and performing an independent review to help validate the AOA process. Based on these analyses and other factors, the TRIO components determined that migrating to a federal shared service provider (SSP) represented the best alternative, and in 2014, DHS selected the Department of the Interior's Interior Business Center (IBC) as the federal SSP for the project. However, because Coast Guard's and TSA's AAs did not fully or substantially reflect all of the characteristics noted above, they are at increased risk that the alternative selected may not achieve mission needs.
DHS also did not fully follow best practices for managing project risks related to its use of IBC on the TRIO project. Specifically, DHS followed three of seven risk management best practices, such as determining risk sources and categories and establishing a risk management strategy. However, it did not fully follow four best practices for defining risk parameters, identifying risks, developing risk mitigation plans, and implementing these plans largely because its guidance did not sufficiently address these best practices. For example, although DHS created joint teams with IBC and provided additional resources to IBC to help address risk mitigation concerns, it did not always develop sufficiently detailed risk mitigation plans that also included contingency plans for selected critical risks. As a result, although IBC's capacity and experience for migrating large agencies the size of Coast Guard and TSA was identified as a risk in July 2014, a contingency plan working group to address this concern was not established until January 2017. By not fully following risk management best practices, DHS is at increased risk that potential problems may not be identified or properly mitigated.
DHS, IBC, Office of Management and Budget (OMB), and other federal oversight agencies identified various challenges that have impacted the TRIO project and contributed to a 2-year delay in the implementation of Coast Guard's and TSA's modernized solutions. These challenges include the lack of sufficient resources, aggressive schedule, complex requirements, increased costs, and project management and communication concerns. To help address these challenges, DHS and IBC established review teams and have taken other steps to assess potential mitigating steps. In May 2017, DHS determined that migrating the solution from IBC to a DHS data center represented the best option and initiated discovery efforts to further assess this as its path forward for the TRIO project.

What GAO Recommends

GAO recommends that DHS more fully follow best practices for conducting an AOA process and managing risks. DHS concurred with GAO's recommendations and described actions it will take, or has taken, in response.