DOD Continues to Take Steps to Strengthen Management of Its Business Systems Modernization, but Long-standing Challenges Remain

DOD continues to take steps to comply with the requirements of the Act  and to satisfy relevant systems modernization management guidance. However, the pace of DOD’s progress in defining and  implementing these key modernization management controls has slowed  compared with the progress the department had made, and we have  reported, each of the last 4 years. In 2008, we reported that the then-current version of the BEA (version 5.0)  addressed, to varying degrees, missing elements, inconsistencies, and  usability issues that we previously identified, but that gaps still remained. For example, it begins to address information assurance by  identifying related laws, regulations, and policies. Notwithstanding these additions  and deletions, BEA 6.0 still does not provide business rules for all business  processes. For example, they  have    selected the Department of the Army’s Defense Knowledge Online to be  BTA’s federated enterprise portal, which is to be the point of access to  information about all DOD and component architectures, and is to allow  users to search and navigate through this information;   established and are using the DOD Architecture Registry System, which is  maintained by ASD(NII)/DOD CIO, as the repository to contain  architecture content;   conducted five pilots at three military departments and two defense  agencies to evaluate various aspects of architecture federation and  develop lessons learned about, for example, approaches for capturing and  managing architecture metadata (Air Force pilot), and enterprise search  and discovery methods (Navy pilot); and   developed guidance on identifying and registering business services and,  as of November 2008, identified and registered 25 business services, such  as a service that provides detailed information on each aircraft at the base  (e.g., an aircraft’s mission capability and maintenance status), and a  service that allows aircraft maintenance data to be retrieved, created,  updated, and removed. DOD Continues to Update Its ETP, but Important Elements Are Still Missing, as Are Individual Component Plans

Among other things, the Act requires DOD to develop an ETP for  implementing its BEA that includes listings of the legacy systems that will  and will not be part of the target business systems environment and  specific time-phased milestones and performance metrics for each  business system investment. In addition, the plan is missing information about some  legacy systems and modernization programs. As we reported last year, the department’s fiscal year 2009 budget  submission included a range of information required by the Act on  business system investments. Without such visibility, DOD components risk  making investment decisions that are inconsistent and not fully grounded  in objective data. Annual reviews. DOD Certification and Approval Decisions Have Been Based on Limited Information

Although DOD has been meeting the Act’s requirement to certify and  approve business system modernization programs, it has at times relied on  limited information in doing so. According to relevant DOD guidance, the economic viability  of system investments is to be analyzed on the basis of reliable estimates  of costs and benefits. The information in DITPR is not always accurate, and thus does not  always provide an adequate basis for informed decision making. Conclusions

The pace of DOD’s progress in defining and implementing key institutional  modernization management controls has slowed relative to each of the  prior 4 years, leaving much to be accomplished. Relatedly, it is important for the department  to obtain independent assessments of the completeness, consistency,  understandability, and usability of the federated family of business mission  area architectures, including associated transition plans, and to share the  results of these assessments with its authorizing and appropriations  committees. In addition, to ensure that DOD continues to implement the full range of  institutional management controls needed to address its business systems  modernization high-risk area, we recommend that the Secretary of  Defense direct the Deputy Secretary of Defense, as chair of the DBSMC  and as DOD’s CMO, to resolve the issues surrounding the roles,  responsibilities, authorities, and relationships of the Deputy CMO and the  military department CMOs relative to the BEA and ETP federation and  business system investment management. Further, to ensure that business system investment reviews and related  certification and approval decisions, as well as annual budget submissions,  are based on complete and accurate information, we recommend that the  Secretary of Defense direct the appropriate DOD organizations to develop  and implement plans for reconciling and validating the completeness and  reliability of information in its DITPR and SNAP-IT system data  repositories, and to include information on the status of these efforts in  the department’s fiscal year 2010 report in response to the Act. Generally, these five requirements are (1) development  of a business enterprise architecture (BEA), (2) development of an  enterprise transition plan (ETP) for implementing the BEA, (3) inclusion  of business systems information in DOD’s budget submission, (4)  establishment of business systems investment review processes and  structures, and (5) approval of defense business systems investments with  obligations in excess of $1 million.
Why GAO Did This Study

Since 1995, GAO has designated the Department of Defense's (DOD) business systems modernization program as high risk, and it continues to do so today. To assist in addressing DOD's business system modernization challenges, the Ronald W. Reagan National Defense Authorization Act for Fiscal Year 2005 (the Act) contains provisions that require the department to take certain actions and to annually report to its congressional committees on these actions. The Act also directs GAO to review each annual report. In response, GAO performed its fifth annual review of DOD's actions to comply with key aspects in the Act and related federal guidance. To do so, GAO reviewed, for example, the latest version of DOD's business enterprise architecture (BEA) and transition plan, investment management policies and procedures, and information in the department's business system data repositories.

What GAO Found

The pace of DOD's progress in defining and implementing key institutional modernization management controls has slowed compared with progress made in each of the last 4 years, leaving much still to be accomplished to fully implement the Act's requirements and related guidance. In particular, the corporate BEA continues to evolve and address previously identified missing elements, inconsistencies, and usability issues, but gaps still remain. For example, while the BEA now identifies information assurance laws, regulations, and policies, it still does not include business rules for all business processes. Further, little progress has been made in the last year in extending (i.e., federating) the BEA to the entire family of business mission area architectures, including using an independent verification and validation agent to assess the components' subsidiary architectures and federation efforts. The updated enterprise transition plan continues to identify systems and initiatives, but important elements are still missing, as are individual component plans. For example, while the plan provides a range of information, such as budgets and performance measures, for key enterprisewide and component-specific investments, it is missing information on identified investments. The fiscal year 2009 budget submission included some, but omitted other, key information about business system investments, in part because of the lack of a reliable comprehensive inventory of all defense business systems. Investment approval and accountability structures have been established for DOD and the Air Force, and related policies and procedures that are consistent with relevant guidance have been partially defined. However, these structures and processes are still lacking for the Navy. Business system investments costing over $1 million continue to be certified and approved, but these decisions are not always based on complete information. For example, key Navy investments have not fully demonstrated compliance with the department's BEA, and their economic justifications were not based on reliable estimates of cost and benefits. In addition, the information in DOD's authoritative repository of system investments that is used to make these decisions is not always accurate. Department officials attributed this slowdown in large part to pending decisions surrounding the roles, responsibilities, authorities, and relationships among key senior leadership positions, such as DOD's Deputy Chief Management Officer and the military departments' Chief Management Officers. Until DOD fully implements these long-standing institutional modernization management controls provided for under the Act, addressed in GAO recommendations, and otherwise embodied in relevant guidance, its business systems modernization will likely remain a high-risk program. As a result, it is important that the department act quickly to resolve pending decisions about key positions.