The pilots, costing over $1  billion in total, were limited in scope and were not intended to provide  corporate solutions to any of the Navy’s long-standing financial and  business management problems. Because of the various inconsistencies in the design and implementation,  the pilots were stovepiped and could not operate with each other, even  though they performed many of the same business functions. Assuming that the project continues to  effectively implement the processes it has adopted, the planned  functionality of the Navy ERP has the potential to address at least some of  the weaknesses identified in the Navy’s financial improvement plan. Since the  program is still in a relatively early phase—it will not be fully operational  until fiscal year 2011, at a currently estimated cost of $800 million—the  project team must be continually vigilant and held accountable for ensuring  that the disciplined processes are followed in all phases to help achieve  overall success. Considering that the project is in a relatively early phase and  DOD’s history of not implementing systems on time and within budget, the  projected schedule and costs estimates are subject to, and very likely will,  change. The Navy determined that the pilots would not meet its overall  requirements and concluded that the best alternative was to develop a new  ERP system—under the leadership of a central program office—and use  efforts from the pilots as starting points by performing a review of their  functionality and lessons learned, eliminating redundancies, and  developing new functionalities that were not addressed by the pilots. Requirements Management Process Following Best Practices

Our review found that the ERP development team has so far followed an  effective process for managing its requirements development. Requirements represent the blueprint that system developers and program  managers use to design, develop, test, and implement a system. However, the currently planned ERP is not intended to  provide an all-inclusive end-to-end corporate solution for the Navy. Going forward, the  Navy faces very difficult challenges and risks in the areas of developing and  implementing 44 system interfaces with other Navy and DOD systems, and  accurately converting data from the existing legacy systems to the ERP. As  previously noted, financial management is a high-risk area in the  department and has been designated as such since 1995. For example, we found that the Navy  does not have a mechanism in place to capture the data that can be used to  effectively assess the project management processes. Additionally, although the Navy ERP program has a verification and  validation function, it relies on in-house subject matter experts and others  who are not independent to provide an assessment of the Navy ERP to  DOD and Navy management. Given DOD’s long-standing  inability to implement business systems that provide users with the  promised capabilities, an independent assessment of the ERP’s  performance is warranted. Recommendations for Executive Action

To improve the Navy’s and DOD’s oversight of the Navy ERP effort, we  recommend that the Secretary of Defense direct the Secretary of the Navy  to require that the Navy ERP Program Management Office (1) develop and  implement the quantitative metrics needed to evaluate project performance  and risks and use the quantitative metrics to assess progress and  compliance with disciplined processes and (2) establish an IV&V function  and direct that all IV&V reports be provided to Navy management and to  the appropriate DOD investment review board, as well as the project  management. We characterized the pilots as failures because the department  spent $1 billion on systems that did not result in marked improvement in  the Navy’s day-to-day operations. Scope and Methodology

To obtain a historical perspective on the planning and costs of the Navy’s  four Enterprise Resource Planning (ERP) pilot projects, and the decision to  merge them into one program, we reviewed the Department of Defense’s  (DOD) budget justification materials and other background information on  the four pilot projects. To determine if the Navy has identified lessons learned from the pilots, how  they are being used, and the challenges that remain, we reviewed program  documentation and interviewed Navy ERP program officials.
Why GAO Did This Study

The Department of Defense's (DOD) difficulty in implementing business systems that are efficient and effective continues despite the billions of dollars that it invests each year. For a decade now--since 1995--we have designated DOD's business systems modernization as "high-risk." GAO was asked to (1) provide a historical perspective on the planning and costs of the Navy's four Enterprise Resource Planning (ERP) pilot projects, and the decision to merge them into one program; (2) determine if the Navy has identified lessons from the pilots, how the lessons are being used, and challenges that remain; and (3) determine if there are additional best business practices that could be used to improve management oversight of the Navy ERP.

What GAO Found

The Navy invested approximately $1 billion in four ERP pilots without marked improvement in its day-to-day operations. The planning for the pilots started in 1998, with implementation beginning in fiscal year 2000. The four pilots were limited in scope and were not intended to be corporate solutions for any of the Navy's long-standing financial and business management problems. Furthermore, because of the various inconsistencies in the design and implementation of the pilots, they were not interoperable, even though they performed many of the same business functions. In short, the efforts were failures and $1 billion was largely wasted. Because the pilots would not meet its overall requirements, the Navy decided to start over and develop a new ERP system, under the leadership of a central program office. Using the lessons learned from the pilots, the current Navy ERP program office has so far been committed to the disciplined processes necessary to manage this effort. GAO found that, unlike other systems projects it has reviewed at DOD and other agencies, Navy ERP management is following an effective process for identifying and documenting requirements. The strong emphasis on requirements management, which was lacking in the previous efforts, is critical since requirements represent the essential blueprint that system developers and program managers use to design, develop, test, and implement a system and are key factors in projects that are considered successful. While the Navy ERP has the potential to address some of the Navy's financial management weaknesses, as currently planned, it will not provide an all-inclusive end-to-end corporate solution for the Navy. For example, the current scope of the ERP does not include the activities of the aviation and shipyard depots. Further, there are still significant challenges and risks ahead as the project moves forward, such as developing and implementing 44 system interfaces with other Navy and DOD systems and converting data from legacy systems into the ERP system. The project is in its early phases, with a current estimated completion date of 2011 at an estimated cost of $800 million. These estimates are subject to, and very likely will, change. Broader challenges, such as alignment with DOD's business enterprise architecture, which is not fully defined, also present a significant risk. Given DOD's past inability to implement business systems that provide the promised capability, continued close management oversight--by the Navy and DOD--will be critical. In this regard, the Navy does not have in place the structure to capture quantitative data that can be used to assess the effectiveness of the overall effort. Also, the Navy has not established an independent verification and validation (IV&V) function. Rather, the Navy is using in-house subject matter experts and others within the project. Industry best practices indicate that the IV&V activity should be independent of the project and report directly to agency management in order to provide added assurance that reported results on the project's status are unbiased.