In particular, they fully incorporate 1  best practice, partially incorporate 2, and do not incorporate the remaining  5. Until this is accomplished, DOD is  increasing the risk that important and beneficial best practices will not be  followed and that DOD business systems investments will not deliver  promised capabilities and benefits on time and within budget. The 5000 Series Largely Incorporates Best Practices Relevant to Any Business Systems Acquisition

Of the 10 best practices that we categorized as relevant to the acquisition of  any business system, whether custom-developed or developed using  commercial packages and products, essentially all have been incorporated  into DOD’s acquisition policies and guidance. In turn, this increases the  risk that acquisition projects will fall short of expectations. The 5000 Series Generally Does Not Incorporate Best Practices Relevant to Commercial Component-Based Business Systems Acquisitions

Of the 8 best practices relevant to acquiring commercial component-based  business systems, few have been incorporated into DOD systems  acquisition policies and guidance. According to these officials, 80 to 90 percent of  the revision has been completed and reviewed and their goal is to publish  the initial version of the revised guidebook by September 30, 2004. DOD’s acquisition policy requires program managers and investment  decision authorities to examine and, as appropriate, adopt best practices. However, other practices that, if followed, could increase the odds of  acquisitions delivering promised system capabilities and benefits on time  and within budget have yet to be similarly included. Therefore, it is important for DOD to treat further revisions of its  acquisition policy and guidance relative to business systems as a priority  and move quickly to incorporate missing best practices and associated  controls for ensuring that the practices are followed. To ensure that the best practices provided for in DOD acquisition  policies and guidance are appropriately followed, we also recommend  that the above recommended plan incorporate steps to include in  DOD’s acquisition policies a provision for measurement and  verification of best practices. As another example,  we do not agree with DOD’s comment that 2 of the best practices that we  recommended for incorporation in its policies and guidance—preparing  users for the impact that business process changes embedded in  commercial components will have on their roles and responsibilities, and  actively managing changes in how users will use new systems—are already  sufficiently contained in the 5000 series. Objectives, Scope, and Methodology

Our objectives were to determine whether the Department of Defense’s  (DOD) revised systems acquisition policies for acquiring information  technology (IT) business systems (1) are consistent with industry best  practices, including those pertaining to commercial component-based  systems, and (2) provide the necessary controls to ensure that the  department’s component organizations adhere to the practices.
Why GAO Did This Study

The way in which the Department of Defense (DOD) has historically acquired its business systems has been cited as a root cause for its limited success in delivering promised system capabilities and benefits on time and within budget. In response, DOD recently revised its systems acquisition policies and guidance to incorporate best practices, including those pertaining to business systems. GAO was asked to determine whether DOD's revised systems acquisition policies and guidance (1) are consistent with industry best practices, including those pertaining to commercial component-based systems, and (2) provide the necessary controls to ensure that DOD component organizations adhere to the practices.

What GAO Found

DOD's revised policies and guidance largely incorporate 10 best practices for acquiring any type of information technology (IT) business system. For example, the revisions include the requirement that acquisitions be economically justified on the basis of costs, benefits, and risks. However, the revisions generally do not incorporate 8 best practices relating to the acquisition of commercial component-based systems. For example, they do not address basing any decision to modify commercial components on a thorough analysis of the impact of doing so or on preparing system users for the business process and job roles and responsibilities changes that are embedded in the functionality of commercial IT products. In total, GAO found that DOD's acquisition policies and guidance fully incorporate 8 of the 18 best practices that they were evaluated against, partially incorporate 5 practices, and do not incorporate the remaining 5 practices. DOD intends to expand its acquisition guidance to incorporate additional best practices by September 30, 2004, but department officials cite other priorities as a reason why they have not been able to complete this effort and could not provide a plan specifying how this will be accomplished. Until DOD's revised policies and guidance incorporate key systems acquisition best practices, the risk that system investments will not consistently deliver promised capabilities and benefits on time and within budget is increased. DOD's revised policies also do not contain sufficient controls to ensure that DOD components appropriately follow the best practices that are incorporated in its policies and guidance. According to acquisition best practices experts, as well as GAO's internal control guidance, controls are effective if they are backed by measurements that are verified. Although the revised policies and guidance require acquisition managers to examine and, as appropriate, adopt best practices, they do not cite what that examination entails. DOD believes existing controls are sufficient, even though these controls do not provide for measuring and validating the practices' use. Without specific requirements to measure and validate the use of best practices, the risk that they will not be followed increases, which, in turn, increases the risk that system investments will not meet expectations.