Each business area is described below. Customs’ major information technology effort is its Automated Commercial Environment (ACE) system. Our objectives were to determine (1) the maturity of Customs’ software development processes and (2) the effectiveness of Customs’ software process improvement program. Few  processes are defined, and success depends  on individual effort. While Customs showed some strengths in this KPA, its many weaknesses render its software project planning processes unrepeatable. According to the SW-CMM, effective software project tracking and oversight, among other things, includes (1) designating a project software manager to be responsible for the project’s software activities and results, (2) having a documented software development plan for tracking software activities and communicating status, (3) following a written organizational policy for managing the project, (4) conducting periodic internal reviews to track technical progress, plans, performance, and issues against the software development plan, (5) tracking the software risks associated with the cost, resource, schedule, and technical aspects of the project, (6) explicitly assigning responsibility for software work products and activities, (7) tracking the sizes of the software work products (or sizes of the changes to the software work products) and taking corrective actions as necessary, and (8) periodically reviewing the activities for software project tracking and oversight with senior management. Also, NCAP 0.1 had strengths in all but five of this KPA’s 24 key practices. Customs Lacks a Software Quality Assurance Process

All of the projects evaluated had extensive and significant software quality assurance practice weaknesses. For example, all three projects had developed software configuration management plans according to a documented procedure. Currently, Customs has no software process improvement program. This model has five phases: Initiating, Diagnosing, Establishing, Acting, and Leveraging—IDEAL. Each of the phases is summarized below. Customs did not staff or fund any other KPA improvement activities at this time. Currently, Customs does not have a software development process improvement program, and it has not taken steps to initiate one. However, its software development processes are ad hoc and sometimes chaotic, and are not repeatable even on a project-by-project basis. Recommendations

We recommend that, after ensuring that its mission-critical systems are Year 2000 compliant but before investing in major software development efforts like ACE, the Commissioner of Customs direct the Chief Information Officer to assign responsibility and authority for software development process develop and implement a formal plan for software development process improvement that is based on the software capability evaluation results contained in this report and specifies measurable goals and time frames, prioritizes initiatives, estimates resource requirements (trained staff and funding) and defines a process improvement management structure; ensure that every new software development effort in Customs adopts processes that satisfy at least SW-CMM level 2 requirements; and ensure that process improvement activities are initiated for all ongoing essential software maintenance projects.
Why GAO Did This Study

Pursuant to a congressional request, GAO reviewed the Customs Service's software development maturity and improvement activities, focusing on: (1) the maturity of Customs' software development processes; and (2) whether Customs has an effective software process improvement program.

What GAO Found

GAO noted that: (1) because of the number and severity of Customs' software development process weaknesses, Customs did not fully satisfy any of the key process areas (KPA) necessary to achieve the repeatable level of process maturity; (2) as a result, its processes for developing software, a complex and expensive component of Customs' systems, are ad hoc, sometimes chaotic, and not repeatable across projects; (3) Customs had some practice strengths in all but one of the five KPAs evaluated (i.e., requirements management, software project planning, software project tracking and oversight, software quality assurance, and software configuration management); however, GAO also found extensive and significant weaknesses in each of these KPAs; (4) some of these weaknesses were systemic, recurring in each of the KPAs; (5) for example, Customs had no written policy for managing or implementing any of the KPAs; (6) none of the projects had: (a) an approved quality assurance plan; (b) documented procedures for determining the project cost, schedule, or effort; or (c) any outside group reviewing or reporting on the project's compliance with defined processes; (7) these weaknesses are some of the reasons for Customs' limited success, for example, in delivering promised Automated Commercial Environment (ACE) capabilities on time; (8) Customs does not have a software development process improvement program, and it has not taken the basic steps to initiate one; (9) these steps, many of which are described in Software Engineering Institute's initiating, diagnosing, establishing, acting, and leveraging model for process improvement, include assigning responsibility and authority for process improvement, establishing a process improvement management structure, defining a plan of action, and committing needed resources; and (10) until Customs establishes an effective process improvement program, its software processes will remain poorly defined and undisciplined, and its software projects are likely to suffer cost, schedule, and performance shortfalls.