As a result of these  problems, as well as the tremendous cost, complexity, and mission  criticality of the modernization program, we designated the program as a  high-risk information technology initiative in 1995, and it has remained on  our high-risk list since that time. Prior Report Noted Weaknesses in FAA’s Software Acquisition Capabilities

In March 1997, we reported that FAA’s processes for acquiring software, the  most costly and complex component of its ATC systems, were ad hoc,  sometimes chaotic, and not repeatable across projects. The Capability Maturity Model Integration Provides a Means of Assessing an Organization’s Ability to Manage Software and System Acquisition and Development

An essential aspect of FAA’s ATC modernization program is the quality of  the software and systems involved, which is heavily influenced by the  quality and maturity of the processes used to acquire, develop, manage, and  maintain them. Our objectives were (1) to evaluate FAA’s  capabilities for developing and acquiring software and systems on its ATC  modernization program and (2) to assess the actions FAA has under way to  improve these capabilities. To assess the actions FAA has under way to improve its system and  software acquisition and development processes, we evaluated process  improvement strategies and plans. FAA Is Performing Most Project Planning Practices, but It Is Not Yet Fully Managing the Process

The purpose of project planning is to establish and maintain plans that  define the project activities. However, these process  improvement activities are not required throughout the air traffic  organizations, and the recurring weaknesses we identified in our project- specific evaluations are due in part to the choices these projects were given  in deciding whether to and how to adopt process improvement initiatives. As a result, FAA is not consistent in its adoption and management  of process improvement efforts, so that individual projects’ costs,  schedules, and performance remain at risk. Without agencywide adoption  of process improvement initiatives, the agency cannot increase the  maturity of its organizational capabilities. Acting on our prior recommendations, in 1999, FAA established a  centralized process improvement office that reports directly to the Chief  Information Officer. Because of this voluntary approach, to date less than half of the projects  listed in FAA’s system architecture have sought appraisals in at least one  process area. As a result, the  Air Traffic Organization does not currently have a policy that requires  organizations and project teams to implement process improvement  initiatives such as the iCMM. Unless the Chief Operating Officer demonstrates a strong commitment to  process improvement and establishes a consistent, institutionalized  approach to implementing, enforcing, and evaluating this process  improvement, FAA risks taking a major step backwards in its capabilities  for acquiring ATC systems and software. That is, FAA may not be able to  ensure that critical projects will continue to make progress in improving  systems acquisition and development capabilities, and the agency is not  likely to proceed to the more advanced capability levels which focus on  organizationwide management of processes. Should this occur, FAA  will continue to be vulnerable to project management problems including  cost overruns, schedule delays, and performance shortfalls. Conclusions and Recommendations

		Conclusions

The Federal Aviation Administration (FAA) has made considerable  progress in implementing processes for managing software acquisitions. However, recurring weaknesses in the areas of verification, quality  assurance, and measurement and analysis prevented the projects from  achieving a basic level of performance in these areas and from effectively  managing these and other process areas.
Why GAO Did This Study

Since 1981, the Federal Aviation Administration (FAA) has been working to modernize its aging air traffic control (ATC) system. Individual projects have suffered cost increases, schedule delays, and performance shortfalls of large proportions, leading GAO to designate the program a high-risk information technology initiative in 1995. Because the program remains a high risk initiative, GAO was requested to assess FAA's progress in several information technology management areas. This report, one in a series responding to that request, has two objectives: (1) to evaluate FAA's capabilities for developing and acquiring software and systems on its ATC modernization program and (2) to assess the actions FAA has under way to improve these capabilities.

What GAO Found

FAA has made progress in improving its capabilities for acquiring software-intensive systems, but some areas still need improvement. GAO had previously reported in 1997 that FAA's processes for acquiring software were ad hoc and sometimes chaotic. Focusing on four mission critical air traffic projects, GAO's current review assessed system and software management practices in numerous key areas such as project planning, risk management, and requirements development. GAO found that these projects were generally performing most of the desired practices: of the 900 individual practices evaluated, 83 percent were largely or fully implemented. The projects were generally strong in several areas such as project planning, requirements management, and identifying technical solutions. However, there were recurring weaknesses in the areas of measurement and analysis, quality assurance, and verification. These weaknesses hinder FAA from consistently and effectively managing its mission critical systems and increase the risk of cost overruns, schedule delays, and performance shortfalls. To improve its software and system management capabilities, FAA has undertaken a rigorous process improvement initiative. In response to earlier GAO recommendations, in 1999, FAA established a centralized process improvement office, which has worked to help FAA organizations and projects to improve processes through the use of a standard model, the integrated Capability Maturity Model. This model, which is a broad model that integrates multiple maturity models, is used to assess the maturity of FAA's software and systems capabilities. The projects that have adopted the model have demonstrated growth in the maturity of their processes, and more and more projects have adopted the model. However, the agency does not require the use of this process improvement method. To date, less than half of FAA's major ATC projects have used this method, and the recurring weaknesses we identified in our project-specific evaluations are due in part to the choices these projects were given in deciding whether to and how to adopt this process improvement initiative. Further, as a result of reorganizing its ATC organizations to a performance-based organization, FAA is reconsidering prior policies, and it is not yet clear that process improvement will continue to be a priority. Without a strong senior-level commitment to process improvement and a consistent, institutionalized approach to implementing and evaluating it, FAA cannot ensure that key projects will continue to improve systems acquisition and development capabilities. As a result, FAA will continue to risk the project management problems--including cost overruns, schedule delays, and performance shortfalls--that have plagued past acquisitions.