Air Traffic Control: Status of FAA’s Modernization Program (GAO/RCED-94-167FS, Apr. To determine if FAA’s estimates were based on good policies and practices, we researched current literature and interviewed project estimating experts to identify the key components of good cost estimating practices; obtained and analyzed FAA’s policies and practices for estimating costs to determine what criteria (directives, orders, instructions, and implementing procedures), if any, FAA has in place to guide managers in developing projects’ cost estimates; assessed FAA’s cost estimating policies, practices, tools, and techniques to determine if they incorporate the key components of good cost and schedule estimating practices advocated by SEI and other experts; and selected FAA’s five largest (based on latest life cycle cost estimates) ongoing ATC modernization projects and one project that was the subject of another GAO review, and interviewed project managers and assessed project documentation on these six projects to determine (1) how the current life cycle cost baseline was estimated and (2) how this estimating approach compared to the SEI project-level questions. To determine whether the actual costs of ATC modernization projects are being properly accumulated, we obtained and reviewed (1) selected reports and testimonies issued by GAO, the Department of Transportation’s Office of the Inspector General, and the Defense Contract Auditing Agency, (2) related policies and procedures issued by the Department of Transportation, (3) applicable accounting standards and guidance, and (4)  applicable OMB directives; reviewed FAA’s policies and procedures governing ATC financial management and interviewed program managers and financial accounting staff to determine (1) their roles and responsibilities for recording and managing ATC cost information and (2) the financial processes used to accumulate and record ATC costs; and reviewed available information for the five largest (based on life cycle cost estimates) ongoing ATC projects and determined if costs are properly accumulated by (1) obtaining available financial information on the projects and identifying the cost elements included and excluded and (2) assessing reconciliation procedures among varying sources of information. We performed our work at the Federal Aviation Administration in Washington, D.C., and the Software Engineering Institute in Pittsburgh, Pennsylvania, between February and December 1996. FAA’s cost estimating processes used on its ATC modernization projects do not meet SEI criteria. These weaknesses are exacerbated by FAA’s practice of presenting cost estimates as precise, point estimates. By doing so, FAA obscures the estimates’ inherent uncertainty and may mislead decisionmakers. Policies and Practices for Estimating Costs Do Not Satisfy “Best Practices” Requisites

FAA’s institutional processes for estimating ATC projects’ costs do not fully satisfy any of the six SEI requisites. Further, what limited information is available on actual cost performance is not an integral part of the project estimating process, and the information on cost estimates that is retained by the central estimating organization is not readily available to the project personnel that are responsible for updating estimates after the initial investment decision has been made. As a result, projects’ use and calibration of the models is inconsistent. Individual ATC Projects’ Cost Estimating Approaches Are Inconsistent and Sometimes Unreliable

Because FAA does not have well-defined, institutional processes for estimating information technology projects’ costs, the approaches used and the reliability of the estimates are inconsistent. The result is uninformed, and thus potentially unwise, investment decisions. They also stated that this report will be useful as FAA strives to improve its cost estimating capabilities. FAA Is Not Adequately Accounting for ATC Project Costs

Agencies are required to maintain adequate systems of accounting and internal controls to provide managers and other decisionmakers with reliable financial information to effectively measure performance and make sound investment decisions. As a result, the full costs of “restructured” ATC projects are understated. FAA does not have a cost accounting system capable of reliably accumulating full project cost information, and therefore cannot reliably estimate future project costs, ensure that investment decisions are sound, or manage projects effectively.
Why GAO Did This Study

GAO reviewed the reliability of the cost information critical to capital investment decisionmaking on air traffic control (ATC) projects, focusing on the Federal Aviation Administration's (FAA) processes for estimating what projects will cost and the related accounting for actual project costs.

What GAO Found

GAO found that: (1) FAA's ATC modernization program's cost estimating processes do not satisfy recognized estimating requisites, and its cost accounting practices do not provide for proper accumulation of actual project costs; (2) the result is an absence of reliable project cost and financial information that the Congress has legislatively specified and that leading public-sector and private-sector organizations point to as essential to making fully informed investment decisions among competing ATC projects; (3) without this information, the likelihood of poor ATC investment decisions is increased, not only when a project is initiated but also throughout its life cycle; (4) with respect to cost estimating, FAA fails to meet five of the six process requisites that the Software Engineering Institute (SEI) says should be institutionally entrenched and consistently used for information technology projects; (5) in the absence of such institutional policies to guide ATC project cost estimating, FAA has adopted a cost estimating process that allows each ATC project to approach cost estimating in whatever manner its estimators choose; (6) the result is inconsistency in the rigor and discipline with which ATC project cost estimates are derived, which in turn means estimates of varying degrees of reliability; (7) when comparing the approaches that six ATC projects used to derive their current official life cycle cost estimates to SEI's project-specific criteria, GAO found that two were too poorly documented to permit any comparative analysis, while none of the remaining four satisfied all of the criteria SEI associates with highly credible estimates; (8) compounding these estimating process weaknesses is FAA's practice of presenting cost estimates as precise, point estimates; (9) by doing so, FAA fails to disclose the estimates' inherent uncertainty and risks, thus further limiting the estimates' decisionmaking value and credibility; (10) with respect to cost accounting, FAA is not accumulating all ATC project costs, and FAA does not have a cost accounting system for capturing and reporting the full cost of its ATC projects; (11) instead, FAA decisionmakers use accounting and financial management systems that omit relevant project costs, such as those associated with FAA project management; and (12) the result is that FAA cannot reliably measure the ATC projects' actual cost performance against established baselines, and cannot reliably use information relating to actual cost experiences to improve future cost estimating efforts.