With an IT budget of $2.1 billion for fiscal year 2008, FAA accounts for  about 83 percent of the Department of Transportation’s IT budget. The Air Traffic Organization is led by  FAA’s Chief Operating Officer. FAA Has Established an EVM Policy for Major IT Investments, but Key Components Are Not Fully Consistent with Best Practices

In 2005, FAA established a policy requiring the use of EVM on its major IT  investments; however, key components of this policy are not fully  consistent with best practices. Without the  level of detail provided by a product-oriented approach, program  managers may not have the information they need to make decisions on  specific program components. Until FAA establishes EVM training requirements  for all relevant personnel (including executives with oversight  responsibilities and program staff responsible for contract management)  and verifies the completion of this training, it cannot effectively ensure  that its program staff have the appropriate skills to validate and interpret  EVM data, and that its executives fully understand the data they are given  in order to ask the right questions and make informed decisions. Case studies of four programs  demonstrated that all are using or planning to use EVM. Until these areas are fully addressed, FAA  faces an increased risk that program managers are not adequately using  earned value to manage their programs. Instead, the program estimates  government costs. SWIM is not currently collecting  EVM data. ERAM also partially implemented each of the three practices for ensuring  that earned value data are reliable. For example:    There were multiple cases in which the contractor reported that no work  was planned or accomplished, yet funds were spent; in other cases, the  contractor reported that work was planned and accomplished, but funds  were credited to the government. FAA Has Taken Steps to Oversee EVM Compliance, but Its Oversight Process Lacks Sufficient Rigor

FAA has taken important steps to oversee compliance with EVM policies  by establishing an oversight office, assessing major systems using defined  evaluation criteria, and demonstrating improved capabilities on most  programs. However, the oversight office’s assessments are not thorough  enough to identify anomalies in contractor data, and its agencywide  progress reports can be misleading, in that the agency’s evaluation process  does not distinguish between systems that collect comprehensive data and  those that do not. As a result, FAA executives do not always receive an  accurate view of the quality of a program’s EVM data when making  investment decisions on that program. The agency has  established policies that require the use of EVM; system acquisition  programs are using earned value data to manage their programs; an  oversight office monitors system acquisition programs’ compliance with  policy and standards; and earned value performance data are being used  by multiple levels of management as they review and manage IT  investment. Appendix I: Objectives, Scope, and Methodology

Our objectives were to (1) assess the Federal Aviation Administration’s  (FAA) policies for implementing earned value management (EVM) on its  information technology (IT) investments, (2) evaluate whether the agency  is adequately using EVM techniques to manage key system acquisitions,  (3) assess the agency’s efforts to oversee compliance with its EVM  policies, and (4) evaluate whether the agency is using EVM data as part of  its IT investment management. ASR-11 partially  met the other practices because, while the program implemented many  key components of an effective EVM system, ASR-11 is limited in what it  can measure and validate.
Why GAO Did This Study

In fiscal year 2008, the Federal Aviation Administration (FAA) plans to spend over $2 billion on information technology (IT) investments--many of which support FAA's air traffic control modernization. To more effectively manage such investments, in 2005 the Office of Management and Budget required agencies to use earned value management (EVM). EVM is a project management approach that, if implemented appropriately, provides objective reports of project status, produces early warning signs of impending schedule delays and cost overruns, and provides unbiased estimates of a program's total costs. Among other objectives, GAO was asked to assess FAA's policies for implementing EVM on its IT investments, evaluate whether the agency is adequately using these techniques to manage key IT acquisitions, and assess the agency's efforts to oversee EVM compliance. To do so, GAO compared agency policies with best practices, performed four case studies, and interviewed key FAA officials.

What GAO Found

FAA has established a policy requiring the use of EVM on its major IT acquisition programs, but key components of this policy are not fully consistent with best practices of leading organizations. Specifically, FAA fully met four and partially met three components of an effective EVM policy. For example, FAA requires its program managers to obtain EVM training, but it does not enforce completion of this training or require other relevant personnel to obtain this training. Until FAA expands and enforces its policy, it will be difficult for the agency to gain the full benefits of EVM. FAA is using EVM to manage IT acquisition programs, but not all programs are ensuring that their earned value data are reliable. Case studies of four programs demonstrated that all are using or planning to use EVM systems. However, of the three programs currently collecting EVM data, only one program is adequately ensuring that its earned value data are reliable. Another program is limited in its ability to ensure data reliability because it was initiated before earned value was required. The third program did not adequately validate contractor performance data. For example, GAO found anomalies in which the contractor reported spending funds without accomplishing work and others in which the contractor reported accomplishing work while crediting funds to the government. Until programs undertake a rigorous validation of their EVM data, FAA faces an increased risk that managers may not be getting the information they need to effectively manage the programs. FAA has taken important steps to oversee program compliance with EVM policies, but its oversight process lacks sufficient rigor. Through its recurring assessments, FAA has reported that most programs have improved their earned value capabilities over time, and that 74 percent of the programs were fully compliant with national standards. However, FAA's assessments are not thorough enough to identify anomalies in contractor data, and its progress reports do not distinguish between systems that collect comprehensive data and those that do not. As a result, FAA executives do not always receive an accurate view of the quality of a program's EVM data when making investment decisions on that program.