According to OMB,  these data are intended to provide a near real-time perspective on the  performance of these investments, as well as a historical perspective. OMB Has Multiple Efforts Under Way to Further Refine the Dashboard and Uses the Dashboard to Improve IT Management

Since our last report, OMB has initiated multiple efforts to increase the  Dashboard’s value as a management and oversight tool, and has used data  in the Dashboard to improve the management of federal IT investments. Specifically, OMB is focusing its efforts in four main areas: streamlining  key OMB investment reporting tools, eliminating manual monthly  submissions, coordinating with agencies to improve data, and improving  the user interface. OMB anticipates these efforts  will increase the Dashboard’s data reliability by ensuring that the agencies  are aware of and are working to address issues. Specifically, OMB analysts are using the Dashboard’s  investment trend data to track changes and identify issues with  investments’ performance in a timely manner. Additionally, according to OMB officials, the  Dashboard is one of the key sources of information that OMB analysts use  to identify IT investments that are experiencing performance problems  and to select them for a TechStat session—a review of selected IT  investments between OMB and agency leadership that is led by the  Federal CIO. Furthermore, according to OMB, these sessions and other OMB  management reviews have resulted in a $3 billion reduction in life-cycle  costs, as of December 2010. Dashboard Ratings Did Not Always Reflect True Investment Cost and Schedule Performance

While the efforts previously described are important steps to improving  the quality of the information on the Dashboard, cost and schedule  performance data inaccuracies remain. Until the agencies submit  complete, reliable, and timely data to the Dashboard and OMB revises its  Dashboard calculations, performance ratings will continue to be  inaccurate and may not reflect current program performance. For example,  Automatic Dependent Surveillance-Broadcast’s schedule performance was  rated “green” on the Dashboard in July 2010, but our analysis showed its  current performance was “yellow” that month. In particular, there were four  primary weaknesses in agency practices that resulted in inaccurate cost  and schedule ratings on the Dashboard: the investment baseline on the  Dashboard was not reflective of the investment’s actual baseline, agencies  did not report data to the Dashboard, agencies reported erroneous data,  and unreliable earned value data were reported to the Dashboard. Table 1 shows the causes of  inaccurate ratings for the selected investments. Until agencies provide more reliable data and OMB improves  the calculations of the ratings on the Dashboard, the accuracy of the  ratings will continue to be in question and the ratings may not reflect  current program performance. Specifically, we are recommending  that:    The Secretary of the Department of Homeland Security direct the CIO to    ensure that investment data submissions include complete and  accurate investment information for all required fields;    comply with OMB’s guidance on updating the CIO rating as soon as  new information becomes available that affects the assessment of a  given investment, including when an investment is in the process of a  rebaseline; and    work with C4ISR officials to comply with OMB’s guidance on updating  investment cost and schedule data on the Dashboard at least monthly. GAO staff who made major  contributions to this report are listed in appendix V.

Appendix I: Objectives, Scope, and Methodology

Our objectives were to (1) determine what efforts the Office of  Management and Budget (OMB) has under way to improve the Dashboard  and the ways in which it is using data from the Dashboard to improve  information technology (IT) management and (2) examine the accuracy of  the cost and schedule performance ratings on OMB’s Dashboard.
Why GAO Did This Study

Each year the federal government spends billions of dollars on information technology (IT) investments. Given the importance of oversight, the Office of Management and Budget (OMB) established a public Web site, referred to as the IT Dashboard, that provides detailed information on about 800 federal IT investments, including assessments of actual performance against cost and schedule targets (referred to as ratings). In the second of a series of Dashboard reviews, GAO was asked to (1) determine OMB's efforts to improve the Dashboard and how it is using data from the Dashboard, and (2) examine the accuracy of the Dashboard's cost and schedule performance ratings. To do so, GAO analyzed documentation on OMB oversight efforts and Dashboard improvement plans, compared the performance of 10 major investments from five agencies with large IT budgets against the ratings on the Dashboard, and interviewed OMB and agency officials.

What GAO Found

Since GAO's first review, in July 2010, OMB has initiated several efforts to increase the Dashboard's value as an oversight tool, and has used the Dashboard's data to improve federal IT management. These efforts include streamlining key OMB investment reporting tools, eliminating manual monthly submissions, coordinating with agencies to improve data, and improving the Dashboard's user interface. Recent changes provide new views of historical data and rating changes. OMB anticipates that these efforts will increase the reliability of the data on the Dashboard. To improve IT management, OMB analysts use Dashboard data to track investment changes and identify issues with performance. OMB officials stated that they use these data to identify poorly performing IT investments for review sessions by OMB and agency leadership. OMB reported that these sessions and other management reviews have resulted in a $3 billion reduction in life-cycle costs, as of December 2010. While the efforts above as well as initial actions taken to address issues GAO identified in its prior review--such as OMB's updated ratings calculations to factor in ongoing milestones to better reflect current status--have contributed to data quality improvements, performance data inaccuracies remain. The ratings of selected IT investments on the Dashboard did not always accurately reflect current performance, which is counter to the Web site's purpose of reporting near real-time performance. Specifically, GAO found that cost ratings were inaccurate for six of the investments that GAO reviewed and schedule ratings were inaccurate for nine. For example, the Dashboard rating for a Department of Homeland Security investment reported significant cost variances for 3 months in 2010; however, GAO's analysis showed lesser variances from cost targets for the same months. Conversely, a Department of Transportation investment was reported as on schedule on the Dashboard, which does not reflect the significant delays GAO has identified in recent work. These inaccuracies can be attributed to weaknesses in how agencies report data to the Dashboard, such as providing erroneous data submissions, as well as limitations in how OMB calculates the ratings. Until the selected agencies and OMB resolve these issues, ratings will continue to often be inaccurate and may not reflect current program performance.

What GAO Recommends

GAO is recommending that selected agencies take steps to improve the accuracy and reliability of Dashboard information and OMB improve how it rates investments relative to current performance and schedule variance. Agencies generally concurred with the recommendations; OMB did not concur with the first recommendation but concurred with the second. GAO maintains that until OMB implements both, performance may continue to be inaccurately represented on the Dashboard.