Planned federal IT spending has now risen to an  estimated $79.4 billion for fiscal year 2011, a 1.2 percent increase from the  2010 level of $78.4 billion. According to OMB, these data are intended to provide a  near real-time perspective of the performance of these investments, as  well as a historical perspective. Another issue with the ratings is that large inconsistencies  exist in the number of milestones that agencies report on the Dashboard. For example,  the Dashboard rated the Law Enforcement Wireless Communication  investment a 10 for cost (less than 5 percent variance) every month from  July 2009 through January 2010. However, our analysis shows the  investment’s cost rating during December 2009 and January 2010 is  equivalent to an 8 (a variance of 10 percent to less than 15 percent). However, investment  data we examined showed the schedule rating should have been a 5  (greater than or equal to 30 days and less than 90 days behind schedule)  from September 2009 through December 2009. Cost and Schedule Ratings Do Not Reflect Current Performance and Wide Variation in Milestone Reporting Exists

A primary reason why the cost and schedule ratings were not always  accurate is that the cost and schedule ratings do not take current  performance into consideration for many investments on the Dashboard,  though it is intended to represent near real-time performance information  on all major IT investments. Having too many milestones may mask recent performance problems  because the performance of every milestone (i.e., historical and recently  completed) is equally averaged into the ratings. Specifically, investments  that perform well during many previously completed milestones and then  start performing poorly on a few recently completed milestones can  maintain ratings that still reflect good performance. Conversely, having too few milestones can limit the amount of information  available to track work and rate performance and allows agencies to  potentially skew the performance ratings. As of July 1, 2010, this updated Dashboard  had not been released. Specifically, in  commenting on a draft of this report, the Federal CIO stated that OMB has  recently chartered a working group comprised of representatives from  several federal agencies, with the intention of developing clear guidance  for standardizing and improving investment activity reporting. Despite these efforts, until OMB upgrades the Dashboard application to  improve the accuracy of the cost and schedule ratings to include ongoing  milestones, explains the outcome of these improvements in its next annual  report to Congress on the Implementation of the E-Government Act  (which is a key mechanism for reporting on the implementation of the  Dashboard), provides clear and consistent guidance to agencies that  standardizes milestone reporting, and ensures agencies comply with the  new guidance, the Dashboard’s cost and schedule ratings will likely  continue to experience data accuracy issues. Use of the Dashboard as a Management Tool Varies

Officials at three of the five agencies we reviewed—DOD, DOJ, and HHS— stated that they are not using the Dashboard to manage their investments,  and the other two agencies, DOE and USDA, indicated that they are using  the Dashboard to manage their investments. Specifically, officials from the three agencies are not using the Dashboard  to manage their investments because they have other existing means to do  so:    DOD officials indicated that they use the department’s Capital Planning  and Investment Control process to track IT investment data—including  cost and schedule. The Federal CIO stated that the Dashboard  has greatly improved oversight capabilities compared to previously used  mechanisms. He also stated that the Dashboard has increased the  accountability of agencies’ CIOs and established much-needed visibility. According to OMB officials, the Dashboard is one of the key sources of  information that OMB analysts use to identify investments that are  experiencing performance problems and select them for a TechStat  session—a review of selected IT investments between OMB and agency  leadership that is led by the Federal CIO. OMB agreed with our recommendation that it develop and issue clear  guidance that standardizes milestone reporting on the Dashboard. GAO staff who made major  contributions to this report are listed in appendix V.

Appendix I: Objectives, Scope, and Methodology

Our objectives were to (1) examine the accuracy of the cost and schedule  performance ratings on the Dashboard for selected investments and (2)  determine whether the data on the Dashboard are used as a management  tool to make improvements to IT investments. To select these agencies and investments, we first identified ten  agencies with large IT budgets as reported in the Office of Management  and Budget’s (OMB) fiscal year 2010 Exhibit 53. However, 4 of the 8 investments had notable discrepancies  on either their cost or schedule ratings.
Why GAO Did This Study

Federal IT spending has risen to an estimated $79 billion for fiscal year 2011. To improve transparency and oversight of this spending, in June 2009 the Office of Management and Budget (OMB) deployed a public website, known as the IT Dashboard, which provides information on federal agencies' major IT investments, including assessments of actual performance against cost and schedule targets (referred to as ratings). According to OMB, these data are intended to provide both a near real-time and historical perspective of the performance of these investments. GAO was asked to (1) examine the accuracy of the cost and schedule performance ratings on the Dashboard for selected investments and (2) determine whether the data on the Dashboard are used as a management tool to make improvements to IT investments. To do so, GAO selected 8 major investments from 5 agencies with large IT budgets, compared its analyses of the selected investments' performance to the ratings on the Dashboard, and interviewed agency officials about their use of the Dashboard to manage investments.

What GAO Found

The cost and schedule ratings on OMB's Dashboard were not always accurate for the selected investments. GAO found that 4 of the 8 selected investments had notable discrepancies on either their cost or schedule ratings. For example, the Dashboard indicated one investment had a less than 5 percent variance on cost every month from July 2009 through January 2010. GAO's analysis shows the investment's cost performance in December 2009 through January 2010 had a variance of 10 percent to less than 15 percent. Additionally, another investment on the Dashboard reported that it had been less than 30 days behind schedule since July 2009. However, investment data GAO examined showed that from September to December 2009 it was behind schedule greater than or equal to 30 days and less than 90 days. A primary reason for the data inaccuracies was that while the Dashboard was intended to represent near real-time performance information, the cost and schedule ratings did not take into consideration current performance. As a result, the ratings were based on outdated information. For example, cost ratings for each of the investments were based on data between 2 months and almost 2 years old. As of July 1, 2010, OMB plans to release an updated version of the Dashboard in July that includes ratings that factor in the performance of ongoing milestones. Another issue with the ratings was the wide variation in the number of milestones agencies reported, which was partly because OMB's guidance to agencies was too general. Having too many milestones can mask recent performance problems because the performance of every milestone (dated and recent) is equally averaged into the ratings. Specifically, investments that perform well during many previously completed milestones and then start performing poorly on a few recently completed milestones can maintain ratings that still reflect good performance. Conversely, having too few milestones limits the amount of information available to rate performance and allows agencies to potentially skew the ratings. OMB officials stated that they have recently chartered a working group with the intention of developing guidance for standardizing milestone reporting. However, until such guidance is available, the ratings may continue to have accuracy issues. Officials at three of the five agencies stated they were not using the Dashboard to manage their investments because they maintain they already had existing means to do so; officials at the other two agencies indicated that they were using the Dashboard to supplement their existing management processes. OMB officials indicated that they relied on the Dashboard as a management tool, including using the Dashboard's investment trend data to identify and address issues with investments' performance. According to OMB officials, the Dashboard was one of the key sources of information that they used to determine if an investment requires additional oversight. In addition, the Federal Chief Information Officer (CIO) stated that the Dashboard has greatly improved oversight capabilities compared to previously used mechanisms. He also stated that the Dashboard has increased the accountability of agencies' CIOs and established much needed visibility.

What GAO Recommends

GAO recommends that OMB report on its planned changes to the Dashboard to improve the accuracy of performance information and provide guidance to agencies that standardizes milestone reporting. OMB agreed with these recommendations, but disagreed with aspects of the draft report that GAO addressed, as appropriate.