Specifically, OMB requires 26 key federal  departments and agencies to provide information to it related to their IT  investments (called exhibit 53s) and capital asset plans and business  cases (called exhibit 300s). a comparison of current performance with a pre-established cost areas for innovation in the areas of customer satisfaction, strategic  and business results, and financial performance;  indication if the agency revisited alternative methods for achieving  the same mission needs and strategic goals;  consideration of issues, such as greater utilization of technology  or consolidation of investments to better meet organizational  goals; an ongoing review of the status of the risks identified in the  investment’s planning and acquisition phases;  identification of whether there is a need to redesign, modify, or  terminate the investment; an analysis on the need for improved methodology (i.e., better  ways for the investment to meet cost and performance goals);   lessons learned;  cost or schedule variances;  recommendations to redesign or modify an asset in advance of  potential problems; and overlap with other investments. Federal Agencies’ Assessments of Major IT Steady State Investments Vary Significantly

Although OMB guidance calls for agencies to develop an OA policy and  perform such analyses annually, the extent to which the five federal  agencies we reviewed carried out these tasks varied significantly. Specifically, DHS and HHS developed a policy and conducted OAs, but in  doing so, they excluded key investments and assessment factors. DHS and HHS Developed an OA Policy and Performed OAs, but Did Not Address All Investments and Key Factors

DHS and HHS had developed policies, which contained all performance  factors identified in OMB’s guidance. Addressing  these factors is important because they help agencies to, among other  things, identify where cost-effective improvements can be made. With regard to why analyses were not performed on all investments and  why those that were conducted did not fully address all factors, DHS and  HHS attributed these shortfalls to the following:
Officials from DHS’s Office of the CIO who are responsible for overseeing  the performance of OAs department-wide told us the components only  performed 16 of the 44 analyses and did not address all key factors (in  the 16 OAs that were performed) because they were not consistently  implementing department and OMB policy as they should have because  it was not a priority. To address these shortfalls, the department  recently took steps to make OAs a priority and to ensure consistent  application of department and OMB policy. Until these agencies assess  all steady state investments and ensure that they are fully assessed  against factors, there is increased risk that these agencies will not know  whether the multibillion dollar investments fully meet their intended  objectives. DOD, Treasury, and VA Did Not Develop Policies or Perform OAs

DOD, Treasury, and VA had not developed a policy for performing OAs  and did not conduct OAs for their 23 steady state investments that have  combined annual budgets of $2.1 billion. Regarding why DOD and VA had not developed policies and are not  performing analyses, officials from those agencies stated that in lieu of  conducting OAs, they assess the performance of steady state  investments as part of developing their annual exhibit 300 submissions to  OMB. Although OMB requires OAs for all steady  state systems, its guidance does not provide a mechanism for ensuring  they are completed and submitted to OMB for review. Treasury officials from the department’s office of the CIO told us they  decided not to perform OAs in 2011 and instead decided to use the time  to develop a policy. However, the officials stated that they did not  anticipate the policy to be completed until the end of this calendar year. Taken together, these  five agencies continue to invest billions of dollars each year on IT steady  state investments without ensuring that they are continuing to meet  agency needs and are delivering value. In addition, we recommend that the Secretaries of Homeland Security  and Health and Human Services direct their Chief Information Officers to  ensure OAs are performed annually on all major steady state investments  and the assessments include all key factors. Further, to ensure that OA policies are developed and that annual  analyses are conducted and to promote transparency into the results of  these analyses, we recommend that the Director of OMB revise existing  guidance to include directing agencies to report on the IT Dashboard the  results from the OAs of their steady state investments. Agency Comments and Our Evaluation

In commenting on a draft of this report, OMB and the five agencies  agreed with our findings and recommendations. Appendix I: Objective, Scope, and Methodology

Our objective was to determine the extent to which selected federal  agencies assess the performance of steady state information technology  (IT) investments in accordance with Office of Management and Budget  (OMB) guidance. To accomplish our objective, we selected the five  agencies (Departments of Defense (DOD), Health and Human Services  (HHS), Homeland Security (DHS), the Treasury (Treasury), and Veterans  Affairs (VA)) that have the largest budgets for major steady state IT  investments; collectively, these investments accounted for approximately  $37 billion annually or about 70 percent of all reported IT operations and  maintenance (O&M) spending. No: if none of the aspects of the key factor were addressed.
Why GAO Did This Study

Of the $79 billion federal agencies budgeted for IT in 2011, $54 billion (about 69 percent) was reported to have been spent on the operations and maintenance of existing legacy IT systems--commonly referred to as steady state investments. Given the size and magnitude of these investments, it is essential that agencies effectively manage them to ensure they continue to meet agency needs. As such, OMB directs agencies to periodically examine the performance of such investments against, among other things, established cost, schedule, and performance goals by performing annual OAs.
GAO was asked to determine the extent to which federal agencies analyze the performance of steady state investments in accordance with OMB guidance. To do so, GAO (1) selected five agencies, DOD, HHS, DHS, Treasury, and VA, which reported spending $4.6 billion annually on major steady state investments; and (2) and compared their fiscal year 2011 OAs to OMB criteria. GAO also analyzed documents and interviewed agency officials regarding any variances as well as their causes.

What GAO Found

Federal agency assessments of the performance of information technology (IT) investments in operations and maintenance (O&M)--commonly referred to as operational analyses (OAs)--vary significantly. Office of Management and Budget (OMB) guidance calls for agencies to develop an OA policy and perform such analyses annually to ensure steady state investments continue to meet agency needs. The guidance also includes 17 key factors (addressing areas such as cost, schedule, customer satisfaction, and innovation) that are to be assessed. The five agencies GAO reviewed varied in the extent to which they carried out these tasks.
The Departments of Homeland Security (DHS) and Health and Human Services (HHS) developed a policy which included all OMB assessment factors and performed OAs. However, they did not include all investments and key factors. In particular, DHS analyzed 16 of its 44 steady state investments, meaning 28 investments with annual budgets totaling $1 billion were not analyzed; HHS analyzed 7 of its 8 steady state investments. For OAs performed by DHS and HHS, both fully addressed approximately half of the key factors. With regard to the DHS and HHS investments that did not undergo an analysis or were not fully assessed against key factors, agency officials said this was due in part to program officials inconsistently applying OMB and agency guidance in conducting OAs and that OAs were not a priority. DHS and HHS have recently begun to take action to make OAs a priority and improve consistency. For example, DHS's chief information officer recently issued a directive requiring all steady state IT investments to conduct analyses annually and plans to assign staff in the office of the chief information officer to review them to ensure they are complete.
The Departments of Defense (DOD), the Treasury (Treasury), and Veterans Affairs (VA) did not develop a policy and did not perform analyses on their 23 major steady state investments with annual budgets totaling $2.1 billion. DOD and VA officials said that they did not have a policy or perform analyses because they measure the performance of steady state investments via development of plans and business cases submitted to OMB (called exhibit 300s) as part of the budget process. While these can be helpful in managing performance and do address aspects of the 17 key factors, they do not address 11 of the key factors. For example, the exhibit 300 does not address reviewing strategic business results and making recommendations to modify or terminate an investment. Treasury officials stated that they did not to perform OAs in 2011 and instead decided to use the time to develop a policy. However, the officials stated that they did not anticipate the policy to be completed until the end of this calendar year.
Overall, these five agencies have steady state investments with a fiscal year 2011 budget of over $3 billion which have not undergone needed analyses. While OMB requires agencies to perform OAs, its existing guidance does not provide mechanisms that ensure the OAs are completed and allow public transparency into the results of the assessments. Until agencies address these shortcomings, there is increased risk that these agencies will not know whether the multibillion dollar investments fully meet their intended objectives.

What GAO Recommends

GAO is recommending that DOD, Treasury, and VA develop an OA policy and conduct annual OAs; and that DHS and HHS ensure OAs are being performed for all investments and that all factors are fully assessed. GAO is also recommending that OMB revise its guidance to incorporate mechanisms to ensure OAs are completed and provide for increased transparency. In commenting on a draft of this report, OMB and the five agencies GAO reviewed agreed with its content and recommendations.