	Conclusions

VA deployed the first two of four releases of its long-term system solution  by its planned dates, therefore providing improved claims-processing  functionality to all regional processing offices, such as the ability to  calculate original and amended benefit claims. In addition, the Agile  process allowed the department the flexibility to accommodate legislative  changes and provide functionality according to business priorities, such as  housing rate adjustments. Due to these delays, VA planned to reprioritize what  functionality would be included in its third release. Until VA  improves these areas, management does not have the visibility it needs to  clearly communicate progress to stakeholders and estimate when future  system capabilities will be delivered. Additionally, reduced visibility and  unresolved issues in its development processes may result in the  department continuing to remove functionality that was expected in future  releases, thus delivering a system that does not fully and effectively  support the implementation of education benefits as identified in the Post- 9/11 GI Bill. Recommendations for Executive Action

To help guide the full development and implementation of the Chapter 33  long-term solution, we recommend that the Secretary of Veterans Affairs  direct the Under Secretary for Benefits to take the following five actions:    establish performance measures for goals and identify constraints to  provide better clarity in the vision and expectations of the project;    establish bidirectional traceability between requirements and legislation,  policies, and business rules to provide assurance that the system will be  developed as expected;    define the conditions that must be present to consider work “done” in  adherence with agency policy and guidance;  implement an oversight tool to clearly communicate velocity and the  changes to project scope over time; and  improve the adequacy of the unit and functional testing processes to  reduce the amount of system rework. This act amended  Title 38 United States Code to include Chapter 33, which provides educational assistance  for veterans and members of the armed forces who served on or after September 11,  2001. To develop the system for its long-term solution, VA is relying on contractor assistance  and is using an incremental development approach, called Agile software development,  which is to deliver software functionality in short increments before the system is fully  deployed. Given the importance of delivering education benefits to veterans and their families, we  were asked to review the long-term solution to    determine the status of VA’s development and implementation of its information  technology (IT) system to support the implementation of education benefits identified  in the Post-9/11 GI Bill and    evaluate the agency’s effectiveness in managing its IT project for this initiative. In doing so, VA provided its regional  processing offices with key automated capabilities to prepare original and amended  benefit claims. While VA did not previously estimate costs for these releases and, as  such, could not track estimated to actual costs, it has reported that about $84.6 million  has been obligated through July 2010. However, VA did not ensure that certain critical tasks were performed  that were expected to be part of the second release. Specifically, it did not complete the  conversion of data from systems in the interim solution to the systems developed for the  long-term solution and did not complete the development of interfaces between the new  system and legacy systems. However, VA intends to provide this  capability after its fourth release or under a separate initiative. VA has established a vision that captures the project  purpose and goals; however, it has not established metrics for the project’s goals or  prioritized project constraints. VA has  also established a plan that identifies how to maintain requirements traceability  within an Agile environment; however, the traceability between legislation, policy,  business rules, and test cases was not always maintained. However,  VA has not yet established criteria for work that is considered “done” at all levels of  the project. In October 2008, VA established its Chapter 33 initiative to develop the capability to  process the new education benefit. The long-term solution was expected to be complete enough to replace the interim  solution by June 2010 and to include additional capabilities to provide a fully  automated end-to-end system to support the delivery of education benefits by  December 2010. To provide effective management for an Agile project, such as the development of the Chapter 33 long-term solution, a key component for success is demonstrating effective use of the Agile practices: working as one team, focusing on business priorities, delivering functionality in short increments, and inspecting and adapting the project as appropriate. While VA has taken an important step to effectively manage its development of the system for processing Chapter 33 educational benefits by establishing a cross-functional team, it has not yet fully ensured business priorities are a focus, demonstrated that it is delivering quality functionality in short increments, or provided mechanisms to enable inspection and adaptation of the project.
Why GAO Did This Study

The Post-9/11 GI Bill was signed into law in June 2008 and provides educational assistance for veterans and members of the armed forces who served on or after September 11, 2001. The Department of Veterans Affairs (VA) is responsible for processing claims for these new education benefits. VA concluded that its legacy systems and manual processes were insufficient to support the new benefits and, therefore, began an initiative to modernize its benefits processing capabilities. The long-term solution was to provide a fully automated end-to-end information technology (IT) system to support the delivery of benefits by December 2010. VA chose an incremental development approach, called Agile software development, which is intended to deliver functionality in short increments before the system is fully deployed. GAO was asked to (1) determine the status of VA's development and implementation of its IT system to support the implementation of education benefits identified in the Post-9/11 GI Bill and (2) evaluate the department's effectiveness in managing its IT project for this initiative.

What GAO Found

VA has made important progress in delivering key automated capabilities to process the new education benefits. Specifically, it deployed the first two of four releases of its long-term system solution by its planned dates, thereby providing regional processing offices with key automated capabilities to prepare original and amended benefit claims. In addition, the Agile process allowed the department the flexibility to accommodate legislative changes and provide functionality according to business priorities. While progress has been made, VA did not ensure that certain critical tasks were completed that were initially expected to be included in the second release by June 30, 2010. For example, the conversion of data from systems in the interim solution to systems developed for the long-term solution was not completed until August 23, 2010. Because of the delay, VA planned to reprioritize the functionality that was to be included in the third release. Further, while VA plans to include full self-service capabilities to veterans, it will not do so in the fourth release as scheduled; instead it intends to provide this capability after the release or in a separate initiative. VA reported obligations and expenditures for these releases, through July 2010, to be approximately $84.6 million, with additional planned obligations of $122.5 million through fiscal year 2011. VA has taken important steps by demonstrating a key Agile practice essential to effectively managing its system development--establishing a cross-functional team that involves senior management, governance boards, key stakeholders, and distinct Agile roles. In addition, VA made progress toward demonstrating three other Agile practices--focusing on business priorities, delivering functionality in short increments, and inspecting and adapting the project as appropriate. Specifically, to ensure business priorities are a focus, VA established a vision that captures the project purpose and goals and established a plan to maintain requirements traceability. To aid in delivering functionality, the department established an incremental testing approach. It also used an oversight tool, which was intended to allow the project to be inspected and adapted by management. However, VA could make further improvements to these practices. In this regard, it did not (1) establish metrics for the goals or prioritize project constraints; (2) always maintain traceability between legislation, policy, business rules, and test cases; (3) establish criteria for work that was considered "done" at all levels of the project; (4) provide for quality unit and functional testing during the second release, as GAO found that 10 of the 20 segments of system functionality were inadequate; and (5) implement an oversight tool that depicted the rate of the work completed and the changes to project scope over time. Until VA improves these areas, management will lack the visibility it needs to clearly communicate progress and unresolved issues in its development processes may not allow VA to maximize the benefits of the system. To help guide the full development and implementation of the long-term solution, GAO is recommending that VA take five actions to improve its development process for its new education benefits system. VA concurred with three of GAO's five recommendations and provided details on planned actions, but did not concur with the remaining two.