VBA Is Following Course Recommended by SEI, but Weaknesses Remain to Be Addressed

As recommended by SEI, VBA is continuing to work on the replacement  initiative at a reduced pace and taking action to address identified  weaknesses in the project’s overall management and software  development processes. For example, VBA has established a new  governance structure and has developed an integrated master schedule  that provides additional time and includes the full range of project  activities. Further, VBA has not yet  institutionalized many of the improvements that it has undertaken for the  initiative. In particular, process improvements remain in draft and have  not been established through documented policies and procedures. Nonetheless, if VBA does not institutionalize these  improvements, it increases the risk that these process improvements may  not be maintained through the life of the project or be available for  application to other development initiatives. By implementing the new governance and organizational structure and  ensuring that the project has priority, VBA partially responded to SEI’s  concerns in this area; however, VBA has not yet taken action with regard  to ownership responsibility for total system and process operating costs,  as SEI advised. However, according to project management officials, these costs do not  include expenditures for in-house development work. VETSNET Is Currently Processing a Portion of Compensation Claims, but Much Work Remains

After more than 10 years of effort, including the recent management,  organizational, and process improvements, VBA has achieved critical  functionalities needed to process and pay certain original compensation  claims using the replacement system, but it remains far from completing  the project. Nonetheless, the system requires further development  before it can be used to process claims for the full range of compensation  and pension benefits available to veterans and their dependents. According to VBA officials, all five of the software applications that make  up the new system are now being used in VA’s 57 regional offices to  establish and process new compensation claims for veterans. As of March  2007, VBA leadership reported that the replacement system was providing  monthly compensation payments to almost 50,000 veterans (out of about   3 million veterans who receive such payments). Specifically, while all five  software applications can now be used to process original compensation  claims for veterans, two of the applications—Awards and FAS—require  further development before the system will be able to process claims for  the full range of benefits available to veterans and their dependents. It has not defined processes and resources for capacity planning  for the project. In addition, although  VBA has improved key processes for managing the software development,  these processes have not yet been institutionalized in defined policies and  procedures, and performance measures of productivity and user  satisfaction have not been developed. Appendix I: Objectives, Scope, and Methodology

Our objectives were to determine (1) to what extent the Department of  Veterans Affairs (VA) has followed the course of action recommended by  the Carnegie Mellon Software Engineering Institute (SEI) and addressed  the concerns that it raised and (2) the current status of the replacement  project, the Veterans Service Network (VETSNET).
Why GAO Did This Study

Since 1996, the Veterans Benefit Administration (VBA) has been working on an initiative to replace its aging system for paying compensation and pension benefits. In 2005, concerned about the slow pace of development, VBA contracted with the Software Engineering Institute (SEI) for an independent evaluation of the project, known as the Veterans Service Network (VETSNET). SEI advised VBA to continue working on the project at a reduced pace while addressing management and organization weaknesses that it determined had hampered the project's progress. GAO was requested to determine to what extent the VETSNET project has followed the course of action recommended by SEI and describe the project's current status. To perform its review, GAO analyzed project documentation, conducted site visits, and interviewed key program officials.

What GAO Found

VBA is generally following the course of action recommended by SEI by continuing to work on the replacement initiative at a reduced pace, while taking action to address identified weaknesses in overall management and software development processes. For example, VBA established a new governance structure for the initiative that included senior management and involved all stakeholders, and it incorporated all critical areas of system development in an integrated master schedule. However, not all of SEI's management concerns have been addressed. For example, SEI advised VBA to ensure that stakeholders take ownership responsibility for the project, including the total system and process operating costs; however, although VBA is tracking costs incurred by contractors, it is not yet tracking and reporting in-house costs incurred by the project. Further, although the project has improved its management processes, such as establishing a process to manage and stabilize system requirements, it has not yet developed processes for capacity planning and management. This will be important for ensuring that further VETSNET development does not lead to delays and slowdowns in processing of benefits. In addition, although the project has established certain performance measures, it has not yet established results-oriented measures for productivity and user satisfaction, both of which will be important for measuring progress. Finally, the process improvements that VBA has incorporated in the replacement initiative remain in draft and have not been established through documented policies and procedures. If VBA does not institutionalize these improvements, it increases the risk that they may not be maintained through the life of the project or be available for application to other development initiatives. After more than 10 years of effort, including the recent management, organizational, and process improvements, VBA has developed critical functionalities needed to process and pay certain original compensation claims using the replacement system, but it remains far from completing the project. According to VBA officials, all five of the major software applications that make up the new system are now being used to establish and process new compensation claims for veterans. In total, the replacement system is currently providing monthly compensation payments to almost 50,000 veterans (out of about 3 million veterans who receive such payments); the system was used to process about 83 percent of all new compensation claims completed in March 2007. Nonetheless, the system requires further development before it can be used to process claims for the full range of compensation and pension benefits available to veterans and their dependents.