DOD Has Outlined the Responsibilities of the New Directorate, but Has Not Conducted a Business Case Analysis to Demonstrate How It Will Achieve Cost Savings

DOD has outlined the areas of responsibility for its Education and  Training Directorate, including consolidation and management of a  number of activities currently performed by the services. However, in its  plans, DOD has not demonstrated through a fully developed business  case analysis how creating a shared service for education and training  will result in cost savings. Further, the  National Defense Authorization Act for Fiscal Year 2013 required DOD to  develop business case analyses for its shared service proposals as part  of its submissions on its plans for the implementation of the DHA,  including, among other things, the purpose of the shared service and the  anticipated cost savings. DOD does not have a fully developed business case analysis for medical  education and training because it has not yet completed the first step of  that analysis, which is to identify specific problems, which, given the  stated purpose of shared services, should be directed toward the  achievement of cost savings. Several of DOD’s other shared service  projects present a clear linkage between (1) a stated problem, (2)  proposed process changes, and (3) an estimate of benefits, costs, and  risks. In contrast, DOD listed the new processes the  Directorate will employ, but it did not explain the problem its proposed  new processes will address, and how they will achieve cost savings. However, senior service officials stated that the Directorate was unlikely  to achieve significant savings and that its creation serves more as a  functional realignment than a cost savings endeavor. Without a business case analysis that links (1) a stated problem, (2)  proposed process changes, and (3) an estimate of benefits, costs, and  risks, the role of the Directorate remains ambiguous, and it is unclear how  DOD will measure its accomplishments and hold the Directorate  accountable for achieving cost savings by sharing training and education  services. DOD Cannot Determine Whether the Consolidation of Training at METC Has Resulted in Cost Savings but Is Taking Action to Improve Some Processes for Evaluating Training Effectiveness

DOD established METC as part of the 2005 BRAC process to provide  interservice training for enlisted service members and to achieve cost  savings. However, DOD is unable to determine whether the consolidation  of medical education and training for enlisted personnel at METC has  resulted in cost savings because it did not establish a baseline for  spending on education and training prior to METC’s establishment. DOD Officials Cannot Demonstrate Whether the Consolidation of Training at METC Has Resulted in Cost Savings

DOD cannot demonstrate whether the consolidation of training at METC  has resulted in cost savings. However, some officials stated they are unsure whether  the services’ transfers were representative of their true costs for the  transferred programs prior to the creation of METC. Additionally, the  funding transfers from the services were not sufficient to fund training at  METC, and the Office of the Assistant Secretary of Defense for Health  Affairs provided additional funding to cover this shortfall. DOD did not establish and monitor baseline cost information as part of its  metrics to assess performance to ensure that the establishment of METC  provided costs savings. DOD Has Designed Processes for Evaluating the Effectiveness of Training and Is Taking Action to Address Accreditation and Survey Response Issues

METC has designed quality assurance processes to provide continuous,  evaluative feedback related to improvements in education and training  support, and is taking action to address issues regarding course  accreditation and the post-graduation survey process. To improve the  level of feedback received from these surveys, METC officials have  begun a pilot process to conduct their own post-graduation surveys, using  an online survey program that can be sent directly to the students’ and  supervisors’ personal email addresses. Conclusions

DHA’s Education and Training Directorate is scheduled to begin  operations in August 2014 to oversee medical education and training  reform, but DOD does not have key information necessary to assess its  progress in realizing the reform effort’s goal of achieving cost savings. In  addition, without baseline cost information prior to future course  consolidation of training at METC and within the Education and Training  Directorate, DOD will be unable to assess potential cost savings. To help ensure that DOD has the necessary information to determine the  extent to which cost savings result from any future consolidation of  training within METC or the Education and Training Directorate, we  recommend that Assistant Secretary of Defense for Health Affairs direct  the Director of the DHA to develop baseline cost information as part of its  metrics to assess achievement of cost savings. Agency Comments

We provided a draft of this product to DOD for comment. Defense Health Care Reform: Additional Implementation Details Would  Increase Transparency of DOD’s Plans and Enhance Accountability.
Why GAO Did This Study

To help address DOD's escalating health care costs, in 2013 DOD established the DHA to, among other things, combine common medical services such as medical education and training. DOD trains its servicemembers for a wide variety of medical positions, such as physicians, nurses, therapists, and pharmacists. DHA's Education and Training Directorate is to oversee many aspects of DOD's medical education and training and is now expected to begin operations in August 2014. GAO was mandated to review DOD's efforts to consolidate medical education and training.
GAO examined the extent to which DOD has (1) conducted analysis to reform medical education and training to achieve cost savings and (2) determined whether the consolidation of training at METC has resulted in cost savings and designed processes to assess its effectiveness. GAO compared DHA implementation plans and METC budget information from fiscal years 2010 through 2012 with best practices and interviewed officials from the DHA, METC, and military services' Surgeons General offices.

What GAO Found

In its 2013 plans for the implementation of the Defense Health Agency (DHA), the Department of Defense (DOD) outlined the responsibilities of a new Education and Training Directorate, but has not demonstrated how its proposed reforms will result in cost savings. The National Defense Authorization Act for Fiscal Year 2013 required DOD to develop business case analyses for its shared service proposals as part of its submissions on its plans for the implementation of DHA, including, among other things, the purpose of the shared service and the anticipated cost savings. Although DOD has stated that the Directorate is a shared service that combines common services and that it will result in cost savings, DOD has not fully developed the required business case analysis for the medical education and training reforms. This is because DOD has not yet completed the first step of the process, which includes identifying the specific problems that the reform is intended to address and thereby achieve cost savings. Unlike the medical education and training reforms, other DOD shared service projects present a clear linkage between (1) a stated problem, (2) proposed process changes, and (3) an estimate of benefits, costs, and risks. For the Directorate, DOD has identified the new processes it will employ, but has not identified the concerns the proposed new processes are intended to address and how they will achieve cost savings. In addition, some officials are unconvinced that the potential cost savings will be achieved, and stated that the creation of the Directorate serves more as a functional realignment than a cost savings endeavor. Without a fully developed business case analysis, it is unclear how DOD will measure any accomplishments and hold the Directorate accountable for achieving cost savings.
DOD is unable to determine whether the consolidation of training at the Medical Education and Training Campus (METC) resulted in cost savings; however, DOD is taking action to improve some of the processes for evaluating the effectiveness of training at METC. DOD co-located medical training for enlisted medical servicemembers at METC as part of the 2005 Base Realignment and Closure Commission (BRAC) process to achieve cost savings, and subsequently, the services decided to consolidate their training. However, some officials stated they were unsure whether all funds were transferred to METC. Furthermore, due to a shortage of military service funds, the Office of the Assistant Secretary of Defense for Health Affairs provided funding for METC in addition to the services' transfers. DOD is unable to determine whether the consolidation of training at METC resulted in cost savings because it did not develop baseline cost information as part of its metrics to assess METC's success. Baseline cost information is a key characteristic of performance metrics critical to ensuring that processes achieve the desired results. Without baseline cost information prior to future course consolidation of training at METC and within the Education and Training Directorate, DOD will be unable to assess potential cost savings. DOD has designed processes to evaluate the quality of training at METC—including processes related to certification rates, accreditation, and surveys. Further, DOD has taken action to improve some processes. For example, to improve the level of feedback received from METC surveys, METC officials have begun a pilot process to conduct their own post-graduation surveys.

What GAO Recommends

GAO recommends that DOD conduct a fully developed business case analysis for the Education and Training Directorate and develop baseline cost information as part of its metrics to assess cost savings for future consolidation efforts. In comments to a draft of this report, DOD concurred with each of GAO's recommendations.