VA  Central Office expected this start-up funding would be repaid by the pilot  locations. Planning and Early Implementation Phases of the Dialysis Pilot Had Significant Weaknesses

There were a number of weaknesses in VA’s execution of the planning  and early implementation phases of the Dialysis Pilot that collectively  could limit the achievement of its goals. VA’s Pilot Location Selection Process Was Not Transparent or Appropriately Documented

While the Dialysis Workgroup reported using several criteria to select the  Dialysis Pilot locations and documented some of these criteria in the  approval documents for the Dialysis Pilot, it did not document how these  criteria were applied or whether it assessed all 153 VAMCs for potential  inclusion in the Dialysis Pilot. VA Did Not Produce Consistent and Comparable Cost Estimates for Pilot Locations

It is not possible to determine whether pilot locations completed reliable  cost estimations because these estimates are not consistent and  comparable. Reliable cost estimates are necessary to ensure that pilot location costs   Generating reliable and  are comparable across the four pilot locations.comparable cost estimates prior to opening the pilot locations was critical  to the early implementation of the Dialysis Pilot in order to ensure that  appropriate site-specific baseline cost estimates were generated that  would allow VA to evaluate the cost of the Dialysis Pilot and ensure that  any cost savings generated by the pilot locations could be accurately  calculated. We found that not all pilot locations used the  validated formulas developed by VA systems redesign engineers to  calculate their cost estimates. VA Did Not Provide Clear and Timely Guidance to Pilot Locations on Start-up Fund Repayment and Cost Savings Calculations

VA Central Office officials did not provide VISN and VAMC officials with  clear and timely written guidance or instructions on how to pay back   start-up funds or how to calculate cost savings from the Dialysis Pilot. Pilot locations are  expected to achieve cost savings through the outpatient dialysis clinics  and to repay their start-up funding. VA’s Dialysis Pilot Has a Stated Mission but Lacks Clearly Defined Performance Criteria and an Evaluation Plan

VA Central Office has not yet determined how it will define success for  the Dialysis Pilot or created clear performance measures linked to the  four Dialysis Pilot goals. When other leading public sector organizations  are engaged in efforts to improve their performance and help their  organizations become more effective—similar to VA’s goals for the  Dialysis Pilot—we found that these organizations commonly take three  steps: (1) define a clear mission and goals, (2) measure performance to  gauge progress toward achieving goals, and (3) use performance  information as a basis for decision making. Despite its  success in defining a clear mission and goals for the Dialysis Pilot, VA  has not developed a clear plan for evaluating the pilot. In March 2012, the VHA  Chief Business Office reported that VA is in the early stages of  establishing an agreement with a leading university research center to  conduct an evaluation of the Dialysis Pilot; however, no target dates were  provided for when this evaluation would begin or what aspects of the  Dialysis Pilot beyond cost-effectiveness it would evaluate. Because VA has not yet developed an evaluation plan or formally defined  performance measures for pilot locations, it does not have access to  consistent and reliable information on the performance of the pilot  locations and may not have this information accessible when it is time to  either make midcourse corrections for the Dialysis Pilot or decide whether  and how to open additional VA-operated outpatient dialysis clinics. VA set four goals for its Dialysis Pilot:   (1) improve quality of care, (2) increase access for veterans, (3) provide  additional dialysis research opportunities, and (4) achieve cost savings for  VA-funded dialysis treatments. In its plan for addressing our recommendations, VA stated that it is  developing a plan for the Dialysis Pilot that will address three of our  recommendations related to (1) the documentation of Dialysis Pilot key  decisions, including the selection of future pilot locations; (2) the creation  of reasonable cost estimates for pilot locations; and (3) guidance for the  repayment of start-up funding and cost savings calculations. Appendix I: Status of VA Dialysis Pilot Locations

Table 1 provides additional information on the current operating status of  the four Dialysis Pilot locations at the Department of Veterans Affairs (VA)  medical centers (VAMC) in Durham, North Carolina; Fayetteville, North  Carolina; Philadelphia, Pennsylvania; and Cleveland, Ohio.
Why GAO Did This Study

Veterans diagnosed with end-stage renal diseasea condition of permanent kidney failurerepresent one of the most resource-intensive patient populations at VA. These veterans are often prescribed dialysis, which is a life-saving and relatively expensive medical procedure that removes excess fluids and toxins from the bloodstream. VA began developing its Dialysis Pilot in 2009 with four goals: (1) improved quality of care, (2) increased veteran access, (3) additional medical research opportunities, and (4) cost savings. Through this pilot, VA will establish four VA-operated outpatient dialysis clinics in communities surrounding select VA medical centers by the end of fiscal year 2012 using start-up funding provided by VA Central Office. Pilot locations are expected to achieve cost savings and to repay their start-up funding. GAO examined VAs planning and early implementation efforts for the Dialysis Pilot, and how VA plans to evaluate the pilot.
GAO reviewed relevant VA documents, including those related to pilot location selection and cost estimation, and spoke with VA officials responsible for overseeing the Dialysis Pilot and representatives from all pilot locations.

What GAO Found

GAO found a number of weaknesses in the Department of Veterans Affairs (VA) execution of the planning and early implementation phases of the Dialysis Pilot. These weaknesses involved pilot location selection, cost estimation practices, and cost savings calculations that could collectively limit the achievement of the pilots goals. Specifically, VA did not do the following:
Appropriately document its pilot location selection process. VA did not maintain a clear and transparent pilot location selection process; it did not document how its criteria for pilot location selection were applied to all 153 VA medical centers (VAMC) or why substitutions in pilot locations were made. However, VA officials reported that several criteria, including dialysis patient prevalence and average treatment costs, were used to select the pilot locations in Durham and Fayetteville, North Carolina; Philadelphia, Pennsylvania; and Cleveland, Ohio.
Produce consistent and comparable cost estimates for pilot locations. VA did not complete consistent and comparable cost estimates for the four pilot locations. Specifically, GAO found several cases where pilot locations did not complete reliable cost estimates because they made changes to formulas and assumptions of the Dialysis Pilot cost estimation model, which was developed by VA systems redesign engineers.
Provide clear and timely guidance on start-up fund repayment and cost savings calculations. VA did not provide Veterans Integrated Service Network and VAMC officials with clear and timely written guidance or instructions on how to pay back start-up funds, or how to calculate cost savings generated by the pilot locations.
VA Central Office has not yet determined how it will achieve its goals for the Dialysis Pilot or created clear performance measures for the pilot locations. Previously, GAO found that leading public sector organizations take three steps to improve their performance and help their organizations become more effective: (1) define a clear mission and goals, (2) measure performance to gauge progress toward achieving goals, and (3) use performance information as a basis for decision making. While VA has defined a clear mission and goals for the Dialysis Pilot, it has only made limited progress in the remaining two steps. In March 2012, VA reported that it was in the early stages of establishing an agreement with a leading university research center to conduct an evaluation of the Dialysis Pilot; however, no target dates were provided for when this evaluation would begin or what aspects of the Dialysis Pilot it would evaluate. Because VA has not yet developed an evaluation plan or defined performance measures for pilot locations, it is not collecting consistent and reliable information on the performance of the pilot locations and thus may not have this information available when it is time to either make midcourse corrections to the Dialysis Pilot or decide whether and how to open additional VA-operated outpatient dialysis clinics. VA officials also told GAO they have developed a limited plan for expanding the Dialysis Pilot despite not having access to performance information on the existing four pilot locations.

What GAO Recommends

Among other actions, GAO recommends that VA improve its Dialysis Pilot by providing guidance for start-up fund repayment, as well as developing an evaluation plan that includes performance measures for the pilot locations. VA concurred with GAOs recommendations and provided an action plan to address them.