VA Dialysis Pilot Evaluation

VA is performing an internal evaluation of the performance of the four  pilot locations by assessing the quality of care—including clinical quality,  patient satisfaction, and access to care—and treatment costs at each pilot  location. Dialysis Pilot Evaluation Contractor Review

In September 2012, VA awarded a 3-year contract to an evaluation  contractor to provide an independent evaluation of its Dialysis Pilot— including a comparison of the clinical quality provided by the four pilot  locations with non-VA dialysis providers and a dialysis per treatment cost  comparison between the pilot locations and the Non-VA Medical Care  Program. VA Has Not Documented Criteria or Plans for Concluding the Dialysis Pilot

Even though it is 5 years into the Dialysis Pilot, VA Central Office has not  yet set a timetable for completing the pilot or documented how it will  determine the success of the four pilot locations. We have previously  outlined best practices for project scheduling that specify project timelines  are critical for managing and measuring an agency’s performance on  projects and that it is necessary for the agency to have a document that  details the rationale used in developing a project timeline. In addition,  we have outlined best practices for designing evaluations that state that  choosing and documenting well-regarded criteria that are used to make  comparisons can lead to strong, defensible conclusions. According to VA’s business analysis of the Dialysis Pilot conducted prior  to opening the pilot locations, VA planned to conclude the Dialysis Pilot   5 years after the pilot locations opened and assumed all four pilot  locations would begin operations at generally the same time. However, in March 2014, VA officials told us they are no longer  operating under the 5-year timeline outlined in this plan, although they  have not updated the timeline included in the Dialysis Pilot business  analysis. According to VA officials, the first  achievement that indicates success is the creation of a model for a  VA-operated, free-standing dialysis clinic that can be replicated by  other VAMCs. VA officials stated that the second  achievement that indicates success is the confirmation of the time  necessary for a pilot location to reach its breakeven point—which VA  has defined as the point when a pilot location achieves cost savings in  the Non-VA Medical Care Program for dialysis services for its  sponsoring VAMC and repays its start-up funding to VA Central  Office. By not clearly communicating  these milestones as criteria for concluding the Dialysis Pilot, the  transparency of VA’s management decisions on pilot location outcomes is  compromised and the Department lacks accountability for ensuring the  success of the Dialysis Pilot. Current Data on Quality of Care and Treatment Costs Are Limited Due to Ongoing Evaluations

VA and the evaluation contractor have been evaluating the Dialysis Pilot  for 2 years; however, neither has concluded its evaluation of the Dialysis  Pilot and data on two of the pilot locations—Philadelphia and Cleveland— is limited due to their delayed opening. As a result, VA recommended that another full 12 months of data from the  Philadelphia and Cleveland pilot locations be analyzed before making  conclusive comparisons of the treatment costs of these pilot locations  with non-VA dialysis providers. In addition, the transparency of the Dialysis Pilot is jeopardized  by VA not clearly documenting the achievements stated by VA officials— the creation of a replicable model for additional VA-operated free- standing dialysis clinics and demonstrating a breakeven point—as criteria  for concluding the Dialysis Pilot.
Why GAO Did This Study

Veterans with end-stage renal disease—a condition of permanent kidney failure—are one of the most resource-intensive patient populations served by the VA and are generally prescribed a life-saving medical procedure called dialysis. In 2009, VA began developing a pilot program at four VAMCs to provide dialysis to veterans in VA-operated, free-standing dialysis clinics largely in an effort to stem rising costs for providing such care in the private sector through the Non-VA Medical Care Program. In May 2012, GAO issued a report identifying several weaknesses in VA's execution of the planning and early implementation phases of the Dialysis Pilot.
GAO was asked to continue its evaluation of the Dialysis Pilot. GAO examined the extent to which VA documented plans for concluding the Dialysis Pilot and the status of data on the quality of care and treatment costs for the four pilot locations.
GAO reviewed relevant documents from VA and the evaluation contractor selected by VA to perform an independent analysis of the pilot locations. GAO also spoke with VA officials responsible for managing the Dialysis Pilot, representatives from all four pilot locations, and evaluation contractor officials responsible for reviewing the performance of the four pilot locations.

What GAO Found

Five years into the Dialysis Pilot, the Department of Veterans Affairs (VA) Central Office still has not set a timetable for completing the pilot or documented how it will determine the success of the pilot locations. GAO previously identified best practices that state that a project timeline is critical for managing and measuring an entity's performance on projects and that choosing and documenting well-regarded criteria that are used to make comparisons can lead to strong, defensible conclusions. Initially, VA planned to conclude the Dialysis Pilot after the pilot locations were all open for 5 years. However, in March 2014, VA officials told GAO they are no longer operating under this timeline but instead plan to conclude the pilot once the pilot locations achieve (1) the creation of a model for a VA-operated, free-standing dialysis clinic that can be replicated by other VA medical centers (VAMC) and (2) the confirmation of the time necessary for a pilot location to reach a “breakeven point.” VA considers that a pilot location has achieved a breakeven point when it repays its start-up funding and the VAMC realizes a cost savings because its treatment cost for dialysis at the pilot location is lower than purchasing care from non-VA dialysis providers. However, VA has not formally documented these pilot location achievements as criteria for concluding the Dialysis Pilot. By not doing so, the transparency of VA's management decisions on pilot location outcomes is compromised and the Department lacks accountability for ensuring the success of the Dialysis Pilot.
The Dialysis Pilot has been under evaluation for 2 years by VA and the contractor it selected to conduct an independent analysis of pilot location quality of care and treatment costs. However, neither has concluded its evaluation. Specifically, VA noted that the delayed openings and initial operational issues of two pilot locations—Philadelphia, Pennsylvania, and Cleveland, Ohio—led to limited data availability, and it recommended another 12 months of data be collected on these two pilot locations before drawing conclusions.

What GAO Recommends

GAO recommends that VA document its plans for concluding the Dialysis Pilot. VA concurred with GAO's recommendation but did not clearly delineate its plans for pilot conclusion.