Enrollment is generally the first step veterans take to obtain health care services, within VA or through community care. VA’s Health Eligibility Center manages the process of accepting applications, verifying eligibility, and determining enrollment, in collaboration with VA medical centers. VA requires veterans’ enrollment applications be processed within 5 business days of receipt, including pending applications that require additional information from the applicant to process. Once enrolled, veterans can access VA health services by scheduling an appointment. VA’s scheduling policy establishes the procedures for scheduling medical appointments, as well as sets the requirements for staff directly or indirectly involved in the scheduling process (e.g., training). A scheduler at the VA medical facility is responsible for making appointments for new and established patients (i.e., patients who have visited the same VA medical center in the previous 24 months), which are then recorded in VA’s electronic scheduling system. VA scheduling policy requires patients who have requested an appointment and have not had one scheduled within 90 days to be placed on VA’s electronic wait list. VA determines wait times at each facility based on outpatient appointment information from its scheduling system. VA is required to publish information on appointment wait times at each VA medical facility for primary care, specialty care, and hospital care and medical services, which it does through two public websites. In November 2014, VA began posting monthly wait times for scheduling appointments at all VA medical facilities. One public website provides links to spreadsheets containing data for each VA medical facility, such as the average wait times for primary, specialty, and mental health care appointments and the number of patients on the electronic wait list. In April 2017, VA created a second public “Access and Quality in VA Healthcare” website to post both patient access data and information on VA medical facilities’ performance on various quality metrics. This website aims to help veterans find wait times at a specific facility. This information would allow veterans and their family members to use the wait-time data on this website to determine the best option for obtaining timely care. In order to receive needed care in a timely manner, veterans may need to obtain care outside of VA medical facilities through one of VA’s community care programs. VA has purchased health care services from community providers through various community care programs since 1945. Veterans may be eligible for community care when they are faced with long wait times or travel long distances for appointments at VA medical facilities, or when a VA facility is unable to provide certain specialty care services. Since 2014, Congress has taken steps to expand the availability of community care for veterans. The Veterans Access, Choice, and Accountability Act of 2014 provided up to $10 billion in funding for veterans to obtain health care services from community providers. The law established a temporary program—called the Veterans Choice Program (Choice Program)—to offer veterans the option to receive hospital care and medical services from a community provider when a VA medical facility could not provide an appointment within 30 days, or when veterans resided more than 40 miles from the nearest VA facility or faced other travel burdens. VA contracted with two third-party administrators (TPA) to establish networks of community providers, schedule veteran appointments with those providers, and pay those providers for services rendered through the Choice Program. In June 2018, the VA MISSION Act of 2018 was enacted to further address some of the challenges faced by VA in ensuring timely access to care. The Act required VA to implement within 1 year a permanent community care program—the Veterans Community Care Program (VCCP). The act identified criteria that all veterans enrolled in the VA health care system would be able to qualify for care through the VCCP; for example, if VA does not offer the care or service needed by the veteran or VA cannot provide the veteran with care and services that comply with its designated access standards. The access standards include appointment wait times for a specific VA medical facility; for example, veterans may be eligible for care through the VCCP if VA cannot provide care within 20 days for primary and mental health care, and 28 days from the date of request for specialty care, unless veterans agree to a later date in consultation with their VA health care provider. VA has taken a number of actions to address our recommendations regarding deficiencies we found in wait-time measurement and implementation of its scheduling policy. For wait-time measurement, these actions included changes to the wait-time measurement definitions, provision and documentation of scheduler training, and improved oversight through audits, all of which have been in a state of flux for the past 6 years. On July 12, 2019, VA provided us additional updates on efforts to implement our related recommendations. This new information fully addresses one of our recommendations. In December 2012, we found that outpatient medical appointment wait times reported by VA were unreliable, and, therefore, VA was unable to identify areas that needed improvement or mitigate problems for veterans attempting to access care. VA typically has measured wait times as the time elapsed between the ‘start date’—a defined date that indicates the beginning of the measurement—and the ‘end date’, which is the date of the appointment. At the time of our 2012 report, VA measured wait times as the number of days elapsed from the start date identified as the desired date—the date on which the patient or health care provider wants the patient to be seen—to the date of the appointment. We found that the reliability of the reported wait-time measures was dependent on the consistency with which schedulers recorded the desired date in the scheduling system, as required by VA’s scheduling policy. However, VA’s scheduling policy and training documents for recording the desired date were unclear and did not ensure consistency. We observed that not all schedulers at VA medical centers that we visited recorded the desired date correctly. Therefore, we recommended that VA either clarify its scheduling policy to better define the desired date, or identify clearer wait- time measures that are not subject to interpretation and prone to scheduler error. VA concurred with the recommendation, which we have identified as among those recommendations that warrant priority attention. Actions VA has taken or is taking to address this recommendation include: changes to the start date and definitions for wait-time measurement, provision and documentation of scheduler training, and improved oversight through scheduler audits. In addition, we are currently assessing new information VA provided in July 2019, which will include obtaining additional evidence and clarification from VA to see whether it has fully addressed our concerns. VA’s Actions to Change Start Dates for Wait-Time Measurement While the terminology for the start dates of the wait-time measurement has changed several times over the past 6 years, we believe that the current definitions of the start dates are substantively the same as those we reviewed—and found to be deficient—in our 2012 report. VA subsequently introduced new terms with similar definitions—from “desired date” to “preferred date”—without fundamentally addressing the deficiency. See table 1 for the changes to and definitions of the start dates for measuring outpatient appointment wait times and wait-time goals since June 2010. As table 1 shows, for new patients and established patients seeking appointments without a return-to-clinic date specified by their provider, VA changed the terminology of the start date to preferred date in its July 2016 scheduling policy from what it had established in its June 2010 policy. However, the definition of preferred date is substantively the same as the definition of desired date in the previous scheduling policy, the latter of which we found to be subject to interpretation and prone to scheduler error in our 2012 report. We continue to believe that the preferred date is also subject to interpretation and prone to scheduler error, which poses concerns for the reliability of wait times measured using the patient’s preferred date. In its updated July 2016 scheduling policy, VA also changed the terminology of the start date to the “clinically indicated date” for established patients whose provider has documented a clinically appropriate return-to-clinic date in the patient’s electronic health record. The clinically indicated date is substantively the same as the definition of desired date for established patients in the previous scheduling directive. While VA has not clarified the definitions of start dates, VA has taken actions intended to improve the accurate recording of the clinically indicated date in three ways: 1. VA requires clinical leadership (such as the Associate Chief of Staff) at each VA medical facility to ensure that providers enter the clinically indicated date in the electronic health record for future appointments; 2. VA standardized the entry of the clinically indicated date in the electronic health record to improve the accuracy of the date, which was implemented across all VA medical facilities as of July 2018; and 3. VA created a technology enhancement to enable the automatic transfer of the clinically indicated date from the electronic health record to the scheduling system. As a result, the scheduler no longer has to retrieve the date from veterans’ electronic health records and manually enter it into the scheduling system. VA reported that this enhancement was implemented at all but three VA medical facilities as of January 2019. In July 2019, VA reported to us that the error rate for the patient indicated date (either the clinically indicated date, or in the absence of that date, the patient’s preferred date) was 8 percent of about 667,000 appointments audited in the most recent biannual audit cycle, ending March 31, 2019. VA cites an almost 18 percent improvement in reducing the number of errors caused by manual entry of the clinically indicated date due to the use of the technology enhancements. VA’s Actions to Provide and Document Scheduler Training Although VA updated its scheduling policy in 2016, we believe the instructions, which form the basis for wait-time measurement, are still subject to interpretation and prone to scheduler error, making training and oversight vital to the consistent and accurate implementation of the policy. VA reported that 97 percent of all staff who scheduled an appointment within 30 days completed the required scheduling training as of July 2, 2019. VA stated that the department will closely monitor compliance with scheduler training completion for the remaining staff. Given the high turnover among schedulers, it is important that VA remain vigilant about scheduler training, ensuring all who need it receive it. VA’s Actions to Improve Oversight through Scheduler Audits VA has taken a number of actions to improve oversight of the scheduling process through biannual scheduling audits at VA medical centers and second level audits, as well as completion of the first system-wide internal audit of scheduling and wait-time data. Biannual scheduler audits. VA’s July 2016 scheduling policy required biannual audits of the timeliness and appropriateness of schedulers’ actions and accuracy of entry of the clinically indicated date and preferred date, the start dates of wait-time measurement as identified by the revised scheduling policy. In June 2017, VA deployed a standardized scheduling audit process for staff at VA medical centers to use. As part of our recommendation follow-up in July 2019, VA reported 100 percent completion of the required biannual scheduling audits in fiscal year 2018. As noted above, VA reported to us that the error rate for the patient indicated date (either the clinically indicated date, or in the absence of that date, the patient’s preferred date) was 8 percent of about 667,000 appointments audited. While VA asserts that errors in the clinically indicated date have decreased, an error rate of 8 percent still yields errors in more than 53,000 appointments audited. Given these errors, we remain concerned about the reliability of wait times measured using preferred date (one part of the patient indicated date), and have requested additional information from VA about these errors. Second level scheduler audits. In November 2018, VA implemented a second-level scheduling audit (Audit the Auditors program), which is overseen by the VA integrated service networks tasked with oversight of VA medical facilities within their regions. Each medical center within a network region is paired with another medical center and they audit each other’s scheduling audit. Throughout the cycle, medical centers share their findings with each other and the network. The goal is to standardize scheduling audit practices across the network and to ensure reliability of the scheduler audit results. According to VA, the first cycle was completed April 30, 2019, by all VA medical centers. First internal system-wide audit of wait-time data and scheduling. In its first internal audit completed in August 2018, VA was unable to evaluate the accuracy and reliability of scheduling and the wait-time data. Specifically, VA was unable to determine the accuracy and reliability of the scheduling and wait-time data, databases, and data flow from the electronic health record and scheduling system to the VA Access and Quality website because they were not able to obtain the rules for calculating wait times. Given our continued concerns about VA’s ability to ensure the reliability of the wait-time data, we plan to obtain additional information from VA about its methodology and assessment of evidence underlying the audit findings. In December 2012, we also found inconsistent implementation of VA’s scheduling policy that impeded VA medical centers’ scheduling of timely medical appointments. Specifically, we found that not all of the clinics across the medical centers we visited used the electronic wait list to track new patients that needed medical appointments as required by VA’s scheduling policy, putting these patients at risk of being lost for appointment scheduling. Furthermore, VA medical centers’ oversight of compliance with VA’s scheduling policy, such as ensuring the completion of required scheduler training, was inconsistent across facilities. Scheduler training was particularly important given the high volume of staff with access to the scheduling system—as of July 2, 2019, VA reported there were approximately 33,000 staff that had scheduled an appointment within the last 30 days. We also found that VA medical centers identified the outdated and inefficient scheduling system as one of the problems that can impede the timely scheduling of appointments and may impact their compliance with VA’s scheduling policy. We recommended VA ensure that VA medical centers consistently and accurately implement VA’s scheduling policy, including use of the electronic wait list, as well as ensuring that all staff with access to the scheduling system completes the required training. VA concurred with this recommendation, which we also have identified as among those recommendations that warrant priority attention. VA’s actions to improve implementation of the scheduling policy, including updated information VA provided in July 2019, fully addresses this recommendation. VA issued an updated scheduling policy in July 2016 that provided clarification on scheduling roles and responsibilities for implementing the policy and business rules for scheduling appointments, such as using the electronic wait list, and required biannual scheduler audits. VA also ensured almost all schedulers received training on the updated scheduling policy and improved oversight through audits, as previously described. In addition, VA plans to rapidly deploy a single nationwide scheduling system that is intended to simplify the operating environment for schedulers and may mitigate challenges identified in our 2012 report. The new scheduling system will be a resource-based system where each provider’s schedule is visible on one screen, instead of requiring the need to toggle through multiple screens as it currently exists. VA plans to roll out the new scheduling system starting in 2020, which is expected to be implemented in coordination with the planned modernization of the electronic health records system across VA facilities. According to VA, the scheduling system will be available for use in advance of the completion of the electronic health record implementation at some sites. In addition to the recommendations we made to improve VA’s wait-time data and implementation of its scheduling policy, we have also made recommendations to address other factors that affect the timeliness by which veterans obtain appointments. These recommendations have targeted VA’s enrollment processes and its management of veterans’ initial requests for care. While VA has taken some steps to address these recommendations, they have not yet been fully addressed. For example, we have found that VA’s wait-time measures do not yet capture the time it takes the agency to enroll veterans in VA health care benefits, or manage a veterans’ initial request for care. In September 2017, we found that VA did not provide its medical centers, who historically receive 90 percent of enrollment applications, with clear guidance on how to resolve pending applications, which led to delays in veteran’s enrollment. For example, we found instances in which pending applications remained unresolved for more than 3 months. We concluded these delays in resolving pending applications, along with previously documented delays due to errors in enrollment determinations, may result in veterans facing delays when obtaining health care services or incorrectly denied benefits. We made several recommendations to address these deficiencies, two of which we determined to be priority recommendations for VA to clearly define roles and responsibilities for (1) resolving pending applications and (2) overseeing the enrollment process. VA has made progress in addressing these priority recommendations by beginning to update, but not yet finalizing, its policies, procedures, and guidance on enrollment processing. In 2017, VA’s Health Eligibility Center began conducting secondary reviews of enrollment determinations. However, in fiscal year 2018, Health Eligibility Center staff found that 18 percent of rejected enrollment determinations and 8 percent of ineligible enrollment determinations that underwent secondary reviews were incorrect. These recommendations remain unimplemented as of July 2019. Once enrolled, we have found that VA’s management of veterans’ initial request for care have led to delays; and although VA has clarified timeliness requirements, it has yet to fully capture the wait veterans experience in scheduling initial appointments. In a number of reports from 2015 to 2018, we found instances in which newly enrolled veterans were not contacted to schedule initial primary care appointments, and did not complete initial primary care appointments and mental health evaluations according to VA timeliness requirements. These delays may be understated in VA data, because VA’s wait-time measures do not take into account the time it takes VA medical center staff to contact the veteran to determine a preferred date (the starting point for wait-time measurement) from the veteran’s initial request or referral. We found that the total amount of time it took for veterans to be seen by providers was often much longer when measured from the dates veterans initially requested to be contacted to schedule an appointment or were referred for an appointment by another provider than when using the veterans’ preferred dates as the starting point. See figure 1 for an example of how the two wait-time calculations differ for an initial primary care appointment. We made several recommendations to VA, including a priority recommendation to monitor the full amount of time newly enrolled veterans wait to be seen by a provider. VA has taken several steps to address the priority recommendation, including revising an internal report to help identify and document newly enrolled veterans and monitor their appointment request status. The report is intended to help VA and its medical centers oversee the enrollment and appointment process by tracking the total time from application to appointment. However, VA is still in the process of enhancing its electronic enrollment system to capture the application date for all newly enrolled veterans. Until the enhancements are implemented, VA may not consistently capture the start date for newly enrolled veterans, which, in turn, affects the reliability of its wait-time data. The priority recommendation remains unimplemented as of July 2019. VA has not implemented several of our recommendations related to the Choice Program that could impact veterans’ timely access to care under the VCCP. These recommendations address (1) establishing achievable community care wait-time goals and a scheduling process consistent with those goals, (2) collecting accurate and complete data to systematically monitor veteran community care wait times, and (3) other factors that could adversely affect veterans’ access to community care. VA has begun taking steps to address these recommendations as it implements the VCCP. Our review of the Choice Program in June 2018 found that despite having a wait-time goal, VA developed a scheduling process for the Choice Program that was not consistent with achieving that goal. The Veterans Access, Choice, and Accountability Act of 2014 required VA to ensure the provision of care to eligible veterans within 30 days of the clinically indicated date or, if none existed, within 30 days of the veteran’s preferred date. However, we found that those veterans who were referred to the Choice Program for routine care because services were not available at VA in a timely manner could potentially wait up to 70 calendar days for care. Under VA’s scheduling processes, this potential wait time included VA medical centers having at least 18 calendar days to prepare veterans’ Choice Program referrals to TPAs and another 52 calendar days for appointments to occur as scheduled by TPAs. Based on this finding, we recommended that VA establish an achievable wait-time goal for the VCCP that will permit VA to monitor whether veterans are receiving community care within time frames that are comparable to the amount of time they would otherwise wait to receive care at VA medical facilities. We also recommended that VA should design an appointment scheduling process for the VCCP that sets forth time frames within which (1) veterans’ referrals must be processed, (2) veterans’ appointments must be scheduled, and (3) veterans’ appointments must occur that are consistent with the wait-time goal VA has established for the program. VA agreed with both recommendations, which remain unimplemented, and officials stated that they are in the process of finalizing metrics to capture wait-time performance and designing an appointment scheduling process. Without specifying wait- time goals that are achievable, and without designing appointment scheduling processes that are consistent with those goals, VA lacks assurance that veterans are receiving care from community providers in a timely manner. In June 2018, we reported that VA could not systematically monitor wait times for veterans accessing care under the Choice Program due to incomplete and inaccurate data. Without complete and accurate data, VA was not able to determine whether the Choice Program was achieving its goals of (1) alleviating the wait times veterans experienced when seeking care at VA medical facilities, and (2) easing geographic burdens veterans may have faced when accessing care at VA medical facilities. We made three recommendations to address VA’s incomplete and inaccurate data related to the Choice Program, and VA is taking steps to implement two of those recommendations. We found that the data VA used to monitor the timeliness of Choice Program appointments captured only a portion of the total appointment scheduling process. Though VA had a 30-day wait-time goal to provide veterans with care under the Choice Program, VA’s timeliness data did not capture (1) the time VA medical centers took to prepare veterans’ referrals and send them to the TPAs, and (2) the time spent by TPAs in accepting the referrals and opting veterans into the Choice Program. For example, we found that it took VA medical center staff an average of 24 calendar days after the veteran’s need for care was identified to contact the veteran, compile relevant clinical information, and send the veteran’s referral to the TPAs. For those same authorizations, it took the TPAs an average of 14 calendar days to accept referrals and reach veterans to opt them into the Choice Program. In 2016, VA also conducted its own manual review of appointment scheduling times and found that wait times could be longer than the 30 days (see fig. 2). Specifically, out of a sample of about 5,000 Choice Program authorizations, VA analyzed (1) the timeliness with which VA medical centers sent referrals to the TPAs, and (2) veterans’ overall wait times for Choice Program care. VA’s analysis identified average review times when veterans were referred to the Choice Program to be greater- than-30-day wait time for an appointment at a VA medical facility. For example, for overall wait times (i.e., the time veterans’ need for care was identified until they attended initial Choice Program appointments), wait times ranged from 34 to 91 days across the 18 VA integrated service networks. The national average was 51 days. In September 2017, VA began implementing an interim solution to monitor overall wait times, but this solution relied on VA medical center staff consistently and accurately entering data on referrals, a process that is prone to error. In June 2018, we recommended that VA establish a mechanism to monitor the overall wait times under the VCCP. VA agreed with this recommendation, and stated that it is developing a monitoring mechanism that will be incorporated into a new system that will be fully implemented across all VA medical facilities by fiscal year 2021. We also reported that the clinically indicated dates included on referrals that VA medical centers sent to the TPAs, which are used to measure the timeliness of care, may not have been accurate, further limiting VA’s monitoring of veterans’ access to care. Our review of 196 Choice Program authorizations found that clinically indicated dates were sometimes changed by VA medical center staff before they were sent to the TPAs, which could mask veterans’ true wait times. We found that VA medical center staff entered later clinically indicated dates on referrals for about 23 percent of the 196 authorizations reviewed. We made two recommendations to improve the accuracy of the Choice Program data. For example, we recommended that VA establish a mechanism under the VCCP that prevents clinically indicated dates from being modified. VA agreed with our recommendation, and stated that a new system will interface with VA’s existing referral package to allow a VA clinician to enter in a clinically indicated date while restricting schedulers from making alterations to it. In June 2018, we also reported that numerous factors adversely affected veterans’ timely access to care through the Choice Program and could affect access under the VCCP. These factors included the following: (1) administrative burden caused by complexities of VA’s referral and appointment scheduling processes; (2) poor communication between VA and its medical facilities; and (3) inadequacies in the networks of community providers established by the TPAs, including an insufficient number, mix, or geographic distribution of community providers. VA has taken steps to help address these factors; however, none have been fully addressed. For example, to help address administrative burden and improve the process of coordinating veterans’ Choice Program care, VA established a secure e-mail system and a mechanism for TPAs and community providers to remotely access veterans’ VA electronic health records. However, these mechanisms only facilitate a one-way transfer of necessary information. They do not provide a means by which VA medical facilities or veterans can view the TPAs’ step-by-step progress in scheduling appointments or electronically receive medical documentation associated with Choice Program appointments. We made five recommendations to VA to address the factors that adversely affected veterans’ access to Choice Program care. VA agreed or agreed in principle with all five recommendations and has taken some steps in response to these recommendations. However, our recommendations remain unimplemented. On June 6, 2019, VA began implementing the VCCP, which created a consolidated community care program. Under the VCCP, VA began determining veteran eligibility based on designated access standards, such as wait-time goals of 20 days for primary and mental health care and 28 days for specialty care and other criteria identified in the MISSION Act. According to VA officials, the implementation of the VCCP also included the use of the new Decision Support Tool—a system that combines eligibility and other information to help veterans, with assistance from VA staff, decide whether to seek care in the community. VA officials previously identified the Decision Support Tool along with another new system—known as the Health Share Referral Management system—as key efforts in addressing many of our recommendations related to VA’s community care wait-time data and monitoring issues. VA expects the Health Share Referral Management system, which will manage community care referrals and authorizations as well as facilitate the exchange of health information between VA and community providers, to be fully implemented across all VA medical facilities in fiscal year 2021. We began work in May 2019 to review VA’s implementation of the VCCP, including how it will address issues such as appointment scheduling. In addition to the actions described above, VA has taken other steps to improve veterans’ access to care by, for example, offering veterans access to routine care without an appointment. We have ongoing work related to same-day services provided in VA primary care and mental health clinics. In order to improve access, VA implemented the same-day service initiative in 2016, and by 2018 offered same-day services in over 1000 facilities. As part of the initiative, VA medical facility staff are directed to address veterans’ primary care and mental health needs that day through a variety of methods, including face-to-face visits, telehealth, prescription refills, or by scheduling a follow-up appointment. Our ongoing work indicates that the six VA medical facilities we visited were generally providing same-day services prior to the initiative; however, according to VA officials, ongoing staffing and space shortages created challenges implementing the initiative. Our ongoing work also indicates that VA does not have performance goals and measures to determine same-day services’ impact on veterans’ access to care. We plan to issue our report on VA’s same-day services initiative in August 2019. In closing, we have identified various weaknesses in VA’s wait-time measurement and scheduling processes over the years. These weaknesses have affected not only VA’s internal delivery of outpatient care, but also that provided through community providers. As we have highlighted here, we have made a number of recommendations to address these weaknesses. VA has taken actions to address our recommendations, but additional work is needed for some. The implementation of enhanced technology, such as a new scheduling system, is crucial and will provide an important foundation for improvements. However, this is not a panacea for addressing all of the identified problems. Moving forward, VA must also continuously ensure that it has clear and consistent policies and processes, adequate oversight, and effective training. Chairman Takano, Ranking Member Roe, and Members of the Committee, this completes my prepared statement. I would be pleased to respond to any questions that you may have at this time. If you or your staff have any questions about this testimony, please contact Debra A. Draper, Director, Health Care at (202) 512-7114 or DraperD@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this statement. GAO staff who made key contributions to this testimony were Sharon Silas (Acting Director), Ann Tynan (Assistant Director), Cathy Hamann, Aaron Holling, Akbar Husain, Kate Tussey, and E. Jane Whipple. Also contributing were Jacquelyn Hamilton and Vikki Porter.
The majority of veterans utilizing VA health care services receive care in VA-operated medical facilities, including 172 VA medical centers and more than 1,000 outpatient facilities. For nearly 20 years, GAO has reported on the challenges VA medical facilities have faced providing health care services in a timely manner. When veterans face wait times at VA medical facilities, they may be able to receive services from VA's community care programs, which VA estimates will be 19 percent of its $86.5 billion in health care obligations in fiscal year 2020. This testimony focuses on GAO's large body of work on veterans' access to care and the status of VA's efforts to address GAO's recommendations, including those from GAO's June 2018 report on VA's community care programs and from GAO's December 2012 report on VA's scheduling of timely medical appointments that VA has provided information on through July 2019. It also includes preliminary observations on related ongoing work. GAO has issued several reports recommending that the Department of Veterans Affairs (VA) take action to help ensure its facilities provide veterans with timely access to medical care. VA has taken a number of steps to address GAO's recommendations to improve wait-time measurement and its appointment scheduling policy. However, additional actions are needed to fully address most of GAO's recommendations. GAO found in 2012 that outpatient appointment wait times reported by VA were unreliable because VA did not ensure consistency in schedulers' definitions of the dates by which wait times were measured. GAO recommended that VA clarify these definitions. VA concurred and has taken a number of actions in response, including improved oversight through scheduling audits. However, VA's first internal audit in August 2018 was unable to evaluate the accuracy and reliability of its wait-time data due to the lack of business rules for calculating them, indicating that additional efforts are needed to address this issue. GAO also found in 2012 that not all facilities GAO visited used the electronic wait list to track new patients that needed medical appointments, as required by VA's scheduling policy. This put patients at risk for being lost for appointment scheduling. GAO recommended VA ensure consistent implementation of its policy, and that all schedulers complete required training. VA concurred, and with the information VA provided in July 2019 GAO considers VA's actions, including updating its scheduling policy and completing scheduler training, sufficient to fully address the recommendation. While improvements to VA's scheduling policy and processes will help ensure veterans receive timely access to care, there are other factors that may also affect access that are not currently reflected in VA's wait-time data. For example, GAO found instances in which the time it took the agency to initially enroll veterans in VA health care benefits was more than 3 months. GAO has also made recommendations to improve appointment scheduling and ensure timely access to care from non-VA providers in VA's community care programs that remain unimplemented. GAO found in June 2018 that the data VA used to monitor the timeliness of the Veterans Choice Program's appointments captured only a portion of the total appointment scheduling process. Although VA had a wait-time goal of 30 days, VA's timeliness data did not capture certain processes, such as the time taken to prepare veterans' referrals and send them to a third-party administrator. GAO found that if these were accounted for, veterans could potentially wait up to 70 calendar days to see a community care provider. VA officials stated that most recommendations will be addressed with new program tools it plans to implement. For example, VA is implementing a system for referral management and appointment scheduling expected to be available in all VA medical facilities by fiscal year 2021. While technology may be an important tool, VA will also need clear and consistent policies and processes, adequate oversight, and effective training to help avoid past challenges. GAO has made a number of recommendations to VA to address timely scheduling and reliable wait-time data for outpatient appointments and through community care. VA generally agreed with GAO's recommendations. As of July 2019, VA has taken actions to fully implement one recommendation discussed in this statement. GAO continues to believe that all of the recommendations are warranted.