Traditional validation surveys provide the  basis for assessing the effectiveness of JCAHO’s hospital accreditation  process in detecting deficiencies in Medicare COPs, which JCAHO- accredited hospitals are treated as meeting. JCAHO’s Pre-2004 Hospital Accreditation Process Often Did Not Detect Serious Deficiencies Found by State Survey Agencies

JCAHO’s pre-2004 hospital accreditation process often did not identify  either hospitals with serious deficiencies or the individual serious  deficiencies found by state survey agencies through CMS’s validation  program. In a sample of 500 JCAHO-accredited hospitals, state agency  validation surveys conducted in fiscal years 2000 through 2002 identified  31 percent (157 hospitals) with serious deficiencies; of these, JCAHO did  not identify 78 percent (123 hospitals) as having serious deficiencies. However, in  its comparison to determine disparity between the two surveys, CMS does  consider whether it is reasonable to conclude that the deficiencies found  by state survey agencies existed at the time JCAHO surveyed the hospital. The number of  serious deficiencies found by CMS’s validation program represents 2  percent of the 11,000 Medicare COPs surveyed by state agencies in the  sample and were found in 157 hospitals. Serious deficiencies in the  COP on physical environment compromise patient safety and health. Potential of JCAHO’s New Hospital Accreditation Process Is Unknown, and Testing Was Limited

The potential of JCAHO’s new hospital accreditation process to improve  the identification of serious deficiencies is unknown because it is too soon  after its January 2004 implementation for a meaningful evaluation; in  addition, JCAHO’s testing of the new process was limited. While unannounced surveys, which are planned for implementation in  2006, have the potential to improve the detection of serious deficiencies,  other features of the new process that JCAHO did not test before  implementation may have limitations that could affect the potential of the  new process to identify problems with patient care. However, periodic performance reviews may not necessarily improve the  detection of deficiencies. The hospitals participating in the pilot test were not randomly selected by  JCAHO. During the pilot test, an observer from JCAHO’s central office  accompanied each surveyor, and the knowledge that they were being  observed may have influenced the surveyors’ actions. CMS Oversight Authority of JCAHO Is Limited

Because of JCAHO’s unique legal status, CMS’s oversight of JCAHO’s  hospital accreditation program is limited in two major ways: Unlike other  accreditation programs with deeming authority, JCAHO does not have to  reapply to CMS to reauthorize its deeming authority, and CMS cannot take  action to address performance problems with JCAHO’s hospital  accreditation program. Recommendations for Executive Action

To strengthen the ability of CMS to identify and report to Congress on  JCAHO’s ability to ensure that the hospitals it accredits protect the safety  and health of patients through compliance with the Medicare COPs, we  recommend that the Administrator of CMS take the following three  actions:    modify the method used to measure the rate of disparity between  validation survey findings and accreditation program findings to provide a  reasonable assurance that Medicare COPs are being met and consider  whether additional measures are needed to accurately reflect an  accreditation program’s ability to detect deficiencies in Medicare COPs;    provide in the annual report to Congress an estimate, based on the  validation survey sample, of the performance of all JCAHO-accredited  hospitals, including the limitations and protocols for these estimates based  on generally accepted sampling and statistical methodologies; and develop  a written protocol for these calculations; and    annually conduct traditional validation surveys on a sample of JCAHO- accredited hospitals that is equal to at least 5 percent of all JCAHO- accredited hospitals. Rather, we focused our evaluation on how well  JCAHO’s hospital accreditation program ensures hospitals’ compliance  with Medicare participation requirements. Appendix I: Scope and Methodology

We examined the extent to which JCAHO’s pre-2004 survey process  identified hospitals with deficiencies and individual deficiencies in  Medicare COPs that were identified by state survey agencies.
Why GAO Did This Study

Hospitals accredited by the Joint Commission on Accreditation of Healthcare Organizations (JCAHO) are considered in compliance with Medicare participation requirements. GAO examined the extent to which JCAHO's pre-2004 hospital accreditation process identified hospitals not complying with Medicare requirements, the potential of JCAHO's new process for improving the detection of deficiencies in Medicare requirements, and the effectiveness of CMS's oversight of JCAHO's hospital accreditation program. GAO analyzed CMS data on hospitals state surveyors found to have deficiencies in Medicare requirements that JCAHO surveyors did not detect, analyzed CMS's measure of JCAHO's ability to detect noncompliance with Medicare requirements, and interviewed JCAHO officials.

What GAO Found

JCAHO's pre-2004 hospital accreditation process did not identify most of the hospitals found by state survey agencies in CMS's annual validation survey sample to have deficiencies in Medicare requirements. In comparing the results of the two surveys, CMS considered whether it was reasonable to conclude that the deficiencies found by state survey agencies existed at the time JCAHO surveyed the hospital. In a sample of 500 JCAHO-accredited hospitals, state agency validation surveys conducted in fiscal years 2000 through 2002 identified 31 percent (157 hospitals) with deficiencies in Medicare requirements. Of these 157 hospitals, JCAHO did not identify 78 percent (123 hospitals) as having deficiencies in Medicare requirements. For the same validation survey sample, JCAHO also did not identify the majority--about 69 percent--of deficiencies in Medicare requirements found by state agencies. Importantly, the number of deficiencies found by validation surveys represents 2 percent of the 11,000 Medicare requirements surveyed by state agencies in the sample during this time period. At the same time, a single deficiency in a Medicare requirement can limit the hospital's capability to provide adequate care and ensure patient safety and health. Inadequacies in nursing practices or deficiencies in a hospital's physical environment, which includes fire safety, are examples of deficiencies in Medicare requirements that could endanger multiple patients. The potential of JCAHO's new hospital accreditation process to improve the detection of deficiencies in Medicare requirements is unknown because the process was just implemented in January 2004. JCAHO plans to move from using announced to unannounced surveys in 2006, which would afford JCAHO the opportunity to observe hospitals' operations when the hospitals have not prepared in advance to be surveyed. In addition, the pilot test of the new accreditation process was of limited value in predicting whether it will be an improvement over the pre-2004 process in detecting deficiencies. Limitations in the pilot test included that hospitals were not randomly selected to participate; that observers from JCAHO accompanied each surveyor, thus possibly affecting surveyors' actions; and that JCAHO evaluated the results instead of an independent entity. CMS has limited oversight authority over JCAHO's hospital accreditation program because the program's unique legal status effectively prevents CMS from taking actions that it has the authority to take with other health care accreditation programs to ensure satisfactory performance. For example, requiring JCAHO's hospital accreditation program to submit to a direct review process or placing the program on probation while monitoring its performance. Further, CMS relies on a measure to evaluate how well JCAHO's hospital accreditation program detects deficiencies in Medicare requirements that provides limited information and can mask problems with program performance, uses statistical methods that are insufficient to assess JCAHO's performance, and has reduced the number of validation surveys it conducts.