CMS used QMs both to provide the public with information on nursing  home quality of care and to help evaluate QIO efforts to address quality-of- care issues, such as pressure ulcers. 3). QIOs Generally Had a Choice among Homes That Volunteered but Did Not Target Assistance to Low-Performing Homes

Although QIOs generally had a choice of homes to select for intensive  assistance because more homes volunteered than CMS expected QIOs to  assist, QIOs typically did not target the low-performing homes that  volunteered. Most QIOs reported in our Web-based survey that they did  not have difficulty recruiting homes, and their primary consideration in  selecting homes from the pool of volunteers was that the homes be  committed to working with the QIOs. In the 7th SOW, CMS did not specify  recruitment and selection criteria for intensive participants, leaving the  development of guidelines to the QIO support contractor, which  encouraged QIOs to select homes that seemed committed to quality  improvement and to exclude homes with a high number of survey  deficiencies, high management turnover, or QM scores that were too good  to improve significantly. The stakeholders we interviewed—including officials of state  survey agencies and nursing home trade associations—generally believed  QIOs’ resources should be targeted to low-performing homes. Nationwide, the homes in the intensive  participant group were less likely than other homes in their state to be  low-performing in terms of the number, scope, and severity of deficiencies  for which they were cited in surveys during that time frame. QIO Contract Flexibility Resulted in Variation in Assistance Provided to Intensive Participants

The 7th SOW contracts allowed QIOs flexibility in the QMs they focused on  and the interventions they used, and while the majority of QIOs selected  the same QMs and most used the same interventions to assist homes  statewide, the interventions for intensive participants and staffing to  accomplish program goals varied. Most intensive participants worked on a subset of the QMs selected by  their QIO—chronic pain and pressure ulcers (see fig. Statewide Interventions Less Variable Than Those for Intensive Participants

The terms of the QIO contract with CMS allowed QIOs to determine the  kinds of quality improvement interventions they offered to homes, and  those selected by QIOs were consistent with an approach recommended  by the QIO support contractor: QIOs generally relied most on conferences  and the distribution of educational materials to assist homes statewide and  on on-site visits to assist intensive participants. For homes statewide, most QIOs (54 percent) reported that  conferences were their most effective intervention, followed by  distribution of educational materials and on-site visits. Of the 15 QIOs that said they would change their approach with  these homes, most (60 percent) would make on-site visits their primary  intervention, while fewer would rely on small group meetings,  conferences, and other interventions. Homes also found particular types of assistance less helpful. QIOs’ Impact on Quality Is Not Clear, but Staff at Homes We Contacted Attributed Some Improvements to QIOs

Although the QIOs’ impact on the quality of nursing home care cannot be  determined from available data, staff we interviewed at most nursing  homes attributed some improvements in the quality of resident care to  their work with the QIOs. Nursing homes’ QM scores generally improved  enough for the QIOs to surpass by a wide margin the modest contract  performance targets set by CMS; however, the overall impact of the QIOs  on the quality of nursing home care cannot be determined from these data  because of the shortcomings of the QMs as measures of nursing home  quality and because confounding factors make it difficult to attribute  quality improvements solely to the QIOs. Subsequently, HHS’s Office of General  Counsel determined that QIO program regulations prohibited QIOs from  providing to CMS the identities of intensive participants. Identify a broader spectrum of measures than QMs to evaluate changes in  nursing home quality. Analysis of State Survey Data

To assess the characteristics of the nursing homes that were selected by  the QIOs for intensive assistance from among the homes that volunteered,  we analyzed 3 years of standard state survey data on deficiencies cited at  nursing homes and compared the results for homes that were assisted  intensively with results for homes that were not; we used this information  to address our first objective. Based on these  rankings, we identified homes in the bottom and top quartile in each state  in each survey.
Why GAO Did This Study

In 2002, CMS contracted with Quality Improvement Organizations (QIO) to help nursing homes address quality problems such as pressure ulcers, a deficiency frequently identified during routine inspections conducted by state survey agencies. CMS awarded $117 million over a 3-year period to the QIOs to assist all homes and to work intensively with a subset of homes in each state. Homes' participation was voluntary. To evaluate QIO performance, CMS relied largely on changes in homes' quality measures (QM), data based on resident assessments routinely conducted by homes. GAO assessed QIO activities during the 3-year contract starting in 2002, focusing on (1) characteristics of homes assisted intensively, (2) types of assistance provided, and (3) effect of assistance on the quality of nursing home care. GAO conducted a Web-based survey of all 51 QIOs, visited QIOs and homes in five states, and interviewed experts on using QMs to evaluate QIOs.

What GAO Found

Although more homes volunteered to work with the QIOs than CMS expected them to assist intensively, QIOs typically did not target their assistance to the low-performing homes that volunteered. Most QIOs' primary consideration in selecting homes was their commitment to working with the QIO. CMS did not specify selection criteria for intensive participants but contracted with a QIO that developed guidelines encouraging QIOs to select committed homes and exclude those with many survey deficiencies or QM scores that were too good to improve significantly. Consistent with the guidelines, few QIOs targeted homes with a high level of survey deficiencies, and eight QIOs explicitly excluded these homes. GAO's analysis of state survey data confirmed that selected homes were less likely than other homes to be low-performing in terms of identified deficiencies. Most state survey and nursing home trade association officials interviewed by GAO believed QIO resources should be targeted to low-performing homes. QIOs were provided flexibility both in the QMs on which they focused their work with nursing homes and in the interventions they used. Most QIOs chose to work on chronic pain and pressure ulcers, and most used the same interventions⎯conferences and distribution of educational materials⎯to assist homes statewide. The interventions used to assist individual homes intensively varied and included on-site visits, conferences, and small group meetings. Just over half the QIOs reported that they relied most on on-site visits to assist intensive participants. Sixty-three percent said such visits were their most effective intervention. Of the 15 QIOs that would have changed the interventions used, most would make on-site visits their primary intervention. Homes indicated that they were less satisfied with the program when their QIO experienced high staff turnover or when their QIO contact possessed insufficient expertise. Shortcomings in the QMs as measures of nursing home quality and other factors make it difficult to measure the overall impact of the QIOs on nursing home quality, although staff at most of the nursing homes GAO contacted attributed some improvements in the quality of resident care to their work with the QIOs. The extent to which changes in homes' QM scores reflect improvements in the quality of care is questionable, given the concerns raised by GAO and others about the validity of the QMs and the reliability of the resident assessment data used to calculate them. In addition, quality improvements cannot be attributed solely to the QIOs, in part because the homes that volunteered and were selected for intensive assistance may have differed from other homes in ways that would affect their scores; these homes may also have participated in other quality improvement initiatives. Ongoing CMS evaluation of QIO activities for the contract that began in August 2005 is being hampered by a 2005 Department of Health and Human Services decision that QIO program regulations prohibit QIOs from providing to CMS the identities of homes being assisted intensively.