Under agreement with CMS, a state survey  agency in each state assesses whether nursing homes meet CMS’s  standards, allowing them to participate in the Medicare and Medicaid  programs. Clinical quality measures. Nursing Home Quality Data Show Mixed Results, Although Data Issues Complicate Ability to Assess Quality Trends

In recent years, trends in four key sets of data that give insight into  nursing home quality show mixed results. Specifically, one of the four  data sets suggests that consumers’ concerns over nursing home quality  have increased, which may indicate a potential decrease in quality, while  the other three sets of data may indicate potential improvement in nursing  home quality. Data on Nursing Home Quality Show Mixed Results

Nationally, in recent years, one of four data sets—number of consumer  complaints—demonstrated a potential decrease in nursing home quality,  while the other three data sets—serious deficiencies cited on standard  surveys, staffing data, and selected clinical quality measures— demonstrated potential quality improvement. Consumer complaints: From 2005 through 2014, the average number  of consumer complaints reported per nursing home increased nationally  from 3.2 to 3.9, a 21 percent increase over the 10-year period. 1.) Data Issues Complicate Ability to Assess Quality Trends

CMS’s ability to use available data to assess nursing home quality trends  is complicated by various issues with these data. Specifically, each of the  four key sets of nursing home data we analyzed have issues that make it  difficult to determine whether observed trends reflect actual changes in  quality, data issues, or a combination of both. CMS officials said these challenges led to reduced state survey agency  capacity to conduct surveys, which could contribute to the decrease in the  number of deficiencies cited on standard surveys. CMS Has Modified Oversight Activities, But Has Not Monitored Potential Effect on Nursing Home Quality Oversight

In recent years CMS has made numerous modifications to its nursing  home oversight activities. Some of these modifications expanded or  added new oversight activities. Other modifications have reduced existing oversight activities. For  example, CMS has made modifications to the federal monitoring survey  program and the Special Focus Facility program. Nursing homes placed in the  SFF program receive additional oversight because of the homes’  history of poor performance. Specifically,  CMS officials said that increasing oversight responsibilities, such as those  required by PPACA, and a limited number of staff and financial resources  at the central, regional, and state levels required the agency to evaluate  its activities and reduce the scope of some activities. Under federal  internal control standards, ongoing monitoring should occur in the course  of normal program operations. Federal  internal control standards require agencies to monitor performance data  to assess the quality of performance over time, and CMS’s ability to do so  is hindered by data issues. Consistent with federal internal control  standards, establishing an effective process for monitoring modifications  of essential oversight activities made at the CMS central office, CMS  regional office, and state survey agency levels—whether positive or  negative—could allow CMS to better understand the effects these  modifications may have on nursing home quality and make improvements  to its own oversight. To help ensure modifications of CMS’s oversight activities do not  adversely affect the agency’s ability to assess nursing home quality and  that effective modifications are adopted more widely, the Administrator of  CMS should establish a process for monitoring modifications of essential  oversight activities made at the CMS central office, CMS regional office,  and state survey agency levels to better understand the effects on  nursing home quality oversight. As we describe in this report, ongoing auditing  of self-reported data is important for ensuring data accuracy; as a result,  whenever self-reported data are used for understanding nursing home  quality—including the new electronic payroll system for collecting staffing  data and data used to calculate clinical quality measures—our  recommendation indicates that HHS should plan for and conduct audits in  a continuing manner. Appendix I: Additional Detail on Analysis

This appendix describes our scope and methodology for examining the  extent to which reported nursing home quality has changed in recent  years and the factors that may have affected any observed changes. For  this examination, we analyzed four sets of quality data from the Centers  for Medicare & Medicaid Services (CMS).
Why GAO Did This Study

To help ensure nursing home residents receive quality care, CMS, an agency within the Department of Health and Human Services (HHS), defines quality standards homes must meet to participate in the Medicare and Medicaid programs. To monitor compliance with these standards, CMS enters into agreements with state survey agencies to conduct on-site surveys of the state's homes and also collects other data on nursing home quality. CMS and others have reported some potential improvements in nursing home quality.
GAO was asked to study these trends. This report examines (1) the extent to which reported nursing home quality has changed in recent years and the factors that may have affected any observed changes, and (2) how CMS oversight activities have changed in recent years. GAO analyzed four sets of CMS quality data—deficiencies cited on standard surveys (2005-2014), consumer complaints (2005-2014), staffing levels (2009-2014), and a sub-set of clinical quality measures (2011-2014)—at both national and state levels. We also reviewed relevant documents, including CMS guidance and Standards for Internal Control in the Federal Government, and interviewed CMS and state agency officials at 5 states selected on factors such as size.

What GAO Found

In recent years, trends in four key sets of data that give insight into nursing home quality show mixed results, and data issues complicate the ability to assess quality trends. Nationally, one of the four data sets—consumer complaints—suggests that consumers' concerns over quality have increased, while the other three data sets—deficiencies, staffing levels, and clinical quality measures—indicate potential improvement in nursing home quality. For example, the average number of consumer complaints reported per home increased by 21 percent from 2005-2014, indicating a potential decrease in quality. Conversely, the number of serious deficiencies identified per home with an on-site survey, referred to as a standard survey, decreased by 41 percent over the same period, indicating potential improvement. The Centers for Medicare & Medicaid Services' (CMS) ability to use available data to assess nursing home quality is complicated by various issues with these data, which make it difficult to determine whether observed trends reflect actual changes in quality, data issues, or both. For example, clinical quality measures use data that are self-reported by nursing homes, and while CMS has begun auditing the self-reported data, it does not have clear plans to continue. Federal internal control standards require agencies to monitor performance data to assess the quality of performance over time.
In recent years, CMS has made numerous modifications to its nursing home oversight activities, but has not monitored the potential effect of these modifications on nursing home quality oversight. Some of the modifications have expanded or added new oversight activities, while others have reduced existing oversight activities. According to CMS, some of the reductions to oversight activities are in response to an increase in oversight responsibilities and limited number of staff and financial resources. However, CMS has not monitored how the modifications might affect CMS's ability to assess nursing home quality. For example, CMS reduced the number of nursing homes participating in the Special Focus Facility program—which provides additional oversight of homes with a history of poor performance—from 152 in 2013 to 62 in 2014. State survey agency officials who conduct surveys for CMS also made modifications which could have either a positive or negative effect on oversight, but CMS does not have an effective mechanism for monitoring. Federal internal control standards require ongoing monitoring as a part of normal program operations; without this monitoring, CMS cannot ensure that any modifications in oversight do not adversely affect its ability to assess nursing home quality.

What GAO Recommends

GAO recommends, among other things, that CMS implement a clear plan for ongoing auditing of self-reported data and establish a process for monitoring oversight modifications to better assess their effects. HHS agreed with GAO's recommendations.