Allowance Rates Vary Across Judges, Even for Typical Claims

		Allowance Rates Have Varied Across Judges and Hearing Offices in Recent Years, Even After Holding Constant a Range of Factors Relevant to the Appeals Process

Allowance rates varied across administrative law judges from fiscal years  2007 through 2015. We analyzed about 3.3 million decisions  made by administrative law judges on adult Social Security disability  appeals over this period. These factors  included characteristics of claimants, judges, and hearing offices, as well  as other factors such as the unemployment rates in a claimant’s state,  that could otherwise explain differences in allowance rates. 5). 6), compared  to a 46 percentage-point estimated range across judges. We also identified factors that did not have such  associations. Claimant representative: Similar to findings in our prior work,  claimants who had a representative—either an attorney or a  nonattorney representative—were allowed at a rate 2.9 times higher  than a typical claimant with no representative. Judges with certain characteristics, such as those appointed in earlier  years, were associated with a greater chance of allowing benefits. SSA also conducts multiple quality assurance  reviews, but some of these reviews may overlap and SSA has not  evaluated them. In fiscal year 2016, nearly 500 ALJs (about one- third) used eBB. SSA officials stated that the agency has been enhancing its  quality review efforts since 2009. In addition, SSA has struggled to sustain all of its quality assurance  reviews due to competing demands for the staff who perform them. We found that reviews conducted by the four entities  have resulted in similar findings, raising questions about the efficiency of  these reviews. Moreover, SSA has not conducted a cost-benefit analysis of the five  reviews. By evaluating the quality assurance reviews to  determine the extent to which each is needed to monitor and improve  accuracy and consistency, SSA would be better positioned to meet its  goals within its resources. (Recommendation  2)

	Agency Comments and Our Evaluation

In commenting on a draft of this report, SSA agreed with our two  recommendations to (1) establish public performance measures for the  accuracy and consistency of administrative law judges’ decisions, and (2)  systematically evaluate its various quality assurance reviews and take  steps to reduce or better manage any unnecessary overlap among them. Appendix I: Objectives, Scope, and Methodology

Our objectives were to assess (1) the extent to which allowance rates  vary across administrative law judges, and any factors that are associated  with this variation, and (2) the extent to which the Social Security  Administration (SSA) has processes to monitor the accuracy and  consistency of hearings decisions. SSA Administrative Data Systems

To analyze information on all adult disability decisions made by  administrative law judges from fiscal years 2007 through 2015—the most  current data available at the time of our analysis—we compiled claims  data from several SSA administrative data systems. Specifically, this system provided information on claim type (i.e.,  Disability Insurance, DI; Supplemental Security Income, SSI; or  concurrent claim); the outcome of the claim (i.e., dismissed, allowed,  or denied) and the date the decision was made; the unique  identification number of the administrative law judge (ALJ) who made  the decision; whether a medical expert or vocational expert attended  the hearing; whether the claimant was represented; the hearing office  where the claim was decided and the type of hearing office (i.e.,  hearing office or National Hearing Center); the claimant’s date of birth;  the primary impairment at the time of the hearing level; the presence  of a secondary impairment; and whether the case was classified as  being a critical case—that is, a case requiring special processing,  such as a terminal illness. We included factors related to SSA’s administration of its disability  programs, such as the hearing office in which the claim was decided,  whether the claim was heard in one of 10 states that do not have a  reconsideration step between the initial state-level Disability  Determination Service decision and a hearing before an ALJ, and  the percentage of pending cases at the hearing office that were  pending for more than 270 days (SSA’s definition of a “backlogged”  case). Because judges’ decisions to allow benefits may be related  to this or other factors, simple univariate comparisons of allowance rates  across judges may reflect characteristics of the cases that judges hear. We  interviewed SSA officials at different levels, including officials at  headquarters, regional, DDS, and field office levels.
Why GAO Did This Study

Individuals who do not agree with the initial decision on a claim for Social Security disability benefits can ultimately appeal the decision by requesting a hearing before one of SSA's approximately 1,500 administrative law judges. However, the rate at which these judges have allowed benefits has varied, raising questions about the reasons for this variation. GAO was asked to review aspects of SSA's oversight of judges' decisions.
This report examines (1) to what extent allowance rates vary across administrative law judges, and factors associated with this variation; and (2) the extent to which SSA has processes to monitor the accuracy and consistency of hearings decisions.
GAO developed a statistical model to analyze SSA data on adult disability decisions made by administrative law judges from fiscal years 2007 through 2015, the most current data available at the time of GAO's analysis; reviewed relevant federal laws, regulations, and agency documents; and interviewed SSA officials and chief judges in SSA's 10 regions, as well as officials from organizations representing judges, disability claimants, and claimant representatives.

What GAO Found

Allowance rates—the rate at which Social Security Administration (SSA) administrative law judges allowed disability benefits to be paid when claimants appealed—varied across judges, even after holding constant certain characteristics of claimants, judges, hearing offices, and other factors that could otherwise explain differences in allowance rates. Specifically, GAO estimated that the allowance rate could vary by as much as 46 percentage points if different judges heard a typical claim (one that was average in all other factors GAO analyzed). SSA officials said that this level of variation is not surprising, given the complexity of appeals and judicial discretion. Nonetheless, the variation declined by 5 percentage points between fiscal years 2007 and 2015 (see figure), a change officials attributed to enhanced quality assurance efforts and training for judges. GAO also identified various factors that were associated with a greater chance that a claimant would be allowed benefits. In addition to characteristics related to disability criteria, such as the claimant's impairment and age, GAO found that claimants who had representatives, such as an attorney or family member, were allowed benefits at a rate nearly 3 times higher than those without representatives. Other factors did not appear related to allowance rates, such as the percentage of backlogged claims in a hearing office.
SSA has various reviews to monitor the accuracy and consistency of hearings decisions by administrative law judges, but some of these reviews may overlap and SSA has not systematically evaluated them. Specifically, SSA conducts five types of quality assurance reviews of hearings decisions, several of which have similar goals and may look at similar claims. SSA has not evaluated the efficiency or effectiveness of these reviews, despite spending at least $11 million on them in fiscal year 2016. Moreover, the agency has struggled to sustain all of its quality reviews due to competing priorities—two of the five reviews were curtailed in 2016 because SSA reassigned staff to help expedite claims decisions. By evaluating which quality assurance reviews are most effective and efficient in improving accuracy and consistency, SSA would be better positioned to meet its goals within its resources.

What GAO Recommends

GAO is making two recommendations, including that SSA systematically evaluate its quality assurance reviews and take steps to reduce or better manage any unnecessary overlap among them. SSA concurred and plans to address them through a comprehensive assessment of its oversight.