	Scope and Methodology

To determine the soundness of IRS’s PDC study as primary support for  IRS’s decision to discontinue contracting out tax debt collection, we  reviewed the study report and supporting documents and other data. To determine what changes IRS has planned or made to its collection  approach based on its PCA experience and the PDC study, we reviewed  program documents and interviewed IRS officials on their processes and  procedures for collecting tax debt. Methodological Errors and Narrow Scope Limit the Study’s Usefulness in Supporting the PDC Decision

According to IRS officials, the PDC study was not originally intended or  designed to be primary support for IRS’s PDC program decision. Even  though other factors, such as potential increases in IRS collection staffing,  were considered, based on our interviews with IRS officials and IRS’s  announcement of the program’s termination, the study results played a  primary role in supporting the decision. Nevertheless, neither we  nor IRS officials know whether the PDC study results and decision on the  program would have differed significantly if the study had been designed  to be primary support for IRS’s PDC program. The Study Result May Be Over- or Understated Because the Sample Was Not Generalizable to the PDC Population

Because IRS did not perform certain analyses and documentation is not  available to do those analyses, it is unclear whether the study’s results are  accurate. Different Study Objectives and Design Would Have Better Supported the Program Decision-Making Process

Because the PDC study had a narrow objective of comparing the results of  collection efforts by IRS and PCAs for the PCA-type cases, its design did  not consider other factors included in federal and other guidance on  conducting program analyses. However, beyond not addressing whether to continue the PDC  program, the study did not analyze alternatives for program scale, such as  expanding the PDC program or scaling it back to a segment of cases that  might be more cost effective for PCAs to work than IRS. In 2004, we  concluded that IRS should do a study in line with federal guidance, such as  comparing the results of using PCAs to the results from using the same  amount of funds to be paid to PCAs in an unconstrained manner that IRS  determined to be the most effective overall way of achieving its collection  goals. IRS Has Not Changed Its Collection Approach Based on Its PDC Experience and Study, and Whether IRS Will Work Certain PCA- Type Cases Is Unclear

		Although the Basis Was Not Documented, IRS Concluded That No PCA Practices Should Be Adopted

In authorizing the PDC program in 2004, Congress required IRS to create a  measurement plan to capture information on successful collection  techniques used by the contractors that IRS could adopt. The PCAs’ best  practices were to be compared with IRS’s collection practices. In an unpublished draft biennial report, IRS said it reviewed PCA best  practices and concluded that none of them were sufficiently better than  IRS’s practice to merit adoption. It said that IRS did not find any immediate  opportunities to adopt PCA practices, but provided no details beyond this  sentence. Conflicting Information on Whether or How IRS Will Work PCA-Type Cases

IRS officials said that IRS had not changed its criteria to start regularly  selecting PCA-type cases to work because the PDC study results were not  sufficient to identify which of the PCA-type cases could be productively  worked. However, IRS officials said that they were surprised by the study  results, which indicated that IRS staff might have better results working  these cases than some of the cases IRS normally works. IRS does not have guidance for managers on the  types of analyses that should be done and documented to support program  decisions. Recommendations for Executive Action

We recommend that the Commissioner of Internal Revenue take the  following three actions:    Establish guidance on the types of analyses that should be done to support  decisions to initiate, renew, or expand programs. Establish a policy requiring documentation for the design, analyses, and  conclusions of studies supporting program changes. IRS disagreed with our finding that its PDC study was not soundly  designed to support its decision on whether to continue contracting out  debt collection. Our report discusses our reasoning in detail, focusing on the  study’s methodological errors, narrow scope, and lack of adherence to  guidance for doing such studies. IRS agreed with our two recommendations dealing with establishing  guidance on analyses to support decisions to initiate, renew, or expand a  program and policies to ensure documentation of such studies. Although IRS’s comments on the draft report said that the need for  completing the pilot case study was overtaken by the development of the  CDA models, in separate technical comments IRS officials said they were  continuing to work the pilot cases and provided no indication that they  would stop working them before CDA is implemented in January 2011.
Why GAO Did This Study

In September 2006, the Internal Revenue Service (IRS) started the private debt collection (PDC) program for using private collection agencies (PCA) to help collect some unpaid tax debts. Aware of concerns that PCAs might cost more than using IRS staff, IRS began studying the collection costs and performance of PCAs and IRS. In March 2009, IRS announced that it would not renew its PCA contracts based on the study and announced plans for increasing collection staffing. As requested, GAO is reporting on whether (1) the study was sound as primary support for IRS's PDC decision and (2) IRS has planned or made changes to its collection approach based on its PCA experience and PDC study. GAO compared IRS's study to federal and other guidance on what should be included in analyses to support program decisions and analyzed IRS's changes given expectations that IRS would consider PCAs' best practices.

What GAO Found

IRS's comparative study of the PDC program was not soundly designed to support its decision on whether to continue contracting out debt collection. Although the study was not originally intended or designed as primary support for the decision, IRS officials nonetheless used it as such. IRS did not have guidance for program managers on the type of analysis that should be done to support decisions to create, renew, or expand programs. IRS had not retained sufficient documentation on the sample used in the study or documented some analyses that would have been helpful if performed. The study results may be overstated or understated because the study sample was not generalizable to the program as a whole. The study had a narrow objective of comparing results for IRS working the same cases as PCAs had, and as a result, the study design did not consider other factors recommended by Office of Management and Budget and other guidance on conducting program analysis. For example, the study did not analyze alternatives to program scale, such as expanding it or scaling it back. Program analysis guidance states that to the extent possible all costs and benefits should be counted and alternative means of achieving a program's goals should be considered. But the study did not identify important costs and benefits, such as whether taxpayers' compliance costs would be different if IRS or PCAs work debt cases. Nevertheless, neither GAO nor IRS officials know whether the study results and decision on the program would have differed significantly if it had been designed to be primary support for IRS's PDC program. In commenting on a draft of GAO's report, IRS disagreed that the PDC study was not soundly designed. GAO stands by its analysis detailing the study's errors, narrow scope, and lack of adherence to guidance. These design and methodology deficiencies limited the study's usefulness in supporting IRS's decision. IRS has not made or planned changes to its collection approach based on its PCA experience and study. In authorizing the use of PCAs, Congress required IRS to report to Congress its measurement plan to identify any of the PCAs' best practices that IRS could adopt to improve its own collection operations. IRS did not continue to report to Congress as required. In an unpublished draft report, IRS asserted that it had reviewed a number of PCA practices and found no immediate opportunities to change its collection approach. IRS did not provide GAO documentation on the study to support that conclusion. In part because PCA-type cases had previously been considered low priority, IRS officials were surprised by the PDC study results, which indicated that IRS staff might have better results working PCA-type cases than some of the cases IRS normally works. IRS officials said that they initiated a pilot study in 2009 to help them decide whether to use IRS staff to work selected types of PCA cases. As GAO concluded its review, IRS provided conflicting information on the role of the pilot study. On one hand, IRS said a collection selection system to be implemented in January 2011 overtook the need for the study. On the other hand, an IRS official said that the results from PCA-type cases were not used in the development of the new case selection system.

What GAO Recommends

GAO recommends that IRS (1) establish guidance on analyses to support program decisions, (2) establish a policy requiring documentation of program studies, and (3) ensure that PCA-type case results are considered for IRS's new case selection model. IRS agreed with the first two recommendations and agreed in principle with the third, which GAO revised to reflect updated information that IRS provided.