The Response Rate Estimate Lacks Support, Is Not Systematically Reevaluated, and Is Not Clearly Incorporated into Planning Efforts

		2010 Census Response Rate Estimate Is Not Fully Supported

The Bureau developed the response rate estimate for the 2010 Census in  2001 but could not demonstrate support for one of the three components  underpinning the estimate. The Bureau Does Not Systematically Reevaluate the Response Rate Estimate

Although the Bureau updated the response rate estimate as a result of a  major redesign of census operations undertaken in April 2008, the Bureau  still lacks procedures for establishing when and how to reevaluate the  response rate estimate. From 2001 through April 2008, the Bureau did not  reevaluate the estimate to determine whether it should be updated, even  though, after establishing the initial estimate in 2001, the Bureau  completed several evaluations from Census 2000 and conducted several  census tests that could have informed the estimate. The Bureau revised the estimate down to 64 percent after announcing that  nonresponse follow-up would be changed from an automated to a paper- based operation. Further, the  Bureau could not demonstrate whether or how the short-form-only  component and the general decline component of the national estimate  were reflected in the life cycle cost model. The Bureau Plans to Incorporate Additional Methods to Increase Mail Response in 2010

		Two of Nine Methods Tested to Increase Response Rate Are Planned to Be Included in the 2010 Census

We determined from reviewing Bureau testing documents that through  various national and field tests and experiments, the Bureau tested nine  methods to increase self-response to the 2010 Census, as shown in table 1. The Bureau currently plans to implement two of these methods—the  replacement questionnaire and two-column bilingual form (see fig. Despite its July 2006 decision, the Bureau recently announced  that it is considering including the Internet option as part of the 2010  Census design; however, the Bureau has not developed plans for further  testing this option. However, Bureau officials did not document the methods that  they considered but decided not to test in the 2010 testing cycle or the  rationale behind those decisions. Although officials said that they considered  cost, complexity of the test, and compatibility of experiments in their  decisions, they did not specify how they defined or weighed these factors  to select and prioritize the nine methods they chose to test. For example, they said that they attend research conferences to learn  about experiences of other organizations that conduct national surveys to  identify potential methods to increase response rate. Both Bureau officials  and some survey experts noted that methods used to increase response in  other surveys may only be indirectly applicable to the decennial census. Nonetheless, testing modifications to methods that the Bureau has  previously considered or tested in earlier studies may yield additional  opportunities for increasing response. Conclusions

Nonresponse follow-up is the largest field operation, and the Bureau  estimates that it will cost more the $2 billion. To control the cost of  nonresponse follow-up, it will be important for the Bureau to devise a  strategy for getting people to return their census forms. Appendix I: Objectives, Scope, and Methodology

The objectives of this report were to (1) assess how the U.S. Census  Bureau (Bureau) develops, supports, and updates the response rate  estimate, and the extent to which the Bureau uses the response rate  estimate to inform its 2010 planning efforts; (2) describe the methods the  Bureau considered for increasing response rates in 2010 and what it did to  test these methods; and (3) assess how the Bureau identifies and selects  for testing methods to increase response rates, including considering other  surveys’ methods for increasing response. Telephone reminder call. Internet. Communications campaign.
Why GAO Did This Study

The U.S. Census Bureau (Bureau) estimates that it will spend at least $2 billion to enumerate households that did not return census forms during the 2010 Census. Increasing the response rate would reduce the number of households that Bureau field staff must visit. To address concerns about reducing the cost of enumerating these households, GAO (1) analyzed how the Bureau develops, supports, and updates the response rate estimate, and the extent to which the Bureau uses the estimate to inform its 2010 planning efforts; (2) described the methods the Bureau considered for increasing response in 2010 and how it tested these methods; and (3) assessed how the Bureau identifies and selects for testing methods to increase response rate, including considering other surveys' methods. To meet these objectives, GAO analyzed the Bureau's documentation for estimating the response rate and selecting for testing methods to increase response, and interviewed experts from other survey organizations.

What GAO Found

The 2010 Census response rate estimate is not fully supported, systematically reevaluated, or clearly incorporated into the life cycle cost estimate and planning efforts for nonresponse follow-up--where census workers visit households that do not return their census forms. Specifically, the Bureau could not demonstrate support for one component underpinning the estimate--a general decline due to decreasing public participation in surveys--because it did not document its decisions or data sources when developing the estimate. The two other estimate components that affect responses are the short-form-only census and the replacement questionnaire. In 2001, the Bureau estimated the 2010 Census response rate to be 69 percent. However, from 2001 through 2008, the Bureau did not systematically reevaluate the estimate or consider test results from this decade to determine if the estimate should be updated. Although the Bureau revised the estimate to 64 percent after a major redesign of its nonresponse follow-up operation in 2008, the Bureau still lacks procedures for establishing when and how to reevaluate and, if necessary, update the estimate. To estimate costs and plan for nonresponse follow-up, the Bureau relies on response rate estimates for local census office types because these estimates reflect geographic differences. Officials said that the local estimates reflect components of the national estimate. However, only one of the three components from the national estimate--the replacement questionnaire--was clearly reflected in the local census office type estimates. Through various national and field tests and experiments, the Bureau tested nine methods to increase 2010 Census response and currently plans to implement two of these methods--the replacement questionnaire and two-column bilingual form. The Bureau also plans to use a communications campaign to increase response and plans to test campaign messages in 2009. In July 2006, the Bureau decided not to include an Internet response option in the 2010 Census. However, the Bureau recently announced that it is again considering including the Internet option in 2010, although it has not developed further plans for testing it. For 2010, the Bureau established test objectives and research questions to identify methods to test for increasing response. However, Bureau officials did not document the methods that they considered but decided not to test or the rationale behind those decisions. Although officials said that they considered cost, complexity of the test, and compatibility of experiments in their decisions, they did not specify how they weighed these factors to select and prioritize the nine methods they chose to test. Officials said that they consider the experiences of other survey organizations to identify potential methods to increase response, but they, along with some experts, noted that such methods may only be indirectly applicable to the decennial census. Nonetheless, testing modifications to methods the Bureau has previously considered or tested, such as testing a variety of telephone reminder messages, may yield additional opportunities for increasing response.