Scope and Methodology

As agreed with your offices, our objectives for this report were to   (1) assess the soundness of the Bureau’s design for the 2004 census test and  whether the Bureau implemented the test consistent with its plans,   (2) review the quality of the Bureau’s IT security practices, and (3) identify  initial lessons learned from conducting the test and their implications for  the 2010 Census. The Bureau Developed a Sound Test Design

The design of the 2004 census test contained many components of a sound  study (see table 1). The Bureau also developed evaluation  plans for each of the test’s 11 research questions, and explained how  stakeholders were involved with the design, as well as how lessons learned  from past studies were incorporated. The Bureau Needs to Implement Better IT Security Practices

The Bureau operated a number of IT systems in order to transmit, manage,  and process data for the test. For example:  IT inventory was not complete. The Bureau did not have a  complete inventory that showed all applications and general support IT  systems associated with the test. For example, the HHCs show promise in  that enumerators were successful in using them to collect data from  nonrespondents and remove late mail returns. The Bureau tested new group quarters procedures in 2004 that were  designed to address the difficulties the Bureau had in trying to identify and  count this population group during the 2000 Census. Thus, as the  Bureau plans for the next field test in 2006 and the 2008 dress rehearsal, it  will be important for the Bureau to ensure the various census operations  are fully functional at the time of the test so they can be properly evaluated. Conclusions

The Bureau is well aware that a successful enumeration hinges on early  research, development, testing, and evaluation of all aspects of the census  design. However, looking toward the future, it will be critical  for the Bureau to diagnose the source of these challenges, devise cost- effective solutions, and integrate refinements and fixes in time to be  assessed during the next field test scheduled for 2006. Define specific, measurable performance requirements for the HHCs  and other census-taking activities that address such important measures  as productivity, cost savings, reliability, durability, and test their ability  to meet those requirements in 2006.
Review and test the wording and formatting of the coverage and  race/ethnicity questions to make them less confusing to respondents  and thus help ensure the collection of better quality data, and ensure  they are formatted the same way on both the HHC and paper versions of  the census form. We will continue to monitor the Bureau’s progress  in resolving these issues and update Congress on a regular basis.
Why GAO Did This Study

A rigorous testing and evaluation program is a critical component of the census planning process because it helps the U.S. Census Bureau (Bureau) assess activities that show promise for a more cost-effective head count. The Bureau conducted a field test in 2004, and we were asked to (1) assess the soundness of the test design and the extent to which the Bureau implemented it consistent with its plans, (2) review the quality of the Bureau's information technology (IT) security practices, and (3) identify initial lessons learned from conducting the test and their implications for future tests and the 2010 Census.

What GAO Found

The Bureau's design for the 2004 census test addressed important components of a sound study, and the Bureau generally implemented the test as planned. For example, the Bureau clearly identified its research objectives, developed research questions that supported those objectives, and developed evaluation plans for each of the test's 11 research questions. The initial results of the test suggest that while certain new procedures show promise for improving the cost-effectiveness of the census, the Bureau will have to first address a number of problems that could jeopardize a successful head count. For example, enumerators had little trouble using hand held computers (HHC) to collect household data and remove late mail returns. The computers could reduce the Bureau's reliance on paper questionnaires and maps and thus save money. The test results also suggest that certain refinements the Bureau made to its procedures for counting dormitories, nursing homes, and other "group quarters" could help prevent the miscounting of this population group. Other aspects of the test did not go as smoothly. For example, security practices for the Bureau's IT systems had weaknesses; the HHCs had problems transmitting data; questionnaire items designed to improve coverage and better capture race/ethnicity confused respondents; enumerators sometimes deviated from prescribed enumeration procedures; and certain features of the test were not fully operational at the time of the test, which hampered the Bureau from fully gauging their performance. With few testing opportunities remaining, it will be important for (1) the Bureau to find the source of these problems, devise cost-effective solutions, and integrate refinements before the next field test scheduled for 2006, and (2) Congress to monitor the Bureau's progress in resolving these issues.