For the 2008 Dress Rehearsal and the 2010 Census, the Bureau awarded  the development of the hardware and software for a HHC to a contractor. The help desk call log revealed that listers most frequently reported issues  with transmission, the device freezing, mapspotting, and large assignment  areas. After address canvassing, the Bureau established a review  board and worked with its contractor to create task teams to address  FDCA performance issues such as (1) transmission problems relating to  the mobile computing environment, (2) the amount of data transmitted for  large assignment areas, and (3) options for improving HHC performance. For example,  the Bureau has not developed an acceptable level of performance for total  number of failed transmissions or average connection speed. Additionally,  the contractor and the Bureau did not use the dashboard specified in the  contract for dress rehearsal activities. However, the “control  panel/dashboard” application was not used during the dress rehearsal. The plan calls for using data such as the number of  HHCs shipped to local census offices, the number of defective HHCs, the  number of HHCs broken during the dress rehearsal address canvassing  operation, the number checked in at the end of the operation, whether  deployment affected the ability of staff to complete assignments,  software/hardware problems reported through the help desk, the amount  of time listers lost due to hardware or software malfunctions, and  problems with transmissions. This testing plan includes a limited  operational field test of address canvassing; however, the plan does not  specify that the dashboard described earlier will be used in this test. Field staff told us that help desk  resources were unavailable on the weekends and that they had difficulty  getting help. Lower than expected productivity has cost implications. Specifically, we revised our recommendation that the Bureau “Specify the  basis for acceptance of the FDCA solution for address canvassing and  when that acceptance will occur—when the Bureau would say it meets its  operational needs and accepts it from the contractor” to “Specify the basis  for determining the readiness of the FDCA solution for address canvassing  and when and how this determination will occur—when the Bureau would  say that the contractor’s solution meets its operational needs.” Also, after  further discussion with Bureau officials, we provided more specific  measures of address and map information successfully collected. Appendix I: Scope and Methodology

Our objectives for this report were to analyze U.S. Census Bureau  (Bureau) and contractor data showing how handheld computers (HHC)  operated and its implications on operations, and examine implications the  redesign may have on plans for address canvassing in the 2010 Census. To  determine how well the HHC worked in collecting and transmitting  address and mapping data, and what data the Bureau and contractor used  in assessing HHC performance during address canvassing, we examined  Bureau documents, observed HHCs in use, and interviewed Bureau and  contractor officials. We  reviewed the Field Data Collection Automation (FDCA) contract, focusing  specifically on what performance specifications and requirements were  included in the contract. The  Bureau acknowledged that data for all address field staff were not  included in its analysis.
Why GAO Did This Study

The U.S. Census Bureau (Bureau) had planned to rely heavily on automation in conducting the 2010 Census, including using handheld computers (HHC) to verify addresses. Citing concerns about escalating costs, in March 2008 the Secretary of Commerce announced a redesign of the key automation effort. GAO was asked to (1) analyze Bureau and contractor data showing how HHCs operated and their impact on operations, and (2) examine implications the redesign may have on plans for address canvassing in the 2010 Census. GAO reviewed Bureau and contractor data, evaluations, and other documents on HHC performance and staff productivity; interviewed Bureau and contractor officials; and visited the two dress rehearsal sites to observe and document the use of the HHCs in the field.

What GAO Found

Census and contractor data highlight problems field staff (listers) experienced using HHCs during the address canvassing dress rehearsal operation in 2007. Help desk logs, for example, revealed that listers most frequently reported issues with transmission, the device freezing, mapspotting (collecting mapping coordinates), and difficulties working with large blocks. When problems were identified, the contractor downloaded corrected software to the HHCs. Nonetheless, help desk resources were inadequate. The Bureau acknowledged that issues with the use of technology affected field staff productivity. After address canvassing, the Bureau established a review board and worked with its contractor to create task teams to analyze and address Field Data Collection Automation (FDCA) performance issues. Although the Bureau recognized that technology issues affected operations, and the contractor produced data on average transmission times, the Bureau and its contractor did not fully assess the magnitude of key measures of HHC performance. GAO previously recommended the Bureau establish specific quantifiable measures in such areas as productivity and performance. Also, the FDCA contract calls for the contractor to provide near real-time monitoring of performance metrics through a "dashboard" application. This application was not used during the census dress rehearsal. The Bureau has developed a preliminary list of metrics to be included in the dashboard such as daily measures on average transmission duration and number of failed transmissions, but has few benchmarks for expected performance. For example, the Bureau has not developed an acceptable level of performance on total number of failed transmissions or average connection speed. Technology issues and the Bureau's efforts to redesign FDCA have significant implications for address canvassing. Among these are ensuring that FDCA solutions for technical issues identified in the dress rehearsal are tested, the help desk adequately supports field staff, and a solution for conducting address canvassing in large blocks is tested. In June 2008, the Bureau developed a testing plan that includes a limited operational field test, but the plan does not specify the basis for determining the readiness of the FDCA solution for address canvassing and when and how this determination will occur.