Several  systems will be used in the 2010 census. In March 2008, we designated the 2010 census as a high-risk area, citing  several long-standing and emerging challenges. Dress Rehearsal Includes Testing of Certain Systems and Operations

In preparation for the 2010 census, the Bureau planned what it refers to as  the Dress Rehearsal. Bureau Is Making Progress in Conducting Key Decennial System Testing, but Lacks Plans and Schedules to Guide Remaining Efforts

The Bureau has made progress in conducting system, integration, and end- to-end testing for the 2010 census, but much remains to be done. The weaknesses in the Bureau’s  IT testing can be attributed, in part, to a lack of sufficient executive-level  oversight and guidance on testing. Without comprehensive oversight and  guidance, the Bureau cannot ensure that it is thoroughly testing its  systems before the 2010 Decennial Census. Bureau Has Performed Many System Testing Activities, but Much Remains to be Done

Through the Dress Rehearsal and other testing activities, the Bureau has  completed key system tests, but significant testing has yet to be performed,  and planning for this is not complete. In addition, the Bureau has not established a master list of interfaces  between key systems, or plans and schedules for integration testing of  these interfaces. Bureau Has Conducted Limited End-to-End Testing as Part of the Dress Rehearsal, but Has Not Developed Testing Plans for Critical Operations

The Dress Rehearsal was originally conceived to provide a comprehensive  end-to-end test of key 2010 census operations; however, as mentioned  earlier, because of the problems encountered with the handheld devices,  among other things, testing was curtailed. Furthermore, as of December 2008, the Bureau has neither established  testing plans nor schedules to perform end-to-end testing of the key  operations that were removed from the Dress Rehearsal, nor has it  determined when these plans will be completed. Although the 2010 Decennial Census is managed by the Decennial  Management Division, the oversight and management of key census IT  systems is performed on a decentralized basis. For example, the executive alert report does not  include the progress of testing activities, and although the dashboard  provides a high-level, qualitative assessment of testing for key operations  and selected systems, it does not provide information on the testing  progress of all key systems and interfaces. Further, the assessment of testing progress has not been based on  quantitative and specific metrics. For example, the status of testing key  operations removed from the Dress Rehearsal was marked as acceptable,  or “green,” although the Bureau does not yet have plans for testing these  activities. The Bureau also has weaknesses in its testing guidance; it has not  established comprehensive guidance for system testing. Any significant change to an existing IT  system introduces the risk that the system may not work as intended;  therefore, testing all systems after changes have been made to ensure the  systems work as intended is critical to the success of the 2010 census. We also interviewed program officials and contractors of key  decennial systems to obtain information on the current status of and plans  for testing activities.
Why GAO Did This Study

The Decennial Census is mandated by the U.S. Constitution and provides vital data that are used, among other things, to reapportion and redistrict congressional seats. In March 2008, GAO designated the 2010 Decennial Census a high-risk area, citing a number of long-standing and emerging challenges, including weaknesses in the Census Bureau's (Bureau) management of its information technology (IT) systems and operations. In conducting the 2010 census, the Bureau is relying on both the acquisition of new IT systems and the enhancement of existing systems. Thoroughly testing these systems before their actual use is critical to the success of the census. GAO was asked to determine the status of and plans for testing key decennial systems. To do this, GAO analyzed testing documentation, interviewed Bureau officials and contractors, and compared the Bureau's efforts with recognized best practices.

What GAO Found

Although the Bureau has made progress in testing key decennial systems, critical testing activities remain to be performed before systems will be ready to support the 2010 census. Bureau program offices have completed some testing of individual systems, but significant work still remains to be done, and many plans have not yet been developed (see table below). In its testing of system integration, the Bureau has not completed critical activities; it also lacks a master list of interfaces between systems; has not set priorities for the testing of interfaces based on criticality; and has not developed testing plans and schedules. Although the Bureau had originally planned what it refers to as a Dress Rehearsal, starting in 2006, to serve as a comprehensive end-to-end test of key operations and systems, significant problems were identified during testing. As a result, several key operations were removed from the Dress Rehearsal and did not undergo end-to-end testing. The Bureau has neither developed testing plans for these key operations, nor has it determined when such plans will be completed. Weaknesses in the Bureau's testing progress and plans can be attributed, in part, to a lack of sufficient executive-level oversight and guidance. Bureau management does provide oversight of system testing activities, but the oversight activities are not sufficient. For example, Bureau reports do not provide comprehensive status information on progress in testing key systems and interfaces, and assessments of the overall status of testing for key operations are not based on quantitative metrics. Specifically, key operations that do not yet have plans developed are marked as making acceptable progress based solely on management judgment. Further, although the Bureau has issued general testing guidance, it is neither mandatory nor specific enough to ensure consistency in conducting system testing. Without adequate oversight and more comprehensive guidance, the Bureau cannot ensure that it is thoroughly testing its systems and properly prioritizing testing activities before the 2010 Decennial Census, posing the risk that these systems may not perform as planned.