The Bureau’s address canvassing operation updates its address list and maps, which are the foundation of the decennial census. An accurate address list both identifies all households that are to receive a notice by mail requesting participation in the census (by Internet, phone, or mailed- in questionnaire) and serves as the control mechanism for following up with households that fail to respond to the initial request. Precise maps are critical for counting the population in the proper locations—the basis of congressional apportionment and redistricting. Our prior work has shown that developing an accurate address list is challenging—in part because people can reside in unconventional dwellings, such as converted garages, basements, and other forms of hidden housing. For example, as shown in figure 1, what appears to be a single-family house could contain an apartment, as suggested by its two doorbells. During address canvassing, the Bureau verifies that its master address list and maps are accurate to ensure the tabulation for all housing units and group quarters is correct. For the 2010 Census, the address canvassing operation mobilized almost 150,000 field workers to canvass almost every street in the United States and Puerto Rico to update the Bureau’s address list and map data—and in 2012 reported the cost at nearly $450 million. The cost of going door-to-door in 2010, along with the emerging availability of imagery data, led the Bureau to explore an approach for 2020 address canvassing that would allow for fewer boots on the ground. Traditionally, the Bureau went door-to-door to homes across the country to verify addresses. This “in-field address canvassing” is a labor-intensive and expensive operation. To achieve cost savings, in September 2014 the Bureau decided to use a reengineered approach for building its address list for the 2020 Census and not go door-to-door (or “in-field”) across the country, as it has in prior decennial censuses. Rather, some areas (known as “blocks”) would only need a review of their address and map information using computer imagery and third-party data sources— what the Bureau calls “in-office” address canvassing procedures. According to the Bureau’s address canvassing operational plan, in-office canvassing had two phases: During the first phase, known as “Interactive Review,” Bureau employees use current aerial imagery to determine if areas have housing changes, such as new residential developments or repurposed structures, or if the areas match what is in the Bureau’s master address file. The Bureau assesses the extent to which the number of housing units in the master address file is consistent with the number of units visible in the current imagery. If the housing shown in the imagery matches what is listed in the master address file, then those areas are considered to be resolved or stable and would not be canvassed in-field. During the second phase, known as “Active Block Resolution,” employees would try to resolve coverage concerns identified during the first phase and verify every housing unit by virtually canvassing the entire area. As part of this virtual canvass, the Bureau would compare what is found in imagery to the master address file data and other data sources in an attempt to resolve any discrepancies. If Bureau employees still could not reconcile the discrepancies, such as housing unit count or street locations with what is on the address list, then they would refer these blocks to in-field address canvassing. However, in March 2017, citing budget uncertainty the Bureau decided to discontinue the second phase of in-office review for the 2020 Census. According to the Bureau, in order to ensure that the operations implemented in the 2018 End-to-End Test were consistent with operations planned for the 2020 Census, the Bureau added the blocks originally resolved during the second phase of in-office review back into the in-field workload for the test. The cancellation of Active Block Resolution is expected to increase the national workload of the in-field canvassing workload by 5 percentage points (25 percent to 30 percent). During in-field address canvassing, listers use laptop computers to compare what they see on the ground to what is on the address list and map. Listers confirm, add, delete, or move addresses to their correct map positions. At each housing unit, listers are trained to speak with a knowledgeable resident to confirm or update address data, ask about hidden housing units, confirm the housing unit location on the map, (known as the map spot) and collect a map spot using global positioning systems (GPS). If no one is available, listers are to use house numbers and street signs to verify the address data. The data are transmitted electronically to the Bureau. The Census Bureau expects that the End-to-End Test for address canvassing will identify areas for improvement and changes that need to be made for the 2020 Census. Our prior work has shown the importance of robust testing. Rigorous testing is a critical risk mitigation strategy because it provides information on the feasibility and performance of individual census-taking activities, their potential for achieving desired results, and the extent to which they are able to function together under full operational conditions. In February 2017, we added the 2020 Census to GAO’s High-Risk List because operational and other issues are threatening the Bureau’s ability to deliver a cost-effective enumeration. We reported on concerns about the Bureau’s capacity to implement innovative census-taking methods, uncertainties surrounding critical information technology systems, and the quality of the Bureau’s cost-estimates. Underlying these issues are challenges in such essential management functions as the Bureau’s ability to: collect and use real-time indicators of cost, performance, and schedule; follow leading practices for cost estimation; scheduling; risk management; IT acquisition, development, testing, and security; and cost-effectively deal with contingencies including, for example, fiscal constraints, potential changes in design, and natural disasters. The Bureau completed in-field address canvassing as scheduled by September 29, 2017, canvassing approximately 340,400 addresses. Most of the listers we observed generally followed procedures. For example, 15 of 18 listers knocked on doors, and 16 of 18 looked for hidden housing units, which is important for establishing that address lists and maps are accurate and for identifying hard-to-count populations. Those procedures include taking such steps as: comparing the housing units they see on the “ground” to the housing units on the address list, knocking on all doors so they could speak with a resident to confirm the address (even if the address is visible on the mailbox or house) and to confirm that there are no other living quarters such as a basement apartment, looking for “hidden housing units”, looking for group quarters such as group homes or dormitories, and confirming the location of the housing unit on a map with GPS coordinates collected on the doorstep. To the extent procedures were not followed, it generally occurred when listers did not go up to the door and speak with a resident or take a map spot on the doorstep. Failure to follow procedures could adversely affect a complete count, as addresses could be missed or a group quarter could be misclassified as a residential address. After we alerted the Bureau to our observations, the Bureau agreed moving forward, to emphasize the importance of following procedures during training for in-field address canvassing. Address canvassing has tight time frames, so work needs to be assigned efficiently. Sometimes this means the Bureau needs to reassign work from one lister to another. During address canvassing, the Bureau discovered that reassigned census blocks sometimes would appear in both the new and the original listers’ work assignments. In some cases, this led to blocks being worked more than once, which decreased efficiency, increased costs, and could create confusion and credibility issues when two different listers visit a house. According to Bureau procedures, listers were instructed to connect to the Bureau’s Mobile Case Management (MCM) system to download work assignments (address blocks) and to transmit their completed work at the beginning and end of the work day but not during the work day. Thus during the work day, they were unaware when unworked blocks had been reassigned to another lister. Bureau officials also told us that the Listing and Mapping Application (LiMA) software used to update the address file and maps was supposed to have the functionality to prevent blocks from being worked more than once, but this functionality was not developed because of budget cuts. For 2020, Bureau officials told us they plan to create operational procedures for reassigning work. According to Bureau officials, they plan to require supervisors to contact the original lister when work is reassigned. We have requested a copy of those procedures; however, the Bureau has not finalized them. Standards for Internal Control in the Federal Government (Standards for Internal Control) call for management to design control activities, such as policies and procedures to achieve objectives. Finalizing these procedures should help prevent blocks from being canvassed more than once. The Bureau conducts tests under census-like conditions, in part, to verify 2020 Census planning assumptions, such as workload, how many houses per hour a lister can verify (also known as a lister’s productivity rate), and how many people the Bureau needs to hire for an operation. Moreover, one of the objectives of the test is to validate that the operations being tested are ready at the scale needed for the 2020 Census. For the 2018 End-to-End Test, the Bureau completed in-field address canvassing on time at two sites and early at one site; despite workload increases at all three test sites and hiring shortfalls at two sites. The Bureau credits this success to better than expected productivity. As the Bureau reviews the results of address canvassing, evaluating the factors that affected workload, productivity rates, and staffing and making adjustments to its estimates, if necessary, before the 2020 Census would help the Bureau ensure that address canvassing has the appropriate number of staff and equipment to complete the work in the required time frame. For the 2020 Census, the Bureau estimates it will have to send 30 percent of addresses to the field for listers to verify. However, at the three test sites, the workload was higher than this estimate (see table 1). At one test site, the percent of addresses verified through in-field address canvassing was 76 percent or 46 percentage points more than the Bureau’s expected 2020 Census in-field address canvassing workload estimate of 30 percent. Bureau officials told us that the 30 percent in-field workload estimate is a national average and is not specific to any of the three test sites. Prior to the test, officials said that the Bureau also knew that the West Virginia site was assigning new addresses to some of the test site’s housing units due to local government emergency 911 address conversion and that the in-field workload would be greater in West Virginia when compared to the other test sites. We requested documentation for the Bureau’s original estimate that 30 percent of the 133.8 million expected addresses would be canvassed in- field for the 2020 Census. However, the Bureau was unable to provide us with documentation to support how they arrived at the 30 percent estimate. Instead, the Bureau provided us with a November 2017 methodology document that showed three in-field address canvassing workload scenarios, whereby, between 41.9 and 45.1 percent of housing units would need to go to the field for address canvassing. The three scenarios consider a range of stability in the address file as well as different workload estimates for in-field follow-up. At 30 percent the Bureau would need to canvass about 40.2 million addresses; however, at 41.9 and 45.1 percent the Bureau would need to canvass between 56 million and 60.4 million addresses, respectively. According to Bureau officials, they are continuing to assess whether changes to its in-office address canvassing procedures would be able to reduce the in-field address canvassing workload to 30 percent, while at the same time maintaining address quality. However, Bureau officials did not provide us with documentation to show how the in-field address canvassing workload would be reduced because the proposed changes were still being reviewed internally. Workload for address canvassing directly affects cost – the greater the workload the more people as well as laptop computers needed to carry out the operation. We found that the 30 percent workload threshold is what is reflected in the December 2017 updated 2020 Census cost estimate that was used to support the fiscal year 2019 budget request. Thus, if the 30 percent threshold is not achieved then the in-field canvassing workload will likely increase for the 2020 Census and the Bureau would be at risk of exceeding its proposed budget for the address canvassing operation. Standards for Internal Control call for organizations to use quality information to achieve their objectives. Thus, continuing to evaluate and finalize workload estimates for in-field address canvassing with the most current information will help ensure the Bureau is well-positioned to conduct addressing canvassing for the 2020 Census. For example, according to Bureau officials, preliminary workload estimates will need to be delivered by January 2019 for hiring purposes and the final in-field workload numbers for address canvassing will need to be determined by June 2019 for the start of address canvassing, which is set to begin in August 2019. Moreover, by February 2019 the Bureau’s schedule calls for it to determine how many laptops will be needed to conduct 2020 Census address canvassing. At the test sites, listers were substantially more productive than the Bureau expected. The expected production rate is defined as the number of addresses expected to be completed per hour, and it affects the cost of the address canvassing operation. This rate includes time for actions other than actually updating addresses, such as travel time. In the 2010 Census the rates reflected different geographic areas, and the country was subdivided into three areas: urban/suburban, rural, and very rural. According to Bureau officials, for the 2020 Census the Bureau will have variable production rates based on geography, similar to the design used in the 2010 Census. The Bureau told us they have not finalized the 2020 Census address canvassing production rates. Table 2 shows the expected and actual productivity rates (addresses per hour) for the in-field address canvassing operation at all three test sites. To ensure address canvassing for the test was consistent with the 2020 Census, Bureau officials told us they included the blocks resolved during the now discontinued second phase of in-office review, into the in-field workload for the test. The Bureau attributed the greater productivity to this discontinued second phase. Bureau officials told us that they believe that listers spent less time updating those blocks because they had already been resolved, and any necessary changes were already incorporated. Moreover, while benefitting from the second phase of in-office address canvassing may be one explanation for why listers were more productive. Bureau officials told us that they are unable to evaluate the differences in expected versus actual productivity for blocks added to the workload as a result of the discontinued second phase because of limitations with the data. However, there could be other reasons as well such as travel time and geography. Standards for Internal Control require that organizations use quality information to achieve their objectives. Therefore, continuing to evaluate other factors from the 2018 End-to-End Test that may have increased or could potentially decrease productivity will be important for informing lister productivity rates for 2020, as productivity affects the number of listers needed to carry out the operation, the number of staff hours charged to the operation, and the number of laptops to be procured. For the 2018 End-to-End Test address canvassing operation, the Bureau hired fewer listers than it assumed it needed at two sites and hired more at the other site. In West Virginia, 60 percent of the required field staff was hired and in Washington, 74.5 percent of the required field staff was hired. Nevertheless, the operation finished on schedule at both these sites. In contrast in Rhode Island the Bureau hired 112 percent of the required field staff and finished early. According to Bureau officials, both the West Virginia and Washington state test sites started hiring field staff later than expected because of uncertainty surrounding whether the Bureau would have sufficient funding to open all three test sites for the 2018 End-to-End Test. When a decision was made to open all three sites for the address canvassing operation only, that decision came late, and Bureau officials told us that once they were behind in hiring and were never able to catch up because of low unemployment rates and the short duration of the operation. According to Bureau officials, their approach to hiring for the 2018 End-to-End Test was similar to that used for the 2010 and 2000 Censuses. In both censuses the Bureau’s goal was to recruit and hire more workers than it needed because of immutable deadlines and attrition. After the 2010 Census we reported that the Bureau had over recruited; conversely, for the 2000 Census the Bureau had recruited in the midst of one of the tightest labor markets in three decades. Thus we recommended, and the Bureau agreed to evaluate current economic factors that are associated with and predictive of employee interest in census work, such as national and regional unemployment levels, and use these available data to determine the potential temporary workforce pool and adjust its recruiting approach. The Bureau implemented this recommendation, and used unemployment and 2010 Census data to determine a base recruiting goal at both the Los Angeles, California and Houston, Texas 2016 census test sites. Specifically, the recruiting goal for Los Angeles was reduced by 30 percent. Bureau officials told us that it continues to gather staffing data from the 2018 End-to-End Test that will be important to consider looking forward to 2020. Although address canvassing generally finished on schedule even while short staffed, Bureau officials told us they are carefully monitoring recruiting and hiring data to ensure they have sufficient staff for the test’s next census field operation non-response follow-up, when census workers go door-to-door to follow up with housing units that have not responded. Non-response follow-up is set to begin in May 2018. According to test data as of March 2018, the Bureau is short of its recruiting goal for this operation which is being conducted in Providence County, Rhode Island. The Bureau’s goal is to recruit 5,300 census workers and as of March 2018, the Bureau had only recruited 2,732 qualified applicants to fill 1,166 spots for training and deploy 1,049 census workers to conduct non-response follow-up. Bureau officials told us they believe that low unemployment is making it difficult to meet its recruiting goals in Providence County, Rhode Island, but they are confident they will be able to hire sufficient staff without having to increase pay rates. Recruiting and retaining sufficient staff to carry out operations as labor- intensive as address canvassing and nonresponse follow-up for the 2020 Census is a huge undertaking with implications for cost and accuracy. Therefore, striking the right staffing balance for the 2020 Census is important for ensuring deadlines are met and costs are controlled. Bureau officials told us that during the test 11 out of 330 laptop computers did not properly transmit address and map data collected for 25 blocks. The lister-collected address file and map data are supposed to be electronically transmitted from the listers’ laptops to the Bureau’s data processing center in Jeffersonville, Indiana. The data are encrypted and remain on the laptop until the laptops are returned to the Bureau where the encrypted data are deleted. Prior to learning that not all data had properly transmitted off the laptops, data on seven of the laptops was deleted. Data on the remaining four laptops were still available. In Providence, Rhode Island, where the full test will take place, the Bureau recanvassed blocks where data were lost to ensure that the address and map information for nonresponse follow-up was correct. Recanvassing blocks increases costs and can lead to credibility problems for the Bureau when listers visit a home twice. Going into address canvassing for the End-to-End Test, Bureau officials said they knew there was a problem with the LiMA software used to update the Bureau’s address lists and maps. Specifically, address and map updates would not always transfer when a lister transmitted their completed work assignments from the laptop to headquarters. Other census surveys using LiMA had also encountered the same software problem. Moreover, listers were not aware that data had not transmitted because there was no system-generated warning. Bureau officials are working to fix the LiMA software problem, but told us that the software problem has been persistent across other census surveys that use LiMA and they are not certain it will be fixed. Bureau officials told us that prior to the start of address canvassing they created an alert report to notify Bureau staff managing the operation at headquarters if data were not properly transmitted. When transmission problems were reported, staff was supposed to remotely retrieve the data that were not transmitted. This workaround was designed to safeguard the data but according to officials was not used. Bureau officials told us that they do not know whether this was because the alert reports were not viewed by responsible staff or whether the alert report to notify the Bureau staff managing the operation was not triggered. Bureau officials told us they recognize the importance of following procedures to monitor alert reports, and acknowledge that the loss of data on seven of the laptops may have been avoided had the procedures that alert reports get triggered and monitored been followed; however, officials did not know why the procedures were not followed. For 2020, if the software problem is not resolved, then officials said the Bureau plans to create two new alert reports to monitor the transmission of data. One report would be triggered when the problem occurs and a second report would capture a one-to-one match between data on the laptop and data transmitted to the data center so that discrepancies would be immediately obvious. While these new reports should help ensure that Bureau staff are alerted when data has not properly transmitted, the Bureau has not determined and addressed why the procedures that required an alert report get triggered and then reviewed by Bureau staff did not work as intended. Standards for Internal Control require that organizations safeguard data and follow policies and procedures to achieve their objectives. Thus, either fixing the LiMA software problem, or if the software problem cannot be fixed, then determining and addressing why procedures that alert reports get triggered and monitored were not followed would position the Bureau to help prevent future data losses. To effectively manage address canvassing, the Bureau needs to be able to monitor the operation’s progress in near real time. Operational issues such as listers not working assigned hours or falling behind schedule need to be resolved quickly because of the tight time frames of the address canvassing and subsequent operations. During the address canvassing test, the Bureau encountered several challenges that hindered its efforts to efficiently monitor lister activities as well as the progress of the address canvassing operation. The Bureau provides data-driven tools for the census field supervisors to manage listers, including system alerts that identify issues that require the supervisor to follow-up with a lister. For the address canvassing operation, the system could generate 14 action codes that covered a variety of operational issues such as unusually high or low productivity (which may be a sign of fraud or failure to follow procedures) and administrative issues such as compliance with overtime and completion of expense reports and time cards. During the operation, over 8,250 alerts were sent to CFSs or about 13 alerts were sent per day per CFS. Each alert requires the CFS to take action and then record how the alert was resolved. CFSs told us and the Bureau during debriefing sessions that they believed many of the administrative alerts were erroneous and they dismissed them. For example, during our site visit one CFS showed us an alert that incorrectly identified that a timecard had not been completed. The CFS then showed us that the lister’s timecard had indeed been properly completed and submitted. CFSs we spoke to said that they often dismissed alerts related to expense reports and timecards and did not pay attention to them or manage them. Bureau officials reported that one CFS was fired for not using the alerts to properly manage the operation. To assist supervisors, these alerts need to be reliable and properly used. Bureau officials said that they examined alerts for errors after we told them about our observation. They reported that they did not find any errors in the alerts. They believe that CFSs may not fully understand that the alerts stay active until they are marked as resolved by the CFS. For example, if a CFS gets an alert that a lister has not completed a timecard the alert will remain active until the CFS resolves the alert by stating the time card was completed. The Bureau’s current CFS manual does not address that by the time a CFS sees the alert a lister may have already taken action to resolve it. Because this was a reoccurring situation, CFSs told us they had a difficult time managing the alerts. Standards for Internal Control call for an agency to use quality information to achieve objectives. Bureau officials acknowledge that it is a problem that some CFSs view the alerts as erroneous and told us they plan to address the importance of alerts in training. We spoke to Bureau officials about making the alerts more useful to CFSs, such as by differentiating between critical and noncritical alerts and streamlining alerts by perhaps combining some of them. Bureau officials told us they would monitor the alerts during the 2018 End-to-End Test’s nonresponse follow-up operation and make adjustments if appropriate. However, while the Bureau told us it will monitor alerts for the non-response follow-up operation, the Bureau does not have a plan for how it will examine and make alerts more useful. Ensuring alerts are properly followed up on is critical to the oversight and management of an operation. If the CFSs view the alerts as unreliable, they could be likely to miss key indicators of fraud such as unusually high or low productivity or an unusually high or low number of miles driven. Moreover, monitoring overtime alerts and the submission of daily time cards and expense reports is also important to ensure that overtime is appropriately approved before worked and that listers get paid on time. Another tool the Bureau uses to monitor operations is its Unified Tracking System (UTS), a management dashboard that combines data from a variety of Census systems, bringing the data to one place where the users can run or create reports. It was designed to track metrics such as the number and percentage of blocks assigned and blocks completed as well as the actual expenditures of an operation compared to the budgeted expenditures. However, information in UTS was not always accurate during address canvassing. For example UTS did not always report the correct number of addresses assigned and completed by site. As a result, Bureau managers reported they did not rely on UTS and instead used data from the source systems that fed into it. Bureau officials agreed that inaccurate data is a problem and that this workaround was inefficient as users had to take extra time to go to multiple systems to get the correct data. Bureau officials reported problems importing information from the feeder systems into UTS because of data mismatches. They said that address canvassing event codes were not processed sequentially, as they should have been, which led to inaccurate reporting. Bureau officials told us that they did not specify that the codes needed to be processed in chronological order as part of the requirements for UTS. Bureau officials said UTS passed the requisite readiness reviews and tests. However, Bureau officials also acknowledged that some of these problems could have been caught by exception testing which was not done prior to production. To resolve this issue for 2020, Bureau officials stated they are developing new requirements for UTS to automatically consider the chronological order of event codes. The Bureau told us they are working on these UTS requirements and will provide us with documentation when they are complete. They also said the Bureau plans to implement a process which compares field management reports with UTS reports to help ensure that the reports have the same definitions and are reporting accurate information. Standards for Internal Control call for an organization’s data be complete and accurate and processed into quality information to achieve their objectives. Thus, finalizing UTS requirements for the address canvassing reporting should help increase efficiency for the 2020 Census by avoiding time consuming workarounds. The Bureau has taken significant steps to use technology to reduce census costs. These steps include using electronic systems to transmit listers’ assignments and address and map data. However, during the address canvassing test, several listers and CFSs at the three test sites experienced problems with Internet connections primarily during training. The West Virginia site, which was more rural than the other sites, experienced the most problems with Internet connectivity. All six West Virginia CFSs reported Internet connectivity problems during the operation. As a work around, CFSs told us that a couple of their listers transmitted their work assignments from libraries where they could access the Internet. Bureau officials stated that the laptops in the 2018 End-to-End Test only used two broadband Internet service providers, which may have contributed to some of the Internet access issues. Bureau officials added that despite the reported Internet connectivity issues, the 2018 End-to- End Test for address canvassing finished on schedule and without any major problems. While this might be true for the test, we have previously reported that minor problems can become big challenges when the census scales up to the entire nation. Therefore, it is important that these issues get resolved before August 2019 when in-field address canvassing for the 2020 Census is set to begin. The Bureau is analyzing the cellular network coverage across all 2020 Census areas using coverage maps and other methods to determine which carrier is appropriate (including a backup carrier) for geographic areas where network coverage is limited. According to Bureau officials, they anticipate identifying the cellular carriers for each of its 248 area census offices by the summer of 2018. The officials said they are considering both national and regional carriers to provide service in some geographic areas because the best service provider in a certain geographic area may not be one of the national providers, but a regional provider. In those cases, listers and other staff in those areas will receive devices with the regional carrier. According to Bureau officials, for the 2020 Census, the ability to access multiple carriers should provide field staff with better connectivity around the country. We also found that there was no guidance for listers and CFSs on what to do if they experienced Internet connectivity problems and were unable to access the Internet. Bureau officials told us that staff in the field can use different methods to access the Internet, such as using home wireless networks or mobile hotspots located at libraries, or coffee shops to transmit data. However, the Bureau did not provide such instructions to listers. In addition, the Bureau also does not define what constitutes a secure Internet public connection. Ensuring data are safeguarded is important because census data are confidential. Bureau officials told us that the Bureau plans to provide instructions to field staff on what to do if they are unable to access census systems and what constitutes a secure Internet connection for the next 2018 End-to-End Test field operation, non-response follow-up. However, the Bureau has not finalized or documented these instructions. Standards for Internal Control call for management to design control activities, such as providing instructions to employees to achieve objectives. Finalizing these instructions to field staff will help ensure listers have complete information on how to handle problems with Internet connectivity and that data are securely transmitted. Some listers had difficulty accessing the Internet to take online training for address canvassing. This is the first decennial census that the Bureau is using online training, in previous decennials training was instructor-led in a class room. According to the Bureau, in addition to the Bureau provided laptop, listers also needed a personal home computer or laptop and Internet access at their home in order to complete the training. However, while the Bureau reported that listers had access to a personal computer to complete the training, we found some listers did not have access to the Internet at their home and were forced to find workarounds to access the training. According to American Community Survey data from 2015, among all households, 77 percent had a broadband Internet subscription. Bureau officials told us they are aware that not all households have access to the Internet and that the Bureau’s field division is working on back-up plans for accessing online training. Specifically, Bureau officials told us for 2020 they plan to identify areas of the country that could potentially have connectivity issues and plan to identify alternative locations such as libraries or community centers where Internet connections are available to ensure all staff has access to training. However, they have not finalized those plans to identify locations for training sites. Standards for Internal Control call for management to design control activities, such as having plans in place to achieve objectives. Finalizing these plans to identify alternative training locations will help ensure listers have a place to access training. The Bureau’s re-engineered approach for address canvassing shows promise for controlling costs and maintaining accuracy. However, the address canvassing operation in the 2018 End-to-End test identified the need to reexamine assumptions and make some procedural and technological improvements. For example, at a time when plans for in- field address canvassing should be almost finalized, the Bureau is in the process of evaluating workload and productivity assumptions to ensure sufficient staff are hired and that enough laptop computers are procured. Moreover, Bureau officials have not finalized (1) procedures for reassigning work from one lister to another to prevent the unnecessary duplication of work assignments, (2) instructions for using the Internet when connectivity is a problem to ensure listers have access to training and the secure transmission of data to and from the laptops, and (3) plans for alternate training locations. To ensure address and map data are not lost during transmission, Bureau officials will also need to either (1) fix the problem with the LiMA software used to update the address and map files or (2) determine and address why procedures that alert reports be triggered and monitored were not followed. Finally, the Bureau has made progress in using data driven technology to manage address canvassing operations. However, ensuring data used by supervisors to oversee and monitor operations are both useful and accurate will help field supervisors take appropriate action to address supervisor alerts and will help managers monitor the real-time progress of the address canvassing operation. With little time remaining it will be important to resolve these issues. Making these improvements will better ensure address canvassing for the actual enumeration, beginning in August 2019, fully functions as planned and achieves desired results. We are making the following seven recommendations to the Department of Commerce and the Census Bureau: Secretary of Commerce should ensure the Director of the U.S. Census Bureau continues to evaluate and finalize workload estimates for in-field address canvassing as well as evaluates the factors that impacted productivity rates during the 2018 End-to-End Test and, if necessary, make changes to workload and productivity assumptions before the 2020 Census in-field address canvassing operation to help ensure that assumptions that impact staffing and the number of laptops to be procured are accurate. (Recommendation 1) Secretary of Commerce should ensure the Director of the U.S. Census Bureau finalizes procedures for reassigning blocks to prevent the duplication of work. (Recommendation 2) Secretary of Commerce should ensure the Director of the U.S. Census Bureau finalizes backup instructions for the secure transmission of data when the Bureau’s contracted mobile carriers are unavailable. (Recommendation 3) Secretary of Commerce should ensure the Director of the U.S. Census Bureau finalizes plans for alternate training locations in areas where Internet access is a barrier to completing training. (Recommendation 4) Secretary of Commerce should ensure the Director of the U.S. Census Bureau takes action to either fix the software problem that prevented the successful transmission of data, or if that cannot be fixed, then determine and address why procedures that alert reports be triggered and monitored were not followed. (Recommendation 5) Secretary of Commerce should ensure the Director of the U.S. Census Bureau develops a plan to examine how to make CFS alerts more useful so that CFSs take appropriate action, including alerts a CFS determines are no longer valid because of timing differences. (Recommendation 6) Secretary of Commerce should ensure the Director of the U.S. Census Bureau finalizes UTS requirements for address canvassing reporting to ensure that the data used by census managers who are responsible for monitoring real-time progress of address canvassing are accurate before the 2020 Census. (Recommendation 7) We provided a draft of this report to the Department of Commerce. In its written comments, reproduced in appendix I the Department of Commerce agreed with our recommendations. The Census Bureau also provided technical comments that we incorporated, as appropriate. As agreed with your offices, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we are sending copies of this report to the Secretary of Commerce, the Under Secretary of Economic Affairs, the Acting Director of the U.S. Census Bureau, and interested congressional committees. The report also will be available at no charge on GAO’s website at http://www.gao.gov. If you have any questions about this report please contact me at (202) 512-2757 or goldenkoffr@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff that made major contributions to this report are listed in appendix II. In addition to the contact named above, Lisa Pearson, Assistant Director; Kate Wulff, Analyst-in-Charge; Mark Abraham; Devin Braun; Karen Cassidy; Robert Gebhart; Richard Hung; Kirsten Lauber; Krista Loose; Ty Mitchell; Kayla Robinson; Kate Sharkey; Stewart Small; Jon Ticehurst; and Timothy Wexler made key contributions to this report.
The success of the decennial census depends in large part on the Bureau's ability to locate every household in the United States. To accomplish this monumental task, the Bureau must maintain accurate address and map information for every location where a person could reside. For the 2018 End-to-End Test, census workers known as listers went door-to-door to verify and update address lists and associated maps in selected areas of three test sites—Bluefield-Beckley-Oak Hill, West Virginia; Pierce County, Washington; and Providence County, Rhode Island. GAO was asked to review in-field address canvassing during the End-to-End Test. This report determines whether key address listing activities functioned as planned during the End-to-End Test and identifies any lessons learned that could inform pending decisions for the 2020 Census. To address these objectives, GAO reviewed key documents including test plans and training manuals, as well as workload, productivity and hiring data. At the three test sites, GAO observed listers conducting address canvassing. The Census Bureau (Bureau) recently completed in-field address canvassing for the 2018 End-to-End Test. GAO found that field staff known as listers generally followed procedures when identifying and updating the address file; however, some address blocks were worked twice by different listers because the Bureau did not have procedures for reassigning work from one lister to another while listers work offline. Bureau officials told GAO that they plan to develop procedures to avoid duplication but these procedures have not been finalized. Duplicating work decreases efficiency and increases costs. GAO also found differences between actual and projected data for workload, lister productivity, and hiring. For the 2020 Census, the Bureau estimates it will have to verify 30 percent of addresses in the field. However, at the test sites, the actual workload ranged from 37 to 76 percent of addresses. Bureau officials told GAO the 30 percent was a nationwide average and not site specific; however, the Bureau could not provide documentation to support the 30 percent workload estimate. At all three test sites listers were significantly more productive than expected possibly because a design change provided better quality address and map data in the field, according to the Bureau. Hiring, however, lagged behind Bureau goals. For example, at the West Virginia site hiring was only at 60 percent of its goal. Bureau officials attributed the shortfall to a late start and low unemployment rates. Workload and productivity affect the cost of address canvassing. The Bureau has taken some steps to evaluate factors affecting its estimates, but continuing to so would help the Bureau refine its assumptions to better manage the operation's cost and hiring. Listers used laptops to connect to the Internet and download assignments. They worked offline and went door-to-door to update the address file, then reconnected to the Internet to transmit their completed assignments. Bureau officials told GAO that during the test 11 out of 330 laptops did not properly transmit address and map data collected for 25 blocks. Data were deleted on 7 laptops. Because the Bureau had known there was a problem with software used to transmit address data, it created an alert report to notify the Bureau staff if data were not properly transmitted. However, Bureau officials said that either responsible staff did not follow procedures to look at the alert reports or the reports were not triggered. The Bureau is working to fix the software problem and develop new alert reports, but has not yet determined and addressed why these procedures were not followed. The Bureau's data management reporting system did not always provide accurate information because of a software issue. The system was supposed to pull data from several systems to create a set of real-time cost and progress reports for managers to use. Because the data were not accurate, Bureau staff had to rely on multiple systems to manage address canvassing. The Bureau agreed that not only is inaccurate data problematic, but that creating workarounds is inefficient. The Bureau is developing new requirements to ensure data are accurate but these requirements have not been finalized. GAO is making seven recommendations to the Department of Commerce and Bureau including to: (1) finalize procedures for reassigning work, (2) continue to evaluate workload and productivity data, (3) fix software problem, or determine and address why procedures were not followed, and (4) finalize report requirements to ensure data are accurate. The Department of Commerce agreed with GAO's recommendations, and the Bureau provided technical comments that were incorporated, as appropriate.