In the United States, election authority is shared by federal, state, and local officials, and election administration is highly decentralized and varies among state and local jurisdictions. Congressional authority to regulate elections derives from various constitutional sources, depending upon the type of election. Federal election laws have been enacted that include provisions pertaining to voter registration, protecting the voting rights of certain minority groups, and other areas of the elections process. States regulate various election activities, including some requirements related to these federal laws, but generally delegate election administration responsibilities to local jurisdictions. Congress has passed legislation in major functional areas of the voting process. For example, HAVA includes a number of provisions related to voting equipment and other election administration activities, including, for instance, requiring at least one voting system equipped for persons with disabilities at each polling place in federal elections. After HAVA was enacted, Congress appropriated more than $3 billion for the EAC to distribute to states to make election administration improvements, such as the replacement of punch card and mechanical lever voting equipment. In addition to HAVA, federal laws have been enacted in other areas of the voting process. For example, the Voting Rights Act of 1965, as amended, contains, among other requirements, provisions designed to protect the voting rights of U.S. citizens of certain ethnic groups whose command of the English language may be limited. In accordance with the act, covered states and jurisdictions must provide written materials—such as ballots or registration forms—in the language of certain “language minority groups” in addition to English, as well as other assistance, such as bilingual poll workers. The responsibility for the administration of elections resides at the state and local levels. States regulate various election activities, such as absentee and early voting requirements and Election Day procedures, but generally delegate election administration responsibilities to local jurisdictions. Some states have mandated statewide election administration guidelines and procedures that foster uniformity in the ways local jurisdictions conduct elections, including the types of voting equipment used. Other states have guidelines that generally permit local election jurisdictions considerable autonomy and discretion in the way they run elections. Although some states bear some election costs, including those associated with voting equipment, local jurisdictions generally pay for most aspects of election administration. Unless states require otherwise, local jurisdictions generally have discretion over activities such as training election officials and, in most states, over the selection and purchase of voting technology. Among other things, local election officials register eligible voters; design ballots; educate voters on how to use voting technology; provide information on the candidates and ballot measures; arrange for polling places; recruit, train, organize, and mobilize poll workers; prepare and test voting equipment for use; and count ballots. States have established alternatives for voters to cast a ballot other than at the polls on Election Day, including absentee voting and early voting. All states and the District of Columbia have provisions allowing voters to cast their ballots before Election Day by voting absentee, with variations on who may vote absentee, whether the voter needs to provide an excuse for requesting an absentee ballot, and the time frames for applying for and submitting absentee ballots. Some states also permit registered voters to apply for an absentee ballot on a permanent basis so that those voters automatically receive an absentee ballot in the mail prior to every election without providing an excuse or reason for voting absentee. In addition to absentee voting, some states allow early in- person voting. In general, early voting allows voters from any precinct in the jurisdiction to cast their vote in person without providing an excuse, before Election Day either at one specific location or at one of several locations. Further, three states and a number of local election jurisdictions in other states conduct vote-by-mail elections, wherein ballots are automatically sent to every eligible voter. For in-person voting on Election Day, election authorities subdivide local election jurisdictions into precincts. Voters generally cast their ballots at the polling places for the precincts to which they are assigned by election authorities. In addition, some states provide jurisdictions the discretion to allow voters to cast their ballots at vote centers, which are polling places at which any registered voter in the local election jurisdiction may vote on Election Day, regardless of the precinct in which the voter resides. Within the polling place, poll workers check in voters and determine their eligibility to vote by verifying their registration using voter lists or poll books—a list of individuals eligible to vote within the voting precinct or local jurisdiction. After checking the voters in, poll workers direct them to a voting booth to mark their electronic or paper ballots, and then voters submit the ballots for counting. The manner in which votes are cast and counted can vary depending on the voting method and technology employed by the jurisdiction. Following the close of the polls on Election Day, election officials and poll workers complete steps such as securing equipment and ballots, transferring paper ballots or electronic records of vote counts to a central location for counting, and determining the outcome of the election. Votes counted include those cast on Election Day, absentee ballots, early votes (where applicable), and valid provisional ballots. While preliminary results are available usually by the evening of Election Day, the certified results are generally not available until a later date. The EAC has responsibility for developing the voluntary voting system guidelines and overseeing the testing and certification of voting systems based on these guidelines. The EAC works in conjunction with NIST and the Technical Guidelines Development Committee (TGDC) to develop the voluntary guidelines. According to the EAC, these guidelines are a set of specifications and requirements against which voting systems, including hardware and software, can be tested to receive a certification from the EAC. According to NIST, the guidelines are intended to ensure that federal testing provides assurance to state and local election officials that the voting systems meet a defined set of requirements. The EAC testing and certification program verifies that voting systems comply with basic functionality, accessibility, and security capabilities established by the voluntary guidelines. Typically, voting system vendors submit their systems to the EAC for testing and certification and the systems are evaluated by EAC-accredited voting system test laboratories against the guidelines. These laboratories make recommendations regarding certification to the EAC. According to the EAC, an EAC-certified voting system means that the voting system has been tested by a federally accredited test laboratory and complies with the guidelines. According to the EAC, prior to its establishment and the creation of its voluntary voting system guidelines, the first set of federal voluntary Voting System Standards were adopted in 1990 by the Federal Election Commission. The National Association of State Election Directors voluntarily assumed the role of accrediting voting system test laboratories and certifying voting systems to the federal standards. In 2002, the Federal Election Commission adopted a new version of the federal standards. After the EAC’s creation, in 2005, the EAC developed and adopted the third iteration of federal standards, in accordance with HAVA, and the standards were renamed the Voluntary Voting System Guidelines (VVSG). This third iteration of federal voting system guidelines was referred to as the 2005 VVSG or VVSG 1.0, as it is called today. According to the EAC, VVSG 1.0 increased security requirements for voting systems and were intended to expand access, including opportunities to vote privately and independently, for individuals with disabilities. In 2006, the National Association of State Election Directors terminated its voting system testing program and subsequently, in 2007, the EAC launched its own testing and certification program. In March 2015, a fourth iteration of the voluntary guidelines was adopted by the EAC, referred to as VVSG 1.1. According to the EAC, VVSG 1.1 clarified the guidelines to improve testability by testing laboratories, among other updates, and focused on areas that could be improved without requiring significant changes to the testing and certification process. In January 2016, the EAC adopted an implementation plan for VVSG 1.1 whereby all new voting systems being tested for certification would be required to be tested against the VVSG 1.1 beginning on July 6, 2017. As of November 2017, no voting systems have been certified using VVSG 1.1. The EAC, NIST, and TGDC are in the process of developing the next iteration of the voluntary guidelines (known as VVSG 2.0), and these guidelines are expected to be issued in late summer 2018. Typically, a lag exists between when guidelines are issued and when they are used for testing and certification. EAC officials stated that it has generally taken about 18 months before the guidelines are ready for use for testing voting systems. This is due in part to the need for the voting system test laboratories to be reaccredited to test to the new voluntary guidelines by the EAC. According to EAC officials, after the guidelines are approved for use, it typically takes 2 to 4 years before voting system vendors can develop voting systems that are ready for testing and certification. Participation in the EAC testing and certification program is voluntary. Each state determines its own standards for voting systems in statute or administrative regulation, which can be based on the voluntary guidelines established by the EAC. Specifically, most states require some level of participation in the EAC testing and certification program as mandated by their state laws or regulations. As of December 2017, 13 states require federal certification of their voting systems, 24 states and the District of Columbia require testing by a federally accredited laboratory or require testing to federal voting system standards, and 13 states have no federal requirements. Some states have their own voting system standards and conduct their own testing and certification to these standards, either in addition to or as an alternative to the federal voluntary guidelines. Vendors that want to supply their voting systems to local jurisdictions and states must comply with state requirements. See appendix II for federal certification and testing requirements by state, including the associated statutes and regulations we reviewed. According to our analysis of the predominant type of equipment used to process the largest number of ballots during the 2016 general election, jurisdictions using optical/digital scan equipment represented the largest estimated share of the population nationwide, followed by jurisdictions using direct recording electronic (DRE) equipment. Specifically, on the basis of our local election jurisdiction survey, we estimate that jurisdictions with about 63 percent of the population nationwide used optical/digital scan equipment as their predominant voting equipment during the election, while jurisdictions with an estimated 32 percent of the population nationwide used DREs. Jurisdictions with less than 1 percent of the population nationwide used paper hand-counted ballots. See figure 1. Within the optical/digital scan equipment category, the most widely used model of optical/digital scan equipment was the precinct count optical/digital scan, with jurisdictions having an estimated 46 percent of the population nationwide using it as their predominant voting equipment. Figure 2 shows the predominant types of voting equipment that were used by jurisdictions during the 2016 general election, broken out by model of equipment used. While many jurisdictions predominantly used one type of voting equipment, some reported using multiple types. Jurisdictions may choose to use more than one type of equipment as a means to process different types of ballots such as absentee or provisional or to provide accessibility options for voters with disabilities. Overall, we estimate that jurisdictions with about 59 percent of the population nationwide used only one type of equipment during the 2016 general election, while jurisdictions with about 37 percent of the population nationwide used multiple types of equipment during the election. Jurisdictions that used two types of equipment are estimated to have about 30 percent of the population nationwide, while those that used more than two types of voting equipment had approximately 6 percent of the population nationwide. See figure 3 for the types of voting equipment used. According to results from our survey of local election jurisdictions, jurisdictions monitored the performance of their voting equipment during the 2016 general election through a variety of methods, such as equipment testing, performance measurement and tracking of malfunctions, and postelection audits and recounts. Such monitoring can provide information to jurisdictions about how their equipment is functioning and help ensure the accuracy of the outcomes of elections and address any identified issues or problems. Results from our survey of local election jurisdictions indicate that the extent to which jurisdictions tested their voting equipment varied by test type. Key types of voting equipment testing include acceptance testing, logic and accuracy testing, and parallel testing. Acceptance testing verifies that new equipment or any equipment that has been outside election administrators’ control (e.g., for repair) conforms to the purchase agreements and is identical to equipment that was tested and certified by state or federal testing organizations. According to our local jurisdiction survey results, jurisdictions with an estimated 49 percent of the population nationwide conduct acceptance testing of their equipment. Logic and accuracy (also known as functional or readiness) testing is performed in advance of an election to determine whether voting equipment will function properly, such as displaying the correct ballot, collecting votes, and tabulating results. Parallel testing is performed on Election Day by running test votes cast with known results, then comparing the actual and expected results. Of these two types of testing, according to our local jurisdiction survey results, logic and accuracy testing was the most widely performed type of testing as jurisdictions with 99 percent of the population nationwide conducted such testing for the 2016 general election. Jurisdictions with an estimated 37 percent of the population nationwide conducted parallel testing. According to our local jurisdiction survey results, jurisdictions monitored the performance of their predominant voting equipment during the 2016 general election using a variety of measures. Accuracy of the equipment in counting votes was tracked, measured, or assessed by jurisdictions having an estimated 87 percent of the population nationwide. Another widely monitored aspect of voting equipment performance was the accuracy of the equipment in recording voter selections before counting— jurisdictions with 78 percent of the population nationwide tracked, measured, or assessed that aspect. Overvotes and undervotes were also widely used measures, with jurisdictions having about 63 and 64 percent of the population nationwide, respectively, tracking, measuring, or assessing those measures. According to the results of our local jurisdiction survey, most jurisdictions did not experience extensive or widespread errors or malfunctions with their equipment during the 2016 general election. We estimate that jurisdictions with 93 percent of the population did not experience equipment errors or malfunctions on a “somewhat” or “very” common basis during the election. Of those that did experience equipment errors or malfunctions of some type on a “somewhat” or “very” common basis, the error or malfunction most frequently encountered was jams or misfeeds. We estimate that this error or malfunction was experienced on a “very common” basis by jurisdictions with about 1 percent of the population nationwide and on a “somewhat common” basis by jurisdictions with about 3 percent of the population nationwide. The next most frequent error or malfunction experienced as a “very” or “somewhat” common occurrence was that equipment response was sluggish or slower than acceptable, which was experienced by jurisdictions with an estimated 3 percent of the population nationwide. State and local election officials also determined how their voting equipment performed and verified election results by conducting postelection audits and recounts. According to 35 out of 46 respondents to our state survey, the state election agency or local election jurisdictions in their states conducted postelection audits or targeted recounts of results from the 2016 general election. On the basis of our local jurisdiction survey, we estimate that jurisdictions with approximately 45 percent of the population nationwide conducted postelection audits or targeted recounts. Among jurisdictions of different size, large jurisdictions had a higher estimated share of their population within jurisdictions that conducted postelection audits or recounts than did medium or small jurisdictions. Specifically, jurisdictions with 82 percent of the population within large jurisdictions conducted postelection audits or recounts. In contrast, an estimated 55 percent and 37 percent of the population within medium and small jurisdictions, respectively, was represented by jurisdictions that conducted postelection audits or recounts. According to the results of our local election jurisdiction survey, jurisdictions using the two main types of voting equipment (DRE or optical/digital scan) experienced mostly similar benefits as a result of using their respective type of predominant equipment. Table 1 shows the top benefits experienced by jurisdictions according to the type of predominant voting equipment used. In addition to the benefits mentioned above, jurisdictions experienced other benefits associated with using their respective type of predominant voting equipment. For example, jurisdictions that had an estimated half or more of the population within jurisdictions using each of the different types of voting equipment also experienced the following benefits from using their equipment: Jurisdictions predominantly using DREs: accessibility for individuals with disabilities or impairments, timely election night reporting, ease of presenting lengthy ballots in a clear and understandable way, protection and preservation of votes cast against potential non- cybersecurity related threats, and customer support and problem resolution assistance from vendor. Jurisdictions predominantly using optical/digital scan equipment: timely election night reporting, ease of troubleshooting or resolving equipment malfunctions during Election Day, preventing or alerting voters of any overvotes or undervotes before ballot is cast, ability to facilitate a postelection audit, security of equipment against outside electronic hacking or intrusion, and ease of conducting routine maintenance. Jurisdictions also experienced challenges while using their predominant voting equipment, although to a lesser extent overall than they experienced benefits. Table 2 shows the top challenges experienced by jurisdictions according to the type of predominant voting equipment used. The next most frequently experienced challenges by jurisdictions were the following (estimates with the values for the 95 percent confidence intervals are shown in parentheses): Jurisdictions predominantly using DREs: cost to maintain voting equipment (an estimated 12 percent; 6, 19); cost to operate voting equipment (8 percent; 3, 14); and ease of conducting routine maintenance (7 percent; 2, 14). Jurisdictions predominantly using optical/digital scan equipment: cost to operate voting equipment (an estimated 11 percent; 7, 15); preventing or alerting voters of any overvotes or undervotes before ballot is cast (9 percent; 2, 23), and ease of connectivity with other election administration systems (e.g., voter registration, election night reporting) (9 percent; 2, 23). On the basis of our local election jurisdiction survey, we estimate that jurisdictions with approximately 96 percent of the population nationwide were very satisfied or generally satisfied with the performance of their predominant voting equipment during the 2016 general election. Specifically, we estimate that jurisdictions with approximately 70 percent of the population nationwide were very satisfied with their voting equipment’s performance and 26 percent were generally satisfied (see fig. 4). Jurisdictions with about 2 percent of the population nationwide were generally dissatisfied or very dissatisfied with the performance of their predominant voting equipment. When comparing satisfaction with the performance of their predominant voting equipment used in the 2016 general election against the performance of their predominant equipment used in the 2012 general election, we estimate that jurisdictions with 67 percent of the population nationwide were just as satisfied with their equipment’s performance in 2016 as in 2012, while 16 percent reported they were more satisfied (see fig. 5). Among jurisdictions that used different predominant types of equipment, jurisdictions that predominantly used optical/digital scan equipment that were more satisfied with their equipment’s performance in 2016 had a larger estimated share of their population (20 percent) compared to jurisdictions that predominantly used DRE equipment (4 percent). On the basis of our review of literature and studies, interviews with election subject matter experts, and analysis of our local election jurisdiction and state surveys, we identified four key factors and related issue areas within them that jurisdictions and states consider when deciding whether to replace voting equipment. After considering the factors, jurisdictions may decide to replace their equipment or continue using their existing equipment. The four key factors we identified are: (1) the need for voting equipment to meet federal, state, and local voting system standards and requirements; (2) the cost to acquire new equipment and availability of funding; (3) the ability to maintain equipment and receive timely vendor support; and (4) the overall performance and features of voting equipment. In our local election jurisdiction and state surveys, we asked election officials to rate issue areas related to each of these factors as to how important they were when determining whether to replace voting equipment and then rank the issue areas in terms of which were “most important” in making the determination. Analysis of the results of our surveys indicates that the 24 issue areas within the four factors vary in their relative importance to jurisdictions and states when determining whether to replace voting equipment. The need for voting equipment to meet applicable federal, state, and local voting system standards and requirements is a factor considered by local election jurisdictions and states when determining whether to replace equipment. At the federal level, HAVA generally requires that voting equipment be accessible to individuals with disabilities. As discussed earlier, HAVA also established the EAC which developed and maintains the voluntary guidelines that voting equipment can be tested against to receive federal certification. In turn, many states have established requirements that voting equipment be federally certified or meet some or all of the standards established by the federal guidelines. According to election subject matter experts we spoke with, in addition to federal requirements and standards, some states have imposed additional requirements that voting equipment must meet or satisfy such as having the capability to present all ballot issues and candidates on one page or presenting ballots in multiple languages, for example. We identified four issue areas related to this factor. Figure 6 shows the importance local jurisdictions and state election officials attributed to the various issue areas within this factor when determining whether to replace voting equipment. For example, the need for equipment to meet state and local requirements and standards was considered “very important” by jurisdictions with 87 percent of the population nationwide and as one of the three “most important” issue areas overall by jurisdictions with 36 percent of the population nationwide. Among the states, this issue area was considered as “very important” by 18 out of the 25 states that indicated having a role in determining whether to replace voting equipment and as one of the three “most important” issue areas overall by 7 out of the 25 states. According to election subject matter experts we spoke with, the costs to acquire new equipment and the availability of funding to pay those costs is a key factor that jurisdictions and states consider when determining whether to replace voting equipment. Acquiring new voting equipment involves a variety of costs and expenses. For example, in addition to the cost of the equipment itself, there can be other associated costs, such as training for poll workers and elections staff on the new equipment and voter outreach and education about the change in equipment, that may be incurred as existing equipment is replaced. These related acquisition and transition costs and expenses are incurred by the jurisdictions and states, which in turn must obtain or allocate resources to cover those costs. We identified four issue areas related to this factor. Figure 7 shows the importance local jurisdictions and state election officials attributed to these issue areas when determining whether to replace voting equipment. For example, the availability of state and local funds was considered “very important” by jurisdictions with 62 percent of the population nationwide and as one of the three “most important” issue areas overall by jurisdictions with 18 percent of the population nationwide. Among the states, this issue area was considered as “very important” by 20 out of the 25 states that indicated having a role in determining whether to replace voting equipment and as one of the three “most important” issue areas overall by 9 out of the 25 states. Given the importance of funding for the acquisition of new voting equipment and the assistance federal HAVA grants have previously provided, we asked states and jurisdictions additional questions in our surveys about their funding practices and the extent to which they have HAVA grant funds remaining to acquire voting equipment. The results from our surveys provided the following additional information about these issues: Use of local and state funding sources for acquisition of new voting equipment: On the basis of our local election jurisdiction survey, we estimate that, among various potential funding sources, jurisdictions with 79 percent of the population nationwide obtain funds to acquire new voting equipment through local general funds or budgets as a direct appropriation. Additionally, we estimate that jurisdictions with 43 percent of the population nationwide use state financial assistance or cost sharing as a source of funds for new equipment. According to the results from our state survey, states have different levels of involvement in providing funds for the acquisition of voting equipment. Over half (24) of the 46 states that responded to our survey indicated that they do not provide any financial assistance or cost sharing to local jurisdictions for equipment acquisition, while 11 indicated that they cover all acquisition costs. Eight states indicated that their state provides some financial assistance or cost sharing with local jurisdictions for equipment acquisition, while 2 states indicated a different type of involvement in funding the acquisition of voting equipment, such as covering only the costs of acquiring accessible voting equipment. Availability of HAVA funds: On the basis of our local jurisdiction survey, we estimate that jurisdictions with 10 percent of the population nationwide had HAVA funds remaining to apply toward the acquisition of new voting equipment, with jurisdictions representing 6 percent of the population only having enough HAVA funds to acquire a portion of the equipment needed. Additionally, we estimate that jurisdictions with 42 percent of the population nationwide had no HAVA funds remaining while jurisdictions with 46 percent of the population did not know whether they had any HAVA funds remaining. Impact of lack of HAVA funds: Among jurisdictions that did not have any HAVA funds remaining or only enough to buy a portion of the equipment needed, jurisdictions with an estimated 36 percent of the population indicated that the lack of HAVA funds had affected their decisions regarding the replacement of voting equipment. Further, jurisdictions with an estimated 57 percent of the population in this subgroup (of jurisdictions that indicated that the lack of HAVA funds affected their replacement decisions) delayed the replacement of voting equipment while jurisdictions with 25 percent of the population in this subgroup were not able to acquire the equipment that would best meet their needs. The ability of local election jurisdictions and states to maintain voting equipment and receive timely vendor support is a factor considered when determining whether to replace equipment, particularly as the equipment ages. Election subject matter experts we spoke with noted the importance of access to replacement parts for existing voting equipment as something jurisdictions and states may consider when determining whether to replace equipment. Without adequate access to replacement parts and technical service, either from vendors or supplied by in-house expertise, it can be difficult for jurisdictions and states to maintain their current equipment at a satisfactory level. We identified five issue areas related to this factor. Figure 8 shows the importance local jurisdictions and state election officials attributed to these issue areas when determining whether to replace voting equipment. For example, the sufficiency of vendor support and problem resolution was considered “very important” by jurisdictions with 81 percent of the population nationwide and as one of the three “most important” issue areas overall by jurisdictions with 7 percent of the population nationwide. Among the states, this issue area was considered as “very important” by 15 out of the 25 states that indicated having a role in determining whether to replace voting equipment but no state considered it as one of the three “most important” issue areas overall. The overall performance and features, both of the existing voting equipment and of potential replacement equipment, is also a factor considered by local election jurisdictions and states when determining whether to replace voting equipment. For example, jurisdictions and states may consider the age of their current equipment and how well it is performing, as well as how its performance compares to that of new equipment available for acquisition. In addition, according to elections literature we reviewed and election subject matter experts we spoke with, jurisdictions and states may also take into account specific features new voting equipment can provide that might better meet their needs. The desired features may vary from jurisdiction to jurisdiction depending on specific needs and circumstances, but such features may include an enhanced ability to process a high volume of absentee ballots, capability to present ballots in multiple languages, or ease for poll workers to set up and for voters to use, for example. We identified 11 issue areas related to this factor. Figure 9 shows the importance local jurisdictions and state election officials attributed to these issue areas when determining whether to replace voting equipment. For example, the overall performance of the voting equipment was considered “very important” by jurisdictions with 83 percent of the population nationwide and as one of the three “most important” issue areas overall by jurisdictions with 20 percent of the population nationwide. Among the states, this issue area was considered as “very important” by 18 out of the 25 states that indicated having a role in determining whether to replace voting equipment while 4 out of the 25 states considered it as one of the three “most important” issue areas overall. Given the potential challenges local election officials have identified with using aging or outdated equipment, in our local election jurisdiction survey we asked jurisdictions when they first used their predominant voting equipment. Based on their responses, we estimate that jurisdictions with over half of the population nationwide used predominant voting equipment in the 2016 general election that was first deployed between 2002 and 2006 (see fig. 10) Jurisdictions with the next largest estimated share of the population (28 percent) used equipment that was first deployed between 2012 and 2016. The five local election jurisdictions we selected to include in our review either replaced their voting equipment between 2012 and 2016 or plan to replace their equipment in time for the 2020 general election. We selected these jurisdictions to obtain variation in, to the extent possible, population of jurisdiction, type of voting equipment replaced and selected, and state involvement in selecting and funding voting equipment replacement, among other factors. Table 3 summarizes information related to voting equipment replacement across the five selected jurisdictions. These jurisdictions illustrate varying approaches that localities have used or are using to replace their voting equipment based on their specific needs, circumstances, and resources. For example, Los Angeles County, California. The county has a large and diverse electorate and is in the process of self-designing its own voting system, which is expected to consist of ballot marking devices that produce paper ballots to be tallied on central count digital scanners. County officials stated that the current design concept for the new equipment is intended to provide greater flexibility in administering elections, provide a more user-friendly and accessible voting experience, enhance accuracy and auditability, and could potentially lower costs for system upgrades if developed as planned. For example, according to officials, the ballot marking device is intended to provide the ease of use of a touch screen interface, which would incorporate features such as scrolling and tapping that are familiar to voters who use mobile devices, and will include a headset, tactile keypad, and other devices for voters with disabilities. It would also allow the county to have ballots with multiple formats and a large number of races. The county’s process for developing and deploying its new voting equipment began in 2009 and has five phases—(1) public opinion and stakeholder baseline research, (2) establishment of voting system guiding principles, (3) system design and engineering, (4) manufacturing and certification, and (5) phased implementation. According to officials, the county has taken a user-centered approach to the design of the new voting equipment that prioritizes the specific needs and expectations of the voters. The county is currently in the manufacturing and certification phase and reported that about $19 million has been expended to develop the new voting equipment as of December 31, 2017. County officials told us they plan to retain ownership of the intellectual property rights of the new voting equipment so that the system remains publicly owned and not proprietary like traditional vendor equipment. The county plans to pilot the new equipment in some early voting locations in 2019 and fully roll it out in 2020. Travis County, Texas. The county began its efforts to design its own voting equipment based in part on findings and recommendations from an election study group it convened in 2009. In 2012, it developed a concept for a DRE with a voter-verified paper audit trail that centered on system security, auditability, and the use of commercial off-the-shelf technology. In September 2017, the county announced that it had decided to no longer pursue building the voting equipment because the proposals it received from vendors and other organizations for developing key components of the equipment were not sufficient to build a complete voting system, among other reasons. According to county officials, the county plans to acquire either DREs or ballot marking devices with precinct count digital scanners from a voting system vendor with the goal that whatever equipment it acquires incorporates some of the key features it had intended for its self-designed equipment. For example, officials stated that the new equipment must produce printed paper records that can be tallied and connected with electronic voting records through an automated process and allow for third party verification of results and better postelection audits. They noted that they are prepared to work with vendors to customize existing equipment to meet the county’s requirements if needed. County officials estimate that the new equipment will cost about $16 million and stated that acquisition will be funded through local bonds. The county issued a request for proposals for the equipment in November 2017 and plans to have it in place for the 2020 election. Anne Arundel County, Maryland. In 2016, the county replaced its DREs with a system in which voters manually mark paper ballots and insert them into precinct count digital scanners which then count them. Maryland requires the use of uniform voting equipment in polling places statewide and the state and counties each pay 50 percent of the costs of acquiring equipment. In 2007, Maryland enacted a law that prohibited the use of a voting system unless the State Board of Elections (SBE) determined that the system provides a voter-verifiable paper record, thereby requiring the state’s DREs to be replaced. According to Maryland SBE officials, state law specifically required the purchase of precinct count scanners so the board did not consider other types of voting equipment. The SBE issued a request for proposals for the new voting equipment in July 2014 and four vendors responded. The board formed an evaluation committee to analyze the technical and financial details of the proposals, and according to officials, the committee hosted a public demonstration to collect feedback on the equipment under consideration and worked with the University of Baltimore to perform usability and accessibility testing on the equipment. The SBE decided to lease rather than purchase the equipment for a number of reasons. For example, officials said that leasing provided increased flexibility to update or replace equipment more frequently and had lower upfront costs. According to SBE officials, the current payment to the vendor for leasing the digital scan equipment statewide is approximately $1.1 million per quarter. SBE and Anne Arundel County officials stated that deployment of the new equipment in the 2016 general election went smoothly with no significant challenges. The state contracted with a third party vendor to conduct a postelection audit of the 2016 general election by using independent software to tally all digital ballot images. The audit confirmed the accuracy of the election results. According to SBE officials, the new equipment’s ability to capture and store digital images of the ballots made this type of audit possible. Anne Arundel County officials stated that the ability to conduct such an audit is one of the main benefits of the new equipment. Lafayette County, Florida. Lafayette County has a small population and, in 2016, replaced its precinct count optical scan equipment with precinct count digital scan equipment. The county formed a consortium with 11 other counties in the state to help acquire its new equipment. According to the county’s Supervisor of Elections, having the consortium approach state officials as a group helped secure HAVA funds to help the counties purchase the voting equipment. In addition, he stated that being a part of the consortium helped the counties negotiate a lower price for their equipment than what they could have obtained individually because they pooled their purchases and acquired a higher volume of machines. According to the Supervisor of Elections, the consortium decided to purchase precinct count digital scanners from the same vendor the counties had used before because county staff were familiar with the vendor and equipment, among other reasons. He stated that the total cost to purchase Lafayette County’s new voting equipment was about $70,000. The Supervisor of Elections said that the digital scanners have features that were an improvement over the county’s previous optical scan equipment. For example, he told us that the new scanners have more robust security features, such as locking panels, seals, and a requirement for a passcode to access the system. He also noted that the scanners digitally capture and store ballot images. The Supervisor of Elections and the two poll workers we interviewed stated that deployment of the new voting equipment went smoothly and the county did not experience any challenges because the new and previous equipment are both precinct count scanning systems. According to the Supervisor of Elections, a postelection audit that was conducted, in which the county manually tallied ballots from a randomly selected race and precinct, found that the results were accurate. Beaver County, Utah. Beaver County has a small population and previously used DREs with a voter-verified paper audit trail. In 2014, Beaver County began conducting vote-by-mail elections and replaced its DREs with central count digital scan equipment to support this change. County officials said that, in 2014, they verbally requested proposals for the new equipment from their current vendor and an elections services company that the county had employed in 2012 to provide training, systems testing, and other support for elections. According to the Deputy Clerk, the county requested proposals from these two entities because county officials were familiar with them and were not aware of other vendors that might submit proposals. Officials stated that the county received a proposal from the elections services company, and selected the company because it was the only bid received and the equipment the company sold met the county’s needs and was federally certified. The county reported that the cost to purchase the equipment was about $46,000. Officials said that they are very satisfied with the performance of the new voting equipment. They noted that conducting vote-by-mail elections and using central count scanners allow them to administer elections from one location on Election Day, which requires less time and resources than having to manage multiple polling places. Officials also stated that the new digital scanners are able to count a high volume of ballots in a short period of time. According to officials, the county conducted two postelection audits for the 2016 general election—one required by the state and another that the county initiated. They reported that both audits validated the election results. See appendix V for additional details about voting equipment replacement in our five selected jurisdictions, including the factors that influenced their decisions to replace voting equipment; selection, acquisition, and implementation of their equipment; and perspectives on the process. On the basis of our survey of state election officials and interviews with officials from selected voting system vendors and subject matter experts—representatives from nongovernmental research and other organizations involved in the field of election administration—we found that these stakeholders have varying perspectives on how the current Voluntary Voting System Guidelines (VVSG 1.0 and VVSG 1.1) and their associated testing and certification processes facilitated or posed challenges to the replacement and development of voting equipment. The states we surveyed and the other selected stakeholders we interviewed primarily had experience with VVSG 1.0. As discussed earlier, the VVSG 1.1 were issued in March 2015, but due to the time it generally takes to implement updates to new guidelines, including developing testing programs, among other things, no systems had been certified under this version of the guidelines as of November 2017. One vendor’s system underwent partial testing using VVSG 1.1 but the vendor withdrew the system before the testing was completed. States and selected vendors and subject matter experts provided varying perspectives on how aspects of the current voluntary voting system guidelines and their associated testing and certification processes facilitate the replacement and development of voting equipment. Generally, stakeholders indicated that the guidelines and processes provide assurance that new equipment meets certain requirements, provide guidance for equipment developers, provide a model for state standards, and provide cost savings for states that do not have to duplicate federal testing. For example, 15 of the 26 state survey respondents said the guidelines provide assurance that new voting equipment meets baseline requirements related to security, functionality, usability, accessibility, and privacy. One of these 15 state respondents noted that if the EAC certified voting equipment against the federal guidelines, he believes it meets the highest election standards and also meets requirements set by his state. Another of these 15 state respondents noted that voting equipment that has been tested using the federal guidelines and certified by the EAC will have a higher level of reliability than equipment that has not met these guidelines or been certified by the EAC. Subject matter experts from one nongovernmental organization noted that states that establish their own voting system standards often use the federal guidelines as a base to help develop their standards because the federal guidelines have comprehensive requirements and are well vetted. Experts from another nongovernmental organization said that the guidelines establish a standard for voting equipment features and performance, which may help small jurisdictions that want to acquire new voting equipment but may not have the expertise to independently evaluate the equipment. Further, officials from most of the vendors we interviewed agreed that the federal standards serve as effective baseline requirements. For example, officials from five of the seven vendors we interviewed said that when they are developing voting systems, the federal guidelines help them define the baseline standards that their systems should meet, and five of the nine subject matter experts said the federal guidelines provide baseline requirements. Further, 4 of the 26 state survey respondents indicated that the current voluntary guidelines help reduce the costs and resources needed for states to test and approve new voting equipment. For example, one of the 4 state respondents reported that states do not have to rely on their own voting system testing laboratories for all aspects of the testing and certification of new voting equipment to meet state requirements because most of the testing and certification relevant to state requirements has already been done by EAC-accredited testing laboratories and the EAC. The official noted that this allows the states to do less testing, which could save them money. The states we surveyed and selected vendors and subject matter experts we interviewed also reported that aspects of the current voluntary voting system guidelines and their associated testing and certification processes could pose challenges to the replacement and development of voting equipment in a number of ways. Specifically, some stakeholders indicated that aspects of the guidelines and processes could discourage innovation in equipment development, could limit the choices of voting equipment on the market because the testing and certification processes take too long, and could be costly for states and vendors. For example, officials representing three of the seven vendors we interviewed said the current federal guidelines may discourage innovation for new voting equipment because they are too specific or overly prescriptive. Officials from one of these three vendors said the current guidelines require a specific oval size on the ballots, prescribing how tall and wide the oval should be. Instead of such requirements, the officials said they would like the guidelines to be more performance-based and state, for example, that voters should be able to successfully mark a ballot a specified percentage of the time. Further, officials from another vendor said that the current guidelines are generally written for the purpose of testing and certifying end-to-end voting systems rather than system components such as ballot marking devices, which are generally developed by smaller vendors. As a result, according to this vendor, smaller vendors may face challenges getting new technology certified and into the market. EAC officials stated that they recognize that the current guidelines should be more flexible because specificity may limit innovation and they believe the updates to the VVSG 2.0 should help address this issue. In addition, some stakeholders said they believed that the voluntary guidelines and associated testing and certification processes take too long, and thus limit the choices of voting equipment on the market and make it difficult to make improvements to existing equipment. For example, officials from 8 of the 27 state survey respondents and three subject matter experts said the guidelines and their respective processes limit the number of voting systems that are available for acquisition. Three of the 8 states and three subject matter experts said, in their view, the EAC testing and certification process takes too long. In addition, according to one subject matter expert, if a jurisdiction wants to make changes to its existing voting equipment, such as incorporating new software, it can be a difficult and lengthy process to certify the modified equipment, and in some cases the entire system must be recertified. Also, an official from one vendor said that the federal certification processes are complicated, onerous, and time-consuming and they discourage vendors from making modifications to their voting systems even though the modifications might improve the systems. EAC officials said they have heard from stakeholders that the certification process takes too long but stated that this perception was more accurate in the years immediately following the EAC’s issuance of the VVSG 1.0 in 2005. They said that if voting equipment has been modified and is ready for testing and there are no significant problems encountered during the testing, certifying modifications should take a few weeks to a few months to complete and full system testing and certification of new systems should take about 6 to 9 months. Further, officials from 4 of the 27 states that responded to our survey said the EAC testing and certification process can be costly. One state election official said that the cost of certification may discourage vendors from developing new systems and pursuing EAC certification for their systems, which could limit their ability to sell or supply their systems to state and local election jurisdictions. In addition, this state election official noted that costly federal certification of voting systems has limited the voting equipment choices for election officials. Further, officials from one vendor said that they submitted a new voting system for EAC testing and certification and spent over $12 million before they learned that there were significant issues with getting their system certified. According to EAC officials, this was an uncommon occurrence that resulted from the vendor submitting a system that needed additional work and was not ready for certification. The vendor decided to withdraw its system from the testing and certification process. Shortly after the adoption of VVSG 1.1 in March 2015, the EAC, in conjunction with NIST and the TGDC, began work to develop the next iteration of the guidelines, VVSG 2.0, and anticipates issuing the new version in late summer 2018. The EAC, NIST, and the TGDC have taken actions to develop VVSG 2.0 that may address some of the issues with the earlier iterations of the guidelines that were raised by stakeholders. For example, they have established goals to guide the VVSG 2.0 development process, established working groups to inform the guidelines, and developed VVSG 2.0 high-level principles and guidelines. According to the EAC and NIST, in August 2014, the Future VVSG Working Group, which consisted of officials from state and local election offices, technical experts in such areas as security and disability, and voting system vendors, among others, began work which culminated in the creation of 12 goals to guide the development efforts for the voluntary guidelines. One goal, for example, states that the guidelines’ requirements should be performance based and technology neutral. The goal statement further elaborates that the guidelines should be free from detailed descriptions of any technology, and that the guidelines should be functional in nature so that they can more easily be redefined as technology changes. Another development goal states that the voluntary guidelines and its testing and certification processes should not impose unanticipated cost burdens onto organizations. These goals are designed to address some of the issues with the current voluntary guidelines identified by the stakeholders we interviewed as posing challenges to the replacement and development of voting systems, such as discouraging innovation because they are too specific and discouraging vendors and other voting system developers from pursuing EAC certification for their systems because the process is potentially costly. After the 12 goals for the voluntary guidelines were developed, the EAC and NIST established a new process for developing the next guidelines that is intended to allow for broader and more transparent stakeholder involvement than prior guidelines’ development efforts. This new process brings stakeholders together through a working group structure to develop the guidelines. According to the EAC, the previous process did not fully allow for stakeholder input or effectively leverage stakeholder expertise in developing the guidelines because comments on the guidelines were solicited from the Standards Board and external stakeholders after most of the work had been done. In 2015, the EAC and NIST established seven working groups to obtain feedback and input from stakeholders early in the voluntary guidelines development process. According to the EAC and NIST, the four constituency and three election cycle working groups were created as a public/private partnership to inform the development of the guidelines and are composed of state and local election officials, representatives from the federal and private sectors, members of standards bodies, EAC committee members, academic researchers, and other interested parties. The working groups are led by EAC and NIST staff, and have more than 600 participants across the seven groups. EAC and NIST officials stated that they have informed election officials and other stakeholders about opportunities to participate on these working groups to share their ideas. The four constituency working groups represent areas related to human factors (accessibility and usability), cybersecurity, interoperability, and testing and are charged with developing guidance or other deliverables related to these four areas. For example, one objective for the human factors working group is to identify gaps or issues with current accessibility and usability requirements for voting. The election cycle working groups—focused on pre-election, election, and postelection activities—develop process models related to election activities. For example, an objective for the election working group is to identify the necessary functionality of election systems needed to administer early voting and Election Day activities. The work by these seven working groups will help inform the development of the voluntary guidelines’ requirements. Table 4 shows the seven working groups and their respective responsibilities. Some of the stakeholders we interviewed participate in these working groups. For example, officials from six of the seven voting system vendors we contacted said they have a representative on one or more of the constituency working groups. Generally, these six vendors said the working groups are a positive feature of the voluntary guidelines’ development process. For example, officials from one vendor said they have been encouraged by the amount of collaboration on the working groups, and officials from another vendor said it is beneficial that vendors are part of the working groups because they bring experience and expertise with designing and developing various types of voting systems. In August 2017, the TGDC adopted high-level principles and supporting guidelines for the VVSG 2.0. These principles and guidelines are intended to provide system design goals and broad descriptions of the functions that make up a voting system, in contrast to the VVSG 1.1 which focused more on device- or system-specific requirements. The VVSG 2.0 will be supplemented by requirements consisting of technical details voting system vendors can use to design devices that meet the new guidelines. The supplemental requirements will also detail test assertions for how the accredited test laboratories will validate that a system complies with the requirements. One of the VVSG 2.0 principles, for example, is that ballots and vote selections should be presented in a clear, understandable way so that they can be marked, verified, and cast by all voters. The corresponding guidelines for this principle focus on ballots being perceivable, operable, and understandable. For example, the guideline for perceivable ballots notes that default voting system settings for displaying ballots should work for the widest range of voters and allow voters to adjust settings and preferences to meet their needs. Another VVSG 2.0 principle is that the voting system should be designed to support interoperability, including having voting devices that can interface with each other. The corresponding guidelines for this principle include using standard data formats and commercial off-the-shelf devices if they meet applicable requirements. According to NIST officials, one goal of the interoperability working group is to develop guidance that will enable election equipment and interfacing software to interoperate more easily and “speak the same language.” NIST officials stated that this goal is intended to allow vendors to build and certify system components instead of a full voting system. These principles are designed to help address some of the issues reported by stakeholders, such as the impact of prescriptive requirements for ballot designs on vendor innovation and the challenges encountered with component certification under the current voluntary guidelines. Further, officials from the EAC told us that one key change with the VVSG 2.0 is that the EAC commissioners no longer have to approve changes to the supplemental requirements and test assertions, which will instead be vetted by the EAC’s Board of Advisors and Standards Board. EAC officials noted that this allows for greater flexibility to make improvements to the requirements and testing process, including making changes in response to technological advancements. Additionally, depending on the situation, the new voluntary guidelines are intended to allow for more streamlined testing and certification processes. For example, EAC officials said that under the new guidelines, if there are modifications that have been made to a voting system that has already been certified, the changes can be tested without having the entire voting system go back through the testing and certification process. According to EAC officials, the next steps in the VVSG 2.0 development process are to share the high-level principles and guidelines with the EAC’s Board of Advisors and Standards Board for further vetting, provide the public the opportunity to comment on them, and provide them to the EAC commissioners for approval. Specifically, before final adoption of the guidelines, both boards are to review and submit comments and recommendations regarding the guidelines to the commissioners. EAC officials anticipate that the EAC boards will likely review and pass resolutions in support of the principles and guidelines in April 2018. Following the board reviews, there will be a 90-day period for public comment on the VVSG 2.0, as required by HAVA. The EAC hopes that the time it typically takes to respond to public comments will be shorter than for prior voluntary guidelines, due to the extensive feedback and comments received and considered by the working groups during the development phase. EAC officials anticipate that the EAC commissioners will vote on the VVSG 2.0 principles and guidelines in August or September 2018, and the VVSG 2.0 will be issued after they are approved. According to EAC and NIST officials, the working groups have begun developing the supplemental requirements for the new guidelines. They said that the requirements are expected to be drafted by the summer of 2018 and test assertions for most voting systems are expected to be developed by the summer of 2019. EAC officials noted that it will likely take 12 to 24 months after the EAC commissioners approve the new guidelines before they are ready for use. EAC officials plan to submit to the EAC commissioners a range of recommended dates to consider for implementation. They added that in developing these dates, including when vendors will be required to test new equipment against the updated guidelines, they must consider various factors such as the time voting equipment vendors will need to build their new equipment to VVSG 2.0, and reaccreditation of voting system test laboratories to ensure they can test to VVSG 2.0. Because of the lag between when the guidelines will be issued and when they will be used for testing and certification, EAC officials stated that it is unlikely that systems will be certified in time to be ready for use in the 2020 election. However, these officials noted that they are available to meet with vendors that would like to start developing equipment based on the new guidelines. We provided a draft of this report to the EAC, NIST, and election offices in the five local election jurisdictions that we selected and their respective states for review and comment. The EAC, two jurisdictions, and two states provided technical comments, which we incorporated in the report as appropriate. NIST, three jurisdictions, and three states indicated that they had no comments in e-mails received from March 1 through March 23, 2018. We are sending copies of this report to the EAC, NIST, election offices in the five selected local jurisdictions and their respective states that participated in our research, appropriate congressional committees and members, and other interested parties. In addition, this report is available at no charge on GAO’s website at http://www.gao.gov. If you or your staff have any questions, please contact Rebecca Gambler at (202) 512-8777 or gamblerr@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made significant contributions to this report are listed in appendix VI. This report addresses the following questions: 1. What types of voting equipment did local election jurisdictions use for the 2016 general election, and what are jurisdiction perspectives on equipment use and performance? 2. What factors are considered when deciding whether to replace voting equipment and what approaches have selected jurisdictions taken to replace their equipment? 3. What are selected stakeholders’ perspectives on how federal voting system guidelines affect the replacement and development of voting equipment, and what actions has the Election Assistance Commission (EAC) taken to update the guidelines? For our first objective, we conducted a web-based survey of officials from a stratified random sample of 800 local election jurisdictions nationwide to obtain information from the jurisdictions on the voting equipment used during the 2016 general election and perspectives on equipment use and performance. In total, we received 564 completed questionnaires for a weighted response rate of 68 percent. We surveyed the officials about the types of voting equipment they used, various characteristics of the equipment used, their perspectives on the benefits and challenges they experienced while using the equipment, and how satisfied they were with its performance during the election. Overall, there are 10,340 local election jurisdictions nationwide that are responsible for conducting elections. States can be divided into two groups according to how they delegate election responsibilities to the local election jurisdictions. One group is composed of 41 states that delegate election responsibilities primarily to counties. We also included the District of Columbia in this group of states. However, even within this group there are some exceptions to how election responsibilities are delegated. For example, there are no counties in Alaska, so the state groups all of its Boroughs and Census Areas into four election regions; and 6 states—Illinois, Maryland, Missouri, Nevada, New York, and Virginia—delegate responsibilities to some cities independently from counties. The group of 41 states and the District of Columbia contains about one-fourth of the local election jurisdictions nationwide. The other group is composed of 9 states that delegate election responsibilities to subcounty governmental units, known by the U.S. Census Bureau as Minor Civil Divisions (MCD). This group of states contains about three- fourths of the local election jurisdictions nationwide. The categorization of the 50 states and the District of Columbia by how election responsibilities are organized is as follows (states in bold delegate election responsibilities to some cities independently from counties): County-level states: Alabama, Alaska (four election regions), Arizona, Arkansas, California, Colorado, Delaware, the District of Columbia, Florida, Georgia, Hawaii, Idaho, Illinois, Indiana, Iowa, Kansas, Kentucky, Louisiana, Maryland, Mississippi, Missouri, Montana, Nebraska, Nevada, New Jersey, New Mexico, New York, North Carolina, North Dakota, Ohio, Oklahoma, Oregon, Pennsylvania, South Carolina, South Dakota, Tennessee, Texas, Utah, Virginia, Washington, West Virginia, and Wyoming MCD–level states: Connecticut, Maine, Massachusetts, Michigan, Minnesota, New Hampshire, Rhode Island, Vermont, and Wisconsin While 27 percent of election jurisdictions nationwide are in states that delegate election responsibilities primarily to counties, according to the 2010 Census, 89 percent of the U.S. population lived in these states. The U.S. population distribution between the two state groups is shown in table 5. The sampling unit for our survey was the geographically distinct local election jurisdiction at the county, city, or MCD level of local government (or, in Alaska, the election region). We constructed our nationwide sample frame of all local election jurisdictions using 2010 decennial Census data and information on local jurisdictions from state election office websites. Census population data were available for all counties, county equivalents, and MCDs. To obtain a representative sample that included a mix of both rural and non-rural jurisdictions, we used a two-level stratified sampling method in which the sample units, or jurisdictions, were broken out into rural and non-rural strata. To do this, we used the U.S. Department of Agriculture’s Economic Research Service’s Rural-Urban Continuum Code (RUCC) system which classifies counties into a nine-category continuum based on their characteristics and location relative to metropolitan areas. The RUCC continuum coding scheme is shown in table 6. To assign a continuum code to each local election jurisdiction, we matched the RUCC county code to each county in the population frame. Cities that are independent local election jurisdictions and spread geographically across one or more counties received the lowest numbered code among the counties which contain them (i.e., most urban). For independent cities that administer their own elections but are contained geographically within a single county, the city received the code assigned to the county. Where necessary, the parent state’s 2010 decennial Census report was checked to make sure all counties that included part of the independent city were identified. MCDs in New England and the Midwest received the code of the parent county that contained them. For our sampling purposes, the rural stratum was defined as all local election jurisdictions with an RUCC code of 7, 8, or 9. The non-rural stratum was defined as all local election jurisdictions with a code of 1, 2, 3, 4, 5, or 6. Of the 10,340 local election jurisdictions nationwide, 70 percent were classified as non-rural while 30 percent were classified as rural. We selected a two-level stratified sample of 800 local election jurisdictions. Using the RUCC codes, we allocated 600 sampling units, or jurisdictions, to the non-rural stratum and 200 to the rural stratum. To obtain a sample that also reflected the population distribution across jurisdictions nationwide, we used the population of the local election jurisdiction as the measure of unit size and selected the sample units within each stratum with probability proportionate to population of the local election jurisdiction, without replacement. We used jurisdiction population size, rather than the number of eligible or registered voters, because these Census data were readily available for all counties and MCDs nationwide. Because the sample was selected with probability proportionate to population size, any jurisdiction (county or MCD) with more than about 225,000 people was selected with certainty. Table 7 shows the breakout of jurisdictions by population size, the total population within each size grouping, and the number of jurisdictions sampled. After selecting the units to be included in our survey sample, we obtained contact information for the chief election official within the jurisdictions selected. To do this, we first collected contact information for local election jurisdictions from state election office websites and other publicly available sources. We then called the jurisdiction offices directly to confirm the accuracy of the information and the appropriate official and e- mail address to which the survey URL and the respondent’s login information for the questionnaire should be sent. We launched our web- based local election jurisdiction survey on March 27, 2017, and made it available to respondents to complete online through July 14, 2017. Log in information to the survey was e-mailed to the chief election official of each sampled jurisdiction. Between April 4, 2017, and July 10, 2017, we conducted follow-up with nonrespondents by phone and e-mail. During this follow-up, we learned that some MCDs in Minnesota contract with their respective counties to carry out election administration responsibilities, including those concerning the use of voting equipment. In these cases, we reassigned and sent the questionnaire for the particular MCD to the appropriate county election official for completion. Finally, we adjusted the sampling weights to compensate for nonresponse using weighting classes within each stratum that were based upon population size of the jurisdictions. All sample surveys are subject to sampling error—that is, the extent to which the survey results differ from what would have been obtained if the whole population had been observed. Because we followed a probability procedure based on random selections, our sample is only one of a large number of samples that we might have drawn. As each sample could have provided different estimates, we express our confidence in the precision of our particular sample’s results as a 95 percent confidence interval. This is the interval that would contain the actual population value for 95 percent of the samples we could have drawn. As a result, we are 95 percent confident that each of the confidence intervals based on our web-based survey includes the true values in the sample population. In addition to the reported sampling errors, the practical difficulties of conducting any survey may introduce other types of errors, commonly referred to as nonsampling errors. For example, differences in how a particular question is interpreted, the sources of information available to respondents, or the types of people who do not respond can introduce unwanted variability into the survey results. We took numerous steps in questionnaire development, data collection, and the editing and analysis of the survey data to minimize nonsampling errors. For example, to inform the development of our questionnaire, we reviewed existing reports and studies about voting equipment and elections, such as those by various national public policy research organizations and professional associations of state and local officials involved in election administration, as well as previous GAO surveys and work related to this issue area. In addition, we interviewed election subject matter experts and representatives from organizations in the field of election administration and voting equipment to obtain their views and perspectives on potential issues and subject areas to consider covering in our questionnaire. We also pretested the draft questionnaire by telephone with officials in 4 local election jurisdictions (3 counties and 1 MCD) of various sizes in 4 states and had the draft questionnaire reviewed by two election experts. We used these pretests and reviews to further refine our questions, develop new questions, clarify any ambiguous portions of the questionnaire, and identify any potentially biased questions, and made revisions, as necessary. Further, during our analysis of the responses, we found that due to a higher level of nonresponse by very small jurisdictions of 2,500 persons or less, some national-level estimates that included responses from jurisdictions of all sizes had wider than desired confidence intervals. To improve the precision of these national-level estimates, we subsequently excluded the very small jurisdictions of 2,500 persons or less from our analysis. Computer analyses were conducted to identify any inconsistencies in response patterns or other indications of questionnaire response errors. All computer syntax was peer reviewed and verified by separate programmers to ensure that the syntax had been written and executed correctly. Unless noted otherwise, the point estimates we report are national-level point estimates representing the experiences, views, and opinions of all local election jurisdictions nationwide with populations greater than 2,500. We also provide some point estimates for jurisdiction population subgroups, such as large jurisdictions (greater than 100,000 persons), medium jurisdictions (25,001 to 100,000 persons), and small jurisdictions (2,501 to 25,000 persons), and jurisdictions that used a particular type of voting equipment, in cases where statistically significant differences exist between the subgroups that may be of interest. The jurisdictions we surveyed were selected with probability proportionate to population size, so rather than expressing the point estimates in terms of the percentage of jurisdictions nationwide that had a specified characteristic, we express the point estimates for the survey responses in terms of the percentage of the population nationwide that resides within jurisdictions that had a specified characteristic. Similarly, in instances where we report point estimates for jurisdiction subgroups, we express the point estimate in terms of the percentage of the population that resides within jurisdictions of that respective subgroup that had a specified characteristic. For our second objective, we used our local election jurisdiction survey as described above to obtain information from jurisdictions about the factors they consider when determining whether to replace their voting equipment. In addition to the local election jurisdiction survey, we also conducted a web-based survey of the state-level election offices in the 50 states and the District of Columbia about issues pertaining to the states’ role in selecting and acquiring voting equipment, including the factors considered when determining whether to replace voting equipment. In total, we obtained 46 responses (a 90 percent response rate). We took the same steps to develop the state questionnaire as we did in developing the local election jurisdiction questionnaire described above. We conducted pretests of our draft state questionnaire by telephone with election officials of 4 states with varying election system characteristics such as type of voting equipment used, population size, use of federal voting equipment certification processes, and age of equipment, among other characteristics. We also had the draft questionnaire reviewed by two election experts. We used these pretests and reviews to help further refine our questions, develop new questions, clarify any ambiguous portions of the survey, and identify any potentially biased questions, and made revisions, as necessary. Prior to fielding our state survey, we contacted the secretaries of state or other responsible state-level officials, as well as officials from the District of Columbia, to confirm the contact information for the director of elections or comparable official for their respective state. We launched our web-based state survey on April 6, 2017, and made it available to respondents to complete online through May 19, 2017. Log-in information to the survey was e-mailed to directors of elections or comparable officials. Between April 12, 2017, and May 16, 2017, we conducted follow- up with nonrespondents by phone and e-mail. The total number of responses to individual questions may be fewer than 46, depending upon how many respondents were eligible or chose to respond to a particular question. For example, survey respondents who indicated that their state did not have a role in determining whether to replace voting equipment were directed to skip all subsequent questions related to the factors considered when determining whether to replace equipment. Because this survey was not a sample survey, there are no sampling errors. However, the practical difficulties of conducting any survey may introduce nonsampling errors. For example, differences in how a particular question is interpreted, the sources of information available to respondents, or the types of people who do not respond can introduce unwanted variability into the survey results. We included steps in both the data collection and data analysis stages for the purpose of minimizing such nonsampling errors. For example, we examined the survey results and performed computer analyses to identify inconsistencies and other indications of error. Where these occurred, survey respondents were contacted to provide clarification and the response was modified to reflect the revised information. A second, independent analyst checked the accuracy of all computer analyses. The scope of this work did not include verifying states’ survey responses with local election officials. For additional perspectives and context on the factors considered by jurisdictions and states when replacing voting equipment, we also used our reviews of existing reports and studies about voting equipment and elections and interviews with election subject matter experts, including representatives from nongovernmental research and other organizations involved in the field of election administration and voting equipment. For our review of existing reports and studies, we reviewed literature covering the period from 2005 through 2017 including general news, trade and industry articles, association and nonprofit publications, and government reports related to voting system technology, specifically on the replacement and development of voting systems and voting system standards or guidelines. For our interviews, we identified and selected nine subject matter experts based on our review of reports and studies on voting equipment, their expertise and work in this area, and recommendations from these and other researchers. These subject matter experts represented the following organizations: (1) Brennan Center for Justice, (2) National Conference of State Legislatures, (3) National Association of Secretaries of State, (4) National Association of Counties, (5) National Association of State Election Directors, (6) Verified Voting, (7) Kennesaw State University Center for Election Systems, (8) Center for Election Innovation and Research, and (9) Election Data Services, Inc. The information we obtained from these experts cannot be generalized; however, these experts provided additional perspectives and information on the factors considered by jurisdictions and states when replacing voting equipment. In addition, we interviewed election officials from five local jurisdictions— Los Angeles County, California; Travis County, Texas; Anne Arundel County, Maryland; Lafayette County, Florida; and Beaver County, Utah— that replaced their voting equipment between 2012 and 2016 or plan to replace their equipment in time for the 2020 general election to learn about the approaches and practices they used and obtain their perspectives on the replacement process. We selected these jurisdictions to reflect variation in, to the extent possible, population of jurisdiction, type of voting equipment replaced and selected, state involvement in selecting and funding voting equipment, and particular practices used to replace equipment (e.g., self-designing equipment, leasing equipment), among other factors. For each jurisdiction, we interviewed—on site or by phone—local election officials, state election officials in the jurisdiction’s state, and individuals who have served as poll workers at the jurisdiction’s polling locations if applicable. While these five jurisdictions are not representative of all local election jurisdictions nationwide that replaced or plan to replace their voting equipment, they provide examples of various approaches for replacing voting equipment and perspectives on key issues with replacing equipment. We corroborated various information we obtained through these interviews by reviewing relevant state statutes and documentation that these jurisdictions provided to us, such as postelection reports, voting system studies, expenditure summaries, and solicitations for vendor proposals to provide voting equipment and services. To address objective 3, we used responses to our survey of state election officials and interviews with seven selected voting system vendors, the nine selected subject matter experts mentioned above, and officials from the EAC and National Institute of Standards and Technology (NIST) to obtain perspectives on how federal voting system guidelines and their associated testing and certification processes affect the replacement and development of voting equipment. We obtained perspectives on the most recent federal voluntary voting system guidelines (Voluntary Voting System Guidelines, versions 1.0 and 1.1) because they are currently being used to federally test and certify voting systems. We selected the seven voting system vendors based on the prevalence of jurisdictions’ use of their equipment, and to obtain variation in the type of voting system manufactured, such as optical scanners and direct recording electronic voting equipment, and whether systems were federally certified, under test to be certified, or not certified. We also wanted to include a company that plans to enter the voting system market and potentially submit its product for federal certification. Based on these criteria, we selected the following voting equipment vendors—Dominion Voting Systems, DFM Associates, Election Systems and Software, Everyone Counts, Hart InterCivic, Open Source Election Technology Institute, and Unisyn Voting Solutions. To determine the actions taken or planned by the EAC to update the federal voluntary voting system guidelines, we reviewed EAC and NIST documents and interviewed officials from the EAC and NIST about these actions. We also interviewed the seven selected voting system vendors about their involvement, if any, in updating the guidelines and their perspectives on these actions. The perspectives of the seven voting system vendors and nine subject matter experts are not generalizable but provide examples of views on the federal guidelines and their associated testing and certification processes from a range of stakeholders. We conducted this performance audit from June 2016 to April 2018 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. We reviewed state statutes and regulations as of December 2017 regarding the testing and certification of voting systems to describe the extent to which state laws and regulations reference federal voting system certification or testing standards and the extent to which states require the use of these standards. As shown in table 8 below, we grouped the state laws into three categories for the purposes of this report: (1) requires full federal certification; (2) requires testing by a federally accredited laboratory and/or testing to federal voting system standards; and (3) no federal requirements. Category 2 includes states that use some aspect of the federal testing and certification program but do not require full certification. A number of states in this category require both testing by a federally accredited laboratory and testing to federal standards, but we included in this category states that had either requirement in state law or regulation. Category 3 includes some states that utilize the federal certification or testing standards to some extent but that do not require certification or testing to meet federal standards by law or regulation. We then sent our categorization to state officials in the 50 states and the District of Columbia and incorporated changes that we received from those officials. To determine the types of voting equipment local election jurisdictions used for the 2016 general election, jurisdiction perspectives on equipment use and performance, and the factors jurisdictions consider when deciding whether to replace voting equipment, we conducted a web- based survey of officials from a stratified random sample of 800 local election jurisdictions nationwide. In total, we received 564 completed questionnaires for a weighted response rate of 68 percent. The questions we asked in our survey are shown below. Our survey was composed of closed- and open-ended questions. In this appendix, we include all survey questions and results of responses to the closed-ended questions; we do not provide information on responses provided to open- ended questions. The tables below represent the estimated percentages of the jurisdictions’ responses to the closed-ended questions. The estimates we report are rounded to the nearest percentage point and are national-level point estimates representing the experiences, views, and opinions of all local election jurisdictions nationwide with populations greater than 2,500. Because our estimates are from a generalizable sample, we express our confidence in the precision of our particular estimates as 95 percent confidence intervals which are also provided in the tables. As the jurisdictions we surveyed were selected with probability proportionate to population size, rather than expressing the point estimates in terms of the percentage of jurisdictions nationwide that had a specified characteristic, we express the point estimates for the survey responses in terms of the percentage of the population nationwide that resides within jurisdictions that had a specified characteristic. For a more detailed discussion of our survey methodology, see appendix I. Question 1 (open-ended question): What is the name, title, telephone number, and e-mail address of the primary person completing this questionnaire so that we may contact someone if we need to clarify any responses? The Election Assistance Commission’s (EAC) Voluntary Voting System Guidelines, Version 1.1, defines commercial off-the-shelf (COTS) products as software, firmware, devices, or components that are used in the United States by many different people or organizations for many different applications other than certified voting systems and are incorporated into the voting system with no manufacturer- or application- specific modification. Examples of COTS components include hardware that can be purchased commercially (e.g., tablet devices, scanners, printers, memory cards or chips, etc.) and integrated as part of voting equipment. The next series of questions asks about your jurisdiction’s integration of COTS components into voting equipment that was acquired from a vendor or self-designed by your jurisdiction. For the purpose of questions 30-36 (the next 7 questions), the term “voting equipment” refers only to the equipment your jurisdiction used to cast and count votes. Question 58 (open-ended question): If you have any additional comments concerning any of the topics covered in this questionnaire, please use the space below. To obtain information on the types of voting equipment used in the 2016 general election and the factors states consider when deciding whether to replace voting equipment, we conducted a web-based survey of state- level election offices in the 50 states and the District of Columbia. The questions we asked in our survey of state election offices are shown below. Our survey was composed of closed- and open-ended questions. In this appendix, we include all survey questions and results of responses to the closed-ended questions; we do not provide information on responses provided to open-ended questions that required manually entered text responses. The tables below represent the frequencies of state responses to the questions. We received surveys from 46 states (a 90 percent response rate), while 5 states did not respond. However, the total number of responses to individual questions may be fewer than 46, depending upon how many states were eligible or chose to respond to a particular question. For a more detailed discussion of our survey methodology, see appendix I. Question 1 (open-ended question): What is the name, title, telephone number, and e-mail address of the primary person completing this questionnaire so that we may contact someone if we need to clarify any responses? Question 46 (open-ended question): If you have any additional comments concerning any of the topics covered in this questionnaire, please use the space below. The five local election jurisdictions we selected to include in our review— Los Angeles County, California; Travis County, Texas; Anne Arundel County, Maryland; Lafayette County, Florida; and Beaver County, Utah— used varying approaches in replacing their voting equipment. Election officials in these jurisdictions and in their respective state election offices provided a range of perspectives on their experiences and the replacement process. Los Angeles County is the most populous local election jurisdiction in the nation. It currently uses hand-marked paper ballots that are tallied using central count optical scan equipment, which has been in place since 2003. Prior to 2003 and dating back to 1968, these same ballots were used for its punch card voting system. The county is in the process of self-designing its own voting system, which is expected to consist of electronic ballot marking devices (BMDs) that produce paper ballots to be tallied on central count digital scanners, and plans to fully implement it in 2020. According to county officials, the overall performance and features of the county’s voting equipment and the need for the equipment to meet potential state and local requirements were among the key factors that influenced the county’s decision to begin the process of replacing its optical scan system. County election officials stated that while the county’s current voting equipment is reliable, accurate, and familiar to voters, the design and the age of the equipment do not offer the technical and functional flexibility necessary to continue to accommodate potential state regulatory changes and the growing and increasingly diverse county electorate. For example, officials stated that the current equipment may not be able to effectively accommodate state mandates that may require changes to ballot formats or length. Specifically, officials said that state legislation enacted in 2015 requires many cities within Los Angeles County to consolidate their elections with the county’s by 2022, and as a result, the number of races and measures on the ballot may exceed the 12-page capacity that the current equipment can accommodate. They also noted that the technical limitations of the equipment present challenges to providing voters with greater voting options, such as early voting or the use of vote centers on Election Day, and features that enhance accessibility and ease of use. The county has developed a design concept and specifications for its new voting equipment and is in the process of soliciting and selecting vendors to manufacture it. It has acquired several functional prototypes of the current design for the new equipment and has outlined the planned in- person voting process using this equipment, as shown in figure 11. According to county officials, the equipment specifications and in-person voting process have not been finalized and continue to be refined. County officials stated that the current design concept for the new equipment is intended to provide greater flexibility in administering elections, provide a more user-friendly and accessible voting experience, enhance accuracy and auditability, and could potentially lower costs for system upgrades if developed as planned: Greater flexibility for administering elections. According to county election officials, the new equipment is designed to provide more flexibility for administering elections and to respond to changing legislative provisions on conducting elections. For example, the California Voter’s Choice Act, which was enacted in September 2016, generally authorizes Los Angeles County to conduct vote center elections beginning in 2020 if certain conditions are met. Officials stated that the proposed new equipment is expected to facilitate the use of vote centers because it would have the capability to electronically retrieve a voter’s ballot regardless of the precinct in which the voter is registered. They also noted that the BMD would allow the county to have ballots with multiple formats and a large number of races. A more user-friendly and accessible voting experience. County election officials stated that the BMD is intended to provide the ease of use of a touch screen interface, which would incorporate features such as scrolling and tapping that are familiar to voters who use mobile devices. The BMD would also allow voters to select from English or the 11 other languages the county plans to support and is designed to include accessibility devices, such as a headset and tactile keypad for voters with vision impairments and other disabilities. Voters would be able to make their selections and cast their paper ballot without having to handle the ballot. Officials stated that these features are expected to allow voters with special needs to use the same equipment as all other voters and cast their votes independently and privately. The county’s proposed design also includes an interactive sample ballot which voters can access from their computers or mobile devices to pre-mark their vote selections, convert to a Quick Response (QR) code, and then scan into the voting equipment to populate their ballots. Officials stated that this feature may help reduce lines by decreasing the time it takes for voters to mark their ballots once they reach the BMD. Enhance accuracy and auditability. The new voting equipment is designed to record vote selections on paper in human readable text. County officials stated that this is expected to more clearly capture voter intent than manually marked ballots, reduce the time and resources needed by county staff to interpret voters’ intent, and increase the accuracy of election results and public trust in the voting process. Officials stated that the new equipment is also expected to improve the county’s auditing capabilities. For example, the digital scanner is designed to allow the county to efficiently audit the results of individual races and measures, including conducting risk-limiting audits in which a specified number of ballots cast for a particular race are reviewed to confirm the election result for that race. According to officials, the county’s current equipment tallies ballots by precinct and does not keep an electronic record of the specific votes cast on individual ballots. As such, it provides the capability of auditing the results by precinct but not individual races at the ballot level. Easier and less costly upgrades. According to county officials, the design of the voting equipment is intended to be modular so that key components can be replaced individually. Officials stated that this is intended to allow the county to more easily update equipment and incorporate technological advances because it will be able to swap out components if more affordable, better technology becomes available on the market. Officials said that the cost of replacing equipment parts is expected to be lower than with traditional voting systems. Los Angeles County’s Voting Systems Assessment Project (VSAP) was established by the Registrar-Recorder/County Clerk in 2009 to help guide the development and acquisition of the county’s new voting equipment. According to county election officials, the VSAP has taken a user- centered approach to the design of the new voting equipment that prioritizes the specific needs and expectations of the voters and incorporates the requirements of county election administrators. Officials also stated that they sought to have a transparent design process that included voter input and participation to help promote public confidence in the new voting equipment. The project has five phases—(1) public opinion and stakeholder baseline research, (2) establishment of voting system guiding principles, (3) system design and engineering, (4) manufacturing and certification, and (5) phased implementation. The county is currently in the manufacturing and certification phase. Officials reported that about $19 million has been expended to develop the new voting equipment as of December 31, 2017. Officials also stated that after the new system is certified, an additional $49 million in state funds from the Voting Modernization Bond Act of 2002 will be available to the county. Table 108 describes the VSAP phases, their associated expenditures and funding sources, and examples of key actions taken or planned in each phase. County officials told us they plan to retain ownership of the intellectual property rights of the new voting equipment so that the system remains publicly owned and not proprietary like traditional vendor equipment. The county also plans to use an open source technology framework wherein the source code for the system software is available for review and use by other election jurisdictions and entities by license. According to county election officials, this will allow other jurisdictions to, for example, have similar systems manufactured for their use. Officials stated that having the county own the system design on behalf of the public and using an open source software model are expected to provide greater flexibility for any jurisdictions using the software to cost-effectively make modifications to the equipment and adapt it to their varying needs and requirements. For example, jurisdictions would no longer be limited to relying on a single manufacturer if they would like to make an enhancement to the equipment or replace parts. Officials noted that there is currently no licensing model or institutional framework in use for a publicly owned elections system. However, they stated that open source technology solutions in other industries have been successfully implemented and administered, and the county’s new system software could potentially be licensed and administered in a similar manner. In addition, county officials stated that they have outlined a clear business plan in the Request for Proposal (RFP) and during various information sessions with vendors which officials believe will help incentivize them to participate in building the system without potentially owning the equipment or its intellectual property rights. Specifically, officials noted that vendors would primarily receive revenue from the services they would provide, such as building the equipment and software platform and providing ongoing maintenance and support, rather than from selling the equipment itself. County officials stated that implementing the new voting equipment and moving to vote center elections in 2020 are changes to administering elections for the county that will require a substantial educational and informational effort. Officials noted that they have involved numerous stakeholders throughout the VSAP process to help effectively prepare for these changes and plan to allocate resources to educate voters and train poll workers. Some of these efforts are already underway. For example, the county has posted information and videos on the planned new voting equipment and process on the VSAP website and has been using the BMD prototype for public demonstrations and internal training on the new voting process. Travis County currently uses direct recording electronic (DRE) equipment without a voter-verified paper audit trail (VVPAT), which has been in place since 2001. The county also has conducted vote center elections since 2011. Starting in 2009, the county took steps to design and build its own equipment, including developing a concept for a DRE with a VVPAT that centered on system security and auditability. In September 2017, the county decided to no longer pursue building the voting equipment and plans to purchase equipment from a vendor. The county plans to have the new equipment in place for the 2020 election. According to county officials, the overall performance and features of the county’s voting equipment was the primary reason for deciding to begin the process of replacing its DREs. In 2009, the Travis County Clerk convened an Election Study Group to assess the county’s current equipment and make recommendations for future equipment. This group was composed of 45 members representing election officials and workers, advocacy organizations, voters with disabilities, computer security experts, academics, and other segments of the community. According to the report that the group issued, most members expressed confidence in the way Travis County conducted elections and in the accuracy of its current equipment. However, they also expressed concerns over the equipment’s age and the lack of a paper trail, which they said decreased voter trust in the system and increased the risk of election equipment tampering. The group noted that the Travis County Clerk’s Office’s use of safeguards and security and testing procedures beyond those required by law helped minimize the risk of tampering. The report recommended that the county move toward using equipment that offers an electronic count and paper record as soon as an alternative that met the county’s requirements became available. The Election Study Group outlined 19 key requirements that Travis County’s new equipment should meet. The requirements included, for example, producing a paper voting record that can be verified by the voter and be used to independently, transparently, and efficiently reconcile an electronic tally in an audit or recount; allowing voters with special needs to vote using the same equipment as other voters; enabling early voting and the use of vote centers; and having reasonable purchase, operational, and system upgrade costs. The group found that no equipment on the market in 2009 met the needs of the county and, as a result, the county began exploring options to design its own equipment. Officials stated that this effort was also intended to provide an alternative to the current vendor model that could reduce maintenance costs and annual licensing fees that are incurred with proprietary systems. In 2012, the county Clerk convened a group of election administrators, usability experts, and academic experts in computer science and statistics, and through a series of discussion sessions, developed the concept for the county’s new system, which they named STAR (Secure, Transparent, Auditable, and Reliable) Vote. STAR-Vote was designed to be centered around a DRE that produces verifiable and auditable paper records. At the polling place, voters would make their selections on a DRE device with a commercial off-the-shelf (COTS) tablet, which would also be equipped with an auditory interface for visually impaired voters and other features to assist individuals with special needs. The voters’ selections would be encrypted and stored on the internally networked DRE devices, and voters would also receive a printed paper record with their choices. After reviewing the paper record and confirming their selections, voters would feed the paper record into a ballot box scanner to cast their vote. Once the polls closed, the devices storing the votes would be transported to receiving stations, where voting data are transmitted for electronic tabulation. The paper records would be available for audit or recount purposes. In addition, county officials stated that the equipment’s proposed encryption technology was designed to potentially allow for the following features without revealing any individual’s vote: Voters would receive a receipt that was attached to their paper records at the polling place and could go online after Election Day and use a code on the receipt to verify that their ballots had been cast and counted. Third parties, such as the League of Women Voters or political parties, could access encrypted voting data to verify that the results the county had reported matched vote totals they had independently derived from the data. The county could conduct risk-limiting audits to verify the consistency between the electronic and printed vote records and test the accuracy of the reported election outcomes. Audits could be conducted on individual ballots or races if needed. In June 2015, the county issued a Request for Information for STAR-Vote to solicit input on the design, development, implementation, and maintenance of the equipment. Based on information gathered from the request, it issued an RFP in October 2016 to solicit proposals from voting system vendors and others for the development and implementation of key components of the equipment for in-person voting. The county also issued a Statement of Intent for the equipment to inform interested parties of the county’s planned approach for the long-term management and support of STAR-Vote. According to these documents, the county planned to own the intellectual property rights for the equipment and provide open source software for its system to the elections community under a licensing agreement, which would allow other jurisdictions to use similar equipment. The Statement of Intent described the formation of a nonprofit organization to manage and support STAR-Vote and sought $25 million in funding from interested parties to complete the development of the open source software components, support the organization’s operating budget for the first 5 years, and provide a cash reserve. The county planned to use these funding commitments and local budget appropriations to develop, build, and deploy the equipment. In September 2017, the county announced that it had decided to no longer pursue developing and building STAR-Vote. The county stated that it received 12 proposals in response to the RFP but they were not sufficient to build a complete voting system. According to county officials, none of the proposals included the election management system for the equipment that would handle ballot definition and the tallying of results, among other related tasks. In addition, officials stated that they received limited responses to their solicitation for financial commitments in the Statement of Intent and thus lacked the necessary funding to develop and build the equipment. Officials noted that the open source software platform they had envisioned was seen by voting equipment vendors as a low-revenue business model in the current elections marketplace. They added that potential participants in a STAR-Vote entity may not have had a clear concept of how its business model might work, which they said was perhaps due to the county’s more limited focus on this aspect when they were initially designing the system. Given these obstacles and the age of the county’s current equipment, the county decided that it needed to move toward acquiring more immediately deliverable voting equipment through a voting system vendor. The county has incorporated some of the features of STAR-Vote into its requirements for new voting equipment. According to county officials, the county plans to acquire either DREs or ballot marking devices with precinct count digital scanners because, in their view, they are accurate (e.g., prevent voter errors, such as overvotes or stray marks on the ballot, and minimize questions about voter intent), allow individuals with disabilities to vote on the same equipment as other voters, support vote center elections, and offer fast reporting of election results. The county also plans to require that its next voting equipment have the following features: A voter-verified, paper list of choices for recount purposes. County officials stated that the equipment must produce printed paper records that can be tallied and connected with electronic voting records through an automated process. This electronic connectivity would allow paper-ballot recounts to be conducted on individual races. Security features that include support for third party verification of results and better postelection audits. According to county officials, the equipment they acquire must allow for third parties to independently verify reported election results and must support risk- limiting audits. Officials stated that they believe there is or will be equipment on the market in the near future that could support these features. They noted that they are also prepared to work with vendors to customize existing equipment to meet the county’s requirements if needed, acknowledging that such additions may increase expenses or require additional time to recertify parts of the voting system. County officials estimate that the new equipment will cost about $16 million and stated that acquisition will be funded through local bonds. County officials said they would like to have the new equipment in place for the 2020 election, which would require them to start deploying it no later than May 2019. The county issued an RFP for the system in November 2017, and officials stated that they plan to assemble a group of stakeholders similar to those who participated in the 2009 Election Study Group, as well as the individuals who designed STAR-Vote, to help evaluate the proposals received. Officials noted that their current equipment is functioning and robust, but that the new equipment must be deployed before the current equipment begins to degrade. In addition, they stated that the May 2019 implementation date is the latest possible date in order to allow sufficient time to educate voters and train county staff and election judges on the new equipment before using it in the 2020 election. Anne Arundel County had used DREs without a VVPAT since 2004 and replaced its equipment in 2016 with a system in which voters manually mark paper ballots and insert them into precinct count digital scanners which then count them. Maryland requires the use of uniform voting equipment in polling places statewide and the state and counties each pay 50 percent of the costs of acquiring equipment. In our state survey, Maryland officials reported that the state determines when voting equipment is to be acquired and selects the type and model of voting equipment that local jurisdictions use. According to the Maryland State Board of Elections (SBE) and Anne Arundel County Board of Elections officials, the need for voting equipment to meet state requirements, the overall performance and features of the equipment, and the ability to maintain the equipment were among the key factors that influenced the state’s decision to replace its equipment. Specifically, in 2007, Maryland enacted a law that prohibited the use of a voting system unless the SBE determined that the system provides a voter-verifiable paper record, thereby requiring the state’s DREs to be replaced. SBE officials said that the passage of the new law was driven primarily by a push from voting advocates to move to new equipment that used paper ballots and provided a verifiable paper trail. Although the law was enacted in 2007, state funding for the new equipment was not available until 2014 due to budgetary constraints. While the change in state law was the main reason for replacing its voting equipment, both SBE and Anne Arundel County officials noted that the state’s previous DRE equipment was nearing the end of its life cycle and various problems had begun to occur more frequently. For example, SBE officials said that nonresponsive touch screens and battery unit failures became more common with the equipment used in the state. In addition, Anne Arundel County officials stated that while their equipment generally performed satisfactorily, some of the touch screens had begun to degrade and develop calibration issues, which resulted in the appearance of incorrectly recording voters’ selections. In addition, county officials said that the equipment could no longer support certain software or security updates, and replacement parts were challenging to acquire. According to SBE officials, state law specifically required the purchase of precinct count scanners so the board did not consider other types of voting equipment. The SBE issued an RFP in July 2014 and four voting system vendors submitted proposals. The SBE formed an evaluation committee to analyze the technical and financial details of the proposals. According to SBE officials, the committee’s members included a state official with expertise on voting systems, a county election director, a county technical specialist, and election experts and researchers, among others. Anne Arundel County election officials stated that the SBE also established various subcommittees to solicit input from county officials as the state made its selection. They said that relevant local elections staff members were involved in the selection process and that in their view, the process had worked well. According to SBE officials, in addition to assessing the vendors’ proposals, the evaluation committee worked with the University of Baltimore to perform usability and accessibility testing on the equipment under consideration. The committee also hosted a public demonstration to collect feedback on such areas as ease of use and confidence that votes were accurately cast. Officials stated that after conducting its assessment of the equipment, the committee presented its findings to the SBE, and in October 2014, the board selected the voting equipment to be acquired based on the committee’s recommendation. Maryland requires equipment to be certified by the EAC and the SBE before use in the state. The selected equipment had been certified by the EAC in July 2014 and was certified by the SBE in December 2014. As part of the certification process, the SBE tested the equipment to ensure that it met requirements in the Maryland elections code, including simulating primary and general elections using ballots typically used by jurisdictions in the state, and reviewed the findings from the public demonstration and usability testing performed during the selection process. The SBE decided to lease rather than purchase the equipment for a number of reasons. Specifically, SBE officials said that leasing provided increased flexibility to update or replace equipment more frequently and had lower upfront costs. In addition, the state did not want to buy new equipment until the implementation of updated federal guidelines. Under the current contract to lease the digital scan equipment, payments are made to the vendor on a quarterly basis. According to SBE officials, the current payment to the vendor for leasing the digital scan equipment statewide is approximately $1.1 million per quarter. SBE officials said that the process to acquire new equipment is inherently challenging, but in their view, the process generally went well. Knowing what type of equipment the state needed to acquire simplified the process and reduced the number of proposals that officials needed to review. Nevertheless, they noted that the process took more of their time and resources than they had anticipated, which presented challenges because the state was holding elections during the same time period it was selecting and acquiring the equipment. However, the SBE met its goal of implementing the new equipment by 2016. SBE and Anne Arundel County officials stated that deployment of the new equipment in the 2016 general election went smoothly with no significant challenges. The officials said they took a number of steps to help ensure a successful rollout. For example, SBE officials said that they established a strong project management team and hired contractors to assist with tracking progress toward key deadlines; drafting policies, procedures, and training manuals; and testing equipment and sending it to the counties. Anne Arundel County officials said that they hired about 40 temporary staff to assist with deploying the new equipment and other tasks during the general election. In addition, they stated that the county conducted extensive election judge training and held mock elections using the new equipment. The officials noted that with the new paper-based system, the county needed to recruit and train more election judges compared to past elections to hand out ballots, show voters how to operate the equipment, and handle provisional voting. The two election judges we interviewed stated that the training they received was very comprehensive and effectively prepared them for Election Day. Both SBE and Anne Arundel County officials stated that additional voter education efforts would have been beneficial. According to SBE officials, the SBE had developed plans for a statewide multimedia effort to educate voters on the new equipment but did not receive funding to implement it. A scaled down effort was carried out instead, which included demonstrating voting equipment at meetings and fairs around the state, producing local media news stories, and posting a video on the SBE’s website on how to use the new equipment. SBE and Anne Arundel County officials stated that the more limited voter education efforts might have contributed to longer lines on Election Day in some polling places because many voters were unfamiliar with the equipment and some had questions or needed assistance with using it. However, these officials noted that voter wait times were not a widespread or significant issue during the general election. The two election judges we interviewed stated that some voters needed help inserting their ballots into the scanner, but observed that voters generally appeared to find the new equipment easy to use. They also noted that some voters commented that paper ballots provided them with reassurance with regards to the security of their vote. SBE and Anne Arundel County officials said that the equipment itself performed satisfactorily in the 2016 general election with only minor problems. For example, state officials said that the scanners jammed occasionally, but this was easily resolved by elections personnel. In addition, most polling locations in the state were allocated only one scanner, so some jurisdictions with two-page ballots, such as Anne Arundel County, experienced lines because of the length of time it took for voters to scan their ballots. Anne Arundel County officials plan to analyze voter registration data to help determine the number of scanners needed at each polling place and share the information with the SBE to help inform allocations for future elections. More generally, SBE officials noted that the new system has less equipment to manage—about 2,600 digital scan units compared to the approximately 18,000 DRE units used statewide in prior elections—so there is less pre-election testing and postelection maintenance that has to be done, saving time and labor for the state and counties. The state contracted with a third party vendor to conduct a postelection audit of the 2016 general election by using independent software to tally all digital ballot images. The audit confirmed the accuracy of the election results. According to SBE officials, the new equipment’s ability to capture and store digital images of the ballots made this type of audit possible. Anne Arundel County officials stated that the ability to conduct such an audit is one of the main benefits of the new equipment. Lafayette County has a small population and, in 2016, replaced its precinct count optical scan equipment with precinct count digital scan equipment. The county formed a consortium with other counties in the state to help acquire its new equipment. According to the county’s Supervisor of Elections, the cost to acquire new equipment and availability of funding and the need to meet state requirements were among the key factors that influenced the county’s decision to replace its voting equipment. He stated that Lafayette County’s optical scanners were approximately 15 years old but were generally in good condition and performed satisfactorily in prior elections. County officials had planned to replace the county’s aging voting equipment by 2018 or 2020, but decided to replace it in 2016 because of the opportunity to join a consortium of counties that formed to acquire new equipment, which the Supervisor stated helped secure funding for and lower the costs of purchasing the equipment. In addition, the Supervisor of Elections said that, to comply with state law, the county needed to acquire a paper ballot system with a BMD to replace the DRE it had used for voters with disabilities. Specifically, as of July 2008, Florida law required all voting in the state to be done using mark-sense paper ballots, which are generally counted using optical or digital scanners, except for voting by individuals with disabilities. Current state law requires jurisdictions to use these paper ballots for accessible voting by 2020. As such, according to the Supervisor of Elections, part of the impetus for acquiring new voting equipment was to replace the county’s DRE to meet the 2020 deadline in the law. The Supervisor of Elections stated that Lafayette County is a small county and does not have much purchasing power. He said that Lafayette County and other small counties in the state formed a consortium to lobby the state for assistance and to leverage their collective purchasing power. The 12-county consortium was established in a 2015 meeting that was attended by county election officials, the Florida Deputy Secretary of State, and the vendor that supplied the counties’ previous voting system. According to the Lafayette County Supervisor of Elections, the consortium decided to purchase precinct count digital scanners from the same vendor the counties had used before because county staff were familiar with the vendor and equipment, and the cost for the equipment was lower than similar equipment from another vendor that some counties in the consortium had considered. In addition, the Supervisor of Elections stated that the digital scanners have features that were an improvement over the county’s previous optical scan equipment. For example, he stated that the new scanners have more robust security features, such as locking panels, seals, and a requirement for a passcode to access the system. He also noted that the scanners have touch screens that flip up and are back-lit, which are easier for voters and poll workers to read and more clearly identify overvotes. Further, he stated the scanners digitally capture and store ballot images. The two Lafayette County poll workers we interviewed confirmed that the new equipment more clearly identified overvotes for them and for voters than did the previous equipment. According to the county’s Supervisor of Elections, having the consortium approach state officials as a group helped secure HAVA funds to help the counties purchase the voting equipment. In addition, he stated that being a part of the consortium helped the counties negotiate a lower price for their equipment than what they could have obtained individually because they pooled their purchases and acquired a higher volume of machines. While the consortium negotiated as a unit, each county has an individual contract with the vendor. The Supervisor of Elections stated that the total cost to purchase Lafayette County’s new voting equipment—which included seven digital scanners, seven BMDs for voters with disabilities, and various system components—was about $70,000. The equipment was acquired primarily with HAVA funds, although he noted that the county allocated about $12,000 in local funds to purchase three additional BMDs. A memorandum of agreement for funding and purchasing the equipment was signed by Lafayette County and the state in November 2015 and, according to the Supervisor of Elections, the equipment was acquired in late 2015 and first used in the March 2016 primary election. The Supervisor of Elections and the two poll workers we interviewed stated that deployment of the new voting equipment went smoothly and the county did not experience any challenges because the new and previous equipment are both precinct count scanning systems. The Supervisor noted that the voting process remained the same for the voter, so extensive voter education efforts were not needed. He stated that Lafayette County did not experience any equipment malfunctions during the November 2016 general election, and a postelection audit that was conducted, in which the county manually tallied ballots from a randomly selected race and precinct, found that the results were accurate. Beaver County has a small population and previously used DREs with a VVPAT. In 2014, Beaver County began conducting vote-by-mail elections and replaced its DREs with central count digital scan equipment to support this change. According to Beaver County officials, the overall performance and features of the equipment and the ability to maintain the equipment were among the key factors in their decision to replace the county’s equipment. Officials stated that the county had been using DREs since 2005 and that by 2013, they had come to the conclusion that the equipment was not very efficient or user-friendly for administering elections. For example, the Deputy Clerk stated that it was time consuming to both set up the equipment and tally the votes, which required collecting and uploading the memory component from each of the DREs. She also noted that the operating software for the equipment’s election management system had become out-of-date and did not have a user-friendly interface. According to the Deputy Clerk, this made it difficult for staff to navigate without detailed training, which was time consuming and costly. In addition, county election officials said that they were unsure about future maintenance and system upgrade costs and decided it would be more cost-effective to spend funds on purchasing new voting equipment rather than on upgrades to equipment with which they were not very satisfied. In 2013, the county decided to begin conducting vote-by-mail elections the following year and to acquire new equipment to support this change. According to county officials, this decision was due to the performance of their DREs and a desire to reduce costs and increase the efficiency of administering elections, among other reasons. Officials said that because the county was moving to vote-by-mail elections and DREs would no longer be needed for each precinct, the county would instead acquire central count scanners designed to count the mail-in ballots it would receive at the county elections office. According to Beaver County officials, the main individuals involved in the process to select and acquire the county’s new voting system included the current Beaver County Clerk, Deputy Clerk, a county information technology official, and the previous county clerk, among others. When the county started the process in 2013, the state had not initiated any efforts to help local jurisdictions acquire new equipment. As such, both Utah and Beaver County election officials said that the state was aware of the county’s decision to replace its equipment but was not involved in the selection and acquisition process. County officials stated that they wanted to acquire central count scanners to support conducting vote-by-mail elections and a BMD for in-person voting at the elections office for individuals with disabilities. Officials said that, in 2014, they verbally requested proposals from their current vendor and an elections services company that the county had employed in 2012 to provide training, systems testing, and other support for elections. According to the Deputy Clerk, the county requested proposals from these two entities because county officials were familiar with them and were not aware of other vendors that might submit proposals. Officials said that the county received a proposal from the elections services company, and selected the company because it was the only bid received and the equipment the company sold met the county’s needs and was federally certified. They stated that one of the challenges they experienced as a small county looking to purchase equipment was that vendors were not actively marketing to them. In addition, the Deputy Clerk noted that she had limited elections and information technology experience when the county started the selection process. However, she said that the election services company was familiar with Utah’s elections code and federal voting system requirements, helped negotiate with the vendor to acquire the new equipment, and educated county staff on the equipment. Beaver County reported that the cost to purchase the equipment—two central count digital scanners, a BMD, and associated system components—was about $46,000. Local funds were used to purchase the scanners and HAVA funds were used to purchase the BMD. According to Beaver County officials, county commissioners approved the procurement of the equipment in spring 2014 and it was first used in the June 2014 primary elections. Beaver County officials stated that they deployed the new equipment in 2014 because it was more manageable to conduct such a transition during a non-presidential election year. They noted that they needed to educate the public about both voting by mail and the new voting equipment. Officials stated that the county used local newspaper ads, social media posts, and direct mailings to provide information on these changes. Officials also posted information on the county’s website and allowed people to observe logic and accuracy testing of the equipment. They noted that educating the public on the new voting method and equipment in smaller elections during 2014 and 2015 helped voters become more comfortable with what to expect for the presidential election in 2016. County officials said that they are very satisfied with the performance of the new voting equipment. They noted that conducting vote-by-mail elections and using central count scanners allow them to administer elections from one location on Election Day, which requires less time and resources than having to manage multiple polling places. Officials also stated that the new digital scanners are able to count a high volume of ballots in a short period of time. They said that, for the November 2016 general election, the vote tallying was completed within an hour of the polls closing, which allowed the county to report results quickly. However, one challenge they experienced was that the new equipment’s data format for election night reporting of results to the state was not compatible with the state’s reporting system. To address this issue, county officials reformatted the data to produce a report that could be uploaded into the state’s system, but cautioned that this may not be feasible for larger jurisdictions. According to officials, the county conducted two postelection audits for the 2016 general election—one required by the state and another that the county initiated. For the state audit, the county hand counted 1 percent of total ballots from a randomized list. In addition, the county conducted its own audit by running all ballots on its other digital scanner to compare results. According to officials, both audits validated the election results. In addition to the contact named above, Tom Jessor (Assistant Director), David Alexander, Carl Barden, Chuck Bausell, Brett Fallavollita, Sally Gilley, Christopher Hatscher, Eric Hauswirth, Richard Hung, Jill Lacey, Serena Lo, Jan Montgomery, Heidi Nielson, Shannin O’Neill, Claire Peachey, Jeff Tessin, and Johanna Wong made significant contributions to this report. We gratefully acknowledge the substantial time and cooperation of the state and local election officials, and stakeholders and experts whom we interviewed.
Much of the voting equipment acquired with federal funds after the enactment of the Help America Vote Act in 2002 may now be reaching the end of its life span, and some states and local election jurisdictions—which number about 10,300 and generally have responsibility for conducting federal elections—have or are considering whether to replace their equipment. GAO was asked to examine voting equipment use and replacement. This report addresses (1) the types of voting equipment jurisdictions used for the 2016 general election and their perspectives on the equipment; (2) factors considered when deciding whether to replace equipment and replacement approaches in selected jurisdictions; and (3) stakeholder perspectives on how federal voting system guidelines affect replacing and developing equipment. GAO surveyed officials from a nationwide generalizable sample of 800 local jurisdictions (68 percent weighted response rate) and all 50 states and the District of Columbia (46 responded) to obtain information on voting equipment use and replacement. GAO also interviewed officials from (1) five jurisdictions, selected based on population size and type of voting equipment used, among other things, to illustrate equipment replacement approaches; and (2) seven voting system vendors, selected based on prevalence of jurisdictions' use of equipment, type of equipment manufactured, and systems certified, to obtain views on federal voting system guidelines. These interviews are not generalizable, but provide insights into jurisdictions' and vendors' experiences. Local election jurisdictions primarily used optical scan and direct recording electronic (DRE), also known as touch screen, equipment during the 2016 general election and were generally satisfied with voting equipment performance. Specifically, on the basis of GAO's nationwide generalizable survey of local election jurisdictions, GAO estimated that jurisdictions with 63 percent (from 54 to 72 percent) of the population nationwide used optical or digital scan equipment as their predominant voting equipment during the election, while jurisdictions with 32 percent (from 23 to 41 percent) of the population nationwide used DREs. In addition, the survey results indicated that accurate vote counting and efficiency of operation were top benefits experienced by jurisdictions for both types of equipment, and storage and transportation costs were a top challenge. Further, GAO estimated that jurisdictions with 93 percent (from 88 to 96 percent) of the population nationwide did not experience equipment errors or malfunctions on a very or somewhat common basis and jurisdictions with 96 percent (from 94 to 98 percent) of the population were very or generally satisfied with the performance of their equipment during the 2016 general election. GAO identified four key factors that jurisdictions and states consider when deciding whether to replace voting equipment—(1) need for equipment to meet federal, state, and local voting system standards and requirements; (2) cost to acquire new equipment and availability of funding; (3) ability to maintain equipment and receive timely vendor support; and (4) overall performance and features of equipment. When replacing equipment, the five jurisdictions GAO selected for interviews used varying approaches based on their specific needs and resources. For example, Los Angeles County, California, which has a large and diverse electorate, is self-designing its own voting equipment and, according to officials, has incorporated a user-centered approach that prioritizes the needs and expectations of its voters. Lafayette County, Florida, which has a small population, joined a consortium of other small counties to help obtain funding and pool purchasing power to replace its equipment. The state election officials we surveyed and the seven selected voting system vendors we interviewed, among other stakeholders, had varying perspectives on how the current voluntary federal voting system guidelines affected the replacement and development of voting equipment. These guidelines can be used to test and certify equipment to verify that it meets baseline functionality, accessibility, and security requirements. The stakeholders we surveyed or interviewed generally indicated that the guidelines and their associated testing processes provide helpful guidance for equipment developers, cost savings for states that do not have to duplicate federal testing, and assurance that certified equipment meets certain requirements. However, some of these stakeholders stated that aspects of the guidelines could discourage the development of innovative equipment and limit the choices of voting equipment on the market. The Election Assistance Commission (EAC), which is responsible for developing the federal guidelines, is updating them with stakeholder input and plans to issue a new version in late summer 2018. GAO incorporated technical comments provided by the EAC and election officials from the selected local jurisdictions and their respective states as appropriate.