As seen in table 1, about 18,000 undervotes were reported in Sarasota  County in the race for Florida’s 13th Congressional District. After the  election results were contested in the House of Representatives, the task  force met and unanimously voted to seek GAO’s assistance in determining  whether the voting systems contributed to the large undervote in Sarasota  County. We found that some of the prior tests and reviews  conducted by the State of Florida and Sarasota County provided assurance  that certain components of the voting system in Sarasota County  functioned correctly, but they were not enough to provide reasonable  assurance that the iVotronic DREs did not contribute to the undervote. Overview of the Voting Systems Used in Sarasota County in the 2006 General Elections

In the 2006 general election, Sarasota County used voting systems  manufactured by ES&S. Consequently, if we  are reasonably confident that the same firmware was running in all 1,499  machines, then we are more confident that the results of other tests,  conducted both by GAO and by others, on a small number of machines can  be used to obtain increased assurance that the iVotronic DREs did not  cause the undervote. Methodology for Firmware Verification Testing

Our methodology to obtain reasonable assurance that the firmware used  on Sarasota County’s iVotronic DREs during the 2006 general election was  the same as that certified by the State of Florida was broken down into  two basic steps: (1) selecting a representative sample of machines, and (2)  verifying that the firmware extracted from the voting machines was the  same as the escrowed firmware that had been certified by the Florida  Division of Elections. We witnessed the rebuild of the iVotronic DRE’s firmware from the source  code that was held in escrow by the Florida Division of Elections and that  was previously reviewed by Florida State University and by us. Casting a ballot from the review screen selection. Process Used in Executing the Ballot Test

A two-person test team conducted the ballot testing. We conducted calibration testing on two different machines that were  used for ballot testing. Conclusions

Our tests showed that (1) the firmware installed in a statistically selected  sample of machines used by Sarasota County during the 2006 general  election matched the firmware certified by the Florida Division of  Elections, and we confirmed that when the manufacturer rebuilt the  iVotronic 8.0.1.2 firmware from the escrowed source code, the resulting  firmware matched the certified version of firmware held in escrow, (2) the  machines properly displayed, recorded, and counted the selections for all  test ballots cast during the ballot testing involving the 112 common ways a  voter may interact with the system to cast a ballot for the Florida-13 race,  and (3) the machines accurately recorded the test ballots displayed on  deliberately miscalibrated machines. The results of these tests did not  identify any problems that would indicate that the iVotronic DREs were  responsible for the undervote in the Florida-13 race in the 2006 general  election. Although the test results cannot be used to  provide absolute assurance, we believe that these test results, combined  with the other reviews that have been conducted by Florida, GAO, and  others, have significantly reduced the possibility that the iVotronic DREs  were the cause of the undervote. At this point, we believe that adequate  testing has been performed on the voting machine software to reach this  conclusion and do not recommend further testing in this area. Given the  complex interaction of people, processes, and technology that must work  effectively together to achieve a successful election, we acknowledge the  possibility that the large undervote in Florida’s 13th Congressional District  race could have been caused by factors such as voters who intentionally  undervoted, or voters who did not properly cast their ballots on the  iVotronic DRE, potentially because of issues relating to interaction  between voters and the ballot. Appendix I: Methodology for Selecting IVotronic DREs for GAO Testing

Each of the three tests—firmware verification, ballot, and calibration— was conducted on a sample of the 1,499 iVotronic DREs that recorded  votes during the 2006 general election in Sarasota County, Florida.
Why GAO Did This Study

In November 2006, about 18,000 undervotes were reported in Sarasota County in the race for Florida's 13th Congressional District (Florida-13). After the election results were contested in the House of Representatives, the task force unanimously voted to seek GAO's assistance in determining whether the voting systems contributed to the large undervote in Sarasota County. In October 2007, GAO presented its findings on the review of the voting systems and concluded that while prior tests and reviews provided some assurance that the voting systems performed correctly, they were not enough to provide reasonable assurance that the voting systems in Sarasota County did not contribute to the undervote. GAO proposed that a firmware verification test, a ballot test, and a calibration test be conducted. The task force requested that GAO proceed with the proposed additional tests. GAO also verified whether source code escrowed by Florida could be rebuilt into the firmware used in Sarasota County. To conduct its work, GAO conducted tests on a sample of voting systems used in Sarasota County during the 2006 general election. GAO witnessed the rebuild of the firmware from the escrowed source code at the manufacturer's development facility. GAO reviewed test documentation from Florida, Sarasota County, and the voting system manufacturer and met with election officials to prepare the test protocols and detailed test procedures.

What GAO Found

GAO conducted three tests on the iVotronic Direct Recording Electronic (DRE) voting systems in Sarasota County and these tests did not identify any problems. Based on its testing, GAO obtained increased assurance that the iVotronic DREs used in Sarasota County during the 2006 general election did not contribute to the large undervote in the Florida-13 contest. Although the test results cannot be used to provide absolute assurance, GAO believes that these test results, combined with the other reviews that have been conducted by the State of Florida, GAO, and others, have significantly reduced the possibility that the iVotronic DREs were the cause of the undervote. GAO's firmware verification test showed that the firmware installed in a statistically selected sample of 115 machines used by Sarasota County during the 2006 general election matched the firmware certified by the Florida Division of Elections. The statistical approach used in selecting these machines lets GAO estimate with a 99 percent confidence level that no more than 60 of the 1,499 iVotronic DREs that recorded votes in the 2006 general election were using different firmware. Consequently, GAO is able to place more confidence in the results of other tests conducted on a small number of machines by GAO and by others, which indicated that the iVotronic DREs did not cause the undervote. GAO also confirmed that when the manufacturer rebuilt the iVotronic DRE firmware from the source code that was held in escrow by the Florida Division of Elections and previously reviewed by GAO and others, the resulting firmware matched the version certified by the Florida Division of Elections. For the ballot test, GAO cast predefined test ballots on 10 iVotronic DREs and confirmed that each ballot was displayed and recorded accurately. GAO conducted the calibration test by miscalibrating two iVotronic DREs and casting ballots on them to validate that the machines recorded the information that was displayed on the touch screen. Based on the results of the ballot and calibration tests, GAO found that (1) the machines properly displayed, recorded, and counted the selections for all test ballots cast during ballot testing involving 112 common ways a voter may have interacted with the system, and (2) the deliberately miscalibrated machines, though difficult to use, accurately recorded the ballot selections as displayed on screen. At this point, GAO believes that adequate testing has been performed on the voting machine software and does not recommend further testing in this area. Given the complex interaction of people, processes, and technology that must work effectively together to achieve a successful election, GAO acknowledges the possibility that the large undervote in Florida's 13th Congressional District race could have been caused by factors such as voters who intentionally undervoted, or voters who did not properly cast their ballots on the iVotronic DRE, potentially because of issues relating to interaction between voters and the ballot.