Florida map
Best fit of Mitofsky data
Frequently Asked Questions
What Went Wrong in Ohio
Trends in Florida voting. A brief report.
Elections Research
Mission Statement
In Response to Douglas Jones' "Mark Sense Ballot Design: Human Factor Lessons from Florida Election 2000"
The letter Tom Paine eventually printed
From the New Deal to the Dirty Deal

Recommended websites

Carter: No doubt Gore won


New Scientific Paper: Exit Poll Discrepancies


John Conyers' Blog


In Response to Douglas Jones'
"Mark Sense Ballot Design:
Human Factor Lessons from Florida 2000"

By Oliver T. Dawshed
Revision 2.0† 3/15/05


To have one of one's papers serve as the sole listed citation for a colleagueís publication is usually one of the more gratifying professional experiences. So it was with high hopes that I began reading Douglas W. Jonesís Mark Sense Ballot Design: Human-Factor Lessons from Florida 2000. [1] Imagine my disappointment in discovering that Jones' decision to cite arose from a fundamental error in statistical understanding; Bushís Fifth Ace: a Crooked Panhandle [2] actually has nothing at all to do with his work. Unfortunately, Jonesís error is not unique to amateur electoral statisticians and needs to be formally addressed.

Let this not be thought to detract from Jones' contributions. As one of the principal authors in the area of voting machine technology, his knowledge of voting machine types is encyclopedic. His articles are always interesting to read. In Mark Sense Ballot Design: Human-Factor Lessons from Florida 2000, he ably reviewed how voting machine type and ballot design varied from county to county in Florida 2000. Remarkably, however, neither he nor other authors who have written on the subject [3, 4] seem to be aware of one another's work, nor of work by Paul Lukasiak[5] which probably preceded and perhaps unconsciously inspired the work of those who followed.

As it happens, Elizabeth Jordan and I looked at the issue of voting machine type and ballot design in early 2001 on behalf of Lukasiak and happen to be familiar with the issue. Thereís no question that confusing ballots lead to more spoiled (or, if one prefers, "miscast"a) votes. Anyone who suffered through the mediaís coverage of the Palm Beach Butterfly Ballot (i.e., any adult who was not comatose during November, 2000) was aware of this point. Furthermore, no one who read and digested the June 2001 report of the US Civil Rights Commission, Voting Irregularities in Florida During the 2000 Presidential Election [6] could have been unaware of the issue of voting machine type. The latter is such a central work in the area that Jones' failure to cite it in this regard (he cites it only over a trivial matter of terminology) is very surprising.†

Indeed, there are many other factors that might affect vote spoilage. Leon County has shown that perhaps the key factor in preventing spoilage is intelligent political leadership, since that directly affects secondary factors such as programs of voter education and maintenance of voting machines. It is striking that the level of educational attainment is not a factor. Key work by Klinkner,[7] cited by Lichtman, [6] provides strong evidence against the hypothesis that "dumb Democrats" spoiled their ballots, while "smart Republicans" didnít. Addendum 2 of Klinkner also disproves that voter experience was a significant factor in ballot spoilage. Indeed, if the votes of educated people were to count more than the votes of the uneducated, or those of older voters were to count more than younger voters, it might call into question the very legitimacy of American government. ††

The reason that Jordan and I chose not to sign on to Lukasiak's work is simply divergence of interests. Lukasiak's work remains worthy of being read (and, one might add, cited). But while ballot machine type and ballot design can affect the average rate of spoilage in a county, if the machine type and ballot design for the top candidates is identical within a county,b these factors cannot affect precinct by precinct variation in spoilage.† Even if the assumptions about homogeneity of machine type and ballot design within a county are violated, if voting machine distribution and ballot confusion is truly random, the results of an election should not be changed. So, machine type and ballot design are factors of primary interest to those interested in voting technology, but they are of secondary interest to those interested in violations of the Voting Rights Act or other electoral misconduct.†

The error that Jones [1] and at least one other writer in the area [3] have stumbled into is known in the technical jargon as a confusion of the variance between groups vs. the variance within groups. That is a fancy way of saying that ballot spoilage differs not only between counties (with their different voting machine types, machine maintenance schedules, ballot types, political leadership, and so on) but also between precincts within counties. Thereís probably no issue of discrimination if 5% of the ballots are spoiled in every precinct of a county. Itís quite a different thing if 20% are spoiled in African American precincts and 1% in white precincts.

As a consequence of conversations with Lukasiak, in 2001 we examined the incorporation of voting machine type and ballot type into our larger analysis of unusual crossover and spoilage but found the effects to be negligible. This wasnít entirely surprising, since there are so many more precincts than there are counties. The statistical power of the analysis of our two-factor model simply swamps the power of the analysis of technology factors.

But we also found something more probative that variance within counties is what most matters: the level of significance rose as more counties were pooled into a larger meta-analysis of crossover and spoilage. In other words, whatever the importance of ballot design and machine type is in influencing spoilage - and one must assume that these are important factors - it does not provide as good an explanation of the observations as our model.†

Precincts tend to be of roughly similar size, while the counties of Florida range from very large to very small. So, the county-by-county eyeball analysis Jones performs using the diagrams of Mark Sense Ballot Design weights small counties too heavily in estimating the mean. To sort out this issue in detail is time intensive, and I am not volunteering to do it. However, weighting by population is probably one issue to look at.†

Mark Sense Ballot Design has other deficiencies. Without producing a catalog of these,d suffice it to say that Jonesí methods are classic post hoc analysis. To remain impartial, an analyst should first identify which counties are statistically aberrant and then let the results speak for themselves in guiding interpretation. By going first to what Jones calls the "human-factor variables" and "specific polling place procedures" (and coming up with almost as many variables as counties on which to test them), the investigator places the result too close to the test. Placing the results too close to model design tempts the investigator's unconscious prejudices to rearrange facts to fit conclusions.e Post hoc methods are so prone to manipulation that the use of them invites suspicion. At the very least, conclusions arrived at by post hoc methods should be challenged by, for example, applying them to other political races or other states. This Jones did not do.

One other point. Jones (apparently in the mistaken belief that Elizabeth Jordan and I are one person, indivisible) stated that "one author" claimed a "massive fraud" had occurred in 2000. In our manuscript, we (two authors, not one) estimated that by one measure about 7100 votes were probably wrongfully converted.c We said that statistical analysis was not proof of fraud, but that the evidence merited further investigation by law enforcement. Characterizing one ballot in a thousand as "massive" and characterizing a call for investigation as an allegation of fraud was inappropriate.† Indeed, contrary to visions of "massive fraud," it doesnít take many ballots to turn tight elections. One in a hundred, or in the case of Florida 2000, one on ten thousand is often enough to change the outcome.†

In sum, Jones' paper is valuable to those who want to understand the nitty-gritty of voting, especially the details of ballot design in Florida 2000. He makes useful recommendations on ballot design. However, he has inexplicably disregarded the very powerful evidence compiled by the US Commission on Civil Rights showing that there was both discriminatory intent and discriminatory effect in Florida 2000. This might have led him to consider other hypotheses that are clearly lacking from his work. There are serious flaws in his methodology, some of which are detailed in footnote d . These deficiencies vitiate the conclusions. The central flaw, however, is the failure to respectfully consider the contributions of others.†

Postscript.  Prior to publication of this piece, I offered it to Professor Jones for comment, saying, "Should you find any statements that are wrong or demonstrably unfair, I will be happy to amend them before publication."† A lengthy correspondence ensued.[12] Jones's response was to deny that Mark Sense Ballot Design had been published, saying, "I consider what I have on the web at the URL you have cited to be an unfinished work, and it has sat, without any attention from me, for the past year and a half." He said, "I do not consider it credible myself . . ." and "I do not give out this URL without strong warnings about the page being junk, an abandoned draft of incomplete work."

Unfortunately, since Jones is one of the few experts on voting technology in the United States, even what he admits is "junk" could taint the debate. I pointed out to Jones that "The fact that you posted a comment about my work on a publicly-funded, publicly-accessible academic website and left it there for a year and a half makes it difficult to accept this explanation. Anyone who was looking for evaluations of my work would have found it and concluded, wrongly, that you had some basis to attack my work. I think you'll agree that it's only reasonable that I should be able to respond just as publicly to comments that may detract from my reputation."††

I suggested to Jones that he withdraw the piece. Sadly, Jonesís response was to deny that leaving a piece on the Internet for 18 months amounted to publication and to claim - falsely - that it was unreachable by search engine.† While at this writing (3/16/05), it does seem that Google does not find Mark Sense Ballot Design, the piece was originally found by Google and is still found on Yahoo!† Jonesís only substantive response was to say that he would disappear the article: "If every a search engine finds the page, I will change its URL immediately (sic)." I pointed out to him that this looked like covering up rather than owning up to an error in judgment. Apparently this persuaded him to leave Mark Sense Ballot Design online but label the document as an "abandoned draft," whatever that means. I'd rather he would un-abandon it, and do the work right.†

As I told Jones, "I want to encourage real discussion and not just score points. I agree that ballot design is important and needs to be analyzed."

But honor and honesty in academia is also important. If we had more, I suspect we'd have a real discussion of why Florida's electoral statistics are so unusual.




Jones, D. W. "Mark Sense Ballot Design: Human-Factor Lessons from Florida 2000." There was also extensive personal correspondence between Dawshed, Jones, and, in cases, Professor Rebecca Mercuri and Paul Lukasiak, dated October 24, 2001; October 25, 2001; October 29, 2001; October 30, 2001, and October 31, 2001 which discussed many of these issues.


Dawshed, O. and E. Jordan. "Bush's Fifth Ace: A Crooked Panhandle."
Online Journal
, July 2001.


Mebane, W. R., Jr. "The Wrong Man is President! Overvotes in the 2000 Election in Florida." April 6, 2004.

Mebane, W. R., Jr. personal communication, December 23, 2004.


Agresti, A. and B. Presnell. "Misvotes, Undervotes, and Overvotes: the 2000 Presidential Election in Florida."


Lukasiak, P. "The Evidence for Ballot Tampering in Escambia County, Florida."
downloaded from Failure is Impossible on July 11, 2001.


U.S. Commission on Civil Rights. Voting Irregularities in Florida During the 2000 Presidential Election (ISBN 0-16-050927-0), June 2001.

U.S. Commission on Civil Rights. Voting Irregularities in Florida During the 2000 Presidential Election, Appendix (ISBN 0-16-050928-9), June 2001.


Klinkner, P. A. "Whose Votes Donít Count?:  An Analysis of Spoiled Ballots in the 2000 Florida Election."


Cupples, D."Citizen Involvement Could Have Prevented Disenfranchisement." Gainesville Sun, June 30, 2001.


Friedman, B. "Whistleblower Affidavit: Programmer Built Vote Rigging Prototype at Republican Congressman's Request!" December 6, 2004.


Dawshed, O. "A Model for Interpreting Voting Patterns with Application to Florida. Rev. 1.0." Draft: November 21, 2004. .


"tompaine2004." Daily Kos, November 24, 2004.


Dawshed, Oliver and Douglas Jones. Correspondence,
March 13, 2005 and March 14, 2005


a One of the valuable points Jones makes is the difference between "spoiled" and "miscast" ballots.† "Spoiled" is the preferred term when the entire ballot is discarded, while "miscast" suggests that a vote in a single race could not be counted, leading to a "residual vote." Jones notes that "residual vote" is another term that has been used to refer to the difference between the vote and the turnout. The value in making the distinction between spoilage and miscast votes or the residual vote is that "spoilage" serves as a kind of inventory control, serving to measure the gap between the number of ballots distributed and turnout. However, as Jones concedes, the substitution of "spoiled" for "miscast" is widespread and that, indeed, the number of ballots discarded is generally unknown. Failing to see the point in reserving a special term for a variable that, in practice, cannot be measured, we do not make the distinction.†

b Keeping track of these assumptions is absolutely crucial in making interpretations. In Ohio 2004, apparently, voting machine type may not have been the same within a county, and ballot positions were rotated in some locations. Even when the type of voting machine and the ballot design is identical, as in Florida 2000, the quality of maintenance of punch card machines or provision of the right sort of marking implement can vary at the precinct level.

c When do electoral shenanigans rise to the level of fraud? Certainly, it is a violation of the public trust and probably of the law to send the worst-maintained punch card machines for the use of "those people," whether "those people" are African Americans or country clubbers. That sort of semi-conscious bias may be less obviously criminal than actual alteration of ballots, but when it is done to people who have suffered historical discrimination, it is a violation of the Voting Rights Act. In the case of Florida 2000, we know the following beyond the shadow of a doubt:

  • There was an environment of lawlessness in Florida 2000, largely recorded by the US Commission on Civil Rights.[6] Without reviewing the whole litany, the most egregious example was that the Secretary of State and Bush campaign co-chair, in open defiance of a court order, ordered the removal of tens of thousands of qualified voters from the rolls.[8] Certain county officials evidenced a disdain for lawful procedure in their dealings with us by failing to abide by the requirements of the Sunshine Law. Further, a specific allegation has been made that a senior Republican official commissioned the creation of software to miscount ballots.[9] Whether this scheme was carried beyond prototyping is not known, nor indeed has the allegation itself been proven. The mere existence of the allegation, however, should trouble us.

  • There was, at the very least, reckless disregard for racial equality in voting. Ballot spoilage in African American precincts was ten times higher than in white precincts, [6] which is a prima facie violation of the Voting Rights Act.

  • There was inexplicable crossover. Subsequent investigation has shown cannot be explained by "Dixiecrat voting" or by a simple model of crossover.[10]† This unusual crossover, although statistically independent from spoilage, appears to occur in the same counties as inexplicable spoilage. One explanation for unusual crossover is electoral tampering.†

To give a simple analogy, if one found the contents of a cabinet in disorder, it would make one wonder what had happened. If one found a window smashed, it would make one suspicious. If one found both simultaneously, one would logically connect the two and suspect that a crime had been committed.†

Of course, if one found a baseball by the smashed window, one might alter one's conclusions. In the case of Florida, we have a smashed window, a looted cabinet, and known lawbreakers - officials of the state of Florida - standing just outside, whistling as they count currency. We certainly donít know that they committed a crime. Labored efforts to come up with alternative explanations for what happened in 2000, however, look like attempts to pretend that a rock is a baseball.

Simply put: how stupid does one have to be to not call a cop?†††

(A footnote to the footnote: An analogy similar to the smashed window and looted file cabinet was posted on Daily Kos [11] previous to my work. I did, however, arrive at it independently).† †

dWell, OK, a brief catalog of the paperís defects: The paper was difficult to follow because there was no consistent usage of significant figures, because fixed notation with many padded zeroes was used rather than scientific or engineering notation, and because it assumed that the reader knows which counties scan/count votes in one central location, vs. precinct-scanning, of ballots.† It also duplicated the category of "overvote with discernable intent." The second usage appeared to refer to deliberate overvotes. It also raised many hypotheses which it failed to address, adding to reader confusion.

However, overcoming these hurdles, the reader will find that the paper also:

  • made assertions of significance without providing any statistical measure of significance to support the claim,

  • though acknowledging the discrepancy between the reported residual vote and the recount data, failed to provide enough information to understand how this might affect the validity of the analysis,

  • as incomplete in calculations, presenting only mismarkings, abstentions, and "other" but ignoring the apparently far larger categories of overvotes that were either deliberate or showed discernable intent. Deliberate overvotes, overvotes with discernible intent, and abstentions could serve as useful internal controls of what sort of variation is normal, †

  • attempted to draw conclusions from very small numbers (for example, comparing roughly 52 underlined votes for counties using broken arrow markers vs. 17 underlined votes for counties using oval markers),

  • inexplicably failed to complete the analysis of one-column vs. two-column counties,

  • failed to look at distributional issues (for example, a disproportionate amount of the Mark Sense residual vote came from a handful of precincts in one county),

  • failed to test the hypothesis on other races or other states,

  • failed to state the basis for the use of (and probably misused) the term "outlier," and

  • failed to compare the importance of a known factor in the residual vote, namely racially-targeted vote spoilage, with the ballot design issues of this paper.

e See A Model for Interpreting Voting Patterns with Application to Florida Rev. 1.0 [10] for an explanation of how we insulated parameter selection from actual testing and otherwise attempted to keep on the straight and narrow. The main point is that we developed the model by working with county-level data, and then applied it to precinct-level data. This separated the model-building process from the obtaining of results.†††


Copyright, © 2005 Oliver T. Dawshed. All Rights Reserved.
Permissions for reproduction or extensive quotation may be obtained from