Slashdot Log In
Paper Trails Don't Ensure Accurate E-Voting Totals
Posted by
CowboyNeal
on Fri Sep 14, 2007 04:25 AM
from the keeping-them-honest dept.
from the keeping-them-honest dept.
An anonymous reader writes "In an new report from the Information Technology and Innovation Foundation they say that paper trails increase costs and can actually reduce the chances a voters' choices are accurately counted. Congress is considering a 'Voter Confidence and Increased Accountability Act of 2007,' which would mandate 'voter-verified' paper audit trails."
Related Stories
Submission: Paper Trails Don't Ensure Accurate E-Voting Totals by Anonymous Coward
This discussion has been archived.
No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
Full
Abbreviated
Hidden
Loading... please wait.
What do you expect ? (Score:5, Informative)
"Rhett Dawson is President and CEO of the Information Technology Industry Council (ITI). Immediately prior to being selected as President of ITI, Dawson was Senior Vice President of Law and Public Policy for the Potomac Electric Power Company. In the Reagan administration, Dawson was Assistant to the President for Operations. At the White House, he managed the staff and decision-making process for President Reagan and was responsible for three White House support units: the White House Office, the Office of Administration, and the White House Military Office. He also was Executive Director of two presidential commissions, the President's Special Review Board (the Tower Board) that investigated the Iran-Contra matter, and the President's Blue Ribbon Commission on Defense Management (the Packard Commission). During the 1980s Dawson was a partner in two Washington law firms. Earlier in his career, he was Staff Director and Chief Counsel for the Senate Committee on Armed Services, Minority Counsel for the Senate Committee to Study Governmental Operations with Respect to Intelligence Activities (the Church Committee), and Minority Counsel for the Joint Committee on Defense Production. He is a member of the statutory Commission on National Guard and Reserve, and he is Vice Chair of the State Department's advisory committee on International Communication and Information Policy. Dawson received his undergraduate degree from Illinois Wesleyan University, where he was recognized in 2001 as the Alumni of the Year. He was awarded his law degree from Washington University."
CC.
Re:What do you expect ? (Score:4, Informative)
The basic idea (and I'm oversimplifying, I apologize) works like this:
1) You go to vote. You are shown a voter ID number on the screen. You are welcome to write it down if you wish.
2) You select your candidate of choice. If you wish, you are given a paper receipt providing cryptographic proof that the voter ID you were shown in step 1 voted for the candidate you chose.
3) If anything goes wrong in steps 1 or 2, complain loudly and immediately. This is equivalent to you not being allowed to enter the voting area or a machine displaying a candidate other than the one you pushed.
4) If you wish, you may opt to receive copies of paper receipts of other votes for other candidates too. (So that someone can't demand to see your receipt to prove you voted for a particular candidate, since you can get a receipt of someone else who voted for any candidate.)
5) When the results are publicized, the total number of votes is checked against the total number of voters. Any voter with a paper receipt not on the final tally knows their vote wasn't counted. (Though they can't prove it was their vote, of course, they can prove that *a* valid vote wasn't counted.)
6) The receipts can be scannable with barcode and groups may, if they wish, stay outside of voting areas and ask voters if they may scan their receipts. A church group, for example, could make sure all of its members votes are counted this way, though they could never be truly sure how each member voted.
Parent
Vote counting 101 (Score:5, Insightful)
There is just one simple, practical, logical rule for machine assisted voting that anyone need remeber:
A machine that prints your choice is at worst a waste of money, a machine that counts your choice is at best a waste of money.
Parent
Re:Vote counting 101 (Score:5, Insightful)
The basic protocols of manual counting have been tried and tested for well over 100yrs. They are not perfect and are suscepible to "retail fraud" (eg: box stuffing, stand over tactics, ect), ANYTHING that can tie an individual to a particular vote opens the door to stand-over merchants. What is worse is that ANY counting machine is suceptible to "wholesale fraud" (eg: one person + one point of attack = flip an entire election any way you want).
The old fashion system is fast, efficient, auditable, well understood and extensively tested - most importantly the human counters MISTRUST each other by design. Before you reinvent the wheel try googling for "election observers" or "secret ballot".
Parent
The problem is like this (Score:5, Insightful)
Also remember that it's not enough to hold on for it for 5 minutes. You must hold on to it all the way to the recounts, at least. If you just prove before leaving that the machine still has your vote, then there's not thing to say someone can't flip the votes in the database later.
The problem is this: any proof of how you voted, can be used for electoral fraud by itself. E.g.:
- Someone else can demand that proof that you voted for their candidate, or else. Let's say Don Corleone, the respectable head of the local mafia group, is running for mayor. If you have your ticket that you can check at a terminal, then so can Don Corleone's goons for you. It makes an electoral racket as simple as a protection racket. You know, you only have one kneecap in each leg, it would be a shame if that were to change. Show your ticket proving that you voted for Don Corleone, and you have our "protection" so it doesn't.
- Outright buying votes. Let's say I've won the lottery jackpot and want to be governor. Or just mayor. It's as this: everyone who shows me a ticket proving that they've voted for me, gets 100$, no questions asked. (And I'll store the crypto token on a database of my own, of course, so several people can't come with the same ticket.) In fact, let's turn up peer pressure a notch: if you can also prove that your spouse (if applicable) and at least one parent or child of voting age also voted for me, you get an extra 100$. You know, just to have old retired moms call their sons and do the "you won't even do that for me?" sobbing act.
- Pure social pressure. E.g., if you're a student still living with your parents, whoppee, they can control who you voted for. You know, under the old principle of, "as long as you're in _my_ house, you'll do what _I_ say, young man. Now let's go to a terminal and you'll prove to me that you voted as I told you to." E.g., if you want to keep working at my office, better "voluntarily" prove that you voted for my favourite candidate.
Etc.
Yeah, I'm sure _you_ would bravely stand your ground, stick to your ideals, and never betray the sanctity of the free democratic voting. Maybe. But considering that elections have been won by a 0.1% lead before, the funny thing is: you don't need to get _everyone_ to cooperate.
Some of those aren't even easy to legislate against. E.g., how would you legislate against parents demanding to see their 21 year old son's ticket?
So, no. Please don't do that. The important thing about votes isn't just that they're counted, but also that they're secret and hard to influence. The moment all that remains is that they're counted, but someone can easily influence the voters and/or check what they voted... well, you might as well not bother pretending it's a democracy any more.
Parent
Re:What do you expect ? (Score:5, Insightful)
I also do not see any reason to abandon paper-based voting, which still is not 100% secure, but much more difficult to 'hack' due to transparency by distribution of control.
CC.
Parent
Re:What do you expect ? (Score:5, Insightful)
I might be able to prove it's in the final tally. You, too, might be able to prove it's in the final tally. 99% of the voting population, however, have not studied cryptography and would have to rely on an expert to check their vote (and, of course, such a system would have to be designed to make it impossible for the voter to prove to someone else which way they voted).
An election is only democratic if the electorate is able to trust it. If I have a magic wand I wave and then pronounce the results, it doesn't matter if I am 100% accurate, because no one will trust it. And they shouldn't trust it, because there is nothing stopping me from simply making up the result.
In a paper election, anyone who doesn't trust the system can observe the entire procedure. They can watch the ballot box, from the point they enter their vote, watch the counting, and watch the reporting. Verification is not limited to the technorati, it is available to every single voter. This is why paper voting remains superior.
Parent
Re: (Score:3, Insightful)
It's about the SCALE of the fraud and TRUST in sys (Score:3, Insightful)
Any hypothetical electronic system, no matter how secure, is vulnerable to basically _universal_, unauditable fraud by a tiny number of conspirators in the right place - as low as 1. Any kind of cryptographic system can be defeated by the guy who actually controls where the actually-compiled source code - and the COMPILER source code - came from. Even in an OSS system, it's awfully ha
Re:It's about the SCALE of the fraud and TRUST in (Score:4, Insightful)
Suddenly your printed receipt is absolutely worthless. Sure, you can rest easy the system correctly registered your vote, but it's the master counting system, and the values it receives, that matters.
Paper ballots require a massive concerted effort with hudreds, or even thousands of conspirators. With Electronic voting, since the code is closed (and even if it was open, we can't ensure that's the code they used in the final machine), it takes one manager with an agenda and a handful of hand-chosen coders to implement it.
There may be a way around this, but I sure as hell don't know what it is.
Parent
Re: (Score:3, Insightful)
3) How would that printout prove anything on how your vote is recorded, if you really wanted to mess up the machine you would display the correct results and record the wrong. If I wanted to add votes the old ways are still the best ways; get the dead to vote.
4) The giving of e
Re: (Score:3, Insightful)
Re:What do you expect ? (Score:5, Insightful)
Parent
Re: (Score:3, Informative)
Board:
Cal Dooley - Masters degree in Management
Jennifer Dunn - BA
Dr. Robert D. Atkinson - Ph.D. in City and Regional Planning
Rhett B. Dawson - undergrad law degree
David Hart - Professor of Public policy
Staff:
Dr. Robert D. Atkinson - Ph.D. in City and Regional Planning
Julie A. Hedlund - M.A. degree
Daniel D. Castro - M.S. in Information Security Technology and Managemen
Who are these people? (Score:4, Insightful)
Worthless article (Score:5, Insightful)
As to why paper trails are bad, they don't say, just that they will publish a paper really soon now. News at 11.
Re:Worthless article (Score:5, Interesting)
I've been interested in the Caltech/MIT Voting Technology Project [caltech.edu] for awhile now, and they have quite a few papers on voting issues, including discrepancies, statistics, technologies, reliability. One paper in particular [caltech.edu] (PDF warning) speaks of a study done on different methods of verifying user voting. For the mock election, they randomly inserted incorrect vote records, and thus tested different methods of auditing, to see how often voters noticed the discrepancy. With the paper trail, only 8% acknowledged any problem (i.e., realized that the machine voted wrong). For an auditing system, that's not at all counterproductive as the topical article would have you believe, but it's still pathetically ineffective.
A different kind of auditing system is advocated in the paper: one using audio via headphones to play back the recorded votes to the voter. 77% of voters caught the errors. Of course with every added system, there is inherent risk -- listening devices, accessibility, etc. -- and, of course, audio auditing is relatively untested, but this seems promising. This, however, assumes that the problem is with voters or the machines making an honest mistake and not with the machines maliciously changing votes. Thus, the best course of action would be to have both paper and audio: one to help the voter, the other to verify recounts and prove unreliability.
Of course, no auditing system, no matter the sophistication or rate of helpfulness, will matter if the machines themselves are designed to be corrupted and the vote counts manipulated. Ultimately, it will be far more beneficial to the American people if, rather than trying to force accountability and regulations on corrupted producing companies bought and paid for by corrupt political crooks, the machines are written and produced, or at least heavily tested, by independent committees.... May I suggest academic committees, such as this Caltech/MIT VTP, or similar groups? Their ultimate goal is to certify reliability, and since academics is far less motivated by money, they're far less likely to be corrupted. Or so goes my theory, anyway.
Parent
Re:Worthless article (Score:4, Insightful)
The biggest concern is not with people making a mistake in recording their votes (though this is a concern, but one which is easily correctable with a good user interface), but with machines which may be tampered with to alter the outcome of the vote.
Even the marbles-in-a-jug thing is easily falsifiable since anyone with two marbles gets two votes, let alone with a hundred marbles.
The idea is that you have to make the "authority" on which vote is which an immutable record. That is to say something which can't be changed after the vote has been cast. There's nothing in the computer world where this is the case. Not even cryptography would suffice since the voting machine does all the cryptography, and it could easily show you one cryptographically signed vote and record a different cryptographically signed vote. If it has everything it needs to do the original signing, it has everything it needs to forge the signing of different data.
This immutable authority is most easily done as a paper trail. The paper can be shown to users through a piece of glass, and once confirmed, be fed into a locked audit box. Unfortunately even this is still vulnerable to a malicious machine continuing on to forge votes between users and feeding those votes into the box. At least the machine couldn't delete existing votes, it would only be able to add to them, and that would show up as more votes registered than votes cast.
So I think the current approach is that each voter would be issued an audit card as they enter the voting booth. The machine doesn't have these, and the user feeds the card into the machine for their vote to be recorded.
Parent
Re: (Score:3, Informative)
Re:Worthless article (Score:4, Informative)
* barring the presence of any bad actors in the simulation.
It's always the caveats that get you. See these guys are interested in "resilience against corruption". They're only interested in cheaper and more accurate because that's the only thing that supports their position.
I have to agree with you, the whole article screams "Industry Shills".
Parent
Anti-Privacy and Anti-Citizen (Score:5, Informative)
Who funds these people?
Yet again ... (Score:5, Insightful)
The voter marks the ballot paper with a pencil. The ballots are counted by hand by human beings.
Completely transparent, complete audit trail, safeguards against all the failure modes discovered over the decades, results within hours, recounts within hours if needed.
Oh, and I expect it's cheaper than all this inappropriate mucking around with computers too. Computers aren't the answer to everything. This is one application in which they have no place.
Re: (Score:3, Insightful)
You enter your options on a screen, and it prints your ballot out for you, and a barcode with a checksum. The ballot is read optically (like it often is now), and the checksum is verified. No match, it's counted by hand.
A system like this ensures no hanging chads, etc. It's 100% verifiable by the voter. The paper trail is just as good as the current system, and can be fully counted by hand. It's impossible for the voter to prove which way he voted; if he takes th
Wait one minute here ok... (Score:5, Insightful)
What I think is problematic in the US is that there is this automatic tendency to automate tasks and thus making it difficult for the people to carry out the task. Case in point the ballots with hanging chads. Why on earth is there such a ballot? Oh yeah so that you can save a few bucks on counting the votes. But who cares that the voter has to take a Phd on casting votes.
To put this in context. India in 2004 put in electronic voting machines for 348 million people http://www.kablenet.com/kd.nsf/Frontpage/A109B59D2C4BCBA380256E9400373E62?OpenDocument [kablenet.com]
I am sure its not perfect, BUT you have to think twice about this. In a country that is mostly poverty stricken and where people can't really read they have a working democratic system and 348 million people can vote electronically. And what was the population of the US? 300 million...
No, the problem here is quite simple the American voting infrastructure. It's not the fault of the people, nor the political system, but the folks who run the voting infrastructure! They need a good "flogging."
Parent
Paper trails have the proven track record (Score:5, Insightful)
1. Not end to end. I can't do cryptography decryption in my head, and the vote verifier at the other end, he can't also do decryptions in his head. So any solution that involved cryptography isn't end to end.
2. One doesn't preclude the other. You can encrypt the electronic vote AND STILL HAVE THE PAPER AUDIT TRAIL to check the machine's cryptographic vote matches the voters intentions.
3. Papertrails, or ballots as we use to call them, have a proven track record of uncovering fraud in voting. To date the fraud in electronic voting is suspect but unproven. It is unlikely that fraud is eliminated in electronic voting, because fraud is *easier* not *harder* to do when votes can be changed so easily and untraceably on mass in a computer. So the lack of uncovering fraud is likely to be a weakness in the auditability of these machines. i.e. we suspect voter fraud because of systematic irregularities in key districts, but nothing can be proved because the lack of paper trail to verify against.
Why does he want unauditable machines? I see from his history that he's a professional technology lobbyists, but I'm curious why the FUD to keep the voting machines unauditable?
Crikey (Score:3, Insightful)
Im shocked. Really.
Up next - 'Republican Party publish report saying the the Republican Party is better than the Democrats'?
The Perfect is the enemy of the Good (Score:5, Insightful)
Of course, there is no perfect solution. We only have adequate solutions.
Condorcet voting is mathematically better than simple tallies or "instant runoff" voting. But does anyone except mathematicians comprehend it? Would switching to it increase our confidence in voting or would people be suspicious and trust voting even less?
Paper is adequate. And what's better, it is something that mere mortals understand. And the attack vectors for paper are reasonbly well understood after more than a century of use of the "Australian" ballot style that we all use today.
The proposal by this group opens the door to FUD and infinite delay, and thus infinite retention of flawed DRE voting machines. Diebold would win, democracy would lose.
Trust in voting systems is very important (Score:4, Insightful)
The likeliness that computers are capable of correctly counting 100,000 perfectly submitted votes more accurately than humans in an ideal world isn't exactly a surprise, but this isn't really the point because the world isn't ideal and it's not realistic.
Even if paper trails are slightly less accurate in the counting (something I'd dispute once factoring in less measurable quantities like corruption of officials and potential hacking), one of the most important advantages of paper trails is that they can be easily understood by virtually everyone who votes. A voter verifies their correct vote is recorded on a slip of paper, places it in a ballot box, and then the votes recorded on the papers in the ballot boxes are counted, with the process being vetted by people who have reasons to make sure it's being done properly. The entire process is completely visible and clear from start to finish.
This is quite different to voting through computer interfaces, where the ability for nearly everyone to understand ends at them pressing a touch-screen. The abstract concepts of what goes on inside the system are very difficult for most people to grasp, unless they have a relatively high education. Furthermore, very few people can verify and confirm that it's working correctly.
Trust of as much of the population as possible is of huge importance in elections, and systems with paper trails are the ones that are easiest for the majority of people to trust.
Paper Trails Ranked By Value (Score:5, Funny)
Medium - When I get $20 out of an ATM - (requires ID, receive receipt)
Low - When I buy a hamburger & fries - (no ID, receive receipt)
Worthless - When I vote - (no ID, no receipt, no confidence)
Re: (Score:3, Insightful)
in other news: (Score:4, Funny)
"In an new report from the Information Technology and Innovation Foundation they say that backups can increase costs and can actually reduce the chances that users data have to be recreated."
Where's the proof? (Score:3, Interesting)
Are there any facts associated with this article? It appears that this is just one group's claim, backed up by nothing other than their opinion.
The facts of the matter are:
with all this, a well mandated, accessible, audited electronic voting system is more secure than previous voting methods. There is no excuse for these companies to have created and sold the craptastic voting machines they did. There is no reason for Diebold, an ATM maker, to have only made voting machines that had no paper trail capabilities. If they tried to sell something like that toa bank, their contract would have been dropped in a heartbeat, but election boards across the country didn't blink an eye. It is time that there be a nationwide standard that works within a degree of certainty. Electronic voting machines with paper audit trails are accessible, human readable, and as secure as anything we currently use. You don't have questions of "Did this voter actually mark a circle?" or "Which of these half erased circles did the voter mean?" or "That chad isn't punched all the way through, so I will just do it for them because I know what they meant." It is very hard for an auditor to see "President: Al Gore" printed on a receipt in human readable form and say that the voter chose George Bush.
Vote By Mail People! (Score:3, Insightful)
Anybody registered to vote, gets checked, then mailed their ballot to their address on file. Signature checks, collected at the DMV, are used to validate votes. Votes are mailed in a double secret envelope that allows verification but does not tie votes to voters.
The counting system is optical scan, is done in one location with security in place there. Audits are performed, and most importantly:
-the voter can verify their own vote
-said vote is human and machine readable
-casting of votes is distributed over time and space.
Finally! A rational analysis! (Score:4, Funny)
It's pretty incredible to see the Slashdot crowd speak of paper trails as if they were some sort of magic talisman ready to right the evils of the election system. Slashdotters of all people should understand that the whole point of digital computation is to improve precision of calculations far beyond what could be achieved by manual counts and paper trails, and that proper application of encryption and communications technologies can entirely reverse the weaknesses of either paper or poorly implemented eVoting.
It's so blindingly simple: a paper backup cannot possibly have the precision needed to resolve a close election. It's physically impossible. So what happens when the paper disagrees with the electronics? When the backup is more flawed from the start what good is it?
I could go on, but wow... it's so refreshing to see this story posted to Slashdot. I just wish the rest of the US would stop and think for a second to demand decent electronic voting systems instead of insisting on a broken solution to the wrong problem.
Anonymity out the window? (Score:3, Insightful)
Of course, as with paper or e-voting, what the final tally shows may not reflect the paper/button press you submitted. Electronic or paper, you still need to trust the vote counters OR be able to verify your vote later.
Congress should have power. Congresspeople should not.
Error (Score:5, Funny)
You misspelled Diebold.
Why the fuck do you guys need the machines? (Score:5, Insightful)
Canada gets away just fine with using paper ballots. When you vote, you use a pencil to put a check in a circle next to the name of the candidate you're voting for. The circle is large and the text is large, to allow those with poor eyesight to get a better view of what's on the ballot, thus reducing mistakes.
What's more, the results for Canadian elections are near-instantaneous. They actually have legislation in place to prevent the media from reporting about the final results in the eastern and central provinces while polling stations are still open in the west! Why the fuck can't the US manage that?
Yeah, the American population is 10 times larger than the Canadian population. But that's irrelevant! Use 10 times as many ballot counters, and the system will scale just as well.
It's a mixed situation here in Europe. Some of our nations use the sensible Canadian method. Others are stupid, and follow the American scheme with doodad voting machines and all that jibberjabber. But really, we should all just use the Canadian method. It's the best, and safest, there is.
Parent
Re:Why the fuck do you guys need the machines? (Score:5, Informative)
Parent
Re: (Score:3, Informative)
Elections which use proportional representation usually allow you to vote for more than just a political party.
Re:Why the fuck do you guys need the machines? (Score:5, Insightful)
Political parties are a big part of the problem here in the states. The framers of our constitution did not anticipate the rise of political parties, and George Washington spoke against them [state.gov] in his farewell address. The two major parties here in the US have consolidated power and intentionally impeded the ability additional parties to have any influence in elections or legislation.
One good example of this is the current rule on filibustering [wikipedia.org] which has made the process to a simple administrative chore requiring a 60% vote to break. No longer can one man halt all other activity against the will of even his own party and stand for what he knows is right. Another good example is the change to eligibility requirements and governance of the presidential debates. Where previously the League of Women Voters [wikipedia.org] maintained a fair and open debate process, now the Commission on Presidential Debates [wikipedia.org], an organization controlled by corporate sponsors [debates.org], has created minimum eligibility requirements [debates.org] that include a 15% share of the popular vote "as determined by five selected national public opinion polling organizations." This puts the requirements out of the reach [votefraud.org] of third party candidates who need the national exposure that the debates would give them to garner that much popular support.
The last reference above has a great comment from Alan Keyes [renewamerica.us] that I feel deserves inclusions here.
Parent
Re: (Score:3, Insightful)
Re:Why the fuck do you guys need the machines? (Score:4, Informative)
Parent
Re:Why the fuck do you guys need the machines? (Score:5, Interesting)
But of course the two major parties benefit from voter apathy as it helps ensure one of them will be voted. So the system is designed to maximize voter apathy.
Parent
Re:Why the fuck do you guys need the machines? (Score:5, Insightful)
Parent
Re:Why the foolishness do you guys need the machin (Score:4, Insightful)
You think freedom can be reduced to a popularity contest?
Up until recently, America was about voting issues, not people.
Some people find it incomprehensible that an elected representative of the people would find himself trying to implement the will of the people, rather than simply assuming that the election gave him license to implement his/her own ideas. (You do hear me muttering under my breath here, yes.)
This is entirely the point of having the people vote on so much.
It has something to do with the DIY mindset that also used to be rather typical of people from the USA.
joudanzuki
Parent
Re:Why the foolishness do you guys need the machin (Score:5, Insightful)
Parent
Re:Why the foolishness do you guys need the machin (Score:4, Informative)
While I was taught about the electoral college, voting system, and government while I was in school, it seems that the "real world" is a lot different. (surprise!) If I remember correctly, the electors don't even have to vote for the candidate with the most ballot votes, they can choose to vote another way, becoming faithless electors.. WTF? And I'm supposed to trust these people? I find a lot of problems with the system.. Personally, it seems that if a candidate receives the majority of the votes (citizen votes, not electoral votes), then they should be the elected official. However, as we've seen in the past, that's not necessarily true.
I have no faith at all in our political system. I vote because I want my voice heard. Apparently I'm not in-tune with popular opinion, though, because I have yet to vote for any of these elected officials..
Parent
Re: (Score:3, Insightful)
Re:It is easily solvable (Score:4, Insightful)
WHY do people keep bringing up this corruption-laden idea of a take-home receipt with your vote printed on it, or some other way of verifying your vote outside the polling place?
As has been stated repeatedly here and elsewhere, taking home a receipt opens the floodgates for corruption. "Bring me a vote for Candidate XXX, and I'll pay you $10!" "Bring me a vote for Candidate XXX, or you might suffer an 'accident' in the near future."
You verify the paper ballot in the privacy of the voting booth; once verified, the ballot drops into a secure ballot box and serves as the permanent record of your vote. No name, no identifying information is on the ballot, and NO RECEIPT.
Parent
I think this is the key: (Score:3, Insightful)
Paper trail limitations: they require other equipment or groups of people to count them for audits or recounts.
Other technology: you have to rely on the original equipment to report the results correctly the first time. This is cheaper and more accurate, as your results are always the same.