[00:00.000 --> 00:03.300]  I'm Nick Farr from HacDC and the Hackerspace Village downstairs.
[00:04.500 --> 00:08.820]  I wanted to let you know that the Hackersmart is now officially opened.
[00:09.280 --> 00:14.040]  We are taking requests for any sorts of random electronic devices that you might need.
[00:14.240 --> 00:18.320]  Chargers, LEDs, diodes, breadboards, AVRs, things like that.
[00:18.480 --> 00:28.680]  We have a lot of stuff in stock and if you've been down to the Hackerspace Village and you've seen all of our cool lineman's handsets, we have over 60 years of lineman's handsets.
[00:28.680 --> 00:38.440]  It's going everything from like magnesium alloy rotary dial sets all the way up to ISDN testing equipment in hardened briefcases and things like that.
[00:38.560 --> 00:39.660]  So please come check it out.
[00:40.100 --> 00:41.860]  We've got a lot of stuff in there that you may need.
[00:41.900 --> 00:45.860]  And if there's something that we don't have that you need, we are taking requests and we are going out and sourcing those.
[00:46.140 --> 00:49.760]  And we should be able to get you pretty much whatever you want within the day.
[00:50.320 --> 00:55.180]  But I have a couple of announcements before I introduce mouse and friends.
[00:55.180 --> 00:55.820]  Can I say that?
[00:56.220 --> 00:59.080]  Can I just automatically like lower everybody else that you're dealing with?
[00:59.200 --> 01:01.140]  Because guys, give mouse a round of applause.
[01:01.320 --> 01:03.660]  She was HacDC's first donor.
[01:07.140 --> 01:11.660]  I literally had the HacDC PayPal button up for five minutes.
[01:11.940 --> 01:13.400]  I didn't tell anybody about it.
[01:13.460 --> 01:14.120]  I didn't email about it.
[01:14.180 --> 01:16.060]  And then mouse just somehow finds it.
[01:16.160 --> 01:17.220]  Like I thought it was an error.
[01:17.220 --> 01:18.820]  I was like, what, did Sandy hack me?
[01:21.660 --> 01:25.800]  But again, I do have a couple of announcements and I'm actually ashamed that I have to read this.
[01:26.960 --> 01:30.620]  Please use restraint and please don't mess with the hotel.
[01:32.200 --> 01:34.440]  I shouldn't have to read this one, you guys.
[01:34.740 --> 01:35.140]  Come on.
[01:35.480 --> 01:40.780]  We're all adults or at least there should be enough adults here to keep the kids in check.
[01:41.560 --> 01:42.280]  All right.
[01:42.640 --> 01:49.840]  So if you... and that's another thing like, you know, 99% of you are responsible, good, you know, people.
[01:50.160 --> 01:53.040]  And it's always the 1% that screws it up for everybody else.
[01:53.240 --> 02:03.840]  But you know, that 99% still has a responsibility to if they see the 1% doing something, to either stop it themselves or inform someone so that they can stop it.
[02:03.840 --> 02:09.780]  The hotel's been really, really good to hope, every hope that has gone on.
[02:09.860 --> 02:11.380]  Can we just give a round of applause for the hotel?
[02:11.800 --> 02:12.360]  Come on.
[02:12.420 --> 02:13.840]  And all the hard work that they're doing.
[02:16.320 --> 02:17.600]  So, come on, guys.
[02:17.780 --> 02:19.500]  Don't, don't, please don't mess with them.
[02:19.580 --> 02:22.380]  I don't want to have to read this card tomorrow.
[02:23.680 --> 02:26.280]  And Friday, the fourth track in Zusa.
[02:27.600 --> 02:29.560]  Well, you're all missing relationship hacking.
[02:30.720 --> 02:32.740]  Electronic voting, relationship hacking.
[02:33.220 --> 02:35.640]  At least I know everybody in this room has their priorities straight.
[02:37.960 --> 02:41.520]  And 3pm, a solar compass for the human domain.
[02:43.440 --> 02:45.440]  I have no idea what that could possibly mean.
[02:45.780 --> 02:46.980]  I'm sorry about that.
[02:47.860 --> 02:50.840]  Again, there's a lot of really cool stuff going on in the second floor pavilion.
[02:50.980 --> 02:52.540]  And actually, I have a special request.
[02:53.100 --> 02:58.840]  Is there anybody up here that has an unopened bottle of Club-Mate that they would like to donate to their favorite MC?
[03:00.820 --> 03:01.760]  You guys suck.
[03:04.720 --> 03:06.460]  Alright, that's all the announcements I have.
[03:07.200 --> 03:09.600]  I know we're running a little bit early, but are you guys ready to go?
[03:10.360 --> 03:13.660]  Alright, without any further ado, a very important issue, e-voting.
[03:13.860 --> 03:15.540]  Guys, give them a big round of applause.
[03:16.140 --> 03:16.760]  Sandy and crew.
[03:17.920 --> 03:19.040]  And I'll see you guys at 3.
[03:23.920 --> 03:24.740]  Hey, no hugging.
[03:26.840 --> 03:28.600]  Okay, so thanks everyone.
[03:28.920 --> 03:40.580]  This is going to describe some work that we did at the University of Pennsylvania last autumn.
[03:41.260 --> 03:47.720]  On behalf, weirdly, of the state of Ohio, although it turns out it affects about 43 U.S.
[03:47.840 --> 03:57.340]  states and random other places throughout the world that have been fortunate or unfortunate enough to have the same technology that we use here.
[03:57.340 --> 04:01.220]  So, I'm going to be talking... I'm Matt Blaze.
[04:01.580 --> 04:04.840]  This is Sandy Clark, also called Mouse, for some reason.
[04:05.920 --> 04:17.480]  She and I are going to give the talk, but we also... this is joint work with Adam Aviv, Eric Cronin, Gaurav Shah, and Micah Sherr, who are all here, and Pavel Cerny, who turned invisible.
[04:17.480 --> 04:24.180]  So, this is an international conference, so just forgive me for a couple of minutes.
[04:24.920 --> 04:36.300]  Those of you who are not voters in the United States sometimes find it a little bit hard to understand just how inherently complicated U.S.
[04:36.760 --> 04:39.500]  elections are compared with almost everywhere else in the world.
[04:39.500 --> 04:45.820]  And what I mean by complicated is that there is not a single, you know, U.S.
[04:46.180 --> 04:48.880]  department of elections and voting.
[04:49.160 --> 04:53.840]  Rather, the system that was designed really at the time of the U.S.
[04:53.980 --> 05:16.820]  Constitution was set up as a highly decentralized distributed system aimed at determining the outcome of everything from national elections for president to federal elections for members of Congress to local and state elections for things like governor and sheriff and,
[05:16.820 --> 05:23.740]  you know, library board and school board and local judgeships and so on.
[05:23.940 --> 05:33.080]  We often have elections for both federal offices and state offices and local offices going on at the same time.
[05:33.080 --> 05:45.680]  And we vote for a higher proportion of our representatives directly than almost any other country of similar size and structure.
[05:45.960 --> 05:51.800]  And so what that means is that when you go into a voting booth, you're not simply checking a box for your candidate.
[05:52.380 --> 06:02.240]  Depending on what election it is, you may be making something like 50 decisions, some of which you may be seeing for the first time when you walk into the voting booth.
[06:02.240 --> 06:12.280]  And we somehow have also expected that at the end of the evening, you know, five minutes after the polls close, we wonder who won.
[06:12.600 --> 06:20.000]  And we tend to have gotten into the habit of knowing the results of elections very, very quickly afterward.
[06:20.000 --> 06:26.420]  So in the United States, we have this highly decentralized system where there's no central authority running things.
[06:26.660 --> 06:29.920]  The local elections are run locally.
[06:30.320 --> 06:32.680]  The federal government sets broad standards.
[06:32.840 --> 06:33.400]  There's the U.S.
[06:33.540 --> 06:35.400]  Constitution involved somewhere in there.
[06:35.860 --> 06:39.740]  And the elections themselves are really, really complex.
[06:39.740 --> 06:51.980]  So as hackers and as computer-oriented geeks, our first reaction to this might be, hey, maybe computers could have a role here.
[06:52.300 --> 06:58.020]  Maybe that could help manage this very complicated decentralized distributed system.
[06:58.180 --> 07:06.200]  And in fact, computers have been used in calculating the outcome of elections since pretty close to the very beginning.
[07:06.200 --> 07:07.820]  So I have a quick question.
[07:08.160 --> 07:10.980]  How many of you have seen this picture before?
[07:12.080 --> 07:14.880]  Does anybody... who recognizes this picture?
[07:14.960 --> 07:15.480]  For a t-shirt.
[07:15.840 --> 07:16.060]  Yeah.
[07:16.340 --> 07:17.960]  So what is this picture?
[07:19.020 --> 07:19.980]  Florida recount.
[07:20.160 --> 07:20.800]  Florida recount.
[07:20.900 --> 07:22.560]  Why were they... what were they recounting?
[07:23.220 --> 07:24.180]  Hanging chads.
[07:24.320 --> 07:25.460]  Hanging chads.
[07:25.660 --> 07:30.180]  A term that I thought only us people who knew punch cards back in the IBM days.
[07:30.820 --> 07:32.300]  So you look like, what, a small...
[07:32.840 --> 07:33.320]  Yeah.
[07:33.600 --> 07:34.080]  Okay.
[07:34.500 --> 07:34.980]  Okay.
[07:35.540 --> 07:35.860]  Great.
[07:36.000 --> 07:36.980]  So you sure get a t-shirt.
[07:38.280 --> 07:38.760]  Okay.
[07:39.180 --> 07:39.640]  The...
[07:40.260 --> 07:45.680]  This became kind of emblematic of American elections circa 2000.
[07:46.160 --> 07:52.780]  For those of you who are a little bit younger, you might recall we had a presidential election in the United States in 2000.
[07:53.320 --> 08:00.160]  A fellow by the name of Bush and a fellow by the name of Gore were running against each other to become president of the United States.
[08:00.160 --> 08:22.080]  The outcome of the election hinged in the end on a couple of precincts in Florida that were very, very close and there were questions raised about the accuracy of the count of this punch card electromechanical ballot count system that had been put into place.
[08:22.080 --> 08:38.600]  And this picture, I'm not sure if the contrast is good, but this is probably familiar to a lot of you because this kind of came to be emblematic of the problem with elections in the United States.
[08:38.600 --> 08:51.880]  This poor man, who's kind of bug-eyed and has thick glasses and is clearly overdressed for the Florida climate, you know, spent studying this card trying to figure out, you know, what does it all mean?
[08:51.880 --> 09:01.940]  And this picture was repeatedly on the front page of newspapers and magazines and so on and it came to symbolize what's wrong with elections.
[09:02.160 --> 09:12.020]  You know, if this poor man has to, well, he's not squinting, but if this poor man has to look so closely at this card, surely there's a better way to do this.
[09:12.020 --> 09:22.020]  But in fact, I think one of the things we've come to realize is that the meaning of this picture and the message that you can take away from this picture has kind of reversed.
[09:23.640 --> 09:35.360]  This actually has come to me and to a lot of people to symbolize the strength of the way votes were counted at the time, in that there was a question about the tabulation system.
[09:35.360 --> 09:54.920]  And what it turned out you could do is, with enough work, examine the evidence and the forensics left behind by the voters and tell whether or not the machines gave an answer that, in fact, reflected what people voted for.
[09:55.000 --> 10:08.400]  So if you didn't trust the machines, you could get an army of people with white shirts and ties, thick glasses and bulging eyes to look at these cards and double check the results of the machines.
[10:08.580 --> 10:19.700]  And I think that was a great strength of the system as it existed then in a lot of places before something called the Help America Vote Act.
[10:20.260 --> 10:32.560]  So after the 2000 election, Congress hurriedly passed a law, because everyone agreed that the situation was terrible, hurriedly passed a law called the Help America Vote Act.
[10:32.720 --> 10:42.240]  And it basically mandated a shift, not to computerize voting directly, but to what was called accessible voting technology.
[10:42.500 --> 10:50.420]  And what that meant was that voting technology needed to be something that the disabled could use.
[10:50.540 --> 10:58.260]  Blind people, mobility impaired people, and so on, could use on equal footing and without assistance with everyone else.
[10:58.260 --> 11:07.080]  Now, an effect of that was that the only systems that met this requirement were heavily computerized.
[11:07.180 --> 11:19.620]  Computerized user interfaces like touch screens that could have audio input and output, and motion sensors, and sip and puff sensors, and so on, attached to them to collect vote input.
[11:19.620 --> 11:29.620]  So touch screen machines came to be kind of considered equivalent to the Help America Vote Act requirements.
[11:30.260 --> 11:36.060]  Now, the Congress did something else at the same time as creating this mandate for new voting technology.
[11:36.060 --> 11:50.200]  They also allocated a huge, by pre-war standards, pile of money for all of the states and all of the local governments to buy new voting equipment.
[11:50.400 --> 12:03.860]  So, with a timeline that basically said, within a couple of years, you need to go and get yourself new voting equipment that meets the Help America Vote Act, but don't worry, because here's a big pile of money that you can use to buy it.
[12:04.000 --> 12:05.220]  So, what do you think happened?
[12:05.780 --> 12:08.260]  Well, that money was going to get spent.
[12:08.660 --> 12:29.580]  And, in fact, there were a number of vendors who, in the best traditions of American entrepreneurship, you know, came forward with voting systems that states could buy to meet the Help America Vote Act requirements, out of their great sense of altruism.
[12:31.160 --> 12:32.720]  Now, what does this mean?
[12:32.880 --> 12:41.840]  Well, there were basically three technologies that were put into place right after the 2000 election.
[12:41.980 --> 12:45.320]  All of them existed beforehand, but they weren't really widely deployed.
[12:45.320 --> 12:50.680]  The most well-known are what are called these... it's one of the weirdest acronyms ever.
[12:50.920 --> 12:57.620]  You'd think touch screen voting machines would be called touch screen voting machines, but they had to invent a new term for them.
[12:57.720 --> 13:01.420]  They're called direct recording electronic machines.
[13:01.540 --> 13:06.760]  They directly record your votes using an electronic user interface, a touch screen.
[13:06.960 --> 13:08.640]  These are called DRE machines.
[13:08.760 --> 13:18.320]  So, if you ever want to impress a local voting official or scare them with how well-informed you are, you can refer to them as DRE machines, and that means those touch screens.
[13:18.600 --> 13:30.120]  These are basically a specialized, you know, computer with a touch screen interface and some fancy case and some firmware that turns it into a voting machine with some storage devices.
[13:30.820 --> 13:38.160]  There are also optical scan ballots, you know, similar to, you know, what you use when you took the SATs way back in my days.
[13:38.360 --> 13:44.100]  Fill in with a number two pencil your election system and then you put it in a little machine that grades your ballot.
[13:44.260 --> 13:47.260]  And if you voted for the right person, you know, you can go on to the next election.
[13:47.260 --> 14:03.020]  You basically fill out a little paper ballot and put that in the polling place into a reader that counts your vote, collects your ballots, puts the paper ballot in a ballot box, and then you voted.
[14:03.300 --> 14:23.320]  Now, the way they make them compatible with the Help America Vote Act is with a technology where you basically take a touch screen interface, except instead of recording your vote, it marks a ballot that then gets put in with the other ballots into the local precinct ballot reader.
[14:24.020 --> 14:36.700]  So, basically, the disabled or anyone else who wants the nice user interface of a DRE can still use the touch screen, but what they produce is a paper ballot that gets put into this ballot counter.
[14:36.700 --> 14:44.340]  The machine itself isn't involved in doing anything other than marking the ballots, and then you have the separate ballot reader for counting them.
[14:45.400 --> 14:52.740]  So, another... the third technology that's widely used are centrally counted versions of this.
[14:52.920 --> 14:55.440]  These are typically used for things like absentee ballots.
[14:55.580 --> 15:13.020]  If you say you're not going to be around on election day, they'll give you a... send you one of these in the mail, you mail it back, and then on election day, they feed all of the absentee ballots into a big, you know, batch scan version of the machine that reads all the ballots and produces an output.
[15:13.300 --> 15:19.280]  So, that's essentially what the technology of voting post-Help America Vote Act looks like.
[15:19.440 --> 15:27.400]  And every state now uses, except, interestingly enough, the state we're in, uses one of these technologies.
[15:27.400 --> 15:30.520]  And we'll get to what New York is doing a little later if we have time.
[15:31.980 --> 15:37.440]  Now, what's also evolved is that there are four vendors of this stuff.
[15:37.480 --> 15:48.680]  If you are a local voting official and you want to buy voting equipment, you can buy it from one of four companies that are currently in business that would be happy to sell it to you.
[15:48.680 --> 15:57.500]  And one of them is called ES&S, another is called Hart InterCivic, another is called Sequoia, and then there's another called Premier.
[15:58.020 --> 16:02.820]  Premier, you may have heard of by their previous name, they used to be called Diebold.
[16:03.500 --> 16:08.620]  And they did a value jet, and now they're called Premier election systems.
[16:09.840 --> 16:14.520]  The two largest... they all produce both DRE and optical scan systems.
[16:14.520 --> 16:19.000]  The two largest are ES&S and Premier, or Diebold.
[16:20.300 --> 16:30.160]  And now, I think it is safe to say that there have been questions raised about whether or not this technology is in fact faithful.
[16:30.440 --> 16:35.340]  Whether or not it really does record what voters asked it to record.
[16:35.540 --> 16:42.200]  And whether or not it's vulnerable or more vulnerable to tampering than the systems that it replaced.
[16:42.200 --> 16:50.300]  That is, you know, can the people in this room put their minds together and figure out how to get themselves elected president?
[16:51.000 --> 16:53.300]  You know, are we better off doing this?
[16:53.380 --> 16:55.440]  Does this somehow favor the geeks?
[16:56.680 --> 17:02.120]  So, in order to address the question of are these secure, there were these systems put in place.
[17:02.320 --> 17:04.200]  You can't just sell voting equipment.
[17:04.360 --> 17:07.140]  You can't, you know, start up Joe's voting equipment store.
[17:07.140 --> 17:14.920]  You have to get your equipment certified by testing labs that it turns out the vendors themselves pay for.
[17:15.680 --> 17:18.520]  There are occasional independent reviews.
[17:18.760 --> 17:26.980]  Invariably, the independent reviews reveal serious questions and then nothing actually changes.
[17:28.360 --> 17:36.480]  The hardware and designs and the software and firmware that run on these systems are closely held trade secrets by all four vendors.
[17:36.700 --> 17:39.680]  So, this is not open source code that you can look at.
[17:40.100 --> 17:43.060]  So, if there are problems, they're hidden from most of us.
[17:43.220 --> 17:52.180]  Which even goes further to feeding these questions about, hey, are these systems actually secure enough to be used in real elections?
[17:53.260 --> 17:55.900]  Now, Ohio is an interesting state.
[17:56.080 --> 17:59.200]  It's one of those states in which elections are very heavily politicized.
[17:59.940 --> 18:00.960]  It tends to be close.
[18:01.200 --> 18:03.620]  It sometimes goes Democrat, sometimes goes Republican.
[18:03.900 --> 18:09.760]  So, elections tend to be controversial, big, very heavily noticed events there.
[18:10.000 --> 18:17.240]  They use a wide range of technology, all approved by the Ohio Secretary of State.
[18:17.240 --> 18:19.840]  It turns out states have a Secretary of State, too.
[18:23.440 --> 18:31.320]  In Ohio, the Secretary of State's job, among other things, is approving the voting technology that's used there.
[18:32.100 --> 18:39.160]  And, of course, they buy the same equipment from three out of the four vendors that every other state is buying from them.
[18:39.220 --> 18:43.360]  There are only four vendors of this stuff, and they're not getting special Ohio voting equipment.
[18:43.360 --> 18:47.120]  They may get the software a little bit customized for them.
[18:47.400 --> 18:52.620]  But, basically, they're buying the same hardware and largely the same software and firmware as everyone else.
[18:53.280 --> 18:57.320]  So, Ohio did an interesting thing last year.
[18:57.480 --> 19:12.320]  Following on the tale of California, which did a comprehensive top-to-bottom review of voting technology that I was involved in, the state of Ohio wanted to do the same thing for their voting technology.
[19:13.120 --> 19:21.740]  And they use, again, three of the systems that are three of the four vendors.
[19:22.080 --> 19:25.080]  California used three of the four vendors, but not the same three.
[19:25.220 --> 19:26.880]  So, there was not complete overlap.
[19:27.100 --> 19:36.820]  One of the vendor systems, ES&S, had never actually been included in an independent review of the entire system as a whole.
[19:37.700 --> 19:44.680]  So, basically, Ohio went to a couple of academic institutions, Penn State and the University of Pennsylvania.
[19:46.400 --> 19:52.460]  I'm at the University of Pennsylvania, and asked us if we'd be willing to do a similar kind of review.
[19:53.300 --> 20:02.860]  So, when I got asked to do this, first of all, I mean, an interesting property is when you get phone calls from the Secretary of State, the receptionist tends to deliver that very quickly.
[20:03.380 --> 20:05.860]  To explain, I wasn't talking about, you know, Condoleezza.
[20:06.520 --> 20:10.620]  But, you know, so everyone thought I was much more important than I actually was.
[20:10.780 --> 20:14.440]  But, it raised this question of, should we actually be involved in this?
[20:14.520 --> 20:15.840]  Is this a good idea to do?
[20:17.220 --> 20:22.720]  And, in particular, the fact that we'd be looking at trade secret code raised some alarm bells.
[20:22.720 --> 20:27.180]  You know, would this constrain us about what we could say or what we could do in the future?
[20:28.580 --> 20:31.380]  Secondly, you know, would we be able to do a good job?
[20:31.600 --> 20:39.660]  You know, they were asking us to look at an entire system that we had never seen before, that we could never have seen before.
[20:39.860 --> 20:46.040]  Hardware, software, firmware, and everything else in a period of about two months.
[20:46.480 --> 20:47.940]  What if we don't find anything?
[20:48.460 --> 20:49.800]  What can you conclude from that?
[20:49.940 --> 20:52.800]  Well, if we find problems, we know there are problems there.
[20:52.960 --> 20:56.000]  But, if we don't find problems, it might just be that we were dumb.
[20:56.560 --> 20:56.960]  Alright?
[20:57.080 --> 20:58.400]  Maybe we just weren't clever enough.
[20:58.540 --> 21:02.100]  Maybe if we had a little more time or, you know, we're a little bit smarter.
[21:02.520 --> 21:06.420]  There's some looming vulnerability that we didn't find.
[21:06.500 --> 21:18.000]  We would hate to have to implicitly put our seal of approval on something that, in fact, we're not saying is secure, we're just saying we didn't find any defects.
[21:18.400 --> 21:27.860]  So, there's a question of whether or not it's ethical to be involved in a review that's inherently, by its nature, going to be incomplete.
[21:28.520 --> 21:34.020]  Of course, you know, you can't tell that software is free of defects.
[21:34.400 --> 21:37.240]  You can only tell that it has defects that you found.
[21:38.000 --> 21:40.920]  Now, you know, not only can't you tell, we can...
[21:40.920 --> 21:45.560]  It's like the only thing in computer science we can prove is that you can't figure out the answer to this.
[21:45.720 --> 21:54.740]  So, this is a really important result that may not be well understood by people reading the outcome of this.
[21:54.880 --> 21:56.440]  So, we were a little bit hesitant.
[21:56.680 --> 22:03.180]  And we ultimately decided, you know, we kind of gritted our teeth and said, okay, this may be worth doing anyway.
[22:03.180 --> 22:14.580]  So, we... the first problem that we solved was a very painful contract negotiation, in which we had to ensure our independence.
[22:15.080 --> 22:36.340]  So, we had to make sure that we would really be able to do an independent review, get access to all of the source code of the system, get access to all of the hardware, have adequate tools for looking at it, conditions that would let us actually do an independent review and not have to take anyone's word for how anything is set up.
[22:36.460 --> 22:40.880]  And most importantly, get... maintain our editorial control over the output.
[22:41.080 --> 22:51.280]  So, one of the things that we were very careful and insistent on was that we wouldn't do it unless we could control the report that we were publishing.
[22:51.600 --> 22:58.080]  That the vendors or political forces wouldn't be able to edit our report before it got published.
[22:58.300 --> 23:10.040]  There would be a published report, would contain our findings, and, you know, no one would be able to censor it after the fact or force us to change any of our findings or anything like that.
[23:10.220 --> 23:22.360]  Now, we did agree that any source code that we were going to use to illustrate things in our report, we would publish in an unpublished annex that would only go to the state.
[23:22.640 --> 23:33.480]  We wouldn't put that in the published report, any actual code snippets from the vendor's secret code, but all of the findings would be in this published report under our control.
[23:33.720 --> 23:36.480]  So, that took the first ten weeks of the project.
[23:36.600 --> 23:38.840]  And the final week, we actually did the work.
[23:40.780 --> 23:42.600]  But we were successful at doing that.
[23:42.740 --> 23:46.280]  So, there were three academic teams working on this.
[23:46.280 --> 23:48.560]  One at Penn State, run by Patrick McDaniel.
[23:48.860 --> 23:52.220]  Another at UCSD WebWise Security, run by Giovanni Vigna.
[23:52.520 --> 23:54.640]  And then a team at University of Pennsylvania.
[23:54.900 --> 23:57.280]  We divided the systems up by vendor.
[23:57.500 --> 24:00.680]  I was lucky, and we got the one that no one had looked at before.
[24:00.860 --> 24:03.400]  So, we got to look at the ES&S system.
[24:03.820 --> 24:09.840]  That, again, nobody had looked at the entire system as a whole before we looked at it.
[24:10.140 --> 24:12.160]  Okay, so, why is this hard?
[24:12.160 --> 24:15.100]  Why is it hard to evaluate a voting system?
[24:15.400 --> 24:17.480]  Why do we get ten weeks to do this?
[24:18.000 --> 24:21.360]  In theory, you know, e-voting is like the simplest problem in the world.
[24:21.660 --> 24:30.740]  You'd imagine that somewhere buried in the voting system, there's a line of code somewhere that says something like, you know, votes candidate equals votes candidate plus one.
[24:31.000 --> 24:33.680]  Or, you know, if it's in C, votes candidate plus plus.
[24:33.940 --> 24:36.840]  Or, you know, if it's COBOL, add one to votes candidate.
[24:36.840 --> 24:40.120]  And, you know, you should be able to do this with GREP, right?
[24:40.380 --> 24:41.520]  Look for this line.
[24:41.760 --> 24:45.760]  If it says, you know, plus two, you know something's wrong.
[24:45.980 --> 24:49.160]  Or, you know, minus one, then you know something's wrong.
[24:49.240 --> 24:55.660]  But if there's that line in there, and it's only in there once, and you can only reach it from one path, then it's a reliable voting system.
[24:55.780 --> 24:58.940]  It's solving this simplest problem in the world.
[24:58.940 --> 25:02.560]  But, you know, in theory, theory and practice are the same.
[25:02.680 --> 25:03.540]  In practice, they're not.
[25:04.980 --> 25:13.600]  So, in practice, you've got a lot more than just a voting machine that's doing the tallying of every time you push the button, adding one to this counter.
[25:13.820 --> 25:16.920]  You have an election management system.
[25:18.300 --> 25:28.760]  Software that defines the ballots, that provisions the computer, the voting computers, the DRE machines and the optical scan systems.
[25:29.080 --> 25:33.920]  The firmware running in the hardware that's sent out to the precincts.
[25:34.140 --> 25:35.340]  There's communication.
[25:35.560 --> 25:44.140]  Basically, it may involve sneaker net, but there's a big communication system for getting the things at the voting locations back to the county headquarters.
[25:44.380 --> 25:50.340]  And maybe it involves modems, or maybe it involves moving memory cards or equipment back and forth.
[25:50.520 --> 25:55.440]  But that system for communicating these results, again, is part of what you have to examine.
[25:55.440 --> 25:57.320]  And then there's this human element.
[25:57.500 --> 25:59.840]  There are procedures that people are following.
[26:00.080 --> 26:02.240]  There are mechanical locks and seals.
[26:02.480 --> 26:04.360]  There are procedures for dealing with the keys.
[26:04.640 --> 26:06.740]  And so on and so on.
[26:07.020 --> 26:11.160]  So, you know, this is actually a pretty complicated system.
[26:11.440 --> 26:14.600]  And the threat model itself is pretty complicated.
[26:15.260 --> 26:15.520]  Right?
[26:15.620 --> 26:24.180]  This is not as simple as most other systems, because we have threats that don't exist in anything other than voting.
[26:24.180 --> 26:25.880]  You have the usual things.
[26:26.100 --> 26:27.380]  Can somebody alter data?
[26:27.580 --> 26:29.180]  Can somebody deny service?
[26:29.540 --> 26:32.260]  Can somebody change the official results?
[26:32.560 --> 26:33.480]  Data integrity.
[26:33.760 --> 26:36.600]  But you also have ballot secrecy issues.
[26:36.800 --> 26:36.860]  Right?
[26:37.080 --> 26:40.980]  One of the attacks against the voting system is figuring out who someone voted for.
[26:41.220 --> 26:47.660]  A system needs to be secure, not just for integrity, but it also has to not have accountability.
[26:47.660 --> 26:55.100]  So, somehow you have to have data integrity, but not accountability back to the individual voter, which is not the usual way you do data integrity.
[26:55.320 --> 26:56.940]  So, it's a complicated thing.
[26:57.600 --> 27:01.600]  So, the system we looked at at Penn is the largest vendor.
[27:01.740 --> 27:06.420]  It's used in 43 states by something like 40% of voters in the U.S.
[27:06.800 --> 27:08.800]  They have both DRE and optical scan.
[27:08.800 --> 27:14.720]  The centerpiece of the system is their DRE machine, which is called an Ivotronic.
[27:14.960 --> 27:17.140]  It's not, in spite of the punctuation, it's not made by Apple.
[27:18.580 --> 27:24.400]  It's basically a custom computer that runs proprietary firmware with a touch screen on it.
[27:24.580 --> 27:38.220]  The models used in Ohio have a printer attachment that, after you select a candidate, it prints out the fact that you've selected the candidate on this continuous roll printer to allow for non-electronic recounts.
[27:38.420 --> 27:42.880]  And that's called a voter verified paper audit trail.
[27:43.160 --> 27:53.940]  These machines are controlled by the special module that basically turns them on for each voter, called a personalized electronic ballot, or a PEB, which is this little module that turns the Ivotronic on.
[27:54.180 --> 27:55.300]  So, here's what it looks like.
[27:55.760 --> 27:57.180]  Basically, a touch screen.
[27:57.180 --> 28:04.060]  That little red bar is the slot with a PEB in it, and this has been enabled for voting.
[28:04.480 --> 28:05.540]  Here's the printer.
[28:05.840 --> 28:06.380]  It's on the side.
[28:06.500 --> 28:08.200]  A close-up of the printer that's on the side.
[28:08.340 --> 28:16.180]  Basically, it prints a little barcode at the end of who you voted for, so that the barcodes can be scanned in the event of a recount.
[28:16.340 --> 28:17.500]  Now, is there a problem with that?
[28:17.960 --> 28:19.560]  Is that actually voter verified?
[28:19.920 --> 28:22.700]  Well, most of us can read barcodes, so why not everyone else?
[28:23.500 --> 28:27.060]  This is the little personal electronic ballot module.
[28:27.060 --> 28:30.320]  There's a border next to it to give you kind of an idea of scale.
[28:30.660 --> 28:34.120]  It's basically an infrared device with a magnet in it.
[28:34.220 --> 28:42.180]  The magnet turns on the computer, and then it communicates by this IRDA protocol through this little window that you can't quite see here.
[28:42.180 --> 28:48.020]  So, the software for the IVOTRONIC DRE is an Intel 386.
[28:49.300 --> 29:03.860]  Runs... has about 88,000 lines of code written in C and assembly, plus there were some libraries that we turned out not to get, and some additional software that's specific to the back-end system for provisioning it and counting the results out of it.
[29:04.860 --> 29:10.300]  There's an optical scan system called the Model 100, is the precinct optical scan counter.
[29:10.660 --> 29:19.240]  Basically, it looks kind of like a fax machine sitting on top of a shredder, although it doesn't actually have a shredder or a fax machine in it, we were assured.
[29:19.980 --> 29:24.520]  It's basically a scanner that sits on top of a ballot box that's locked.
[29:24.660 --> 29:26.240]  It'll accept or reject your ballot.
[29:26.240 --> 29:33.780]  If you try to vote for two people for president, it'll reject your ballot rather than just saying, okay, thanks, and taking a spoiled ballot.
[29:33.880 --> 29:38.440]  So, it has better usability than just putting it, dropping it in a ballot box by itself.
[29:38.680 --> 29:43.240]  And the thing that scans your ballot also counts your vote for the precinct.
[29:43.960 --> 29:55.680]  And there's an internal memory card that holds both the ballot definition that it accepts and the tally that's going on during the election.
[29:55.680 --> 29:57.360]  So, here's what one of them looks like.
[29:57.480 --> 30:01.380]  Again, it's sort of, you know, fax machine shredder looking thing.
[30:01.540 --> 30:04.080]  And there's a ballot sticking in the front.
[30:04.220 --> 30:10.260]  And then it just uses these little PCMCIA form factor memory cards for it.
[30:10.460 --> 30:18.800]  Now, the software for that, it's also an Intel 8086 class processor with about 30,000 lines of code and, again, code in the back end.
[30:19.140 --> 30:21.220]  And there's a central count system.
[30:21.360 --> 30:22.980]  And that was the fun one to evaluate.
[30:23.980 --> 30:26.220]  It's this high-speed back scanner.
[30:26.440 --> 30:27.640]  We had them send us one.
[30:27.880 --> 30:29.640]  It had to be delivered by truck.
[30:30.720 --> 30:33.600]  One of us got a back injury trying to deliver it.
[30:33.680 --> 30:34.060]  Hi, Adam.
[30:34.200 --> 30:35.220]  Hope you're feeling better.
[30:37.420 --> 30:38.760]  This is what the thing looks like.
[30:38.820 --> 30:43.740]  And the cool thing about it is it's designed to process, you know, all of the ballots for an election in one day.
[30:43.880 --> 30:50.620]  So, the ballots move in really fast into the ballot box, except they didn't actually give us the ballot box that catches them.
[30:50.620 --> 30:57.780]  So, what ends up happening is the ballots go in and then shoot out through the side of them and kind of hit the wall.
[30:57.980 --> 31:01.240]  And if you're standing on the other side of it, you can get like a really deep paper cut.
[31:01.380 --> 31:04.620]  We were trying by the end of it, putting little saw blades through it and things like that.
[31:04.720 --> 31:05.260]  See what it could do.
[31:05.920 --> 31:09.480]  That's an Intel 686 processor, about 22,000 lines of code.
[31:09.660 --> 31:10.960]  Plus, again, code in the back end.
[31:11.060 --> 31:12.760]  So, what's this back end that I keep talking about?
[31:12.760 --> 31:18.080]  So, the back end system for running the elections is called Unity.
[31:18.480 --> 31:20.780]  It's their election management suite.
[31:21.000 --> 31:30.220]  Again, it does ballot definition, provisioning of the Ivotronics, the DREs, the precinct optical scan counters, and the central count systems.
[31:30.400 --> 31:42.580]  And then, collecting the results from all of those precincts, producing a tally, printing out the official results for the counties, different races in the county, and then audit in and recount.
[31:42.920 --> 31:45.020]  Basically, this is a software package.
[31:45.420 --> 31:49.200]  It runs on a Windows PC in the county office.
[31:49.540 --> 31:53.040]  It's designed to run on a single PC, but what a lot of...
[31:53.040 --> 32:03.100]  But, you know, since you're doing a lot of different voting machines, what a lot of counties typically do is use Windows networking and a shared file system on this database to read...
[32:03.100 --> 32:05.460]  to allow multiple machines to run on this.
[32:05.620 --> 32:11.480]  So, there's this little Windows box or network of Windows boxes running this special software suite back in the county.
[32:11.740 --> 32:13.000]  So, what's this look like?
[32:13.140 --> 32:15.940]  Well, this is a standard... again, a standard Windows platform.
[32:16.200 --> 32:23.440]  It's about 400,000 lines of code written in C, C++, SQL, Visual Basic, and COBOL.
[32:23.620 --> 32:25.360]  And that was exciting to discover.
[32:27.020 --> 32:30.080]  And again, it includes... you get this with customized modules.
[32:30.220 --> 32:34.060]  If your county only bought DREs, you don't get the optical scan stuff.
[32:34.060 --> 32:41.280]  But you can mix and match and get whatever modules for counting and provisioning and ballot definition you need.
[32:41.440 --> 32:43.700]  Just to give you an idea of the data flows in it.
[32:43.780 --> 32:46.240]  This is the data flow throughout the software.
[32:46.560 --> 32:49.180]  What you can kind of take away from this is it's complicated.
[32:50.800 --> 33:08.640]  But basically, the basic architecture is there's provisioning data, ballot design, programming the individual devices, loading them onto the devices, and then at the end of the day, going back to the election management system from the precinct devices to get the tallies.
[33:08.880 --> 33:11.700]  So, with that, what do we do?
[33:11.840 --> 33:13.820]  So, let me turn this over to Sandy.
[33:15.140 --> 33:16.420]  Yeah, I get to tell you the fun stuff.
[33:19.440 --> 33:22.680]  So, the first thing, of course, we had to do was set up a secure environment.
[33:23.920 --> 33:29.720]  And we turned one of our offices into a skiff and we changed all the locks and we got in trouble with the University locksmith again.
[33:32.220 --> 33:42.360]  And we took possession of a whole bunch of safes because everything we worked on, every piece of paper we printed out, our lab notebooks, everything had to be locked in a safe every time we left the skiff.
[33:43.100 --> 33:48.140]  This source code, ESNS stuff, was protected better than Cheney's undisclosed location.
[33:50.240 --> 33:51.320]  So, what did we do?
[33:51.920 --> 33:53.400]  Well, there were seven of us.
[33:54.080 --> 33:57.720]  We had less than ten weeks and we had 700,000 lines to go through.
[33:58.600 --> 34:04.820]  No methodology, no checklist that we could have made up was going to be able to get through all of this.
[34:05.000 --> 34:06.920]  So, basically, we did triage.
[34:07.140 --> 34:12.980]  We divided everything up into parts and everybody sat down and said, okay, how do I attack this?
[34:14.300 --> 34:16.040]  So, what exactly did we find?
[34:17.800 --> 34:19.260]  Well, here's a list.
[34:24.230 --> 34:28.360]  Now, that's not all ESNS, some of that's Hart and Premier, but you get the idea.
[34:29.150 --> 34:37.130]  The thing that surprised us is that there wasn't a place where we looked, that we didn't find a vulnerability we could compromise.
[34:37.540 --> 34:42.860]  There wasn't an attack that we tried where we weren't successful at carrying it all the way through.
[34:43.750 --> 34:49.840]  And this is in spite of all of the security procedures that ESNS has in place on these machines.
[34:49.840 --> 34:53.960]  And those security procedures include physical security, there are locks.
[34:54.650 --> 34:56.110]  It includes tamper evidence seals.
[34:56.810 --> 35:01.730]  The top of the line DREs all have a hardware authentication token.
[35:02.070 --> 35:04.360]  There's also passwords and there's encryption.
[35:04.960 --> 35:09.730]  So, why didn't these layers of security protect the system?
[35:11.000 --> 35:12.770]  Let's look at a couple of these individually.
[35:13.310 --> 35:14.360]  We'll start with the locks.
[35:15.020 --> 35:18.830]  This is a lock by the same manufacturer that makes the ESNS.
[35:19.460 --> 35:21.880]  This is not from a voting machine, this picture is actually from eBay.
[35:23.210 --> 35:26.940]  They are cheap cabinet or filing cabinet wafer locks.
[35:27.340 --> 35:31.050]  You can buy them anywhere, you can get the keys anywhere, you can get the blanks anywhere.
[35:31.480 --> 35:36.150]  By the way, I should point out, we know this because they forgot to ship us the keys for a couple of weeks.
[35:36.570 --> 35:37.190]  So, we ended up...
[35:37.190 --> 35:38.670]  The only way we were able to use them.
[35:41.830 --> 35:46.190]  So, the thing about this is that these cheap wafer locks, you can pick them with a paper clip.
[35:46.770 --> 35:57.380]  And since ESNS uses two keys, they use one to control the electronics and one to open and lock up the ballot box, we had to pick the locks just to turn the machines on in the first place.
[35:59.130 --> 36:02.090]  The fact that they use two keys is the real problem here.
[36:02.520 --> 36:05.150]  You see, they only use two keys.
[36:05.540 --> 36:11.880]  Every machine they sell, whether it's in New York, California, Alaska, Hawaii, uses these same two keys.
[36:12.360 --> 36:12.920]  Okay?
[36:14.860 --> 36:17.670]  That doesn't give me much confidence in their physical security.
[36:19.720 --> 36:22.210]  So, what about the tamper evidence seals?
[36:22.460 --> 36:23.500]  That's another layer, right?
[36:25.020 --> 36:26.050]  ESNS is hardware.
[36:26.050 --> 36:31.590]  The touchscreens and the optical scanners use three different kinds of removable media.
[36:31.840 --> 36:36.150]  They use CF cards, they use PCMCIA cards, and they use ZIP disks.
[36:36.750 --> 36:45.670]  And Ohio's elections procedures specifically state that once that media is inserted into the slot, the slot has to be covered with one of these tamper evidence seals.
[36:46.790 --> 36:49.500]  So, how good are they at detecting evidence of tampering?
[36:49.770 --> 36:51.290]  Here's your chance to earn a T-shirt.
[36:51.650 --> 36:54.360]  Somebody give me a way to remove these seals undetectably.
[36:54.360 --> 36:55.000]  Steam!
[36:55.550 --> 36:56.360]  Air dryer!
[36:57.340 --> 36:58.110]  Okay, wait, wait.
[36:58.170 --> 36:58.750]  One at a time.
[36:58.900 --> 37:00.090]  I've got a couple of T-shirts to get out.
[37:00.400 --> 37:01.000]  Air dryer!
[37:01.440 --> 37:01.590]  Steam!
[37:02.420 --> 37:03.380]  Steam works great.
[37:03.570 --> 37:04.190]  WD-40.
[37:04.290 --> 37:04.980]  Peel it back!
[37:05.190 --> 37:06.790]  WD-40 is another really good one.
[37:06.960 --> 37:07.810]  Come get your T-shirts, guys.
[37:08.520 --> 37:08.920]  T-shirts.
[37:09.270 --> 37:10.190]  Why are you doing it?
[37:11.420 --> 37:12.650]  That's like the WD-40.
[37:14.950 --> 37:17.420]  You know what else works really well is cold.
[37:17.420 --> 37:19.360]  Cold actually works better than heat.
[37:21.900 --> 37:24.290]  And, of course, you can always just buy them.
[37:24.810 --> 37:26.690]  The name of the manufacturer is on the labels.
[37:26.860 --> 37:27.540]  Just order a bunch.
[37:34.910 --> 37:38.470]  But notice that the label is over the CF slot.
[37:38.610 --> 37:40.570]  But there's nothing protecting that port.
[37:41.170 --> 37:45.990]  These machines have a lot of open, accessible, completely unprotected ports.
[37:48.610 --> 37:50.830]  Would people actually pull out this plug?
[37:51.250 --> 37:57.830]  Well, Secretary of State Brunner told us that until she had seen this photograph, she hadn't understood what her own people had been telling her.
[37:58.170 --> 38:04.270]  Her election observers had watched at the last Ohio election, normal voters walk up on election day and pull out that plug.
[38:05.110 --> 38:06.670]  And, of course, what does that do?
[38:06.830 --> 38:08.230]  That disables the audit trail.
[38:08.230 --> 38:10.550]  So much for a paper record of your ballot.
[38:11.590 --> 38:16.250]  And, of course, it also gives you access to the printer port so you can print whatever you want on the audit trail.
[38:19.770 --> 38:23.850]  And, in addition, some of the machines have Ethernet ports, modem ports.
[38:24.210 --> 38:25.430]  You can connect to them.
[38:25.510 --> 38:28.210]  Some of them are hidden behind the crappy locks, but that's not an issue.
[38:30.090 --> 38:36.290]  So another problem with these touchscreens is that they are seriously easy to recalibrate.
[38:37.910 --> 38:40.750]  I don't know if you can see on the screen, there's a tiny little X.
[38:41.550 --> 38:44.030]  But they calibrate just like any touch screen.
[38:44.890 --> 38:47.830]  You poke the X and it moves somewhere else, and you poke it somewhere else.
[38:48.050 --> 38:52.150]  So what happens if you poke that X in the wrong place on purpose?
[38:53.870 --> 38:59.310]  You can make it impossible for this machine to record votes for a particular candidate.
[38:59.750 --> 39:04.750]  And this is so simple to do, we found it within the first five minutes of turning the machine on for the first time.
[39:07.790 --> 39:14.430]  Now, a major part of ES&S's touch screen security is that little box with the red stripes that you see in the slot.
[39:15.030 --> 39:18.750]  That's that personal electronic ballot, the PEP.
[39:19.110 --> 39:22.250]  That is ES&S's hardware authentication token.
[39:22.650 --> 39:25.470]  And you need it to boot the machines to turn them on in the first place.
[39:25.650 --> 39:27.090]  You need them to upload the firmware.
[39:27.430 --> 39:29.350]  You need it to install a ballot definition.
[39:29.590 --> 39:34.650]  You need it to open the polls, zero out the vote totals, close the polls, collect all the results at the end of the election.
[39:35.130 --> 39:37.890]  You've got to have one of these, and you can't buy them.
[39:38.730 --> 39:39.410]  We did try.
[39:40.910 --> 39:42.630]  But in our world...
[39:42.630 --> 39:43.970]  We can't buy them, you can roll them.
[39:44.050 --> 39:44.170]  Yeah.
[39:44.530 --> 39:48.790]  Well, in the hacker community, if you can't buy something and you've got to have one, how do you get it?
[39:49.110 --> 39:49.590]  All together?
[39:49.810 --> 39:50.950]  You make it yourself.
[39:51.210 --> 39:54.030]  And we made ours with a PDA and a magnet.
[40:01.890 --> 40:06.230]  So, there's a photo of Micah turning all of the passwords to Everest.
[40:09.330 --> 40:15.250]  So, the documentation for ES&S lists three kinds of these personal electronic ballots.
[40:15.830 --> 40:19.690]  The first one is a blue one, and that's a voter PEP.
[40:19.810 --> 40:21.910]  And that allows one voter to cast one ballot.
[40:22.330 --> 40:26.190]  The second one has a red stripe on it, and that's the supervisor PEP.
[40:26.330 --> 40:32.450]  That allows a poll worker to turn the machines on, to open and close the polls, and to collect all the votes.
[40:32.930 --> 40:36.810]  There's a third kind of PEP with a yellow stripe, and that's a reset PEP.
[40:36.810 --> 40:44.670]  It can be used either by the poll worker or by a vendor's representative, and it resets the machine back to its default state if there's a problem.
[40:45.930 --> 40:56.210]  But imagine our surprise, since this is such an important part of the security of these touchscreens, imagine our surprise when we found that there's a fourth kind of PEP.
[40:59.330 --> 41:10.790]  You see, it's not listed in the documentation, but there's this PEP that allows you to have complete root access to the machine, and it doesn't require a password.
[41:12.650 --> 41:14.610]  That's my definition of a backdoor.
[41:16.010 --> 41:19.010]  I don't know what color it is, but ours was gray.
[41:22.590 --> 41:28.890]  So, I think it's pretty common knowledge that the touchscreens are broken.
[41:29.550 --> 41:35.090]  I think if you ask most people, they would say that you really can't trust the security on these touchscreens.
[41:35.350 --> 41:38.370]  And that just leaves us the optical scanners.
[41:40.310 --> 41:43.510]  But there are a lot of problems with the optical scanners, too.
[41:44.890 --> 41:46.290]  So, let's get into a few.
[41:46.610 --> 41:54.650]  Some of them have to do with the passwords and the encryption that are used by the vendors.
[41:56.070 --> 42:10.890]  In the case of both the precinct individual ballot scanner and the central election headquarters big batch scanner, both of them will upload and run firmware without asking for a password.
[42:11.310 --> 42:19.870]  So, if you can get access to the removable media, you can put your own firmware on there, stick it in the slot, and it will upload it and it will run it, no questions asked.
[42:20.710 --> 42:24.810]  They also do not do any form of encryption on that removable media.
[42:24.810 --> 42:35.990]  So, once those votes are collected and you can get access to the media, you can alter the votes, the results of that election, and there's no way for anyone to be able to tell that you've done it.
[42:40.080 --> 42:45.320]  So, here's a picture, by the way, of the big batch scanner running our Everest firmware.
[42:47.720 --> 42:49.840]  So, what about the paper ballots?
[42:50.280 --> 42:51.960]  I mean, that's pretty much all that's left us.
[42:53.000 --> 42:54.660]  Well, there are problems with that, too.
[42:55.360 --> 43:02.400]  You see, because of those cheap, crappy wafer locks, it's trivially simple to get access to the glass surface of the scanner.
[43:02.660 --> 43:08.320]  So, all you have to do is open the scanner up and put a Post-it down where you don't want the votes to be counted.
[43:08.700 --> 43:11.020]  Paint something reflective, nail polish or something on it.
[43:13.560 --> 43:18.720]  And then, of course, it's trivially simple to forge the paper ballots as well.
[43:19.640 --> 43:25.800]  The paper ballots rely for their security on something called non-read ink.
[43:26.420 --> 43:32.720]  And you'll find this ink on security certificates like stock certificates.
[43:32.940 --> 43:33.920]  You'll see it on deeds.
[43:34.160 --> 43:36.240]  You'll see it on checks or cashier's checks sometimes.
[43:36.640 --> 43:40.840]  Usually, it's wavy lines or an abstract pattern in a yellow ink or a red ink.
[43:42.880 --> 43:47.120]  And it relies for its security on the fact that that ink is invisible to the scanner.
[43:47.560 --> 43:51.120]  So, on this ballot, some of these boxes are printed in regular ink.
[43:51.280 --> 43:53.720]  And some of those boxes are printed in security ink.
[43:54.060 --> 43:57.000]  The one at the top is in the security ink.
[43:57.840 --> 44:09.280]  And when you put that ballot through the scanner, either the big batch one or the individual precinct one, the scanner can't see marks where the security ink are and it just passes that ballot right through.
[44:09.620 --> 44:15.780]  But if you try and make a photocopy of the ballot or you try and print out your own version of it, that prints out in regular ink.
[44:15.960 --> 44:18.880]  The scanner sees boxes where it shouldn't see boxes.
[44:19.100 --> 44:23.140]  And it spits it out and it prints across that LCD screen, possibly counterfeit.
[44:24.200 --> 44:27.620]  So, how do you forge a ballot that's got this high security ink?
[44:28.540 --> 44:29.500]  With whiteout.
[44:32.180 --> 44:38.980]  All you have to do is get rid of the little security boxes, make as many copies as you want, and the scanner can't tell the difference.
[44:39.240 --> 44:42.080]  This ballot was accepted as authentic by both of our scanners.
[44:43.800 --> 44:49.200]  Now, you might think that an alert election observer might notice that those boxes are missing.
[44:49.520 --> 44:50.460]  They might not.
[44:50.460 --> 44:54.420]  Remember, Matt talked about the fact that every single precinct selection is different.
[44:54.600 --> 44:57.980]  And every single ballot looks different because my dog catcher isn't the same as yours.
[44:58.720 --> 45:02.220]  And my school board proposal isn't the same as somebody else's.
[45:03.240 --> 45:05.280]  And also, but they might not notice.
[45:05.440 --> 45:07.180]  I mean, they've been up since 5:30 in the morning.
[45:08.840 --> 45:14.940]  But if you want to be picky, and we like to be picky, you've got to put that security ink back.
[45:16.100 --> 45:17.360]  Well, you can't buy it.
[45:17.580 --> 45:24.260]  There are a couple of places where you can get a hold of the yellow ink or the red ink in small quantities.
[45:24.580 --> 45:27.780]  But the black ink is really, really hard to come by.
[45:28.520 --> 45:31.540]  So once again, in our world, if you can't buy it, how do you get it?
[45:32.000 --> 45:33.100]  You make it yourself.
[45:33.100 --> 45:39.600]  We made ours with the absolutely cheapest color inkjet cartridge we could find at the local CVS.
[45:40.480 --> 45:46.840]  You see, another name for this security ink is composite ink.
[45:48.000 --> 45:49.620]  This is an ugly splotch, I know.
[45:49.760 --> 45:50.740]  I got a C plus in art.
[45:51.360 --> 45:53.040]  The point is, it's a proof of concept.
[45:53.320 --> 45:55.260]  That is invisible to the scanner.
[45:55.500 --> 45:58.640]  And that ballot was accepted as authentic by both of the scanners.
[46:00.700 --> 46:02.680]  And you know what this ink is?
[46:03.000 --> 46:06.460]  This is the stuff your home inkjet printer makes when you run out of black.
[46:07.520 --> 46:09.980]  It just takes the color inkjet cartridge.
[46:10.200 --> 46:11.760]  It mixes all the colors up together.
[46:11.900 --> 46:14.940]  And all I did was stick a paintbrush in all the holes and smear it around.
[46:14.940 --> 46:17.540]  And that is security ink.
[46:19.920 --> 46:20.580]  So...
[46:23.740 --> 46:26.100]  So what did we learn from all of this?
[46:27.260 --> 46:38.680]  The first thing that we learned is that the certification procedures in place to guarantee that vendors' voting machines and their whole systems are secure just don't work.
[46:39.080 --> 46:40.560]  You know what these procedures are?
[46:41.120 --> 46:43.360]  Are the comments in the same format?
[46:43.700 --> 46:46.340]  Do the variables follow a naming convention?
[46:46.340 --> 46:49.060]  Is there only one return statement per function?
[46:50.040 --> 46:52.820]  I don't see how that enhances the security at all.
[46:56.310 --> 47:07.970]  The second thing that we learned is that the security of a system that is intended or needs to be secure can't be added on as an afterthought.
[47:07.970 --> 47:17.710]  You can't just add a couple of locks and a seal and throw a password in and claim that the system is secure.
[47:18.090 --> 47:25.010]  If you need it to be secure, then you've got to design security in from its inception.
[47:25.010 --> 47:30.730]  Oh, by the way, the touch screen voting machines have hard-coded passwords.
[47:32.230 --> 47:42.610]  We found a comment next to a hard-coded password that said, I kid you not, this comment said, we hard-coded this so that hackers couldn't use it.
[47:49.890 --> 47:51.510]  Yeah, you can't make this stuff up.
[47:54.890 --> 48:02.270]  The third thing that we learned is that, well, we weren't the only ones looking at the system.
[48:02.990 --> 48:09.470]  Along with the academic teams, Ohio hired a group of commercial software testers.
[48:10.290 --> 48:17.850]  And they have a methodology and they've got a business that goes around trying to determine the security of software systems.
[48:18.350 --> 48:22.210]  And they didn't find anything that we didn't find.
[48:22.850 --> 48:24.870]  But we found a lot that they didn't.
[48:25.230 --> 48:32.590]  So why did our ad hoc, thrown-together method find so much more than they did?
[48:33.330 --> 48:35.650]  We think it's because we approached it like hackers.
[48:38.740 --> 48:45.700]  We sat down, every one of us with our little piece, and we basically said, how do I attack the system?
[48:47.320 --> 48:48.140]  If I...
[48:50.060 --> 48:53.280]  If I poke at it here, which way does it move and what does that tell me?
[48:55.840 --> 48:59.200]  And it allowed us to go off on tangents if it looked interesting.
[49:00.140 --> 49:02.560]  And that allowed us to make some of our best finds.
[49:02.780 --> 49:05.120]  For example, the Palm Pev emulator.
[49:05.900 --> 49:10.340]  The first thing we noticed when we unboxed that touch screen is that there's no on-off switch.
[49:10.880 --> 49:14.600]  But if you stick that little box in that little slot, the machine comes on.
[49:14.740 --> 49:14.980]  Wow!
[49:15.220 --> 49:16.680]  Okay, there's a switch in there somewhere.
[49:17.100 --> 49:20.380]  If you put the wrong one in, it comes on, but it gives you an error message.
[49:20.520 --> 49:21.000]  Wrong Pev.
[49:22.440 --> 49:24.520]  So that means there's some communication going on.
[49:24.860 --> 49:28.920]  And so the next step was just to try and decode that communication.
[49:31.080 --> 49:33.560]  Another example is finding the ballots.
[49:33.980 --> 49:37.840]  The reason we found we could forge ballots was because we tried to make a photocopy.
[49:37.980 --> 49:42.500]  And we sent it through and it spit it out and said, possibly counterfeit.
[49:42.660 --> 49:44.940]  So we thought, okay, the papers are the same color.
[49:45.420 --> 49:47.280]  The weight...
[49:47.280 --> 49:49.060]  This is paper and this is card stock.
[49:51.080 --> 49:56.920]  But looking closely at the ballot, you could see that some boxes didn't look exactly the same as the other.
[49:57.000 --> 49:58.480]  It's a little fuzzier, not quite as clear.
[49:59.740 --> 50:00.840]  Okay, what does that mean?
[50:01.520 --> 50:04.720]  And that tangent led us to that.
[50:04.720 --> 50:15.740]  And then the one other thing that we did is that we stepped back and looked at this not as this problem with this machine and this problem with that one.
[50:15.960 --> 50:18.240]  But as pieces of an entire system.
[50:18.960 --> 50:26.700]  And we were able to find our major discovery, that of viral propagation.
[50:27.380 --> 50:30.600]  Because we said, we've got a vulnerability here.
[50:30.700 --> 50:31.940]  We've got this piece of a puzzle.
[50:32.240 --> 50:34.920]  How do we use that to attack this piece?
[50:37.080 --> 50:56.680]  And so the fourth discovery, and this is the most important, is that we were able to actually demonstrate an attack that a single person, a single voter could do at a single machine that would virally propagate all the way back to the election central headquarters.
[50:57.380 --> 50:58.400]  of the county.
[51:02.280 --> 51:08.660]  And what this means is that not only could an attacker change the votes on one machine.
[51:09.200 --> 51:13.120]  Not only could a malicious poll worker alter the results for a precinct.
[51:13.260 --> 51:25.660]  But one person in one podunk little precinct, somewhere in the middle of nowhere, can upload malicious code onto a machine, which will then write itself onto the removable media.
[51:25.660 --> 51:30.480]  Which at the end of the election gets taken out of that voting machine and sent back to the headquarters.
[51:30.800 --> 51:34.800]  Where it gets put into that windows box running the election management software.
[51:35.180 --> 51:39.520]  Where it takes advantage of a number of vulnerabilities and writes itself onto that.
[51:39.980 --> 51:43.220]  And thereby alters the results for the election of the entire county.
[51:43.220 --> 51:54.500]  And if you don't think that's important, please remember that in 2004, the entire country waited to find the results of who got Ohio's electoral votes.
[51:54.780 --> 52:00.700]  And who got Ohio's electoral votes depended on the results of Cuyahoga County.
[52:00.700 --> 52:05.680]  One single county made all the difference in who became president.
[52:07.180 --> 52:08.760]  So, you know, it's true what they say.
[52:09.320 --> 52:11.200]  One voter really can make a difference.
[52:22.380 --> 52:26.560]  So this is not part of our mandate, not part of our contract with Ohio.
[52:26.800 --> 52:30.640]  But it is kind of an educated opinion of ours.
[52:30.800 --> 52:32.140]  What do we do about this?
[52:35.520 --> 52:37.000]  We're going to have to be really careful.
[52:37.400 --> 52:39.880]  These machines are in use right now.
[52:40.180 --> 52:41.940]  And they will be in use in November.
[52:43.180 --> 52:45.440]  Now, we are hearing from a number of places.
[52:46.080 --> 52:48.300]  Ohio, for one, is getting rid of their touchscreens.
[52:49.760 --> 52:57.280]  We've got a request from Texas, of all places, to send them a report about which optical scanners would be better than...
[52:57.280 --> 52:58.940]  Our opinion on the optical scanners.
[52:59.140 --> 53:01.480]  Because they want to get rid of their touchscreens.
[53:01.480 --> 53:02.880]  Texas, of all places.
[53:03.120 --> 53:04.560]  So maybe we're making a difference here.
[53:07.560 --> 53:11.680]  But in the short term, all we can really do is be very careful.
[53:11.920 --> 53:20.500]  So I would like to ask all of you who are voting age and U.S. citizens to become poll workers at your next election.
[53:23.940 --> 53:26.020]  Your counties and your precincts need you.
[53:26.200 --> 53:31.000]  Most poll workers are geriatric and wouldn't recognize a problem if they saw it.
[53:31.000 --> 53:33.940]  But at least your technical expertise could be of use.
[53:35.640 --> 53:38.620]  In the long term, we've got to build a better mousetrap.
[53:40.380 --> 53:41.620]  And that's our dog.
[53:52.680 --> 53:53.640]  Questions are welcome.
[53:53.980 --> 53:55.420]  Thank you.
[53:55.480 --> 53:56.160]  Thank you.
[53:56.180 --> 53:58.240]  We have a live for like two questions.
[53:58.680 --> 53:59.040]  Or...
[53:59.040 --> 53:59.160]  Yeah.
[54:00.640 --> 54:01.580]  Use the mic, please.
[54:02.460 --> 54:04.280]  And state your social security number.
[54:04.760 --> 54:04.840]  Yeah.
[54:15.290 --> 54:18.510]  Social security is all Fs, which I assume is true for everyone here in the room.
[54:20.270 --> 54:26.310]  I participated in the accessibility testing of election machines in California.
[54:26.890 --> 54:28.070]  For the top to bottom?
[54:28.270 --> 54:28.450]  Yeah.
[54:28.590 --> 54:28.890]  Oh, cool.
[54:28.910 --> 54:30.190]  And the subsequent one.
[54:30.610 --> 54:42.970]  And one of the things we found, just an interesting sidelight, is that the audio interface for a blind voter is unintentionally, one assumes, being broadcast over RF.
[54:43.310 --> 54:48.750]  So that at almost any distance from the voting machine, you can listen in to what the person's hearing and how they're voting.
[54:48.770 --> 54:49.470]  You chose.
[54:49.750 --> 54:49.950]  Right.
[54:50.170 --> 54:50.850]  So that's...
[54:51.470 --> 54:51.910]  Yeah.
[54:52.030 --> 54:53.570]  That's exactly right.
[54:54.170 --> 54:54.230]  Yeah.
[54:54.350 --> 54:56.830]  And so you can't help but wonder what else is going...
[54:56.830 --> 54:57.910]  ...is available over RF.
[54:57.910 --> 55:03.470]  But I guess my point is that we found massive inaccessibilities in these machines.
[55:03.670 --> 55:13.590]  So to the extent that the disabled community is being used as something of a stalking horse or Judas goat for driving the DRE computer industry...
[55:13.590 --> 55:20.230]  There's been this very interesting co-option of the disabled community by the voting machine vendors.
[55:20.230 --> 55:20.350]  Right.
[55:20.470 --> 55:26.050]  And my point is that there's a certain amount of hardening of attitudes on both sides.
[55:26.050 --> 55:37.170]  And it's perhaps time, again, to consider convening a more conciliatory set of workshops between those two communities so that we can find common ground.
[55:37.330 --> 55:39.610]  Because people with disabilities are citizens as well.
[55:39.770 --> 55:39.850]  Yep.
[55:39.850 --> 55:42.010]  And they want secure and accurate elections.
[55:42.150 --> 55:42.270]  Right.
[55:43.090 --> 55:43.430]  Thank you.
[55:45.630 --> 55:53.510]  You probably know about a writer named Greg Palast who did quite the expose on what went down in Florida in 2000.
[55:53.890 --> 55:55.590]  And I spoke to him yesterday.
[55:56.030 --> 56:13.720]  And he cautions that possibly an even bigger problem than I guess what you're talking about would be the equivalent of black boxes and recording the votes is now ES&S and these same companies have been hired to actually scan voter registration applications.
[56:14.380 --> 56:14.880]  Yeah.
[56:14.900 --> 56:21.380]  And they get to decide who's accepted, who isn't, and they interface it with people's social security records.
[56:21.820 --> 56:26.920]  And whether or not they suddenly have turned into an illegal immigrant even if they were born here.
[56:27.020 --> 56:27.460]  Right.
[56:27.460 --> 56:30.360]  We actually have proof of that because it's being done in Philadelphia right now.
[56:30.520 --> 56:31.820]  But not as part of this.
[56:31.940 --> 56:32.600]  Not as part of the study.
[56:32.720 --> 56:48.660]  And so as far as solutions go, one thing might be to start with Barack Obama and go all the way down the chain of people running for office and put a verbal blow torch up their ass to start talking about voting fraud and coming out against it and promising a fight.
[56:48.660 --> 56:57.500]  Let me just point out, one of the weird things that's happened is that somehow problems with voting machines got identified as a Democrat issue.
[56:58.220 --> 57:02.500]  And problems with voter registrations has been identified as a Republican issue.
[57:02.740 --> 57:09.080]  And it became kind of senselessly partisan because this doesn't actually obviously affect one party or the other.
[57:09.660 --> 57:21.620]  So, but, you know, it ended up becoming polarized both between vendors and the disabled community and also between Republicans and Democrats in ways that have made it very difficult to make progress.
[57:21.620 --> 57:24.020]  That's starting to thaw considerably.
[57:24.840 --> 57:27.720]  So, maybe we'll see some progress on that point.
[57:27.920 --> 57:31.080]  I want to also point out one other quick thing I should have mentioned before.
[57:31.320 --> 57:34.580]  One of the properties of an election is that there's a loser.
[57:35.100 --> 57:35.420]  Right?
[57:35.500 --> 57:42.880]  The point of an election is not just to figure out who won, but it's to convince the loser and the loser's supporter that they lost.
[57:43.120 --> 57:49.720]  To the extent that there are these problems with voting machines, it makes it a lot harder to convince the loser to go away.
[57:49.720 --> 57:58.800]  So, these problems, even if they're not being actually exploited, the fact that they're present is going to seriously compromise confidence in elections.
[57:59.220 --> 58:00.560]  So, I think you get the last word.
[58:00.560 --> 58:03.840]  What's wrong with the New York State voting machines?
[58:04.660 --> 58:06.800]  Simple lever machines, mechanical...
[58:06.800 --> 58:10.020]  So, the quick answer to that is New York did this smart thing.
[58:10.240 --> 58:12.380]  It never bought any Help America Boat Act machines.
[58:12.920 --> 58:13.600]  And now...
[58:13.600 --> 58:19.760]  But the downside of that is they haven't actually been manufactured for something like 45 years.
[58:20.020 --> 58:23.520]  And there's like one guy with duct tape who knows how to repair them.
[58:24.160 --> 58:27.240]  So, at some point they're going to need to replace them.
[58:27.720 --> 58:29.140]  So, anyway, thanks.
[58:29.300 --> 58:29.520]  Thanks.
[58:29.520 --> 58:29.900]  Thanks, Amy.
[58:40.530 --> 58:40.770]  Thanks.
[58:46.580 --> 58:47.020]  Wow.
[58:50.980 --> 58:58.320]  In a few minutes will be our next discussion called Wikipedia, You Will Never Find a More Wretched And this is a haunted hive of scum and villainy.