[00:00.000 --> 00:00.320]  Hang around.
[00:02.260 --> 00:11.340]  All right, but without further ado, it's my great pleasure to introduce the panel, Crypto War II, Updates from the Trenches.
[00:11.820 --> 00:13.880]  Please give it up for Matt Blaze,
[00:17.680 --> 00:20.520]  as well as Sandy Clark.
[00:23.320 --> 00:23.920]  Okay.
[00:25.440 --> 00:27.540]  So, thanks, everybody.
[00:31.060 --> 00:36.980]  I was going to have to introduce myself, but fortunately he did that, so I don't have to remember anything about me.
[00:38.060 --> 01:07.840]  All right, so this is basically a talk about the crypto wars that are going on right now, and kind of an update on some of the different technical things that are going on, and in particular some of the issues that are likely to emerge going forward as we sort of blunder between the political process and the technical process.
[01:08.020 --> 01:13.940]  How many of you, just by a quick show of hands, saw any of the Republican Convention last night?
[01:14.020 --> 01:14.620]  So a bunch of you did.
[01:14.800 --> 01:24.200]  You might have noticed, you know, aside from the words said at it, you might have noticed that that is a very different process than the way we do technical talks.
[01:24.900 --> 01:30.660]  You know, you could very easily say, oh, I am not watching a technical conference on C-SPAN.
[01:30.780 --> 01:51.880]  So there's sort of an impedance mismatch between what the policy community thinks of as the way of resolving controversial issues, which is, let's build a consensus, let's balance interests, and the way the technology resolves issues, which is, let's figure out who's right.
[01:52.780 --> 02:04.100]  And both of those have value for different things, and crypto is one of those things that for the last 25 years, those two approaches have been clashing head-on.
[02:04.100 --> 02:06.560]  So what is this about?
[02:07.400 --> 02:16.080]  Well, 20 years ago, in what we now call Crypto War 1, this was about wiretapping, specifically.
[02:16.460 --> 02:36.580]  And it was about preserving the ability of law enforcement and, to some extent, the security agencies, like the NSA, to perform wiretaps in the face of a new threat to wiretapping technology called cryptography.
[02:36.940 --> 02:52.720]  And you might argue that, well, that's not worth preserving, that the whole point of cryptography is to prevent things like law enforcement and security agencies from being able to access communication.
[02:52.720 --> 03:00.540]  But the, you know, agencies like the FBI and the NSA don't see it quite that way.
[03:01.000 --> 03:05.060]  They think of these things as being critically important for national security.
[03:05.300 --> 03:08.340]  And they have a big voice in the political process.
[03:08.700 --> 03:31.320]  And where all this started was really most visibly back in 1992, when AT&T, which ironically was my employer at the time, I worked for Bell Labs, introduced this product called the TSD 3600, because AT&T comes up with very catchy names for their products.
[03:31.980 --> 03:34.660]  TSD stands for Telephone Security Device.
[03:35.060 --> 03:45.490]  And this is, this was basically a device, a little hardware box, that would sit between your telephone base and your telephone handset.
[03:46.020 --> 03:52.300]  This was back in the days when telephones were wired and had a handset, a separate handset.
[03:52.440 --> 03:54.120]  Ask your parents about that.
[03:54.860 --> 04:11.770]  The, and this was a little box that would sit in between the phone and the handset, and do something that a consumer device available to the public had never actually been able to do before.
[04:12.090 --> 04:16.810]  Which was encrypt digitally the contents of the phone call.
[04:16.990 --> 04:22.570]  And it was actually designed by the same group in AT&T that builds secure phones for the government.
[04:22.870 --> 04:27.350]  So it was, you know, it had pretty good hardware design and a fairly well thought out crypto design.
[04:27.350 --> 04:36.930]  And basically what it would do is cut off the handset when you pressed the red button on the thing.
[04:37.430 --> 04:43.810]  And start a modem connection to another phone that also had one of these boxes.
[04:44.250 --> 04:51.890]  And once you've started a modem connection, you've now got a digital communications link with your other side instead of an analog communications link.
[04:51.890 --> 04:57.990]  And cryptography is really easy, relatively speaking, to do on digital data.
[04:58.170 --> 05:02.150]  So this device would basically digitize your phone audio.
[05:02.630 --> 05:06.590]  And do a Diffie-Hellman key exchange with the other device.
[05:07.050 --> 05:09.690]  Display a hash of the key.
[05:10.410 --> 05:15.570]  And that will be important for ensuring that there's no man-in-the-middle attack.
[05:15.570 --> 05:22.690]  And then session encrypt with the data encryption standard.
[05:22.970 --> 05:26.990]  Which was the state of the art cipher that was available at the time.
[05:28.110 --> 05:35.130]  Now, so basically this device would DES encrypt phone calls.
[05:35.130 --> 05:42.790]  And when this came out, AT&T basically sold them for $1,400 each.
[05:43.810 --> 05:45.750]  And you have to buy two of them.
[05:46.050 --> 05:49.370]  So that's... because otherwise you're just talking to yourself.
[05:49.910 --> 05:57.610]  So you can imagine that this was not really a huge commercial success in 1992.
[05:57.610 --> 06:01.290]  But it really got the government's attention.
[06:01.290 --> 06:11.510]  The government, I think, a calm way to put it would be that the government had a complete freakout when this came out.
[06:11.610 --> 06:15.770]  Because they understood that the price of technology tends to go down.
[06:15.990 --> 06:26.450]  And that this ability to encrypt phone calls would eventually become something that didn't cost $1,400 a pop.
[06:26.650 --> 06:29.790]  But rather would be something that would be easily done in everything.
[06:29.790 --> 06:34.910]  And they worried that this would cause wiretaps to basically become obsolete.
[06:35.190 --> 06:36.490]  And this was in 1992.
[06:36.490 --> 06:43.050]  They predicted that, you know, within a few years, this device will make wiretapping completely useless.
[06:43.430 --> 06:44.590]  So what happened?
[06:44.770 --> 06:58.870]  Well, basically the President of the United States called the President of AT&T and said, We really don't like this thing.
[06:59.930 --> 07:08.250]  And maybe you can be persuaded to build a device that allows for wiretapping.
[07:08.390 --> 07:10.370]  And here's what we propose that you do.
[07:10.630 --> 07:14.190]  Rip out the DES chip that's in these devices.
[07:14.190 --> 07:20.690]  And instead replaced them with a new government designed chip called the Clipper chip.
[07:21.270 --> 07:25.250]  Or the Microtronix MYK78T.
[07:25.910 --> 07:35.450]  And this was basically a drop-in replacement for DES that introduced a new and actually more secure cipher algorithm.
[07:35.450 --> 07:38.450]  That had one little asterisk next to it.
[07:39.010 --> 07:45.350]  Which is that it implemented a new feature called key escrow.
[07:46.090 --> 07:57.790]  And what key escrow meant was that when you send the initialization vector to the other device, which is part of the crypto exchange, what you also send is an encrypted copy of the session key.
[07:57.790 --> 08:04.850]  Encrypted with a key that's held, split into two places, but that was a detail, by government agencies.
[08:05.190 --> 08:12.650]  And that could be decrypted if there is a court order presented to the agencies that hold the device.
[08:13.290 --> 08:28.510]  And AT&T said, You want us to take all of these devices, rip out the DES chip, re-engineer it to include this new Clipper chip, and you want us to do that and pull this device off the market that we've been working so hard on?
[08:29.130 --> 08:29.790]  Okay, sure.
[08:31.970 --> 08:37.270]  And the carrot was that the government said, and if you do this, we'll buy a lot of them.
[08:37.590 --> 08:42.590]  And so basically the AT&T did this, and the government bought pretty much all of them.
[08:43.550 --> 08:49.150]  Right, except for... and they've now reached the surplus market, and you can get one now too for about 50 bucks.
[08:51.330 --> 08:53.650]  So Clipper was pretty controversial.
[08:54.730 --> 08:58.150]  Those of you around probably remember the Clipper chip controversy.
[08:58.150 --> 09:03.790]  It became sort of emblematic of this conflict between the technology world and the government world.
[09:04.430 --> 09:07.050]  And, you know, Clipper had a number of problems.
[09:07.510 --> 09:12.130]  One of the problems was that the key escrow feature could be easily defeated.
[09:12.750 --> 09:25.610]  And I worked for AT&T at the time, and sort of to the embarrassment of people in the division that made the Clipper phone, I looked at the sort of protocols for this and so on, which were published.
[09:25.830 --> 09:32.370]  And I worked in the Bell Labs part of AT&T, which causes problems rather than solving them.
[09:32.370 --> 09:41.230]  And found a way to defeat the key escrow feature while still using the cipher.
[09:41.790 --> 09:47.130]  And that would basically make the whole point of this thing fairly useless.
[09:47.430 --> 09:50.190]  And that kind of killed Clipper.
[09:50.830 --> 09:55.610]  But Clipper, that wasn't actually the problem with Clipper.
[09:55.610 --> 10:10.010]  They could have redesigned it a little bit and made some small changes to this to make it harder to defeat the escrow feature and have it work properly.
[10:10.190 --> 10:12.110]  But it still had other fundamental problems.
[10:12.270 --> 10:20.110]  So one of the most obvious fundamental problems is should we trust the government with these keys?
[10:20.110 --> 10:25.030]  But there's another, you know, should we trust them even if they're doing their job properly?
[10:26.110 --> 10:32.210]  Should we trust the system itself to hold these keys?
[10:32.670 --> 10:50.090]  And should we actually require designers of things like this, that is, anything that uses encryption, to include key escrow features when that might make the design more cumbersome?
[10:50.270 --> 10:59.810]  Particularly at a time in history when cryptography in software, which has zero marginal cost, was just starting to become practical.
[11:00.510 --> 11:23.950]  And basically we spent the entire rest of the decade arguing between the technical community that said, look, no matter how well you design this system, it's going to make cryptography weaker and it's going to make it more expensive and it's going to discourage the deployment of cryptography in what is ultimately going to become our critical infrastructure.
[11:24.550 --> 11:36.150]  And until the end of 1999, cryptography was officially discouraged by the U.S. government in consumer products.
[11:36.150 --> 11:39.510]  It was considered a sort of suspect technology.
[11:40.130 --> 11:52.930]  And we didn't include cryptography in a lot of the basic standards that we're relying on today to run the Internet.
[11:52.930 --> 12:02.050]  We were designing a lot of those basic standards back in the 1990s when the government was very loudly saying, don't include cryptography, that's only for bad guys.
[12:02.050 --> 12:06.930]  At the end of 1999, the government abruptly said, oh, you know what?
[12:07.010 --> 12:07.530]  We were wrong.
[12:08.050 --> 12:09.510]  Go use cryptography.
[12:09.610 --> 12:10.750]  Forget about these restrictions.
[12:11.650 --> 12:13.510]  You know, cryptography is important.
[12:14.270 --> 12:28.790]  And Crypto War One, as we've come to call it before we knew to number the crypto wars, came to an abrupt end with this sort of unilateral surrender.
[12:29.530 --> 12:31.890]  And that brings us to Crypto War Two.
[12:32.270 --> 12:35.270]  And you might think Crypto War Two started in earnest.
[12:35.610 --> 12:39.980]  When would you imagine Crypto War Two would have started if something happened in the year 2000?
[12:40.870 --> 12:46.990]  Was there any large event shortly after the year 2000 that might have affected some of this that you can think of?
[12:47.210 --> 12:49.670]  What year would you say that event might have happened in?
[12:50.250 --> 12:53.170]  2001, say September-ish, you know, in the autumn?
[12:53.170 --> 12:53.590]  Right.
[12:53.890 --> 12:56.330]  So that turned out not to really change much.
[12:56.570 --> 13:01.810]  There was an initial relatively half-hearted attempt to put crypto restrictions in the Patriot Act.
[13:02.010 --> 13:03.710]  But it didn't move forward.
[13:04.930 --> 13:13.050]  The government probably could have, the FBI and the NSA probably could have gotten that included in the Patriot Act if they had pushed harder, but they didn't.
[13:13.050 --> 13:25.370]  So we recognized very soon thereafter that cryptography was important enough that we actually need to start encouraging it as a matter of national policy.
[13:26.850 --> 13:37.830]  This piece, uneasy piece, the Crypto Cold War, continued on until about 2010 to 2014, somewhere in that range.
[13:37.830 --> 13:58.430]  When the director of the FBI, the current director, James Comey, started making statements in congressional testimony and to the press and so on, talking about a new problem that was remarkably similar to the problem that they talked about when the AT&T phone was introduced in the early 1990s called Going Dark.
[13:58.430 --> 14:06.430]  And basically, the way that they describe this is, unfortunately, the law hasn't kept pace with technology.
[14:06.430 --> 14:10.430]  And this disconnect has created a significant public policy problem.
[14:10.650 --> 14:12.650]  We call it Going Dark.
[14:12.990 --> 14:14.570]  And what it means is this.
[14:15.170 --> 14:17.470]  Those charged... I'm adding some dramatic emphasis.
[14:17.730 --> 14:27.690]  Those charged with protecting our people aren't always able to access the evidence we need to prosecute crime and prevent terrorism, even with lawful authority.
[14:27.690 --> 14:33.030]  We have the legal authority and intercept and access communication and information pursuant to court order.
[14:33.170 --> 14:36.190]  But we often lack the technical ability to do so.
[14:36.350 --> 14:41.690]  So this was basically repeating all of the things that were said in the early 1990s.
[14:42.670 --> 14:46.390]  So basically, Comey is saying this in congressional testimony.
[14:46.390 --> 14:59.970]  He's putting a lot of the chips that the FBI has behind this priority of figuring out a way to break cryptography.
[15:00.550 --> 15:03.970]  And we think that this would mean wiretapping is shut down.
[15:04.170 --> 15:14.650]  Well, if we take a look at the recent wiretap reports, basically, the amount of cryptography actually encountered in wiretaps rounds down to zero.
[15:15.930 --> 15:17.930]  So it looks as if...
[15:17.930 --> 15:29.770]  You know, they're talking about this going dark, but when we look at what actually happens in wiretaps, there's remarkably little encryption actually being encountered in wiretaps.
[15:30.870 --> 15:32.790]  So what's going on here?
[15:33.110 --> 15:33.310]  Right?
[15:33.490 --> 15:35.210]  How do we reconcile these two facts?
[15:35.290 --> 15:42.010]  So one possibility, and, you know, sort of techno-libertarians like to say, well, it's just that the FBI is lying, jackbooted thugs.
[15:42.350 --> 15:46.810]  They want to take our modems or whatever it is.
[15:47.030 --> 15:51.630]  They can take my DES and my AES out of my cold, dead hands, etc.
[15:52.830 --> 15:59.110]  Or, you know, it's possible that we're talking across purposes a little bit and about slightly different things.
[15:59.310 --> 16:05.010]  So when we think of wiretapping, we think of cryptography often in the context of wiretapping.
[16:05.010 --> 16:12.690]  And this is basically a picture from IEEE Security and Privacy that they used for the cover of an issue on surveillance.
[16:13.270 --> 16:16.490]  And they showed, you know, they asked an artist to draw a picture of wiretapping.
[16:16.630 --> 16:19.790]  And this is what wiretapping looks like according to the IEEE.
[16:20.530 --> 16:30.390]  Somebody in a van and overdressed for the desert going up to a box with headphones and dark glasses.
[16:30.950 --> 16:34.590]  And we can tell it wasn't an engineer who drew this because these are power lines.
[16:37.970 --> 16:55.450]  But basically what wiretapping is, you know, this is, although some of the technical details are wrong, the basic idea that you use the phone infrastructure, the communications infrastructure, to get your data is correct.
[16:55.830 --> 17:02.030]  This is basically how surveillance, electronic surveillance of communications works.
[17:02.570 --> 17:07.950]  The infrastructure itself, the phone lines, the data lines, what have you, are tapped.
[17:07.950 --> 17:10.610]  And that's where you get the clear text of communication.
[17:10.870 --> 17:19.050]  Back in previous generations, you'd use this stuff called a loop extender and a little dial number recorder and so on.
[17:19.990 --> 17:24.390]  But the cases that we're seeing aren't actually about communication.
[17:24.390 --> 17:52.410]  And if we look at the Apple versus FBI case, where after the San Bernardino shooting, the government was talking about an iPhone from one of the shooters that had data stored on it, encrypted in a way that the government couldn't get to it to collect whatever evidence might have been on it.
[17:52.410 --> 18:00.130]  And they asked Apple if Apple could assist them in recovering this data.
[18:00.270 --> 18:03.110]  Apple said, basically, we can't do this easily.
[18:03.830 --> 18:26.390]  And this started both a technical and a policy battle that was ultimately ended only after a third-party company that hasn't been definitively identified approached the FBI and said, you know, we can build you a solution that can reverse engineer out the key material from this phone for the particular model of iPhone,
[18:26.710 --> 18:31.090]  and basically mooted the issue.
[18:31.350 --> 18:39.090]  So whether or not Apple could be compelled to do that, if they could, was never actually resolved legally.
[18:39.330 --> 18:43.750]  But you'll notice that this case was not actually about the contents of communication.
[18:43.750 --> 18:47.870]  It was rather about data that was stored on a device.
[18:48.230 --> 19:11.310]  And that's where we are seeing most of the actual examples, not from the wiretaps that we've seen, where there's good reporting that's showing there's very little encryption actually out there, but rather devices that are being seized from people.
[19:11.510 --> 19:19.230]  And if cryptography works well, it will prevent somebody who gets hold of that device from being able to get to the data on it.
[19:19.350 --> 19:21.390]  That's ultimately what it's for.
[19:21.750 --> 19:23.830]  So Crypto War 1 was about communications.
[19:24.170 --> 19:30.030]  Crypto War 2 is starting to be very much, although not exclusively, about storage.
[19:31.090 --> 19:33.410]  So I want to talk about failure a little bit.
[19:33.410 --> 19:47.370]  As a computer security specialist, and I know a lot of people in this room are also computer security specialists, I have the unique privilege of being in the one branch of computer science that's making negative progress.
[19:48.150 --> 19:50.950]  Things are actually getting worse rather than better.
[19:51.270 --> 19:55.290]  And we fail at almost every layer of abstraction.
[19:55.290 --> 20:01.050]  We occasionally fail with the cryptography itself, with the algorithms and the protocols.
[20:01.050 --> 20:08.930]  We often fail with the implementation of these algorithms, with the code itself.
[20:09.270 --> 20:17.610]  And we almost always fail when we put systems together as a whole, and when we look at how they are actually installed and used.
[20:17.730 --> 20:32.110]  By the time we go from a conceptual cryptography application to an actual system that uses it, we almost always find that there was some unanticipated flaw.
[20:32.770 --> 20:48.590]  And this is the reason that what the government is asking for, which is cryptography with an added flaw, is something that scares a lot of people in the technical community quite a bit.
[20:48.590 --> 20:59.730]  because this is essentially saying, you know, the only people on the planet who think that computer security is too good are the FBI.
[21:03.230 --> 21:08.970]  So back doors of the kind that the government have been asking for are pretty dangerous and ineffective.
[21:08.970 --> 21:20.050]  So one of the reasons for this is that we are currently in what could only be characterized as a cyber security crisis.
[21:20.390 --> 21:29.270]  And I'm sorry I used the C word, but in the policy circles they actually think that cyber security has meaning.
[21:29.270 --> 21:48.170]  And we are arguably, you know, in a situation where we are seeing failure of our critical infrastructure and the systems that we depend on for finance, for our personal data, and actually for the government itself literally on a daily basis.
[21:48.350 --> 21:49.970]  And what do we do about it?
[21:50.030 --> 21:56.270]  Well, we have basically two tools that address these things.
[21:56.270 --> 21:59.730]  The fact that we don't know how to build systems that are secure.
[22:00.010 --> 22:02.150]  One of them is the use of cryptography.
[22:03.050 --> 22:07.950]  And that's useful because it allows us to reduce the attack surface of the system.
[22:08.110 --> 22:11.250]  It allows us to trust fewer components even if they do fail.
[22:11.670 --> 22:16.390]  And the other is to make systems as simple and have as few extra features as possible.
[22:16.930 --> 22:29.330]  And unfortunately, a back door basically attacks both of the only two tools that we really have for building high-reliability systems at scale.
[22:29.810 --> 22:31.690]  Crypto and simplicity.
[22:31.690 --> 22:33.810]  It makes both of those things weaker.
[22:34.330 --> 22:35.790]  Okay, so where does that leave us?
[22:35.970 --> 22:46.630]  So a bad answer to this question is saying that, well, it means that society has to pick between law enforcement and security.
[22:47.410 --> 22:54.730]  And that's going to be a fairly hard and unreliable way to approach this.
[22:54.730 --> 23:02.930]  So we've basically got, from a policy-making point of view, two problems that I've distilled into sort of policy speak.
[23:03.190 --> 23:06.990]  The first problem is that we can't afford more security vulnerabilities.
[23:06.990 --> 23:09.990]  Our systems are failing left and right.
[23:10.210 --> 23:30.370]  And the second problem is that access, which has come to be called lawful access, whether you like it or not, is maybe or is or maybe will be getting harder, partly because of the proliferation, and yes, they use arms control language, of cryptography.
[23:30.810 --> 23:40.210]  And one way to resolve this that technologists often favor is to say, well, problem one is the only important problem.
[23:41.690 --> 23:44.690]  But the FBI definitely doesn't see it that way.
[23:44.870 --> 23:49.290]  And they have the ears of policymakers who are empowered to do things like pass laws.
[23:49.550 --> 23:51.770]  And some of those laws we may not like.
[23:52.790 --> 23:56.690]  And so one question is, can we solve these two problems simultaneously?
[23:58.490 --> 24:13.850]  So what I'd like to look at is the technology, some of the technology briefly, about what we will do if cryptography wins and cryptography starts to proliferate, what is actually going to happen?
[24:14.090 --> 24:21.470]  Now, we've seen some evidence that the FBI is not actually going dark.
[24:21.510 --> 24:29.470]  If we look at the Apple San Bernardino case, we saw that one of the things that the FBI did was hacked into the device.
[24:29.890 --> 24:32.210]  They didn't actually break the crypto algorithms.
[24:32.450 --> 24:35.190]  They didn't use a deliberate backdoor feature.
[24:35.450 --> 24:41.230]  They did what lots of us do, which is say, was something wrong with this?
[24:41.550 --> 24:43.530]  And find something wrong with this.
[24:43.930 --> 24:45.070]  And they found it.
[24:45.230 --> 24:47.350]  Who is surprised that they found it?
[24:47.870 --> 24:49.130]  Yeah, probably nobody.
[24:52.450 --> 25:05.030]  So what we're seeing is an increase in the use, a quiet increase in the use of exploitation of devices, rather than infrastructure-based wiretapping.
[25:05.430 --> 25:18.230]  So Sandy Clark, who's actually, by the way I should say, about to finish her PhD and is on the job market, and you should be, like, trying to kidnap her and get her to...
[25:18.230 --> 25:21.010]  and pay her enormous amounts of money to come work for you.
[25:21.690 --> 25:23.790]  And so you can give me a cut of that after...
[25:24.690 --> 25:25.930]  when that succeeds.
[25:26.610 --> 25:27.090]  Yeah.
[25:27.090 --> 25:53.250]  He has been studying the vulnerability life cycle, and was part of a paper that we wrote for a law review on looking at what a world in which device exploitation becomes the norm will look like, with Steve Bellevin, me, and Susan Landau, that was published in a law review,
[25:53.310 --> 25:56.390]  which is not usually where computer scientists publish papers.
[25:56.390 --> 26:14.690]  And Sandy has been looking particularly at some of the technical issues in the vulnerability life cycle as it exists, and maybe can make some predictions about how... what a world like this might look like, and we better be prepared for it, because that is likely to be the world that we end up with.
[26:14.930 --> 26:18.630]  So, Sandy, why don't you take that over while people are trying to hire you?
[26:19.890 --> 26:20.550]  Hold on.
[26:21.490 --> 26:21.970]  We're one ahead?
[26:22.410 --> 26:22.890]  Yeah.
[26:24.710 --> 26:28.430]  So, you know, Director Comey was right in one thing that he said.
[26:28.710 --> 26:32.070]  The law really has not kept up with the technology.
[26:33.970 --> 26:48.170]  But, of course, that biggest argument is that they are afraid that they are going to need access that they have the legal authorization to get, but that they haven't got the technological ability.
[26:48.650 --> 26:59.050]  And what we are arguing about here in this section of the talk is that there will always be a vulnerability that they can use.
[27:01.550 --> 27:06.250]  And this is because... we base this on two very simple observations.
[27:06.250 --> 27:08.090]  So, are you paying attention, class?
[27:08.450 --> 27:09.650]  Repeat after me.
[27:10.130 --> 27:11.150]  Observation one.
[27:11.750 --> 27:14.070]  Everything is made of software.
[27:14.350 --> 27:14.750]  Ready?
[27:15.550 --> 27:18.250]  Everything is made of software.
[27:19.050 --> 27:20.170]  Observation two.
[27:20.910 --> 27:23.010]  Software is insecure.
[27:23.790 --> 27:26.950]  And it will always be insecure.
[27:27.410 --> 27:30.230]  Now, I'm not naive here, okay?
[27:30.230 --> 27:41.550]  I recognize that it has gotten orders of complexity harder to exploit vulnerabilities than it used to be back in the Morris worm days.
[27:41.850 --> 27:47.030]  Back in the days when you found a buffer overflow and within 30 minutes you had an exploit.
[27:47.210 --> 27:54.450]  And now you find something and you've got three weeks to three months before you've got a working exploit.
[27:57.130 --> 28:07.150]  Researchers on all sides of the line with all shades of hats are bemoaning the fact that they're getting harder to find and they're getting harder to exploit.
[28:07.350 --> 28:27.790]  And you only have to look at things like Pinkie Pie's wonderful set of attacks in, what, 2012 in the Pone to Own contest, which chained together six vulnerabilities in order to get through Chrome to see this order of magnitude difficulty than it used to be.
[28:28.770 --> 28:37.830]  And we've got things now that are used all the time, like we've got fuzz testing and we've got sandboxes and someday we might even have formal verification.
[28:40.230 --> 28:51.810]  But we've also, let's see, December of 2014, all of the security-related blogs called that year the year of the breach.
[28:52.370 --> 29:00.310]  And December 2015, all of the security blogs called that year the year of the breach.
[29:00.630 --> 29:03.590]  And guess what they're saying about 2016?
[29:05.850 --> 29:08.290]  And why is this?
[29:09.890 --> 29:14.410]  Because no device that we have exists in isolation.
[29:15.370 --> 29:18.110]  We're not talking about a program.
[29:18.970 --> 29:25.330]  We're not talking about a single piece of hardware or a single piece of firmware.
[29:25.550 --> 29:33.290]  We're talking about an actual ecosystem with a bunch of interacting organisms, some of which are beneficial and some of which are hostile.
[29:33.790 --> 29:39.970]  And it's so complex that we don't really understand it at all and we sure as hell don't know how to secure it.
[29:43.390 --> 29:48.270]  Not only that, every device we've got is now connected to everything else.
[29:48.710 --> 29:54.990]  Your smart refrigerator is talking to your smart car so that you don't forget to buy cheese on the way home.
[29:55.210 --> 29:59.410]  But it's also talking to your fitness device so that you don't eat too much of it.
[30:02.210 --> 30:10.290]  And it's not just the fact that you've got this one device or this one piece of software, this one operating system by this one developer.
[30:10.830 --> 30:15.970]  But all of those contain shared libraries.
[30:16.890 --> 30:19.570]  Legacy code, which is a real problem.
[30:20.450 --> 30:22.630]  We'll talk more about that in just a second.
[30:23.250 --> 30:27.470]  They contain massive third-party apps.
[30:27.470 --> 30:30.290]  And they're all configured differently.
[30:31.090 --> 30:36.810]  Every single one of those adds complexity and possibly another attack vector.
[30:37.750 --> 30:39.530]  And you only need one.
[30:40.330 --> 30:42.150]  You only need one.
[30:43.790 --> 30:48.210]  But one of the arguments is that it's ODAs that they want.
[30:48.470 --> 30:52.670]  It's got to be something that no one knows about and nobody protects.
[30:52.690 --> 30:55.850]  And I would argue that that is not the case.
[30:55.850 --> 30:59.250]  But that ODAs are getting harder to find.
[30:59.590 --> 31:02.110]  And, of course, they're getting much more expensive.
[31:03.890 --> 31:07.550]  But if you actually look at the numbers, and I'm a scientist.
[31:07.550 --> 31:08.570]  I like numbers.
[31:10.230 --> 31:14.690]  Here's what the numbers of ODAs looked like in 2012.
[31:17.610 --> 31:20.510]  And here's what they looked like last year.
[31:28.780 --> 31:29.820]  We've got...
[31:29.820 --> 31:35.800]  Just for a second, let's go back to ODAs in legacy code.
[31:37.240 --> 31:38.780]  We have Shellshock.
[31:38.920 --> 31:39.980]  We have Heartbleed.
[31:40.080 --> 31:40.980]  We have Poodles.
[31:40.980 --> 31:48.460]  And if you were paying attention last week, we have a Windows printer attack that has sat dormant in Windows since Windows 1995.
[31:49.440 --> 31:55.600]  And if you're running Windows on any of your systems and you haven't patched it in the past week, you're f*cked.
[32:01.180 --> 32:03.800]  That'll be bleeped for the TV version, yeah.
[32:06.450 --> 32:13.680]  Last year, according to Secunia, there were 16,081 vulnerabilities disclosed.
[32:13.680 --> 32:18.480]  Of those, 84% were patched on the day of disclosure.
[32:18.820 --> 32:27.120]  Now, that does not mean that they weren't known about by people outside of the developers before they were patched.
[32:27.120 --> 32:29.780]  Because they very well could have been but kept secret for a while.
[32:30.320 --> 32:40.480]  A lot of the companies that find and develop exploits nowadays sell them privately for a while and then report them patched.
[32:40.480 --> 32:42.500]  I don't want to talk to Vupen anyone.
[32:44.420 --> 33:03.780]  But even if the developers knew about these vulnerabilities, that still leaves 2,500 vulnerabilities for which there was no patch when they were released.
[33:06.520 --> 33:10.460]  Of those, about 1,100 of them were browsers.
[33:12.140 --> 33:14.840]  Chrome, Firefox, and Safari predominantly.
[33:15.560 --> 33:17.260]  Oh, and Internet Explorer, sorry.
[33:18.760 --> 33:20.200]  Right, I'm going to forget that one.
[33:21.180 --> 33:32.800]  Of those 2,500 vulnerabilities, of the ones that affected Microsoft, 79% of those affected third-party applications, not Microsoft code directly.
[33:34.540 --> 33:36.400]  So this is an ecosystem problem.
[33:37.200 --> 33:38.540]  I'm not going to talk much more about that.
[33:38.640 --> 33:40.120]  If you want to know, you can read my dissertation.
[33:44.100 --> 33:53.320]  But the key here is that these are available for our law enforcement people to use when they need to.
[33:53.320 --> 33:57.580]  But how do we do that as technologists and still protect our systems?
[34:00.600 --> 34:02.800]  There are some technical issues here.
[34:03.000 --> 34:13.000]  And one of the attacks that we get from our three-letter agencies is that, yes, they could develop an exploit, but there's no guarantee that it would work.
[34:13.000 --> 34:24.240]  Well, there's no guarantee that their non-exploit, non-computer, non-device-related techniques work in an investigation either.
[34:24.240 --> 34:31.560]  So it's irrational for them to expect that level of functionality.
[34:33.620 --> 34:46.440]  Anything that our law enforcement agencies do develop for this purpose, they need to be developed in a modular and adaptable sort of architecture.
[34:48.120 --> 34:59.000]  Because one of the essentials to protect the population of the world is that we actually fix whatever vulnerabilities are out there when we become aware of them.
[34:59.920 --> 35:05.520]  And so none of these vulnerabilities should actually have that long of a lifetime.
[35:05.520 --> 35:16.960]  And so there needs to be the ability to rope in a different payload into your dropper or switch to a different type of attack.
[35:21.300 --> 35:26.540]  These...any sort of vulnerability that they're going to use has to be carefully targeted.
[35:28.080 --> 35:33.200]  It has to collect only the information that the wiretap allows.
[35:33.540 --> 35:40.060]  And it has to collect it only on the person specified, person or people specified in the warrant.
[35:41.940 --> 35:51.480]  It also, like many other parts of surveillance or a criminal investigation, is going to involve reconnaissance.
[35:51.480 --> 35:55.040]  It's going to involve a lot of setup work.
[35:56.260 --> 35:59.940]  But so does anything else they do.
[36:00.120 --> 36:01.300]  And how are they going to do that?
[36:01.420 --> 36:02.660]  The same way that we do, right?
[36:03.380 --> 36:05.480]  You follow the person around for a while.
[36:05.660 --> 36:07.140]  You find out what device they're using.
[36:12.460 --> 36:15.540]  They...there are multiple ways to do this.
[36:15.780 --> 36:19.460]  We just need to work through how we do it and how we protect it.
[36:19.460 --> 36:21.780]  And of course, they're already doing it.
[36:22.460 --> 36:25.460]  As Matt mentioned, there was the iPhone case.
[36:26.400 --> 36:28.220]  But there's also the Silk Road investigation.
[36:28.240 --> 36:31.640]  If you're not familiar with it, you should read up on...on the techniques that they used.
[36:31.740 --> 36:32.520]  There will be a talk here.
[36:32.840 --> 36:33.540]  Oh, that's right.
[36:34.260 --> 36:44.220]  So definitely attend that talk and it will give you some more insight into the...the hacking that our law enforcement agencies are already doing.
[36:45.980 --> 36:48.140]  And...and another very recent one.
[36:48.240 --> 36:52.080]  Have you been paying attention to the Kick-Ass Torrance takedown?
[36:52.300 --> 36:53.620]  That was yesterday.
[36:56.160 --> 36:57.420]  Um...here's what they did.
[36:58.200 --> 37:09.060]  Here...here are the really involved technical tools that they did to...to get the information that they needed for their affidavit and for their arrest.
[37:09.860 --> 37:12.280]  They used the Wayback Machine.
[37:12.560 --> 37:19.820]  And they went...and they found all of the DNS names that Kick-Ass Torrance servers were registered to.
[37:20.380 --> 37:22.180]  And then they used WHOIS.
[37:22.420 --> 37:25.680]  And they found out who hosted those machines.
[37:26.000 --> 37:27.680]  And they found those IP addresses.
[37:28.060 --> 37:30.200]  And then they used social engineering.
[37:31.520 --> 37:35.020]  Kick-Ass Torrance made its money by selling advertising.
[37:35.380 --> 37:37.700]  If you ever have...um...used a torrent.
[37:37.800 --> 37:38.980]  And I'm sure none of you ever have.
[37:39.700 --> 37:42.720]  You'll notice that...that there's the slow way to download.
[37:42.980 --> 37:44.580]  And there's the...you want a faster way?
[37:44.700 --> 37:45.220]  Click here.
[37:46.720 --> 37:47.720]  Um...those...want a faster way?
[37:47.840 --> 37:51.560]  Click here's...are advertising and it's paid for.
[37:51.560 --> 37:57.720]  So the FBI created a...um...advert for...want a study in the U.S.?
[37:58.060 --> 37:58.840]  Click here.
[38:01.740 --> 38:04.080]  Um...and...they...they paid for five days worth of advertising.
[38:05.120 --> 38:07.440]  And on...this was not a watering hole attack.
[38:07.700 --> 38:11.160]  They did not include in...in this any sort of malware whatsoever.
[38:11.540 --> 38:16.840]  They simply...used it to get the banking information that they needed.
[38:16.840 --> 38:18.660]  Because where...where were they gonna pay?
[38:21.160 --> 38:26.160]  Um...and...from that they were able to eventually dig out...um...email addresses.
[38:26.400 --> 38:29.660]  And they were able to tie those email addresses into a particular person.
[38:31.180 --> 38:34.240]  Because criminals, like everybody else, make mistakes.
[38:39.920 --> 38:42.200]  So...what do we do to protect ourselves?
[38:42.400 --> 38:44.980]  What are the technical issues that we might have to worry about?
[38:48.140 --> 38:50.600]  Um...first, that argument that it...there's no guarantees.
[38:51.160 --> 38:51.640]  Okay.
[38:51.860 --> 38:53.160]  We've talked about that one.
[38:54.740 --> 38:56.620]  The next two are the most important.
[38:57.540 --> 39:02.040]  The risk of detection and the risk of proliferation are real dangers.
[39:04.400 --> 39:11.460]  A few years ago, there was a very sophisticated wire tap on a Greek phone system.
[39:13.000 --> 39:15.700]  And it was against a cell phone operator.
[39:17.620 --> 39:23.680]  And...it was found out because there was a bug in the actual attack software that was installed on the phones.
[39:24.260 --> 39:28.300]  And the bug...caused some text messages not to be delivered.
[39:28.840 --> 39:32.240]  And those text messages...gave off error messages.
[39:32.240 --> 39:38.680]  And the error messages allowed them to find that there was malware on their phones.
[39:38.920 --> 39:41.040]  And then it was tracked to law enforcement.
[39:43.000 --> 39:45.980]  Um...the moral of the story is, you know, don't document your mistakes.
[39:49.760 --> 40:01.340]  But...but this led to some...a very bad, um, um, um, odour for the people who...who were involved in that, um...in that project, in that operation.
[40:02.380 --> 40:08.820]  And the risk of proliferation can be described in one word.
[40:08.820 --> 40:10.220]  Stuxnet.
[40:12.600 --> 40:18.620]  This was probably the most carefully designed piece of malware ever.
[40:20.560 --> 40:26.360]  It had a very small, very restricted in scope target.
[40:26.720 --> 40:28.680]  And it still got out.
[40:29.580 --> 40:44.940]  The belief that we can put a back door in a system, and still protect that system...is...is...is...a Darwin award-winning level of...of hubris.
[40:46.400 --> 40:49.120]  Um...we can't do security right.
[40:49.800 --> 40:55.480]  To...to assume that you will be able to create something perfect.
[40:56.460 --> 41:02.140]  Or...to underestimate your adversary...is...probably the most dangerous thing that you can do.
[41:03.860 --> 41:07.340]  No matter how smart you are...somebody is smarter.
[41:07.800 --> 41:11.140]  And somebody is going to create...to see something that you didn't see.
[41:11.300 --> 41:21.400]  Because we cannot write a piece of software...design a system...connect it to any other systems...or use it...without making certain assumptions.
[41:22.720 --> 41:27.300]  And every successful exploit in the world is a violation of those assumptions.
[41:29.420 --> 41:31.220]  And it's always going to be that way.
[41:36.960 --> 41:43.400]  So...designing an exploit...the way to make sure that even something like proliferation or detection...
[41:45.840 --> 41:53.160]  damages...does the least amount of damage...is that we put some sort of time...delay...or time...time bomb in it.
[41:53.440 --> 41:58.520]  So that it...it stops at a certain...after a certain period and is no longer...um...valid.
[41:58.640 --> 42:00.320]  And no longer makes people vulnerable.
[42:00.820 --> 42:04.740]  That's pretty...a self-destruct...I think is pretty much the only way that we can do that.
[42:05.980 --> 42:08.820]  And the last argument is that this is expensive.
[42:11.460 --> 42:12.940]  So they don't want to do it.
[42:15.780 --> 42:17.540]  But it should be expensive.
[42:18.720 --> 42:20.840]  It has to be difficult to do.
[42:21.100 --> 42:22.300]  It cannot scale.
[42:23.900 --> 42:27.440]  Otherwise, we have the hoovering up of everyone's information...
[42:27.440 --> 42:30.500]  and then never...um...letting it go away.
[42:30.740 --> 42:36.040]  We've got the collection...and surveillance of innocent people.
[42:36.620 --> 42:43.120]  And we've got...all of the dangers that...that...that could possibly create.
[42:43.540 --> 42:52.760]  We want...a rule of law...where those with power...have checks and balances.
[42:54.200 --> 43:03.320]  And in...in the case of...exploiting software, hardware, firmware, whatever...expense is that check and balance.
[43:06.150 --> 43:10.410]  And the most important of those is that it must not scale.
[43:12.370 --> 43:14.850]  And there are ways to make it not scale.
[43:16.090 --> 43:22.450]  Have you read about the...the recent 3D printing of a finger in order to unlock an iPhone?
[43:23.570 --> 43:25.270]  That's a beautiful example.
[43:25.410 --> 43:26.450]  Let's do more of that.
[43:26.970 --> 43:28.350]  It's only one finger.
[43:29.310 --> 43:32.150]  And all the other people who use iPhones are still safe.
[43:33.870 --> 43:35.470]  So...those are the technical issues.
[43:35.950 --> 43:38.950]  But now we go back to...the policy issues.
[43:39.210 --> 43:40.510]  And that's the hard part.
[43:40.670 --> 43:40.950]  Okay.
[43:42.770 --> 43:47.810]  So...so...where that leaves us is...the...you know, there's some technical issues that...that Sandy identified.
[43:47.810 --> 43:51.790]  But it's clear that these technical issues aren't insurmountable.
[43:52.110 --> 44:00.490]  Because what we're seeing is an increase in precisely the use of exploits by our law enforcement agencies.
[44:00.950 --> 44:04.630]  It clearly does not work in every case with their current resources.
[44:05.250 --> 44:08.210]  But it clearly works in some of the cases.
[44:08.430 --> 44:09.070]  And they're doing it.
[44:09.230 --> 44:11.790]  And the interesting thing is they're doing it quietly.
[44:11.790 --> 44:28.850]  The number of times the U.S. Congress has addressed the question and passed a law about exploit use by law enforcement in wiretapping or evidence collection is zero.
[44:30.430 --> 44:37.830]  There has been a change called Rule 41, if you are interested in the rules... Federal Rules of Criminal Procedure.
[44:38.030 --> 44:49.830]  That was basically an administrative change that allowed the use of malware with a simple court order on large scale.
[44:50.210 --> 44:53.970]  And it was sort of quietly done without actually Congress being involved.
[44:53.970 --> 45:00.510]  So one of the things is that this world that Sandy described is pretty scary.
[45:00.850 --> 45:04.130]  But it's actually the world that we're moving into right now.
[45:04.330 --> 45:11.490]  And we're moving into that world with very, very little public discussion of what the implications of that are.
[45:11.610 --> 45:15.350]  And of what we should be doing moving forward.
[45:15.750 --> 45:19.110]  So there's some harder than the technical issues are the policy issues.
[45:19.110 --> 45:30.150]  Like, what legal authority is required for a government agency to hack into a device or to hack into the computer or endpoint used by somebody?
[45:30.290 --> 45:31.970]  Is it a wiretap order?
[45:32.130 --> 45:34.690]  Is it a regular search warrant?
[45:34.910 --> 45:37.070]  Is it an administrative subpoena?
[45:40.870 --> 45:43.650]  Who decides whether it's allowed?
[45:43.650 --> 45:49.630]  And that's largely an unsettled legal question right now, which is pretty scary given that it's being done.
[45:51.090 --> 45:53.410]  Where should the expertise for this live?
[45:53.570 --> 45:55.570]  There are lots of agencies that collect evidence.
[45:55.790 --> 45:58.870]  Some of them are things like the FBI that have large resources.
[45:59.390 --> 46:01.250]  Some of them are state and local agencies.
[46:01.770 --> 46:08.410]  Should there be some central clearinghouse for this, like the FBI crime lab?
[46:08.530 --> 46:11.550]  Or should every agency be left on its own to figure this out?
[46:11.550 --> 46:21.950]  Those are, again, public policy questions and cost questions that we really haven't discussed in either the political world or the technical world.
[46:22.330 --> 46:27.490]  There's what's called... this is a word that I learned in the crypto wars... equities.
[46:27.730 --> 46:32.330]  What does equities mean in the context of cryptography and computer security?
[46:32.650 --> 46:34.830]  Balancing one set of interests against another.
[46:34.830 --> 46:42.270]  And here there's an equities question of the government discovers a vulnerability in something used by a target.
[46:42.530 --> 46:45.610]  But that's also used by the rest of us.
[46:45.750 --> 46:47.330]  And by the way, that's pretty much everything.
[46:48.470 --> 46:52.630]  Should they report that to the vendor or should they hold on to it and keep it secret?
[46:53.010 --> 46:58.890]  And there's something called the vulnerabilities equities process that currently exists in the government.
[46:58.890 --> 47:01.330]  And basically, we don't really know anything about it.
[47:02.650 --> 47:07.150]  How much are we willing to pay for government exploit development tools?
[47:07.490 --> 47:10.570]  And, you know, is this good public policy?
[47:10.750 --> 47:18.110]  Is it a good public policy to have, on the one hand, the government in trying to encourage security of our critical infrastructure?
[47:18.110 --> 47:22.750]  And on another hand, trying to figure out how to exploit it?
[47:23.150 --> 47:27.230]  So the question is, where are we going to end up?
[47:28.810 --> 47:40.470]  And really, there are relatively easy, and by easy I mean hard, but not out of the question, engineering issues and operational issues.
[47:40.910 --> 47:46.070]  There are harder, I think, policy issues.
[47:46.070 --> 47:59.310]  And one of the reasons they're hard is because they've been virtually undiscussed in the technical community and almost completely undiscussed in the policy and law community.
[47:59.750 --> 48:05.190]  And one of the challenges is that this is moving forward without any of that discussion.
[48:05.190 --> 48:16.270]  And as the government gets good at it, we're going to suddenly discover that this was, this might have led us to a world that we don't want.
[48:16.410 --> 48:17.870]  So we need to start talking about this.
[48:18.070 --> 48:20.970]  So basically, there are more questions than answers here.
[48:22.110 --> 48:24.050]  But fortunately, we have time for questions.
[48:33.040 --> 48:35.780]  And by the way, I can't tell if there's anyone at the microphone.
[48:35.780 --> 48:39.780]  So, because they're blinding me here, which I assume is coming from like the NSA.
[48:40.140 --> 48:40.460]  Yeah.
[48:41.600 --> 48:42.080]  Sure.
[48:42.680 --> 48:43.160]  Okay.
[48:44.620 --> 48:46.240]  So if there's somebody there, talk.
[48:46.560 --> 48:46.640]  Yeah.
[48:46.680 --> 48:47.280]  Wilhelm Baker.
[48:47.560 --> 48:48.240]  Hi, Matt.
[48:48.440 --> 48:48.620]  Hi.
[48:48.960 --> 48:49.300]  Yeah.
[48:49.500 --> 49:06.320]  It just occurred to me that, you know, one of the other coincidences between the first crypto war and the second, is they both coincided with the FBI spending an awful lot of time on an investigation of either the president or a candidate for the presidency.
[49:07.020 --> 49:10.520]  And maybe those are not entirely unrelated.
[49:10.860 --> 49:16.800]  The FBI having a hobby horse and the policy people being targeted for prosecution.
[49:17.620 --> 49:18.120]  Yeah.
[49:18.300 --> 49:20.860]  I mean, that level of speculation is above my pay grade.
[49:25.160 --> 49:35.160]  So, you're pointing out that there's been, you know, the FBI saying that they're going dark, and, you know, there's all this worry, but it's, you know, you can show that, you know, it rounds down to zero.
[49:36.040 --> 49:41.160]  What do you think is the real motivation for pushing for these sort of things?
[49:41.660 --> 49:44.060]  I tend to find it's more out of their own laziness.
[49:44.400 --> 49:46.920]  The investigations are still possible.
[49:47.160 --> 49:48.340]  They want to make it easier, though.
[49:48.340 --> 50:02.740]  I mean, look, I think there are... it's really, really easy to look at this from a sort of technical point of view and say, look, you guys are stupid, or you guys have evil motives, or so on.
[50:03.240 --> 50:15.020]  And, you know, regardless of what I personally believe about that, that set of assumptions won't actually allow us to make progress, right?
[50:15.020 --> 50:30.100]  If you say, your solution to this problem is the FBI should be disbanded because they're lazy or they're doing something wrong, that's simply not going to actually work, even if it's... it doesn't matter if it's true in our public policy debate.
[50:30.300 --> 50:44.200]  I think, you know, the assumption that we probably have to make, whether you agree with it or not, is that the FBI is at least being straightforward in that it does actually have some of the problems that it says it has.
[50:45.100 --> 50:47.240]  There may also be another issue here.
[50:47.520 --> 50:54.500]  From talking to friends who are in the FBI, there's... they're restricted on a lot of resources.
[50:55.540 --> 51:01.300]  You think that they've got masses and masses of cyber money being thrown at them, but...
[51:01.300 --> 51:02.000]  Cyber money?
[51:02.140 --> 51:02.660]  Yeah, exactly.
[51:02.920 --> 51:04.960]  But there may not be that.
[51:05.120 --> 51:12.540]  They may... they may actually... we actually may need to go through Congress to get more allocation... money allocated to them for this purpose.
[51:13.540 --> 51:14.020]  So...
[51:17.450 --> 51:18.450]  Is there anyone else?
[51:18.750 --> 51:20.010]  Yeah, I've got a quick one.
[51:20.110 --> 51:21.510]  My name is Matthew Weaver.
[51:21.510 --> 51:24.570]  First of all, thank you so much for both of you.
[51:24.630 --> 51:27.210]  It's an incredible overview of the landscape.
[51:27.510 --> 51:46.190]  I'd love to hear what each of you think is the single most important thing we should start doing, like, tomorrow morning, around this piece of the problem, where we move from crypto wars to figuring out what it looks like when what we're really doing is compromising devices to get evidence,
[51:46.430 --> 51:46.530]  right?
[51:46.610 --> 51:47.930]  Like, what do we do starting tomorrow?
[51:47.930 --> 51:49.610]  Yeah, no, so...
[51:49.610 --> 51:53.370]  The number one thing that I would say is engage with this.
[51:53.630 --> 52:03.230]  Pay attention to the... pay attention to those idiots at the political level that are actually making the decisions that will affect what our policies are.
[52:04.530 --> 52:22.170]  And, you know, for example, the Rule 41 changes, which basically allow an administrative change that allows large-scale device exploitation, happened incredibly quietly within sort of the administrative office of the courts.
[52:22.230 --> 52:23.950]  It didn't even happen in Congress.
[52:24.110 --> 52:29.350]  And they solicited technical input very late and got very little of it.
[52:29.510 --> 52:44.330]  So, having smart technical folks, which I would count every single person in this room as, engaged in this issue, talking to people, educating them, we're in a crisis, you're trying to make it worse, is really important.
[52:44.770 --> 52:47.310]  Volunteer in all of your local political offices.
[52:47.570 --> 52:50.890]  Be a technical expert for your local politicians.
[52:51.190 --> 52:52.390]  They need it.
[52:52.970 --> 52:58.410]  And if they can get your sort of expertise, it's not hard to do.
[52:58.990 --> 53:01.930]  You just have to go and actually do it.
[53:02.110 --> 53:05.330]  My Congressman is Rush Holt, who is pretty damn good.
[53:06.650 --> 53:10.510]  So, I can say, but I know some of our academic...
[53:10.510 --> 53:11.010]  We're giving the hook.
[53:11.170 --> 53:11.570]  Oh, sorry.
[53:11.690 --> 53:12.030]  Sorry.
[53:12.250 --> 53:12.910]  There are people.
[53:13.070 --> 53:13.470]  Yeah.
[53:13.710 --> 53:17.930]  So, I want to follow up on that question that that gentleman asked, which is a very good one.
[53:18.110 --> 53:20.870]  So, Matt and Sandy, I know you both are in academia land.
[53:20.870 --> 53:22.570]  I think this topic should be this...
[53:22.570 --> 53:22.690]  Not much longer.
[53:23.430 --> 53:24.190]  I think this...
[53:24.830 --> 53:28.650]  I think this topic got to be integrated into the undergrads and computer science...
[53:28.650 --> 53:32.490]  I teach a course in technology and policy, and I think every school should have it.
[53:32.510 --> 53:34.230]  I ain't seeing this happening anywhere else.
[53:34.490 --> 53:35.690]  Well, go talk to folks.
[53:35.830 --> 53:36.870]  We're getting the hook, so...
[53:36.870 --> 53:37.810]  So, fast.
[53:38.090 --> 53:38.550]  You want one more?