[00:00.000 --> 00:04.960]  He's a great ally of free and open-source software and the head of the Software Freedom Law Center.
[00:05.640 --> 00:16.480]  He gave a talk at NYU a couple years back, which was the sort of spark for the beginning of the Freedom Box, which is a distributed set of servers everybody can run in their house.
[00:16.700 --> 00:24.700]  And it inspired the folks who developed Diaspora, the open and shared alternative, distributed alternative to Facebook.
[00:25.020 --> 00:28.720]  So it is my pleasure to introduce Dr. Eben Moglen.
[00:28.900 --> 00:29.620]  Give it up.
[00:38.590 --> 00:39.530]  Thank you.
[00:39.690 --> 00:42.010]  It's a great pleasure to be here.
[00:44.670 --> 00:58.530]  I apologize for the preview in Forbes Online and the resulting Slashdot conversation about whether I understand the first law of robotics or not, which was very entertaining to me.
[00:59.630 --> 01:07.250]  But let me try to start from scratch while still making it as interesting as possible.
[01:08.470 --> 01:29.410]  The free software movement, which is now 30 years old, if you start from Richard's original mulling over his concerns about unfree operating systems in the MIT AI lab.
[01:29.410 --> 01:51.350]  The free software movement is reaching its moment of junction with the great river of the Internet freedom movement, which is going to be the dominant political and technology movement of our time, in which all our boats are finally going to reach the sea.
[01:52.330 --> 02:04.510]  But in the process of becoming first the free software movement and then the great river of the Internet freedom movement, we are, of course, as always, standing on the shoulders of giants.
[02:04.510 --> 02:21.990]  Many of the giants who affected the thinking of those of us who began worrying about the freedom of software decades ago, many of the giants on whose shoulders we stood were authors of science fiction.
[02:21.990 --> 02:24.750]  They were the great vision.
[02:25.170 --> 02:37.950]  They were the great visionaries of the post-Second World War imaginative literature, which coped with the problem of the runaway technology that had transformed the world.
[02:37.950 --> 02:51.630]  You will recall that after the first terrible use of nuclear weapons in the world, Albert Einstein said, we have changed everything except the way men think.
[02:52.690 --> 03:15.450]  And the culture of the post-war Western world and the culture of the post-war Eastern world, too, if you read as we did science fiction from both sides of the Iron Curtain, the culture of the post-war world was very heavily affected by the attempt to understand the implications of technology as imaginative authors,
[03:15.450 --> 03:31.830]  including Ray Bradbury, who recently left us, and many others, as imaginative authors tried to cast in new idealized forms moral, ethical problems with technology out of control.
[03:31.830 --> 03:41.290]  And the literature that they wrote deeply affected me when I was young and growing up, and Richard, and many, many others.
[03:42.530 --> 04:00.250]  I want to go back to that now because I believe it's time for us to acknowledge yet again how much they foresaw those writers of the post-war world, and how much they helped us to foresee, but how little we have helped ourselves to avoid it.
[04:00.250 --> 04:20.970]  One of the staples of that science fiction of the 1960s that we read so avidly growing up was that by now, the middle of the first quarter of the 21st century, by now, it was assumed that human beings would be living in society commensally with robots.
[04:22.010 --> 04:35.750]  And many, many people tried to imagine the nature of that commensal biological coexistence between us and the robots, the androids we had built.
[04:38.250 --> 04:54.870]  Everybody understood that there were enormous ethical and moral dilemmas implicit in our living with robots, as there would also be enormous changes in the texture and fabric of ordinary human life from day to day.
[04:54.870 --> 05:14.050]  And the two elements, the nature of human life as lived in the company of robots, and the nature of the ethical and moral dilemmas implied by the attempt to do so, were fertile ground for some of the very greatest fiction that was written in that time.
[05:14.050 --> 05:38.530]  Not least of which was Isaac Asimov's attempt to understand how we would confront the problem of runaway technology in life with robots, which produced, as many here will recall, as warmly as I do, all of the stories and the novels which were built out of the U.S. Robotics Corporation and its positronic-brained creations.
[05:46.900 --> 05:54.840]  There, of course, from the beginning, the assumption was that robots would be humanoid.
[05:55.220 --> 05:57.520]  And as it turns out, they're not.
[05:58.400 --> 06:01.920]  We do, after all, live commensally with robots now.
[06:02.240 --> 06:04.300]  We do, just as they expected.
[06:04.880 --> 06:08.520]  But the robots we live with don't have hands and feet.
[06:08.520 --> 06:14.040]  They don't carry trays of drinks, and they don't push the vacuum cleaner.
[06:14.900 --> 06:18.320]  At the edge condition, they are the vacuum cleaner.
[06:20.080 --> 06:23.780]  But most of the time, we're their hands and feet.
[06:24.560 --> 06:26.200]  We embody them.
[06:27.160 --> 06:29.540]  We carry them around with us.
[06:30.140 --> 06:32.520]  They see everything we see.
[06:32.520 --> 06:35.480]  They hear everything we hear.
[06:35.900 --> 06:41.360]  They're constantly aware of our location, position, velocity, and intention.
[06:42.480 --> 06:45.580]  They mediate our searches.
[06:45.580 --> 06:47.820]  That is to say, they know our plans.
[06:47.820 --> 06:49.700]  They consider our dreams.
[06:50.280 --> 06:53.520]  They understand our lives.
[06:55.240 --> 07:01.220]  They even take our questions, like, how do I send flowers to my girlfriend.
[07:02.220 --> 07:05.680]  Transmit them to a great big database in California.
[07:06.400 --> 07:11.120]  And return us answers offered by the helpful wizard behind the curtain.
[07:12.240 --> 07:14.500]  Who, of course, is keeping track.
[07:15.620 --> 07:23.320]  These are our robots, and we have everything we ever expected to have from them.
[07:24.060 --> 07:26.080]  Except the first law of robotics.
[07:28.000 --> 07:30.420]  You remember how that went, right?
[07:30.760 --> 07:46.420]  Deep in the design of the positronic intelligence that made the robot were the laws that governed the ethical boundary between what could and could not be done with androids.
[07:48.660 --> 07:59.420]  The first law, the first law, the one that everything else had to be deduced from, was that no robot may ever injure a human being.
[08:01.160 --> 08:12.560]  Robots must take orders from their human owners, except where those orders involve harming a human being.
[08:14.320 --> 08:33.020]  That was assumed to be the principle out of which, at the root, down by the NAND gates, of the artificial neurophysiology of robot brains, down there, where the simplest idea is.
[08:33.340 --> 08:36.940]  You remember, for Descartes, it was cogito ergo sum.
[08:37.180 --> 08:40.480]  For the robot, it was no robot must ever harm a human being.
[08:40.740 --> 08:48.100]  We are living commensally with robots, but we have no first law of robotics in them.
[08:48.100 --> 08:53.360]  They hurt human beings every day, everywhere.
[08:54.200 --> 09:02.320]  Those injuries range from the trivial to the fatal to the cosmic.
[09:03.380 --> 09:07.560]  Of course, they're helping people to charge you more.
[09:08.300 --> 09:10.460]  That's trivial, right?
[09:11.000 --> 09:20.380]  They're letting other people know when you need everything from a hamburger to a sexual interaction to a house mortgage.
[09:21.260 --> 09:38.960]  And, of course, the people on the other end are the repeat players whose calculations about just how much you need whatever it is and how much you'll pay for it are being built by the data mining of all the data about everybody that everybody is collecting through the robots.
[09:40.540 --> 09:43.020]  But it isn't just that you're paying more.
[09:45.720 --> 09:55.820]  Some people in the world are being arrested, tortured, or killed because they've been informed on by their robots.
[09:57.760 --> 10:08.800]  Two days ago, the New York Times printed a little story about the idea that we ought to call them trackers that happen to make phone calls rather than phones that happen to track us around.
[10:08.960 --> 10:15.520]  They were kind enough to mention the topic of today's talk, but they didn't mention the talk.
[10:16.100 --> 10:22.920]  And this morning, the New York Times has an editorial lamenting the death of privacy and suggesting legislation.
[10:24.060 --> 10:28.040]  Here's the cosmic harm our robots are doing us.
[10:28.040 --> 10:32.520]  They are destroying the human right to be alone.
[10:33.700 --> 10:38.100]  They are destroying the human right to do your own thinking.
[10:39.000 --> 10:47.760]  They are destroying the human capacity for disappearing into ourselves.
[10:49.580 --> 10:54.000]  Robots are changing humanity as the literature said they would.
[10:54.000 --> 10:57.900]  They are changing humanity quite deeply.
[10:58.740 --> 11:04.800]  And the way that they are changing humanity is not to make it more human.
[11:07.380 --> 11:12.780]  Instead, android quality is rubbing off on us.
[11:13.860 --> 11:18.840]  Which was, of course, always implicit in the literature when it turned dark.
[11:18.840 --> 11:24.580]  That we might not be able to tell the difference after a while.
[11:25.420 --> 11:28.800]  Between the replicants and ourselves.
[11:31.880 --> 11:33.980]  So, we've got a problem.
[11:35.940 --> 11:40.420]  I've tried to define the problem space in this talk.
[11:40.620 --> 11:43.920]  I don't propose that we can solve the problem this afternoon.
[11:43.920 --> 11:45.300]  We can recognize it.
[11:47.660 --> 11:54.080]  As I get older and grayer and further from that boy who read that science fiction.
[11:54.320 --> 11:57.780]  I realize that the rest of my life is going to be about this.
[11:58.140 --> 12:01.560]  And probably, therefore, some part of the rest of yours.
[12:01.560 --> 12:03.140]  If you see it the way I do.
[12:04.240 --> 12:10.080]  We have to retrofit the first law of robotics into everything.
[12:11.860 --> 12:14.860]  This is not going to be simple.
[12:16.500 --> 12:24.840]  The Slashdotters who wanted me to remember that the purpose of the first law of robotics was trained in the positronic brain.
[12:25.300 --> 12:26.760]  Well, of course, they were right.
[12:26.920 --> 12:29.060]  That was the happy, imaginative part.
[12:29.300 --> 12:31.080]  You remember why that was?
[12:32.100 --> 12:38.420]  The assumption that Isaac Asimov made was that human beings would be afraid of robots.
[12:39.300 --> 12:44.040]  That they would be afraid to allow their children to be tended by robots.
[12:44.220 --> 12:45.940]  Or to have them in their homes.
[12:46.940 --> 12:58.900]  And therefore, without some assurance of the complete engineered in from the very beginning quality of will never hurt a human being.
[12:58.900 --> 13:02.300]  That robots would not be adopted.
[13:03.580 --> 13:09.040]  That the capitalist motive of the robot maker.
[13:10.940 --> 13:11.780]  The U.S. robotics corporation.
[13:12.820 --> 13:25.820]  That the capitalist needs of the robot maker to create a safe market in which consumers would accept robots in their homes and with their children.
[13:25.820 --> 13:32.160]  Would require an absolute guarantee of engineered in safety.
[13:32.160 --> 13:34.320]  Will never harm you.
[13:40.530 --> 13:43.070]  Isaac Asimov was a great New Yorker.
[13:44.490 --> 13:51.010]  I was privileged to grow up in his city while he lived here and bump into him every once in a while.
[13:51.010 --> 13:59.350]  As you know from the Foundation trilogy, he had really, at the bottom, a very gemurtlich sense of all of this.
[13:59.750 --> 14:07.350]  Trantor was really the grand concourse and good jewish family values were really enough to save the galaxy.
[14:11.810 --> 14:16.630]  Unfortunately, this was the visionary part of the science fiction.
[14:16.910 --> 14:18.250]  And it isn't true.
[14:18.950 --> 14:27.170]  It was much easier to get people to hang robots around the necks of their children than anybody ever imagined.
[14:28.330 --> 14:33.730]  And it didn't require any promise that they would never, never, never harm anybody.
[14:33.730 --> 14:36.930]  All it required was little shiny things.
[14:37.750 --> 14:41.090]  Made by Count Dracula, the king of the undead.
[14:42.230 --> 14:50.090]  The purpose of the undead, as you know, is to make evil beautiful.
[14:51.430 --> 14:52.830]  That's what the undead do.
[14:53.730 --> 15:01.850]  They turn evil into something so erotically attractive you can't keep your hands off it.
[15:01.850 --> 15:05.630]  And you don't mind having its hands on you.
[15:08.030 --> 15:09.130]  And he did that.
[15:09.810 --> 15:11.290]  The king of the undead.
[15:13.590 --> 15:14.530]  He's dead now.
[15:16.210 --> 15:19.070]  But they didn't throw his boot into the Danube.
[15:19.150 --> 15:20.830]  And there was no silver bullet.
[15:21.050 --> 15:22.830]  And there was no stake through the heart.
[15:23.030 --> 15:25.090]  And the undead still are with us.
[15:26.050 --> 15:28.110]  They're improving the screen.
[15:29.010 --> 15:30.450]  And such like.
[15:32.110 --> 15:37.870]  Until another king of the undead who can build damned beautiful things is ready.
[15:39.530 --> 15:41.430]  But that's all it took.
[15:42.910 --> 15:50.410]  And now we put those things around our children's necks and we send them off to be harmed.
[15:50.410 --> 15:54.350]  By the robot with whom they live.
[15:56.570 --> 16:02.610]  We have the problem that Einstein was talking about.
[16:02.790 --> 16:07.370]  We have changed everything except the way people think about this.
[16:08.730 --> 16:13.330]  The heuristics that humanity brings to the net.
[16:13.830 --> 16:20.330]  Are heuristics which assume that they know the direction in which danger comes at them.
[16:20.710 --> 16:21.730]  And they do not.
[16:22.950 --> 16:26.430]  So much has happened very quickly.
[16:27.450 --> 16:31.170]  It isn't that we haven't warned about it.
[16:32.050 --> 16:37.710]  The free software movement gave its warning all the way along.
[16:38.650 --> 16:43.590]  Very rationalistically, scientifically in hackerspeak.
[16:43.950 --> 16:44.910]  However.
[16:45.490 --> 16:52.230]  And our great problem was always how were we gonna get people who didn't hack on things.
[16:52.230 --> 16:57.150]  To understand the importance of being able to hack on things.
[16:58.330 --> 17:03.790]  It was a really tough political lift for symbol makers.
[17:04.250 --> 17:08.830]  To explain to people who didn't interact with code at all.
[17:09.050 --> 17:13.270]  Why the freedom of code was their freedom at the end of the day.
[17:13.270 --> 17:14.790]  We knew it was.
[17:15.090 --> 17:19.030]  Because we grew up with I, robot.
[17:20.150 --> 17:22.170]  We knew it was.
[17:22.490 --> 17:31.110]  Because we grew up understanding that humanity had many different ways for creating unethical technology.
[17:32.390 --> 17:39.870]  And that we were going to have to find ways to embed ethics in the technology.
[17:39.870 --> 17:43.770]  Mr. Stallman could not have been clearer about that.
[17:44.790 --> 17:49.110]  But his clarity wasn't universally accessible by any means.
[17:50.250 --> 17:55.350]  It wasn't imaginatively available to every child in the world.
[17:55.610 --> 17:59.470]  It was only available, may I say it, to us.
[18:01.130 --> 18:06.190]  Now we have to operationalize in a profounder way.
[18:07.230 --> 18:18.850]  Because we aren't merely worried about whether there will be code available for operating systems for people who work in artificial intelligence laboratories.
[18:19.090 --> 18:21.330]  Or even for students.
[18:22.430 --> 18:29.330]  Now we have to worry about how to retrofit the first law of robotics into objects that are hurting people.
[18:34.100 --> 18:47.360]  Mostly, mostly, mostly, we have an ethical and moral problem to describe and to set the outside limits for.
[18:47.640 --> 18:58.400]  We have to be able to express to all the people with whom we interact, though they are not necessarily technical in the same ways that we are.
[18:58.400 --> 19:10.920]  We have to be able to express to them what the ethical limits of technologies are with which they are already familiar in ethically compromised form.
[19:12.580 --> 19:16.920]  And, of course, we have some technology work to do as well.
[19:18.840 --> 19:33.680]  Where the two things cross, where we are required to do technology work as well as explain to people the nature of the ethical limits of the technologies around us, we have our biggest problem.
[19:33.680 --> 19:36.740]  And the most immediately urgent.
[19:38.040 --> 19:47.600]  We cannot retrofit the first law of robotics into robots that have been designed to resist our modifying them.
[19:49.160 --> 19:52.700]  This is a fairly simple point, I understand.
[19:53.800 --> 20:12.480]  We tried five years ago in GPL3 to make it with sufficient clarity that everybody who understood the implications could come along with us and do some work in helping to avoid the situation of the locked down robot.
[20:14.220 --> 20:27.160]  We made very little progress because people who are now beginning to realize that they should have supported the anti-lockdown efforts in GPL3 didn't do so at the time.
[20:27.740 --> 20:31.540]  Maybe still haven't done so as powerfully as they should have done.
[20:31.540 --> 20:42.040]  And in the meantime, a whole range of monoliths grew up around society that are very much in love with the idea of the robot you can't retrofit.
[20:43.160 --> 20:46.740]  And it does harm to people every day.
[20:47.320 --> 20:51.980]  And you can't retrofit it because the cover's welded shut.
[20:52.440 --> 20:57.040]  And it's booby-trapped by the DMCA and lots of other things.
[20:58.380 --> 21:04.280]  You can go to jail for trying to retrofit freedom into a robot under the wrong conditions.
[21:04.760 --> 21:11.020]  If that might mean the robot might sing a copyrighted song without permission of the composer.
[21:11.860 --> 21:18.620]  Or show you a movie that you haven't paid for enough times yet.
[21:23.980 --> 21:35.880]  So, first thing we're going to have to do is take with much greater seriousness the job of building a coalition to assure that retrofitting is possible.
[21:36.140 --> 21:42.480]  That it is neither legally nor technologically prohibited to make things safer.
[21:45.240 --> 21:47.700]  This shouldn't be required in a democracy.
[21:49.040 --> 21:51.380]  This shouldn't even be required in capitalism.
[21:51.380 --> 21:55.420]  If you own a thing, it should be your right to make it safer.
[21:57.140 --> 21:58.100]  Don't you think?
[21:59.220 --> 22:00.100]  Ah, well.
[22:01.340 --> 22:05.040]  You see how fundamentally we've lost our way.
[22:05.940 --> 22:15.860]  So we need a few things and we don't need to be altogether unwilling to adopt other people's vocabulary in order to get them.
[22:16.960 --> 22:28.380]  About this, for example, it seems to me that we deserve to be as strong for owners' rights as other people are for their entitlement to have offshore trust funds and other things.
[22:29.300 --> 22:29.400]  Right?
[22:29.720 --> 22:31.160]  We own this stuff.
[22:31.280 --> 22:31.580]  Okay?
[22:31.820 --> 22:32.720]  Back off.
[22:33.820 --> 22:46.020]  Where it's our stuff, we're entitled to modify it if we want to make it safer, if we want to share safety improvements with other people, so they can modify what they own too.
[22:46.180 --> 22:47.000]  That's a right.
[22:51.750 --> 23:01.970]  We need to be very clear that how things work is associated with the quaint concept of the ownership of the thing.
[23:02.810 --> 23:05.650]  If I own it, the way it works should be the way I want it to.
[23:06.730 --> 23:29.290]  The Software Freedom Law Center submitted an exemption request in the Library of Congress DMCA exemption proceedings this year, urging the Library of Congress to declare that it is not prohibited circumvention of means of access control to replace the operating system in a mobile or other computing device you own.
[23:31.350 --> 23:41.390]  I'm very grateful to Aaron Williamson of the SFLC for his extraordinary work in preparing and testifying on behalf of that exemption request.
[23:41.690 --> 23:45.350]  We're going to back it this time as strongly as we can.
[23:45.350 --> 24:00.030]  We hope the Library of Congress will see the wisdom of declaring that in this free market country you are free to modify devices that you've bought with money and that you are quaintly regarded as owning.
[24:02.470 --> 24:08.210]  It shouldn't require any more argument than that, but if it does, we have to double down and keep arguing.
[24:09.190 --> 24:18.370]  We have to point out that if devices are unsafe, it is a legal obligation to permit us to make them safer.
[24:19.130 --> 24:42.250]  If you sell an unsafe slicer in a delicatessen or an unsafe automobile and you attempt to prevent people from modifying those devices to make them safer, if you're actually out there actively interfering with attempts to make them safer, then when people get hurt you should be liable.
[24:51.530 --> 25:00.690]  If we press hard enough on that point, we will scare even Count Dracula, King of the Undead, in his grave.
[25:01.650 --> 25:14.390]  Where he should be very frightened because he has interfered with more attempts to make his products safer by more people than any other undead maker in history.
[25:15.850 --> 25:30.310]  We need to establish the proposition that when people get hurt, when somebody's responsible for that, they pay for it if they have attempted to prevent us from preventing the harm.
[25:30.310 --> 25:33.190]  This is not the first law of robotics.
[25:33.190 --> 25:35.250]  This is the first law of being U.S. robotics, okay?
[25:36.630 --> 25:39.010]  It's your ass on the line.
[25:40.890 --> 25:42.110]  Everybody's got to know that.
[25:42.230 --> 25:53.810]  And by everybody, I distinctly mean to include certain parties called Verizon and AT&T.
[25:53.810 --> 26:05.310]  Nowhere in the world are the network operators more aggressive about prohibiting us from increasing the safety of devices.
[26:06.270 --> 26:12.970]  Nowhere is there a more concentrated opposition to GPL3 than in the U.S. network operator duopoly.
[26:17.450 --> 26:41.710]  Now, we know, thanks to last week's news, confirming what we already knew but it hadn't been printed in the New York Times yet, that millions of times a year, people with a tin star are requesting the real-time location or the contents of messages or the nature of the traffic between tracking devices and the networks.
[26:41.710 --> 26:43.810]  We know that now.
[26:44.550 --> 26:54.610]  That is to say, we know exactly how far down the road of suppressing civil liberties the robots are taking us.
[26:56.990 --> 27:06.130]  Of course, improving your civil liberties is not necessarily regarded by other people as making you safer.
[27:08.970 --> 27:21.430]  So, not all the time when we insist upon improving our civil liberties by retrofitting into devices our first law, you shall not harm the user of the device.
[27:23.330 --> 27:31.410]  We're going to be told that what we're doing isn't making people safer because it makes terrorists safer too, or some such nonsense.
[27:35.560 --> 27:43.200]  The truth of the matter is, products must not harm the people who buy and use them, regardless of whether the people who buy and use them are nice people.
[27:45.400 --> 27:52.100]  When a kid gets his hand injured by a delicatessen slicer, we don't ask ourselves whether he's a good kid or not.
[27:56.420 --> 28:03.480]  We don't even ask whether he was a little bit impaired by something when it happened.
[28:04.560 --> 28:12.620]  Because the manufacturer who makes an article inherently dangerous is responsible for the harm it does.
[28:12.780 --> 28:27.040]  And if, for example, it doesn't have two hands switching and somebody's hand gets hurt, whether they were a nice guy or a bad guy or whether they were planning to sabotage the factory on the weekend is not a relevant concern.
[28:30.430 --> 28:34.870]  The IT architecture of the next period is set.
[28:35.050 --> 28:39.470]  And pretty much everywhere I go in the world, everybody understands it.
[28:39.650 --> 28:40.770]  They recognize it.
[28:40.870 --> 28:42.590]  It's called cloud to mobile.
[28:44.050 --> 28:45.170]  What does that mean?
[28:45.950 --> 28:49.210]  It means robots reporting at headquarters.
[28:50.230 --> 28:57.930]  Tossing your data overhead from where they collect it to where it is stored, wherever that is.
[28:58.630 --> 29:11.710]  If you're a lawyer who worries about privacy, that's about the same as saying, first it will be at the robot and then it will be in whatever legal system in the world gives you the least protection for it.
[29:12.470 --> 29:19.810]  And the most academic commercial advantage to the guy keeping it for you.
[29:21.190 --> 29:31.590]  In 2006, I gave a talk to MySQL annual developers meeting about why it's good to store it yourself instead of storing things other places.
[29:32.710 --> 29:50.270]  But I was still in the grip of the belief that we were all going to be fine and I spent more time talking about technologies of memory in relation to freedom than what I should have said, which was if you don't store it yourself, it's going to be stored by a guy taking advantage of you deeply,
[29:51.110 --> 29:56.470]  eradicating your privacy, and making you the android of him.
[30:00.000 --> 30:05.400]  I probably would even have chosen the word android, which had nothing to do with computer software at the time.
[30:06.640 --> 30:07.780]  But there we are.
[30:08.940 --> 30:09.960]  But there we are.
[30:10.220 --> 30:11.600]  Cloud to mobile.
[30:12.000 --> 30:15.980]  What it means is from unsafety to unsafety unless we do it right.
[30:17.900 --> 30:22.940]  It's kind of us to refer to the NYU talk in 2010 about freedom in the cloud.
[30:22.940 --> 30:30.540]  I wanted then to set out some ideas about how we had gotten into that part of the mess and how we might get out of them again.
[30:31.480 --> 30:47.760]  On that particular point, let me just say about Freedom Box that as of very soon now, by which I mean single digit days, Debian will be natively supporting the plug server called the Dream Plug and from it a variety of other plug servers.
[30:47.760 --> 31:11.680]  And Freedom Box will have moved into being Debian privacy, and we will be trying to deliver best possible privacy tools to every architecture, everywhere, all the time, and particularly to small, effective, power-sipping plug servers that can replace routers everywhere and make the network safer.
[31:12.880 --> 31:16.700]  For which work I am endlessly grateful to B.
[31:16.780 --> 31:24.020]  Dale Garby and James Vazil and Nick Daly and many others who have been hacking on Freedom Box over the last 18 months.
[31:25.300 --> 31:26.400]  But what I...
[31:32.170 --> 31:51.030]  But what I and you know is that no matter what we do to make the network safer and to make server-side improvements, let us call them, we must at the mobile end be capable of delivering safety, security, and privacy to people on the things they really use,
[31:51.110 --> 31:52.910]  the robots they really live with.
[31:53.170 --> 31:57.430]  It won't do us any good to try and compete with U.S. robotics and Count Dracula by saying you can buy a beige little box and plug it in on a wall at home.
[32:04.850 --> 32:14.950]  We have to get into the galaxy in your pocket or the galaxy will have no freedom in it, no matter what they do on Trantor.
[32:16.050 --> 32:27.190]  So this raises questions beyond merely how are we going to get the code in the box or how are we going to define what the code is.
[32:27.350 --> 32:28.570]  We're all real good at that.
[32:28.570 --> 32:32.250]  And I'm actually quite optimistic that we can hold up the technical end.
[32:32.250 --> 32:35.550]  We've been holding up the technical end all the way along.
[32:35.950 --> 32:45.190]  The free software movement has contributed a lot of freedom to the Internet freedom movement it is becoming and we're going to continue to contribute all the way until we win.
[32:47.310 --> 32:55.930]  But what we have to do, beyond all the stuff we're good at, is to do the thing that human beings haven't been good at so far.
[32:55.930 --> 33:05.830]  We have to be really alive to the danger and we have to teach people that safety must be put in now, after we've already launched the boat.
[33:06.090 --> 33:06.570]  Preach.
[33:07.130 --> 33:08.470]  Well, okay.
[33:08.910 --> 33:10.290]  Preaching is part of it.
[33:11.750 --> 33:16.250]  But preaching is effective where there's adequate dogma.
[33:16.410 --> 33:20.590]  That is to say, where we really understand the doctrine of what we're preaching.
[33:20.590 --> 33:24.130]  And so we have a little intellectual heavy lifting to do.
[33:24.310 --> 33:28.330]  What does it really mean to talk about hurting people?
[33:29.510 --> 33:46.430]  What does it really mean to talk about not hurting people, or guaranteeing non-hurt to people, in this complex environment in which, one, the robot's cognition has to be reduced to the level of our desire.
[33:46.430 --> 33:49.410]  It should not be listening to me when I didn't tell it to.
[33:49.850 --> 33:54.310]  It shouldn't be informing people of my location when I haven't said it can.
[33:54.590 --> 33:55.830]  And so on.
[33:56.590 --> 34:05.670]  But where that dialogue cannot possibly consist of punching okay or no on dialogue boxes on something every tenth of a second through a lifetime.
[34:06.530 --> 34:11.890]  We need to understand what services can be safely offered and which ones can't.
[34:11.890 --> 34:19.990]  Or rather, how service design itself must be altered in order to produce safety for users.
[34:22.070 --> 34:33.030]  Location-directed or location-aware or location-based services are terribly important and terribly dangerous.
[34:34.010 --> 34:44.270]  And the primary problem is the real-time ascertainment of the location of human beings by those with power.
[34:45.090 --> 34:51.110]  It does very little good, in other words, to describe regulatory approaches to such services.
[34:51.470 --> 34:58.190]  Because the regulatory approach will always be engineered by government to say, you shouldn't do this unless you're us.
[34:59.090 --> 35:02.050]  Or, you shouldn't do this unless we want you to.
[35:02.610 --> 35:12.630]  Or, you shouldn't do this unless there's a court order or other authoritative communication telling you to start turning over real-time location data about human beings.
[35:13.650 --> 35:31.850]  The very senior U.S. government official who told me back in March, well, we've learned now that we need to have a robust social graph of the United States, is reflecting the learning of all the governments on Earth in the last 14 months.
[35:32.090 --> 35:39.150]  When, with great suddenness, they all discovered that what they wanted was a robust social graph of their societies.
[35:39.790 --> 35:43.350]  You understand, of course, that we could put this in plain English for the people around us.
[35:43.350 --> 35:49.870]  We could say, this means the United States government intends to keep a list of everybody every American knows.
[35:52.230 --> 35:55.810]  That's not what we used to quaintly refer to as a free society.
[35:56.790 --> 35:59.950]  In fact, that's what I would call a dangerous neighborhood.
[36:02.310 --> 36:12.950]  And in that dangerous neighborhood, which it may not be possible to prevent ourselves from living in unless we learn how to exercise democracy really effectively about this.
[36:14.050 --> 36:16.810]  Which is going to mean a lot of preaching and a lot of teaching.
[36:17.250 --> 36:28.410]  But in that very dangerous neighborhood, we have to understand that things that inform at headquarters where we are, are serious problems.
[36:30.230 --> 36:38.570]  I must admit that I find it kind of reassuring how naively confident Americans are.
[36:39.910 --> 36:49.990]  As I grew up to manhood and I started traveling around the world, I discovered that in most of, indeed, all of the societies I went to except my own, people didn't really think that what they said on the telephone was private.
[36:52.390 --> 36:56.070]  And my friends in the Soviet Union were particularly aware of this, of course.
[36:56.730 --> 37:01.190]  I was, too, when I lived there briefly, in the late 70s.
[37:03.150 --> 37:23.150]  What seems to me so amazing is that it is possible to sell people a thing and say, you've got a personal assistant inside this object, and you can talk to her, her, of course, in English, and say whatever you want, and we'll take it back to some warehouse data center somewhere,
[37:23.210 --> 37:25.450]  and then we'll send you an answer and tell you what to do.
[37:27.070 --> 37:29.930]  If the KGB had tried this, it would not have worked.
[37:37.770 --> 37:40.750]  But for Count Dracula, the king of the undead, it was a snap.
[37:45.180 --> 37:45.580]  Extraordinary.
[37:46.020 --> 37:47.200]  And very worrisome.
[37:48.000 --> 37:50.120]  Because how are we going to take it away from people?
[37:51.180 --> 37:51.580]  Right?
[37:51.680 --> 37:52.420]  I mean, what are you going to say?
[37:52.420 --> 37:58.820]  You know, you should go back to not wanting that anymore because, in truth, it's the KGB inside your mind?
[38:00.120 --> 38:08.900]  Because you're contributing everything that you would ever say to anybody who was helping you to a great big database of everything located in the world of the undead?
[38:10.840 --> 38:18.180]  This is not a thing you would expect to have a hard time convincing people not to do unless they were already doing it.
[38:18.820 --> 38:23.700]  And there is an awful lot of effort going into making people comfortable doing it right now.
[38:25.120 --> 38:36.080]  Which means that we're going to have to have strong arguments and good technology and really powerful moral conviction.
[38:37.680 --> 38:47.500]  Now, obviously, we can explain to people why you shouldn't leave your children in the custody of robots that haven't been engineered never to hurt a human being.
[38:48.780 --> 38:50.000]  We can do that.
[38:50.800 --> 38:56.280]  It's going to feel a little counterintuitive to people, but we're going to have to say it.
[38:59.050 --> 39:12.330]  We're going to have to remind people that the great imaginative literature about the king of the undead tells us that he can't come into our houses unless we invite him across the threshold.
[39:15.330 --> 39:24.290]  And we're going to have to ask ourselves and parents everywhere, you don't want to invite him in, do you?
[39:25.610 --> 39:26.350]  Not really.
[39:29.070 --> 39:31.010]  But you see, it's all about convenience.
[39:31.750 --> 39:32.550]  And prettiness.
[39:32.910 --> 39:34.050]  And coolness.
[39:34.510 --> 39:36.430]  And the sexiness of technology.
[39:36.630 --> 39:39.090]  And we know more about the sexiness of technology than anybody.
[39:39.270 --> 39:46.210]  And most of us, I speak at least only for myself, if it weren't for the sexiness of technology, there would be little sexiness in my life.
[39:53.710 --> 39:54.510]  Notwithstanding which.
[39:55.450 --> 39:56.150]  Notwithstanding which.
[39:56.290 --> 39:58.610]  There is a time when the evil is too beautiful.
[40:00.150 --> 40:02.210]  And we have to do something about it.
[40:03.670 --> 40:06.130]  As far as I'm concerned, this isn't a project.
[40:07.370 --> 40:12.070]  It isn't a, this is what we need to do right now, and then we'll be done with it.
[40:12.390 --> 40:14.970]  Unfortunately, this is a way of life for us.
[40:18.200 --> 40:21.160]  Retrofitting the first law of robotics is going to take a long time.
[40:21.300 --> 40:25.000]  Because they're building robots every day without it.
[40:25.660 --> 40:34.120]  And they're getting people more and more accustomed to the idea that you carry around a brain that isn't yours, and it thinks about you for other people.
[40:35.640 --> 40:43.160]  That you have cognitive facilities that don't work for you in your pocket all the time, on the bed table every night.
[40:44.460 --> 40:48.430]  That the tracker is always there pretending to be a phone.
[40:51.020 --> 40:58.240]  That you're wearing the one ring that binds us all to them.
[41:01.770 --> 41:04.210]  It's really hard.
[41:04.930 --> 41:06.470]  It's really hard.
[41:07.770 --> 41:10.630]  We're going to have to be very committed to this.
[41:10.830 --> 41:13.710]  This is the meaning of our part of the freedom movement.
[41:14.710 --> 41:17.690]  This is the part we're going to have to be responsible for.
[41:19.750 --> 41:24.070]  Because there are billions of people on earth who are going to be trapped.
[41:26.130 --> 41:27.490]  And we know why.
[41:28.250 --> 41:29.510]  And we know how.
[41:30.710 --> 41:37.090]  And we haven't quite figured out how to describe it to them in ways that will help them stay safe.
[41:38.730 --> 41:43.770]  But if we don't, it's going to be very dark.
[41:45.310 --> 42:04.950]  And all that hopeful science fiction that came from our attempt to believe that we could think our way into safety after building the bomb will have turned out to be true enough about the bomb.
[42:06.730 --> 42:12.510]  But not so true about the robots we are becoming.
[42:14.010 --> 42:15.150]  Thank you very much.
[42:47.980 --> 42:48.730]  Thank you.
[42:48.830 --> 42:50.350]  I would be happy to take some questions.
[42:50.350 --> 42:52.450]  I have a couple of small questions.
[42:52.770 --> 43:04.980]  How long do you think it will be before the taxing authorities start using the tracking devices to identify the states of residence of people and send them bills for state taxes?
[43:06.390 --> 43:17.270]  Well, as you will recall, when the easy pass first came into use, there were promises that it would not be used to send people speeding tickets by arithmetic calculation.
[43:17.270 --> 43:22.020]  A promise which lasted roughly a decade before it began to be broken.
[43:22.870 --> 43:28.410]  I do not think that that's a wrong general magnitude.
[43:29.200 --> 43:37.330]  The use of automated cameras for many kinds of enforcement are coming at high speed.
[43:38.180 --> 43:46.800]  And tax enforcement is at least a possible long-term use of certain kinds of very simple data mining.
[43:47.440 --> 43:52.460]  The likelihood, I think, is that some jurisdiction somewhere will start doing it at some point.
[43:52.700 --> 44:09.700]  You are aware that New York City has been known to send inspectors at Christmas time across the malls on the other side of the Hudson River to take pictures of people's license plates who are shopping in New Jersey for Christmas Hall at possible loss of use taxes to New York State.
[44:09.700 --> 44:24.180]  It will eventually, particularly once near-field communication inside the phone is how you pay, be possible to give the geographical reference for every payment and associate taxes with it on an automatic basis.
[44:24.480 --> 44:31.230]  How long out into the future that is, I don't know, but it's not more than double the time it takes before somebody begins trying it.
[44:31.230 --> 44:35.900]  And why are back doors not illegal in welded shut devices?
[44:36.840 --> 44:43.160]  They seem to be a great source of security risk and unsafety.
[44:43.420 --> 44:46.180]  Well, it's not illegal to make bad and insecure software.
[44:46.500 --> 44:51.060]  If it had been, of course, Mr. Gates would not be the leading philanthropist in the world.
[44:52.300 --> 44:53.820]  He'd be just another inmate.
[44:53.820 --> 44:57.420]  But why then has product liability law not kept up with that?
[44:57.540 --> 45:01.230]  And why have, you know, why have the lemon laws not been successful?
[45:01.420 --> 45:03.040]  Well, lemon laws are a little different.
[45:03.420 --> 45:03.600]  Okay.
[45:04.000 --> 45:04.900]  Or product liability.
[45:05.250 --> 45:23.920]  The answer in general to the nature of the product liability situation is when you buy things, rip the shrink wrap on things, open the boxes, sign the service agreements, the guy contracting with you has indemnified himself against all possible outcomes to the fullest extent that the best possible legal engineering can do.
[45:24.120 --> 45:28.660]  We need to add some law, as the New York Times was recommending this morning.
[45:28.800 --> 45:33.500]  I don't know that I agree particularly with the editorial writer about where to go first.
[45:33.680 --> 45:45.700]  But we're going to need to make privacy law in some important ways that we learn to make environmental law under the Nixon administration of all times and places.
[45:46.580 --> 45:48.730]  Privacy is an ecological problem.
[45:49.690 --> 45:54.000]  The harm done by the robots is an ecological problem.
[45:54.440 --> 46:01.520]  One of the things that goes wrong is that most people, including my students, tend to think of privacy as between me and somebody.
[46:01.860 --> 46:06.980]  They don't recognize that when they tag a photograph on Facebook they're doing an ecological harm to a third party.
[46:07.500 --> 46:24.020]  When we begin to understand more concretely the third party effects of the way we destroy our own privacy with the robots and therefore harm other people, whether it's our children, our friends, our families, our churches, or whatever, then the legal situation will clarify.
[46:24.140 --> 46:36.620]  Because it will become evident, as it became evident about tobacco, that part of the difficulty is the third party consequences that can't be dealt with by signing a contract that says, we won't sue about it.
[46:39.150 --> 46:44.370]  So most of your examples of safety have been specifically privacy.
[46:44.490 --> 46:48.690]  Are there other areas of safety that you have in mind?
[46:49.310 --> 47:00.950]  When we begin to think about the taxonomies of services, we will discover harms that are the result of each of the kinds of failures of privacy.
[47:00.950 --> 47:07.750]  I teach in my classroom, I'm going to be very brisk about this, but if you look elsewhere on the net you'll see larger explanations of what I mean.
[47:07.930 --> 47:11.930]  I teach in my classroom that privacy is really three things at minimum.
[47:12.650 --> 47:17.430]  Secrecy, that is messages whose content is obscure to all but intended parties.
[47:18.310 --> 47:26.550]  Anonymity, that is messages whose content is available but whose sending and receiving points are obscure to all but permitted parties.
[47:26.550 --> 47:34.210]  And autonomy, which is our ability to decide and act in the world free of violations of secrecy and anonymity.
[47:34.770 --> 47:49.030]  When you take those as the taxonomy of privacy constituents and then begin to work out in the world the nature of the dangers through the services, you get to many different kinds of harms.
[47:49.030 --> 48:08.450]  There are physical harms that result from intentional violent conduct mostly undertaken by states, but not only undertaken by states, sometimes also by criminal actors, non-state actors, on the basis of information available in the net as a result of harmful behavior by robots or androids.
[48:09.010 --> 48:15.930]  There are also examples of economic harms, pricing harms, insurance harms, and so on.
[48:15.930 --> 48:28.670]  There are harms in politics that result in the chilling of protected speech, or the chilling of religious activity as a consequence of the misbehavior of robots or androids.
[48:28.870 --> 48:30.530]  A whole range of harms.
[48:30.730 --> 48:38.770]  I wouldn't want to go so far as to say that every conceivable kind of harm that the law protects people against is done by robots, but many are.
[48:38.770 --> 48:48.570]  And part of our difficulty in elucidating our arguments is going to have to be a more complete analysis and taxonomy of the nature of the harms.
[48:50.930 --> 49:00.550]  I myself am working on an open-source personal robot project now, which will grant people to totally improve the safety of the robot they are going to use or have.
[49:00.550 --> 49:06.630]  So, I believe that it's the way to go to, you know, maybe implement the first law of robotics.
[49:06.910 --> 49:09.510]  So, I would like, you know, your opinion on that.
[49:09.730 --> 49:20.390]  Well, obviously, technology we make ourselves because it's safe and throw into things is a fundamental tool of what we're going to need to accomplish.
[49:20.390 --> 49:25.110]  It was what I had in mind in calling for Freedom Box to try and deal with routing and serving.
[49:25.490 --> 49:30.650]  And we certainly need client-side safety or mobile-side safety.
[49:30.990 --> 49:41.850]  And we have, if anything, a world in which I do believe if we marshal our arguments and build our coalitions correctly, we've an opportunity actually to get it done.
[49:42.110 --> 49:48.930]  As I say, I think we might be able to abate the legal threats against us for doing it in DMCA and other places.
[49:48.930 --> 49:56.930]  I think we will be able to bring some influence to bear with the network operators to move them in other directions.
[49:57.330 --> 50:08.390]  I think old-line IT companies that sell to enterprise are going to turn out to be at least as interested in getting their stacks into other people's mobile handsets as we are for various reasons.
[50:08.890 --> 50:18.590]  I think there are a number of possible allies for us, and I think as a free software movement that has been careful to look for coalitions it could build, we will be able to build some.
[50:18.590 --> 50:23.230]  At the end of the day, of course, the safe technology is what we make and share ourselves.
[50:28.260 --> 50:43.020]  If we do manage to put our own software inside the robots, don't we still have the problem that now you have an unapproved device attached to the network, and you don't have the servers from the network that you don't control?
[50:43.240 --> 50:45.080]  No, I don't think that should be the outcome.
[50:45.280 --> 50:53.300]  We were careful in GPL3 to make rules which prohibited the network operator from booting devices from the network simply because they were modified.
[50:53.300 --> 51:02.480]  If people had taken our approach to that seriously, I say again, half a decade ago, we could have spared ourselves a lot of this nonsense.
[51:02.800 --> 51:08.540]  Mr. Stallman was right, comma, as usual, comma, about that, right?
[51:08.540 --> 51:27.400]  The correct answer here is that we told AT&T in the modified order and judgment at the end of the antitrust cycle that broke up AT&T in 1980, that it had to permit the attachment of uncertified but nonetheless non-harmful devices to the network.
[51:27.400 --> 51:32.120]  This same proposition has to be the case with respect to mobile networks.
[51:32.300 --> 51:32.660]  That's all.
[51:32.780 --> 51:34.580]  If the device is harmful, you can boot it.
[51:34.640 --> 51:41.920]  But if it isn't actually harming the network's operation, then there's no legal right to keep it off.
[51:42.160 --> 51:44.180]  That's the direction in which we need to go.
[51:45.400 --> 51:47.820]  How do we get there?
[51:48.080 --> 51:58.540]  Well, one idea is we get more free software under licenses that require that until they can't do what they do without us.
[51:58.720 --> 52:01.680]  That's the strategy we used in some other contexts.
[52:02.180 --> 52:12.600]  Another possibility is we get telecoms regulators to understand that they have the same reason for that that they had for insisting on that rule in the area of the wired network.
[52:12.940 --> 52:18.200]  Another possibility is we have to get our elected representatives to know that's what we want.
[52:18.760 --> 52:23.800]  The minute they know it's what we want enough not to vote for them unless we get it, we'll get it.
[52:26.040 --> 52:33.700]  So, somewhat related to that, getting the first law of robotics into electronic devices is definitely important.
[52:34.160 --> 52:36.580]  But how do we get that into corporate law as well?
[52:36.900 --> 52:40.160]  You don't, of course, because it's not in people.
[52:40.400 --> 52:45.600]  And while you may not believe that corporations are people, you can certainly believe they're made of people.
[52:45.600 --> 52:55.060]  This problem, how to get people to behave with intrinsically virtuous conduct and intention, is a long-standing human problem.
[53:00.460 --> 53:11.640]  The great English lawyer Edward Cook said in his famous reports, Corporations cannot be excommunicates because they have no souls.
[53:13.280 --> 53:20.320]  A point which, while it was a narrow technical point in the case in which he was offering it, is also largely true, you understand.
[53:21.640 --> 53:29.820]  The king of the undead, Count Dracula, was particularly good, as you notice, in getting corporations to do evil things through beauty.
[53:29.820 --> 53:36.780]  He got the music industry in the world almost to destroy itself under the pretense that he was saving it.
[53:37.740 --> 53:47.620]  His ghost, in charge of the Apple Computer Corporation, has been doing it to the magazine publishers of the world, even when he was already temporarily descended into the grave.
[53:48.740 --> 54:00.180]  The problem of getting the corporations to do it right is not a problem that we can deal with by law, if our long history of the legal regulation of collective enterprise is any indication.
[54:00.500 --> 54:07.520]  I think we really are trying to engineer for safety, despite the natural tendency of human beings to do wrong.
[54:11.940 --> 54:17.060]  I had a regulatory question, again, somewhat related to some of the previous questions.
[54:17.060 --> 54:22.140]  It's probably going to be both the last regulatory question and the last non-regulatory question, so go for it.
[54:23.680 --> 54:33.560]  So there appear to be times when legislatively there's a sort of deterministic acquiescence to how technology functions.
[54:33.560 --> 54:53.360]  And so we see this, for example, in exceptions that are made to something like the ECPA, where the legislation says we make an exception when certain kinds of data are required to be collected in order to provision service in an acceptable manner or what have you.
[54:53.880 --> 55:08.060]  And this seems to create a sort of perverse incentive for the development of technology that promotes the promulgation and development of technology that collects more information necessarily.
[55:10.340 --> 55:17.100]  And I wanted to know what your thoughts were to sort of, you know, maybe historically contextualize that.
[55:17.100 --> 55:23.460]  See, I don't think that the problem of collecting too much is now primarily a byproduct of careless regulation.
[55:23.460 --> 55:31.880]  The problem of collecting too much arises from the fact that the future of capitalism is in the data mining and control of its customers.
[55:32.440 --> 56:00.980]  When capitalist societies begin to depend more on consumption than they do on production for their overall economic health, and the GDP of the United States has been primarily consumption for more than 15 years now, then knowing how to control consumers becomes as important as knowing how to control productive machines was when the GDP was still primarily based upon production rather than
[56:00.980 --> 56:01.580]  consumption.
[56:01.960 --> 56:11.900]  What is happening is that we are automating and instrumenting the portion of the economy which is most powerfully important, namely consumption.
[56:11.900 --> 56:21.980]  Collecting information about consumers is the same thing as knowing how the mill machines worked to the capitalist of the 19th century.
[56:22.340 --> 56:31.780]  So what we are confronting is not the accident of perverse incentives created as a byproduct of unintended consequences of regulation.
[56:31.780 --> 56:38.800]  We're facing the fact that doing the wrong thing is the basis of future economic growth.
[56:38.800 --> 56:42.800]  We're not trying to destroy future economic growth.
[56:43.220 --> 56:46.080]  We're trying to compromise among values.
[56:46.380 --> 56:48.620]  That's what regulation is.
[56:48.780 --> 56:50.000]  It's a messy process.
[56:50.400 --> 56:52.800]  Of course it has unintended consequences.
[56:53.060 --> 56:56.760]  Naturally, it creates perverse incentives because it's imperfect.
[56:57.240 --> 57:00.860]  But understanding the objective has to be clear.
[57:01.300 --> 57:05.260]  Productive capitalism met ecological constraints.
[57:05.260 --> 57:08.120]  It dirtied the water too much.
[57:08.200 --> 57:09.860]  It dirtied the air too much.
[57:10.020 --> 57:18.160]  It created too many difficulties that had to be constrained as negative externalities produced by production.
[57:18.600 --> 57:27.620]  The attempt to regiment and govern consumption through data mining raises ecological problems we must solve.
[57:28.000 --> 57:28.700]  Thank you all.