[00:55.750 --> 00:56.350]  All right.
[00:56.470 --> 00:58.650]  Welcome, everyone, to our two-clock session.
[00:59.730 --> 01:03.190]  Just a reminder that put your phones on vibrate.
[01:03.610 --> 01:05.850]  We have exits in the case of emergency.
[01:06.910 --> 01:10.090]  Look for someone with a radio or a blue volunteer shirt.
[01:11.630 --> 01:12.570]  Stay hydrated.
[01:12.750 --> 01:13.450]  It's hot today.
[01:13.870 --> 01:16.030]  And, you know, have fun.
[01:16.290 --> 01:18.490]  Right now, I'd like to introduce Davide Semenzin.
[01:19.190 --> 01:22.450]  And he's going to be talking about why building digital libraries matters.
[01:22.450 --> 01:23.150]  Thank you.
[01:23.330 --> 01:23.410]  Davide.
[01:24.470 --> 01:25.150]  Oh, wow.
[01:25.230 --> 01:25.670]  The lights are bright.
[01:25.750 --> 01:26.150]  Hey, everybody.
[01:29.200 --> 01:29.980]  Thank you.
[01:30.480 --> 01:31.040]  All right.
[01:34.540 --> 01:35.260]  Of course.
[01:35.780 --> 01:36.200]  One sec.
[01:38.260 --> 01:38.940]  Oh, okay.
[01:39.040 --> 01:39.200]  Whatever.
[01:39.500 --> 01:40.280]  I'll freestyle this.
[01:41.240 --> 01:41.680]  All right.
[01:42.200 --> 01:42.720]  Hey, everybody.
[01:42.860 --> 01:43.440]  Thank you for coming.
[01:43.820 --> 01:44.800]  My name is Davide.
[01:46.420 --> 01:49.520]  I am a software engineer by trade.
[01:49.700 --> 01:52.580]  I studied computer science, like many of you, I guess.
[01:52.580 --> 01:56.520]  I spent most of my career working in distributed systems.
[01:56.780 --> 02:05.700]  I currently work at AWS, but I spent a good chunk of about five years working at the Internet Archive.
[02:06.120 --> 02:12.980]  And there, I worked building the platform to digitize books at scale.
[02:14.140 --> 02:17.140]  And it's work that I think about a lot.
[02:17.140 --> 02:20.340]  And I think it's very meaningful to me.
[02:20.620 --> 02:27.760]  And so, while I remain associated with the archive, I also just always really liked books.
[02:27.780 --> 02:30.980]  And I always really cared about libraries and the humanities.
[02:30.980 --> 02:33.950]  And so, I don't know if these are credentials enough.
[02:34.220 --> 02:35.420]  But, you know, we're all here.
[02:35.620 --> 02:36.360]  We'll try to make it fun.
[02:36.700 --> 02:37.880]  And I have a statue.
[02:38.340 --> 02:40.740]  The Internet Archive is a nonprofit organization.
[02:40.740 --> 02:43.380]  So, they don't have stock options to give you.
[02:43.500 --> 02:45.740]  So, after three years to stay there, they make a little statue of you.
[02:45.980 --> 02:46.920]  And that's mine.
[02:47.260 --> 02:48.420]  Holding that this is fine, dog.
[02:49.380 --> 02:49.720]  Anyway.
[02:50.940 --> 02:55.580]  So, I'm here today to talk to you about building digital libraries.
[02:56.480 --> 02:56.880]  And...
[02:56.880 --> 02:57.440]  Okay.
[02:57.520 --> 02:58.000]  Let's get real.
[02:58.320 --> 03:00.020]  Like, you saw the title of this talk.
[03:00.280 --> 03:00.900]  You're all here.
[03:01.650 --> 03:07.180]  And that means that, in a sense, you wanted to engage with this idea of a digital library.
[03:07.380 --> 03:08.460]  So, it's a real question.
[03:08.460 --> 03:09.000]  I'm asking you.
[03:09.280 --> 03:09.980]  What's a library?
[03:10.620 --> 03:11.020]  What...
[03:11.740 --> 03:13.200]  What do you think a library is?
[03:14.660 --> 03:15.780]  I really want to hear it.
[03:17.220 --> 03:17.620]  Any...
[03:17.620 --> 03:18.140]  Any takers?
[03:18.560 --> 03:20.680]  I can use a 3D printer.
[03:21.320 --> 03:21.720]  Really?
[03:21.760 --> 03:23.060]  You can use a 3D printer in your library?
[03:23.120 --> 03:23.700]  That's awesome.
[03:25.900 --> 03:26.680]  3D printer?
[03:27.560 --> 03:29.020]  A website with all the books.
[03:29.120 --> 03:30.160]  A website with all the books.
[03:30.200 --> 03:30.780]  That's really good.
[03:31.100 --> 03:32.580]  I like your notion of a library.
[03:33.180 --> 03:33.680]  Go ahead.
[03:40.360 --> 03:41.020]  Go ahead.
[03:41.380 --> 03:41.760]  Exactly.
[03:41.860 --> 03:45.100]  It's a place where you can retrieve things that is well organized.
[03:45.300 --> 03:48.060]  Actually, that is as close to my definition as it gets.
[03:48.380 --> 03:52.600]  I think it's an accessible and organized collection of materials.
[03:55.200 --> 03:58.480]  I think accessible is more important than organized.
[03:59.780 --> 04:04.500]  Libraries, it's better if they are organized, but you really have to be accessible.
[04:04.740 --> 04:05.900]  Otherwise, they're just collections.
[04:07.300 --> 04:11.240]  And it's important that they are a collection of materials, not just of stuff.
[04:11.440 --> 04:13.540]  Otherwise, a landfill would be a library, right?
[04:13.680 --> 04:18.660]  Like, there has to be some inerrant value in whatever it is that we're collecting.
[04:20.220 --> 04:29.280]  And so, I want to try and answer this question of why do digital libraries matter by going through a couple of other questions.
[04:29.640 --> 04:34.620]  In a way, asking why of anything is a little bit of a philosophical question, right?
[04:34.760 --> 04:39.920]  Which is just a fancy way of saying that you have to answer a whole bunch of other questions before you get there.
[04:39.920 --> 04:46.440]  And so, we have a working notion of what is a library, and we're going to come back to this.
[04:46.940 --> 04:49.960]  The next question would be, who builds libraries?
[04:50.440 --> 04:56.900]  And, you know, you can think of your classic library, like your library down the street.
[04:57.260 --> 04:58.060]  That's a library.
[04:58.060 --> 05:06.460]  You can think of the library in maybe high school or university or your law firm or the Internet Archive.
[05:06.900 --> 05:09.400]  Private libraries are libraries as well.
[05:09.660 --> 05:12.720]  And, you know, people have been building libraries since the dawn of time.
[05:12.720 --> 05:15.360]  So, the ecosystem is really robust.
[05:16.220 --> 05:20.340]  There are government and government-like entities building libraries.
[05:21.120 --> 05:23.320]  Almost every country has a national library.
[05:24.240 --> 05:26.340]  And, well, then there are companies.
[05:27.600 --> 05:32.280]  Companies have libraries but aren't necessarily libraries.
[05:32.560 --> 05:44.760]  If you think of a certain search engine who spent a very significant amount of money building a very large digital collection of books, they're not really a library because they're not accessible.
[05:45.060 --> 05:49.960]  The collection exists, but it is accessed on their own terms.
[05:49.960 --> 05:53.300]  Maybe a snippet at a time, maybe pay for play, maybe not at all.
[05:54.560 --> 05:58.540]  So, I'm kind of on the verge on that one, but we're going to put it in there.
[05:58.880 --> 05:59.600]  And then there is you.
[05:59.920 --> 06:00.940]  There's like me, you.
[06:01.240 --> 06:02.180]  We have like a cell phone.
[06:02.360 --> 06:03.740]  Your cell phone is a digital library.
[06:04.080 --> 06:10.000]  In this world, we're all engaging in the creation of digital libraries.
[06:10.120 --> 06:12.560]  So, this matters for a variety of reasons.
[06:12.640 --> 06:16.720]  And not the least of it is that you're taking part in it, whether you like it or not.
[06:16.720 --> 06:31.860]  So, of all of the people who are engaging willingly or unwillingly in the construction of digital libraries, I think the Internet Archive is maybe one of the most important or certainly notable examples.
[06:31.860 --> 06:43.320]  And if you do not know what the Internet Archive is, the top of the line description is that it is a digital library that has existed for about 26 years.
[06:43.420 --> 06:45.340]  I think it was 1996, right?
[06:45.760 --> 06:46.860]  May 10th, 1996.
[06:47.580 --> 07:01.480]  And it is known for this one pretty amazing piece of software called the Wayback Machine, where you can go back in time and see what websites look like before they inevitably change or disappear.
[07:02.340 --> 07:05.760]  And it is headquartered in this beautiful building in San Francisco.
[07:06.160 --> 07:07.980]  It used to be a Christ Scientist Church.
[07:07.980 --> 07:11.560]  And I kid you not, the reason we bought it is because it looked like our logo.
[07:11.760 --> 07:12.360]  This is a true story.
[07:15.620 --> 07:17.320]  Anyway, a lot of things work that way.
[07:18.300 --> 07:25.460]  So, I don't want to dwell too long on what the Archive is, because there is like somebody else who can do it a lot better.
[07:26.020 --> 07:27.900]  And that is archive.org.
[07:28.080 --> 07:29.120]  So, just visit the website.
[07:29.480 --> 07:31.980]  All the stuff is in there, a whole hundred petabytes of it.
[07:34.420 --> 07:42.840]  Really, I want to move on to what I think is maybe like the closest part to my heart, which is really the how, because I am an engineer.
[07:43.180 --> 07:47.880]  And I spent many years of my life building the tool to build the library.
[07:48.080 --> 07:52.260]  So, the question of how do you build a library with purpose at a certain scale?
[07:52.500 --> 07:53.740]  Well, I can answer that.
[07:54.040 --> 07:55.540]  And it is pipeline.
[07:55.780 --> 07:56.760]  It is not very surprising.
[07:57.400 --> 08:00.220]  So, I am going to talk about this only a little bit.
[08:00.660 --> 08:07.000]  I am going to leave time at the end for questions, so I can maximize my chances of saying something that is of interest to you guys.
[08:07.820 --> 08:11.020]  But at a high level, there are just two boxes.
[08:11.300 --> 08:12.300]  There is a purple box.
[08:12.420 --> 08:16.540]  That is stuff that happens on the scanner in the world.
[08:16.700 --> 08:19.540]  There are hundreds of the scanners scattered all over the world.
[08:19.900 --> 08:25.080]  And the blue stuff is what happens inside our, call it cloud.
[08:25.500 --> 08:29.900]  So, the way this works is that we start with a scanner.
[08:29.900 --> 08:36.160]  We receive a book and then we have to somehow transform it into a digital artifact.
[08:36.500 --> 08:39.180]  Well, the first step is to take pictures of it.
[08:39.360 --> 08:42.640]  And so, we have devised these devices.
[08:42.940 --> 08:45.000]  These are all built by us.
[08:45.140 --> 08:46.380]  All developed by us.
[08:47.100 --> 08:50.700]  Some of them in different eras.
[08:50.920 --> 08:53.820]  And they all cater to different use cases.
[08:54.480 --> 08:56.300]  You have this full-frame scribe.
[08:56.300 --> 08:58.840]  It's a very large kind of contraption.
[08:59.380 --> 09:03.600]  It's got a very big cradle.
[09:03.600 --> 09:09.720]  So, you use this for very large books that have maybe difficult spines, like dictionaries and stuff like that.
[09:10.940 --> 09:15.960]  It's also very good for digitizing very delicate objects.
[09:16.620 --> 09:28.000]  We have a mandate of doing non-destructive scanning, which makes things a little more complicated than if you could just rip the spine off and turn this into just a page-turning exercise.
[09:29.920 --> 09:32.040]  There aren't so many of these full-frame scribes.
[09:32.120 --> 09:32.820]  They're very expensive.
[09:33.140 --> 09:34.500]  They're custom-built.
[09:34.620 --> 09:35.660]  They're bulky.
[09:36.360 --> 09:39.200]  Our real blockbuster is the tabletop scribe.
[09:39.220 --> 09:40.540]  We built a lot of these.
[09:40.660 --> 09:43.700]  And all of these are built off of consumer hardware, consumer cameras.
[09:44.520 --> 09:46.040]  They're easy to manufacture.
[09:46.280 --> 09:47.300]  They're easy to transport.
[09:48.040 --> 09:50.380]  And they provide pretty high quality.
[09:51.420 --> 09:52.240]  And then...
[09:52.240 --> 09:55.260]  So, we use these for really, like, the mass digitization stuff.
[09:55.920 --> 09:59.680]  And then there is, like, the fold-out scribe, which is, like, this table with a...
[09:59.680 --> 10:01.280]  It's a glorified table with a camera, really.
[10:01.680 --> 10:05.060]  But the camera is very precisely positioned, and it's very, very high quality.
[10:05.660 --> 10:11.040]  And so you use this for maps, atlases, things of that sort, like, really large format.
[10:12.940 --> 10:13.820]  Anyway, so...
[10:14.420 --> 10:15.740]  I put the slide in here.
[10:15.740 --> 10:21.260]  It's really not that important, but this is the software that I spent five years developing, so I just figured, like, it should be in the presentation.
[10:21.840 --> 10:24.100]  This is just the software that we use, right?
[10:24.260 --> 10:25.280]  We run on the scanners.
[10:25.640 --> 10:30.620]  And what it really does is it collates a bunch of different functions together.
[10:30.820 --> 10:33.260]  It's kind of like a Swiss Army knife of book scanning.
[10:33.780 --> 10:43.620]  It can process metadata, load data, try to figure out if there are errors in the way the book was scanned before we send it.
[10:44.640 --> 10:48.320]  And then, ultimately, it takes care of uploading to the archive.
[10:48.520 --> 10:52.580]  And I'm going to come back to these themes later because none of that is actually simple.
[10:53.000 --> 11:00.840]  Anyway, after we have successfully uploaded all of the data in the archive, a bunch of stuff starts happening.
[11:00.960 --> 11:01.560]  That's kind of cool.
[11:01.760 --> 11:05.300]  So first, we preprocess all of this stuff.
[11:05.420 --> 11:15.460]  So we disqueue it, we autocrop it, we make sure that we have all of the derivative formats for images so that we can move to the next step, which I'm going to talk about in a second.
[11:15.900 --> 11:25.600]  And then, generally, we modify the structural metadata so that we have an easy-to-modify book.
[11:25.600 --> 11:33.680]  And the reason that matters is because we have a sort of human-in-the-loop stage called republishing.
[11:33.920 --> 11:44.780]  And that is when a human actually revises what the automation has done and makes sure that the book is good and that everything has been, like, worked out properly.
[11:46.380 --> 11:51.340]  And it uses, like, another piece of homegrown software like this.
[11:52.600 --> 12:02.820]  And sometimes, you know, at this stage we find errors and there is this, like, oof, really painful rigmarole that we need to go through to send back the book to the original scanner where it was originated, fix it.
[12:02.920 --> 12:04.460]  It's very expensive, very complicated.
[12:04.680 --> 12:06.100]  It's borderline magic.
[12:08.660 --> 12:09.840]  So weird it is.
[12:09.920 --> 12:27.280]  But anyway, provided that the human says, okay, everything looks good, the book is checked in, and we move into a post-processing phase where we actually apply all of those crops and those disqueues and all of those transformations.
[12:28.540 --> 12:37.220]  And we do a bunch of other metadata extraction on the images and color correction, all this sort of stuff.
[12:37.700 --> 12:46.560]  And then the output of this stage is a set of files plus a description of the structure of the book plus the metadata.
[12:46.560 --> 12:51.620]  And then from here, we derive all of the other formats that we distribute.
[12:51.840 --> 12:53.700]  PDF, EPUB, text.
[12:53.940 --> 12:54.600]  We OCR it.
[12:54.760 --> 12:56.120]  I'm going to talk about OCR later.
[12:57.060 --> 13:00.820]  Anyway, and so this is how you end with a digital book in short.
[13:01.020 --> 13:04.240]  If you guys want to know, like, more details, we can talk about this at the end.
[13:04.240 --> 13:06.960]  So this is kind of like at a high level what the building blocks are.
[13:07.340 --> 13:13.160]  So what's kind of interesting about this whole thing is what was hard about it.
[13:13.400 --> 13:23.780]  And I entirely, arbitrarily divided the challenges that we had into whether they pertain to the bits or whether they pertain to the atoms.
[13:23.780 --> 13:35.720]  So in terms of the bits, what I call the logical challenges, if any of you has worked for any length of time on distributed systems, this is not going to come.
[13:35.820 --> 13:36.400]  It's a big surprise.
[13:36.660 --> 13:44.600]  But in a highly process-oriented environment, catching and correcting errors as soon as possible is of critical importance.
[13:44.600 --> 13:49.860]  Because from the moment an error is made and the page is turned, a clock starts ticking.
[13:50.140 --> 13:53.720]  And with every passing second, correcting that error becomes more expensive.
[13:56.280 --> 14:07.260]  I talked about this rigmarole that we have to go through if we figure out, I don't know, two, three, four days later, that a book that was scanned has a page that was blurry or something.
[14:07.420 --> 14:09.980]  You know, you have to go figure out where that book is, figure out what the page is.
[14:10.040 --> 14:10.920]  It takes like a lot of time.
[14:12.580 --> 14:16.980]  Metadata curation, also kind of a thorny problem.
[14:17.480 --> 14:21.240]  Mostly because books are...
[14:21.240 --> 14:24.560]  And this is a theme that will come back.
[14:25.120 --> 14:27.760]  Books are an expression of human endeavors.
[14:28.020 --> 14:29.780]  And so they are very, very different from each other.
[14:29.940 --> 14:30.960]  They don't follow a schema.
[14:31.260 --> 14:33.400]  And computers really like schemas.
[14:33.600 --> 14:36.200]  And so sometimes it's really difficult to wrangle things that way.
[14:36.940 --> 14:42.980]  And obviously you have to maintain large databases of metadata and Brewster hates databases.
[14:43.660 --> 14:43.840]  Anyway.
[14:46.060 --> 15:05.940]  Maybe one of the biggest real challenges that we had, and by real I mean stuff that will ruin your day if it doesn't work, was this problem of delivering industrial-level process reliability over what it is fundamentally an unreliable backend.
[15:06.160 --> 15:07.440]  And so let me talk about this for a second.
[15:07.440 --> 15:12.960]  There are very good reasons why the Internet Archive is, quote-unquote, unreliable.
[15:14.960 --> 15:27.140]  I think that by my estimation, the Internet Archive is available 99.9% of the year, which is a whole lot better than your average library, which is available hardly 50% of the time.
[15:27.760 --> 15:31.880]  But for a digital native, 99.9% is not so great.
[15:31.880 --> 15:40.060]  And so, I mean, I think if you tally it at the end of the year, like the Archive maybe was offline for an hour a year, but it's not continuous.
[15:40.180 --> 15:41.160]  It's in blips.
[15:41.580 --> 15:45.780]  And if you're borrowing a book, that really doesn't matter that much.
[15:46.220 --> 15:46.680]  Sorry.
[15:47.340 --> 15:53.300]  But if you're trying to upload a book, it starts really, really disrupting your process.
[15:53.300 --> 15:59.540]  And as it turns out, you know, people who do book scanning, I mean, that's their job.
[15:59.540 --> 16:02.160]  And if people cannot do their job, they don't get paid.
[16:02.260 --> 16:04.540]  And if they don't get paid, they rightfully get very angry.
[16:04.840 --> 16:15.580]  And so, we had to figure out a way to make sure that all of this system that we had devised kept running, even when, you know, if the Archive became a big hole in the ground for a while.
[16:16.500 --> 16:25.000]  And again, if you have worked on distributed systems for any amount of time, the kind of solutions that we came up with, they're not particularly groundbreaking.
[16:25.000 --> 16:28.720]  It's like a lot of retries, a lot of back-off, lots of caching everywhere.
[16:29.600 --> 16:37.260]  But it surprised me in retrospect how much time I spent trying to, you know, fix this kind of problems.
[16:37.920 --> 16:44.880]  Oh, another error, like something that really plagued me was people who put error messages inside the 200.
[16:44.880 --> 16:47.100]  Okay, don't do that.
[16:47.760 --> 16:51.000]  Anyway, networking was also a big problem at times.
[16:51.180 --> 16:57.900]  Like, if you're beaming, like, very large amounts of data over the open Internet, at some point, something is bound to go wrong.
[16:58.040 --> 17:03.880]  And sometimes, like, something goes wrong in an AES that's completely outside of your control, so what do you do?
[17:05.900 --> 17:07.680]  Build a lot of caching, mostly.
[17:08.380 --> 17:10.900]  But again, like, this is not particularly surprising.
[17:12.140 --> 17:18.260]  The stuff about atoms, like, about the physical world was, like, what to me was the newest.
[17:18.380 --> 17:21.260]  Because as a software engineer, I am not really used.
[17:21.260 --> 17:27.340]  I think I have some difficulty dealing with the fact that the physical world has constraints.
[17:28.060 --> 17:32.820]  And so, you know, if you're scanning any amount of books, you have to move them around.
[17:32.820 --> 17:33.940]  And they're bulky and they're heavy.
[17:34.120 --> 17:37.540]  So you need, like, pallets and boxes and forklifts and containers.
[17:37.780 --> 17:39.720]  And, like, it's a whole thing.
[17:39.720 --> 17:42.440]  And you have to keep track of where everything is.
[17:43.060 --> 17:45.240]  You need to write software for that, obviously.
[17:46.480 --> 17:51.900]  But, you know, software kind of assumes, like, an idealized version of the world.
[17:52.060 --> 17:54.040]  And sometimes the world doesn't behave that way.
[17:54.160 --> 17:57.040]  So, you know, book disposition was a problem at some point.
[17:57.140 --> 17:58.020]  Where is the stuff?
[17:59.140 --> 18:00.020]  Hard problem.
[18:01.200 --> 18:02.300]  I'm going to come back to this.
[18:02.420 --> 18:10.920]  But what I pompously call the variability of the input space really means that books come in all sorts of really wacky formats.
[18:11.140 --> 18:11.920]  Like, a lot.
[18:12.080 --> 18:19.160]  And very quickly, whatever assumption you make about a book other than it exists is going to be proven wrong at some point.
[18:19.220 --> 18:19.620]  It's crazy.
[18:20.840 --> 18:22.660]  Sometimes books are just broken.
[18:23.460 --> 18:24.440]  Pages are missing.
[18:24.860 --> 18:27.260]  There is ambiguity about how to deal with them.
[18:27.260 --> 18:34.840]  Sometimes they have pop-ups, fold-outs, all sorts of things that, like, will make them not fit in our scanners.
[18:35.840 --> 18:41.400]  And then I kind of put it here in the physical world because, like, mostly, like, stuff, like, fails physically.
[18:41.400 --> 18:45.400]  But I mentioned that we built this stuff out of consumer hardware.
[18:45.560 --> 18:48.240]  And consumer hardware is consumer hardware.
[18:48.480 --> 18:50.900]  And sometimes it does not behave as advertised.
[18:50.900 --> 18:52.340]  Or sometimes it doesn't behave at all.
[18:53.500 --> 18:57.180]  So, surprising how much time I had to spend debugging drivers.
[18:58.000 --> 19:01.120]  More than I cared, but it is what it is.
[19:03.260 --> 19:07.940]  So, now that I'm here and I have a pulpit, I actually want to address a little pet peeve of mine.
[19:08.120 --> 19:13.240]  So, a couple of years ago, maybe a couple of months ago, maybe a year ago, this video made rounds on the Internet.
[19:13.400 --> 19:14.860]  And here you see Eliza.
[19:15.680 --> 19:16.340]  She's awesome.
[19:16.480 --> 19:19.680]  She works as a book scanner at the San Francisco Scanning Center.
[19:19.680 --> 19:22.380]  She's one of our best scanners, one of the fastest.
[19:22.580 --> 19:26.180]  In fact, I used to use her as a benchmark.
[19:26.620 --> 19:29.940]  She's an extremely thorough and capable person.
[19:29.940 --> 19:32.060]  And she's really, really good at her job.
[19:32.240 --> 19:34.700]  So, this video made the rounds on Twitter.
[19:35.300 --> 19:47.300]  And if you have not been on Twitter, what you need to know about Twitter is that Twitter is a place for nuance where everybody's very calm and where everybody really takes their time thinking about what they're about to say.
[19:48.060 --> 20:00.780]  And so, when they saw this video, there is a particular type of tech bro who will not be able to resist the temptation of optimizing a thing that he just learned the existence of.
[20:00.860 --> 20:01.820]  And it's invariably a dude.
[20:02.560 --> 20:08.280]  And so, it's the kind of person who will say, oops, why isn't this automated?
[20:08.820 --> 20:09.980]  Why do you have a person?
[20:10.160 --> 20:13.720]  And this really, really annoys me to no end.
[20:13.840 --> 20:15.800]  And so, I just want to talk about this for a second.
[20:15.800 --> 20:19.380]  Because the fact that we have a person in there, that is not a coincidence.
[20:19.540 --> 20:22.200]  It's not because a robot could do it better.
[20:22.440 --> 20:24.620]  So, first of all, there are...
[20:25.380 --> 20:31.620]  If you were to put a robot where that person is, well, first of all, you have to account for this.
[20:31.820 --> 20:32.800]  I mentioned books.
[20:32.920 --> 20:34.660]  They come in all sorts of shapes.
[20:34.760 --> 20:35.540]  They are big.
[20:35.620 --> 20:36.220]  They are heavy.
[20:36.320 --> 20:36.800]  They are bulky.
[20:36.880 --> 20:37.580]  Some of them are fluffy.
[20:37.800 --> 20:38.540]  Some of them are tiny.
[20:38.540 --> 20:39.880]  Some of them have foldouts.
[20:40.060 --> 20:41.620]  Some of them have pop-ups.
[20:42.400 --> 20:46.680]  Some of them, this happened to me, have tomato sauce in the cover.
[20:47.360 --> 20:48.960]  This is a real thing.
[20:50.040 --> 20:51.740]  It's good luck automating that.
[20:52.000 --> 20:55.920]  Like, give a computer, give a robot the task of turning that.
[20:56.020 --> 20:58.140]  And then, you know, texts are not just books.
[20:58.140 --> 21:04.920]  I talk about books a lot, but these are palm leaves, inscripted palm leaves from Bali.
[21:05.080 --> 21:09.620]  This is still a very common type of medium in Southeast Asia.
[21:10.080 --> 21:11.440]  And, you know, this is very valuable text.
[21:11.560 --> 21:13.160]  We want to be able to digitize it.
[21:13.260 --> 21:13.820]  It's a text.
[21:14.000 --> 21:15.360]  We must be able to digitize it.
[21:15.440 --> 21:17.380]  We designed a system to digitize everything.
[21:18.080 --> 21:21.560]  There is no point in trying to automate this.
[21:24.280 --> 21:30.180]  And, actually, it's not just that there is no point trying to automate it.
[21:31.180 --> 21:34.400]  A person does a lot more than what you see.
[21:34.540 --> 21:36.460]  A person does a lot more than turning pages.
[21:36.640 --> 21:39.240]  A person disambiguates ambiguity.
[21:39.760 --> 21:42.940]  A person can figure out when metadata is wrong.
[21:43.200 --> 21:47.400]  A person in this loop is a feature.
[21:47.620 --> 21:48.620]  It is not a bug.
[21:49.380 --> 21:52.940]  And, if you actually want to run the numbers, I will not run the numbers here for you.
[21:53.260 --> 21:55.620]  But, if you actually want to automate...
[21:56.200 --> 21:58.000]  Say, okay, I don't care about the palm leaves.
[21:58.080 --> 22:00.000]  I don't care about, like, the tomato sauce.
[22:00.320 --> 22:02.000]  I only care about Harmony books.
[22:02.300 --> 22:03.780]  And, like, you know, tiny little paperbacks.
[22:04.240 --> 22:05.480]  I just want to scan those.
[22:05.680 --> 22:08.920]  Well, there's only two things you can do.
[22:09.080 --> 22:14.980]  Either you build your own entire automated solution.
[22:15.280 --> 22:16.700]  Good luck building your robot.
[22:17.480 --> 22:19.880]  Or, you just get something off the shelf.
[22:20.040 --> 22:21.180]  And, there are a few commercial vendors.
[22:21.440 --> 22:22.740]  And, like...
[22:22.740 --> 22:30.500]  If you actually run the numbers, what comes out is that the only metric that automation at this scale improves is cost.
[22:32.140 --> 22:32.660]  So...
[22:32.660 --> 22:37.560]  And, since we're on the topic of pet peeves, people are sometimes...
[22:37.560 --> 22:40.580]  Somehow, like, I've been asked about this all the time.
[22:40.580 --> 22:43.440]  I don't know why, but people are super fascinated with OCR.
[22:44.820 --> 22:47.060]  And, unfortunately, I never have anything to say.
[22:47.160 --> 22:48.720]  Because OCR, to me, was...
[22:48.720 --> 22:52.020]  For who doesn't know, OCR is optical character recognition.
[22:52.080 --> 22:57.100]  It's how you turn photos into, like, text files.
[22:57.460 --> 22:58.880]  Which is very important, but...
[22:58.880 --> 23:03.800]  It is, at least for Latin alphabets, it is a solved problem.
[23:03.800 --> 23:06.960]  I never had to worry about OCR not working, ever.
[23:07.340 --> 23:07.900]  So, I'm sorry.
[23:08.000 --> 23:09.000]  I don't know anything about OCR.
[23:09.220 --> 23:10.120]  I'm the wrong person to ask.
[23:10.500 --> 23:10.800]  Anyway.
[23:11.600 --> 23:17.500]  So, in terms of the takeaways from, you know, how do you build a digital library?
[23:17.580 --> 23:18.840]  And then we can move on to why.
[23:20.640 --> 23:21.440]  I think...
[23:22.840 --> 23:27.320]  I think what was hard was not what I thought was gonna be hard.
[23:27.500 --> 23:30.560]  And what was actually hard, I could never even conceptualize.
[23:31.240 --> 23:34.100]  And so, to me, the lesson is, like, maybe a pretty common one.
[23:34.240 --> 23:35.760]  But, you know, I guess it bears repeating.
[23:36.120 --> 23:37.160]  Don't try to be too smart.
[23:37.300 --> 23:38.460]  Just, like, iterate quickly.
[23:38.720 --> 23:43.780]  You can never, kind of, like, really figure it all out in your head before you actually try it.
[23:45.320 --> 23:45.720]  Anyway.
[23:45.720 --> 23:49.380]  So, this is how you build a big library.
[23:49.520 --> 23:51.820]  How you build, like, a digital library at scale.
[23:52.040 --> 23:54.040]  But I want to circle back to this notion.
[23:54.420 --> 24:01.540]  Like, what we started is, a library is, you know, a collection of items that have some value.
[24:01.920 --> 24:10.140]  And the fact that I have spent some time here talking about, you know, a big pipeline and all of these scanners that can digitize a lot of books.
[24:10.140 --> 24:17.200]  It makes, I don't want to make it feel like this is a game of numbers.
[24:17.380 --> 24:18.500]  It certainly is not a game.
[24:18.660 --> 24:20.440]  And it is certainly not a game of numbers.
[24:20.760 --> 24:23.880]  You will pardon my romanticism here.
[24:24.100 --> 24:28.940]  But building a library is an act of love.
[24:29.220 --> 24:34.360]  It's, like, there is something profoundly noble in wanting to preserve knowledge for knowledge's sake.
[24:35.800 --> 24:44.220]  And so, I want to spend just a couple of minutes talking about what is, like, what does this look like at the end of the day?
[24:44.260 --> 24:46.920]  Because we talked about the how, but what is it really?
[24:47.120 --> 24:55.740]  So, a digital library is the ability for you, at any minute of the day, to load up this beautiful edition of the Hypnerotomachia Polyphili.
[24:55.740 --> 24:59.840]  This is a 1499 edition.
[25:00.360 --> 25:03.460]  It's a book by Juan Francesco Colonna.
[25:04.040 --> 25:08.980]  It was published by Aldous Manusius in Venice.
[25:09.300 --> 25:14.740]  This publisher was a scholar, the guy who invented the semicolon and italics.
[25:15.620 --> 25:16.880]  Italics, italics, whatever.
[25:18.280 --> 25:21.440]  And this is a beautiful book, beautifully illustrated.
[25:21.760 --> 25:23.200]  Nobody really knows what it's about.
[25:23.580 --> 25:25.780]  It's kind of written in Latin and Greek, and it's unclear.
[25:26.020 --> 25:35.280]  It, like, talks about this, like, dreamy adventure where Polyphilos tries to get to this woman.
[25:35.740 --> 25:38.860]  Anyway, it's full of, like, weird symbology.
[25:39.260 --> 25:41.160]  And you may ask, why would I want to know about this?
[25:41.240 --> 25:42.400]  Well, first of all, because it's awesome.
[25:42.560 --> 25:45.320]  Second of all, because maybe you've been reading a book called The Code of Four.
[25:45.700 --> 25:46.320]  Great book.
[25:46.400 --> 25:47.240]  I highly recommend it.
[25:47.340 --> 25:48.220]  It's a book about a book.
[25:49.080 --> 25:50.320]  You could expect that from me.
[25:50.920 --> 25:52.420]  Or what about this?
[25:52.720 --> 25:54.020]  This is, I think, a 16.
[25:54.160 --> 25:55.680]  I have, like, I have it in my notes here.
[25:56.180 --> 25:56.940]  This is...
[25:56.940 --> 25:57.240]  Oh, yeah.
[25:58.680 --> 26:00.860]  1575 Tractatus on Hieroglyphics.
[26:01.520 --> 26:03.920]  The illustrations in here are awesome.
[26:04.060 --> 26:07.260]  This was sponsored by the Medici family in Florence.
[26:07.600 --> 26:09.020]  Like, pretty much the whole Renaissance.
[26:11.300 --> 26:12.420]  What about...
[26:12.420 --> 26:17.320]  Oh, this is a collection of emblems and related poems.
[26:17.320 --> 26:18.800]  166 of them.
[26:20.800 --> 26:22.840]  Was that, like, too classical for you?
[26:22.900 --> 26:23.500]  What about this?
[26:23.580 --> 26:24.320]  Do you know what this is?
[26:24.700 --> 26:42.360]  About, maybe, a hundred years before Pete Mondrian made these colors kind of mainstream, Oliver Byrne, a mathematician, published this book where he used colors to elucidate some concepts in Euclid's Elements.
[26:42.360 --> 26:44.380]  And this is just a beautiful, beautiful book.
[26:44.460 --> 26:45.560]  And you can access it anytime.
[26:47.600 --> 26:51.460]  Or if you're feeling a little gory, you have this book about death.
[26:51.580 --> 26:52.640]  It's a dance of death.
[26:52.820 --> 26:57.640]  There are, like, poems and prose and illustration about death, if you're feeling so inclined.
[26:57.880 --> 26:59.060]  Is that not morbid enough?
[26:59.280 --> 27:00.380]  What about the Old Testament?
[27:01.260 --> 27:03.400]  It's a pretty beautiful edition.
[27:04.700 --> 27:07.120]  There's a pretty beautiful edition of the Koran, also.
[27:07.320 --> 27:09.600]  This is, I think, from early 1900s.
[27:09.680 --> 27:13.520]  This was from, I think, 14... 1599.
[27:14.880 --> 27:16.800]  Also, talk about the variability in books.
[27:16.820 --> 27:18.140]  This is left to right.
[27:19.560 --> 27:22.180]  Screwed a whole bunch of stuff up when I had to implement that.
[27:23.540 --> 27:27.040]  This is the Divine Comedy, as verbose as Dante wanted it to be.
[27:27.220 --> 27:27.780]  No, I'm kidding.
[27:27.920 --> 27:30.200]  This is just the introduction.
[27:30.780 --> 27:33.360]  The Divine Comedy is written in rhyme.
[27:33.640 --> 27:34.360]  It looks pretty different.
[27:34.460 --> 27:37.060]  But this is, like, beautiful, beautiful text.
[27:37.640 --> 27:42.580]  Or what about this tractatus on how to avoid premature burial?
[27:42.800 --> 27:45.560]  Apparently, in the mid-1800s, this was a concern.
[27:47.220 --> 27:49.820]  And, you know, why do digital libraries matter?
[27:49.820 --> 27:52.760]  Also, because, like, our memories are short.
[27:52.940 --> 27:57.240]  And sometimes we forget some of the weird ideas that we have or some of the bad ideas that we have.
[27:57.780 --> 28:00.040]  Introducing this tractatus on phrenology.
[28:00.340 --> 28:07.920]  Phrenology is the study of the shape of the head to determine the characteristics and the character of a person.
[28:08.320 --> 28:10.140]  Something that, hopefully, we move down from.
[28:11.040 --> 28:12.200]  But you can access it.
[28:13.700 --> 28:15.120]  There's this beautiful...
[28:15.120 --> 28:15.600]  Oh, yeah.
[28:15.720 --> 28:16.440]  Talk about edgy.
[28:16.440 --> 28:19.760]  This is a beautiful book.
[28:20.120 --> 28:30.780]  It's a series of thoughts from American pilgrims in Brazil.
[28:33.740 --> 28:35.840]  Oh, this is pretty interesting.
[28:36.160 --> 28:38.820]  This is the first book about cryptography.
[28:38.820 --> 28:40.700]  It's from the early 1800s.
[28:40.700 --> 28:43.600]  So, to this crowd, maybe pretty interesting.
[28:44.800 --> 28:46.400]  But I know what you really want.
[28:48.540 --> 28:50.120]  I think this explains itself.
[28:51.160 --> 28:52.560]  Available right now.
[28:54.020 --> 28:55.560]  I know what you'll be reading tonight.
[28:56.060 --> 28:57.700]  Or maybe, what about this?
[28:57.780 --> 29:03.100]  This is a 1943 book from the U.S. Army on how to get tough.
[29:03.100 --> 29:10.120]  And inside, there are, like, a bunch of, like, really good advices if you want to have a fun night out at the bar.
[29:11.520 --> 29:11.920]  Anyway.
[29:12.300 --> 29:12.980]  Oh, okay.
[29:13.060 --> 29:13.920]  This is actually pretty awesome.
[29:15.260 --> 29:21.020]  As I was preparing this presentation, I was going through a bunch of items in the archive.
[29:21.200 --> 29:23.500]  And I was with a very good friend of mine.
[29:24.100 --> 29:25.080]  And he's Thai.
[29:25.080 --> 29:29.280]  And I just pulled up some of the Thai content that we have.
[29:29.360 --> 29:31.180]  Because I remember we had a scanning center in Thailand.
[29:31.840 --> 29:36.860]  So, these are funerary books from Thailand.
[29:37.120 --> 29:44.520]  So, the tradition is that when somebody gets cremated, it's the day that their...
[29:44.520 --> 29:50.080]  It's the last day that their body exists in its, you know, known form.
[29:50.080 --> 30:01.040]  And it is tradition that for each person that gets cremated, the family or the friends, they get together and they make a little book that kind of contains a bunch of scriptures from texts.
[30:01.040 --> 30:06.060]  And it contains just, like, pictures and, like, a memory of somebody's life.
[30:06.060 --> 30:07.800]  And this is how they are remembered.
[30:07.960 --> 30:20.840]  And, I mean, if that wasn't amazing enough in itself, my friend found one of his ancestors in there and, like, learned stuff as I was preparing for this presentation.
[30:21.660 --> 30:22.840]  Libraries in action.
[30:23.100 --> 30:25.560]  You never know what matters to whom.
[30:25.680 --> 30:28.980]  But, sure enough, this mattered a whole deal to him.
[30:29.900 --> 30:31.560]  This was the text in question.
[30:32.920 --> 30:34.100]  Or, what about this?
[30:34.240 --> 30:40.960]  These are books that were sent to us at the archive from a library in Hong Kong.
[30:41.100 --> 30:44.400]  These are political dissent pamphlets.
[30:44.400 --> 30:50.340]  These are things that, you know, somebody has a vested interest to make sure disappear because they matter.
[30:51.580 --> 30:54.380]  And, you know, we talked about, like, the palm leaves.
[30:54.920 --> 31:08.120]  The Internet Archive actually started a palm leaves wiki where not only you find the pictures that we have digitized, they also get transcribed and they can be referenced and linked if you're a researcher.
[31:08.920 --> 31:11.020]  Palmleaf.org is where you can find it.
[31:11.060 --> 31:11.500]  It's pretty cool.
[31:12.560 --> 31:16.840]  And, you know, a lot of these palm trees, palm leaves, they come from Bali.
[31:17.380 --> 31:19.260]  Bali is pretty rock and roll.
[31:19.600 --> 31:22.120]  They, as a country, decided to go full digital.
[31:22.360 --> 31:26.800]  And so, the Balinese digital library is much bigger than palm leaves.
[31:26.920 --> 31:28.740]  There are, like, all sorts of artifacts in here.
[31:29.920 --> 31:36.920]  A digital library also means that you, you know, not only can travel the world, but, like, learn more about your neighborhood.
[31:37.080 --> 31:38.800]  This is the Noi Valley Voice.
[31:39.040 --> 31:42.620]  It's a local newspaper from where I live.
[31:42.700 --> 31:47.900]  And we have all of the additions dating back to, I think, the mid-70s.
[31:49.960 --> 31:51.360]  So, you know, okay.
[31:51.500 --> 31:54.860]  I've shown you a bunch of, like, cool little things.
[31:55.460 --> 32:01.040]  And I want to talk about, for a second, you know, what is a library?
[32:01.440 --> 32:05.340]  We know it's organized and it's a collection of all these cool things.
[32:05.700 --> 32:09.000]  There is one thing that I left out that is, I think, very important I want to stress.
[32:10.140 --> 32:11.340]  Who knows what this is?
[32:13.680 --> 32:14.460]  Any takers?
[32:14.860 --> 32:16.080]  Mark knows what this is.
[32:16.080 --> 32:18.320]  This is the Boston Public Library.
[32:19.440 --> 32:23.760]  This was created by Andrew Carnegie, certainly not a communist.
[32:25.600 --> 32:29.220]  Full expression of American capitalism.
[32:29.640 --> 32:31.260]  You know what it says on the front?
[32:32.120 --> 32:33.960]  I took this picture about a month ago.
[32:34.220 --> 32:35.540]  It says, free to all.
[32:36.520 --> 32:37.980]  Free to all.
[32:38.140 --> 32:42.700]  The fact that the library is free is fundamental to the way it operates.
[32:42.700 --> 32:44.440]  Free is a feature.
[32:44.700 --> 32:46.520]  It's maybe even the killer feature.
[32:49.360 --> 32:54.760]  So, I want to talk about why free is important.
[32:55.200 --> 32:58.160]  And it's, in a way, it's part of what makes the library.
[32:58.460 --> 33:00.700]  Because there are...
[33:01.360 --> 33:04.680]  Okay, so first of all, there is a matter of equality.
[33:05.220 --> 33:07.720]  Knowledge is not equally distributed.
[33:07.720 --> 33:14.800]  And sometimes, you know, this inequality falls on social fault lines.
[33:15.800 --> 33:21.420]  But be that as it may, the role of a library is to act as an equalizer.
[33:22.900 --> 33:26.340]  And it cannot do that if it's pay for play.
[33:26.560 --> 33:27.540]  It just simply cannot.
[33:29.860 --> 33:31.680]  A library is also important...
[33:32.360 --> 33:39.580]  It's important that it is free because it means that it doesn't depend on the money of patrons to exist.
[33:39.820 --> 33:45.620]  And it needs to exist because knowledge is easily lost.
[33:46.040 --> 33:48.080]  You remember what happened in Alexandria.
[33:48.340 --> 33:49.860]  I mean, we weren't there, but you know.
[33:52.280 --> 33:58.440]  And, you know, a library needs to celebrate the diversity of humankind.
[33:59.740 --> 34:04.180]  And you cannot be tethered to a business model and do that.
[34:04.320 --> 34:05.840]  Like, the incentives, they simply don't align.
[34:07.040 --> 34:08.120]  And ultimately...
[34:09.500 --> 34:14.660]  And I think in this day and age, it is particularly painfully obvious that we need this.
[34:14.660 --> 34:17.920]  But a library is a repository of trust.
[34:18.660 --> 34:21.420]  And, you know, having texts that we can trust.
[34:21.760 --> 34:23.500]  Having texts the...
[34:23.500 --> 34:25.800]  Having original sources.
[34:26.180 --> 34:27.560]  That's really, really important.
[34:28.380 --> 34:36.160]  And so, every library, every free public library fulfills all of these things.
[34:36.500 --> 34:40.020]  But digital libraries fulfill even more.
[34:40.180 --> 34:44.560]  And so, we can finally get to talk about why digital libraries matter.
[34:44.560 --> 34:45.200]  There's digital.
[34:45.920 --> 34:49.180]  And I want to propose a few...
[34:49.180 --> 34:50.160]  There's five of them.
[34:50.280 --> 34:51.760]  Five reasons why.
[34:52.040 --> 34:54.720]  The first one is that this is a Brewsterism.
[34:56.060 --> 34:57.700]  Availability drives preservation.
[34:58.000 --> 35:03.140]  Which means, fundamentally, if nobody uses it, it will be lost.
[35:03.400 --> 35:06.460]  And I'm going to break news to you guys.
[35:06.680 --> 35:08.040]  People are on the Internet.
[35:08.900 --> 35:13.320]  So, if we are to have a library, it better be on the Internet.
[35:13.320 --> 35:15.060]  Because that's where people look for things.
[35:15.200 --> 35:19.320]  And if you cannot find something, functionally, it's the same as if it didn't exist.
[35:19.500 --> 35:21.820]  And if it doesn't exist, it will be lost.
[35:23.360 --> 35:28.100]  And, in this sense, availability is a forcing function for preservation.
[35:29.700 --> 35:34.000]  Although, preservation is maybe the more important of the two functions.
[35:35.140 --> 35:37.020]  But I don't want to dwell on that.
[35:38.180 --> 35:46.720]  And so, digital libraries, by virtue of existing, have the ability to perpetuate these digital artifacts that are all so important.
[35:46.720 --> 35:51.660]  I put together a little diagram here.
[35:52.540 --> 36:02.640]  It turns out that physical and digital artifacts have, like, specular properties on many dimensions.
[36:02.640 --> 36:13.960]  Some, you know, I will break no ground saying that, sorry, a digital book is very easy to copy, whereas a physical book is very hard to censor.
[36:16.400 --> 36:19.780]  Physical books require almost no energy.
[36:20.220 --> 36:22.280]  They rarely run out of battery.
[36:22.760 --> 36:27.840]  So, you know, you could engage in a debate about, oh, which one is better?
[36:27.840 --> 36:29.240]  Well, that's not the point.
[36:29.420 --> 36:31.840]  The point is that these are two of the same thing.
[36:32.020 --> 36:33.460]  They're two sides of a new coin.
[36:33.780 --> 36:50.980]  If you forgive my bellic metaphor, I think that with e-books, we are in a little bit of the same place where we were when airplanes were first invented and used in war.
[36:51.440 --> 36:58.780]  So, at the beginning, you know, airplanes were used for reconnaissance and mostly for fighting the same war with a new tool.
[36:58.780 --> 37:04.520]  But over time, what really became evident is that airplanes are an entirely new dimension.
[37:04.860 --> 37:20.100]  And I think with books, we are in a similar situation where this new digital artifact and these digital libraries that we're building, they really can offer features that were unaccessible before, even inconceivable.
[37:20.100 --> 37:21.440]  Think about accessibility.
[37:21.580 --> 37:25.680]  That doesn't just mean, you know, scaling fonts up and down, and that's also very important.
[37:26.020 --> 37:27.960]  It means also read aloud.
[37:28.300 --> 37:34.000]  It means, like, the ability for people who are impaired in various ways to access materials.
[37:34.000 --> 37:40.520]  And it also means the ability to access these items when you cannot physically go to the library.
[37:40.780 --> 37:45.940]  And a lot of you in here are able-bodied, but all of you in here lived through COVID.
[37:46.220 --> 37:48.860]  And at some point, we couldn't go to the library.
[37:49.020 --> 37:51.680]  And this suddenly became a really, really important accessibility.
[37:53.400 --> 37:59.780]  And since we're not tethered to the atoms, you know, digital libraries can really help there.
[38:00.740 --> 38:03.400]  Searching and indexing, that's another fantastic feature.
[38:03.640 --> 38:08.900]  Like, at the Internet Archive, my friend Giovanni implemented a full-text search engine.
[38:08.900 --> 38:19.580]  And right now, for free, right a second, you can go to archive.org, and in the homepage, it lets you search the full text of 40, 50, however many million texts.
[38:19.780 --> 38:22.420]  There is, like, patents and papers and books and what have you.
[38:22.560 --> 38:25.320]  You can search all of that instantly, for free, right now.
[38:25.720 --> 38:30.400]  This was, to me, this is mind-boggling still, because I use this thing almost every day.
[38:30.620 --> 38:32.600]  And whenever it works, I'm like, what?
[38:33.460 --> 38:37.840]  It's, I mean, I also, like, know, like, how much goes into making this system.
[38:38.040 --> 38:43.480]  But the fact that we can realistically have it, and we can have it for free, that's just amazing.
[38:45.180 --> 38:46.520]  Oh, and then there is sharing.
[38:46.720 --> 38:50.940]  And I want to talk about sharing, because sharing is not just, like, your little button, right?
[38:51.020 --> 38:54.260]  And the button matters, like, sending stuff to your friends, that's really important.
[38:55.040 --> 38:58.580]  But there is something to be said about sharing in the economics of scale, right?
[38:58.580 --> 39:02.740]  It's one of the properties of digital artifacts is that they are very light, right?
[39:02.800 --> 39:06.860]  Once you created it, there is almost no cost in carrying it around.
[39:07.220 --> 39:26.700]  And under the right predications, let's say, it is entirely conceivable that after the digital artifact has been created, it can facilitate things like interlibrary loans, like you no longer have to send the actual book.
[39:27.240 --> 39:33.780]  You can, there is this service, I don't recall the name, but you can ask a library to send you, like, a chunk of a book.
[39:34.520 --> 39:36.100]  That can also be sped up.
[39:36.400 --> 39:43.540]  You can really supercharge the sharing infrastructure of libraries with, by building a digital library.
[39:44.960 --> 39:54.340]  There's obviously multimedia, like, embedding, like, the ability of really experiencing texts and learning in a way that we haven't before.
[39:54.340 --> 39:57.140]  I, for instance, for one, I really struggle.
[39:57.340 --> 39:59.800]  I really like maths, but I'm not good at it.
[40:00.360 --> 40:05.860]  And I really struggle learning it from books, but if I can, like, see stuff that moves, that really helps me.
[40:06.040 --> 40:08.160]  So that's a classic example of that.
[40:09.060 --> 40:11.460]  And then there's linking.
[40:11.580 --> 40:21.800]  I think this is another, like, killer feature that I think if you have spent any amount of time doing research on Wikipedia, this will speak to you.
[40:21.800 --> 40:24.220]  So this page is for Martin Luther King.
[40:24.400 --> 40:30.020]  And at the end, down here, you have a quotation for Cutler S.
[40:30.080 --> 40:30.620]  Abuse of Power.
[40:30.760 --> 40:35.740]  Well, at the archive, we went and fixed two million of these links.
[40:35.760 --> 40:39.800]  And now they deep link right into the page where the citation is for.
[40:40.680 --> 40:44.980]  You can have access to all of this wealth of information at a click.
[40:45.140 --> 40:46.900]  That's what a digital library can do.
[40:46.980 --> 40:57.860]  That's why it matters, because it really expands your ability to be creative and to do research and to learn.
[40:58.400 --> 41:13.680]  And then, ultimately, the reason why a digital library matters is that in this world that we came to live in, we are making very heavy use of digital assets, but somehow we don't really have the infrastructure for it.
[41:13.760 --> 41:17.380]  And so I don't know if you guys have heard about fake news, but that seems to be a problem.
[41:18.160 --> 41:24.200]  First source, like original sources that are trustworthy are hard to come by.
[41:24.660 --> 41:27.720]  Libraries are not just places.
[41:27.900 --> 41:29.120]  Libraries are organizations.
[41:29.120 --> 41:31.520]  They are people and they are reservoirs of trust.
[41:31.860 --> 41:39.400]  Having a lender of last resort that you can trust that will have the right untampered original file may really matter.
[41:40.520 --> 41:46.980]  And also, you know, lots of copies keep things safe, but what happens if nobody has the copy?
[41:47.080 --> 41:49.060]  We have to have a lender of last resort.
[41:49.240 --> 41:54.320]  Somebody who has the file that all the torrents resolve to.
[41:56.860 --> 41:58.600]  Anyway, I want to just wrap this up.
[41:58.860 --> 42:03.760]  I hope I made a case for why building digital libraries matter.
[42:03.760 --> 42:09.860]  But I want to make it further in the sense of like, how does this like affect your life right now?
[42:10.240 --> 42:12.880]  Well, here's a couple of ideas.
[42:13.400 --> 42:15.910]  Reading, like knowing that this resource exists.
[42:16.520 --> 42:19.540]  I'm not talking about the Internet Archive, but there is tons of other digital libraries.
[42:19.620 --> 42:21.980]  And I encourage you to discover them and even build them.
[42:23.420 --> 42:30.840]  But if you don't take even an active role in building one, you can certainly be a patron and consume it.
[42:30.840 --> 42:42.040]  So be reminded and be aware of the fact that there is like this joyous resource that you can use and you can just like delve into.
[42:42.780 --> 42:47.960]  And you know, at least on the archive, you can upload your digital files, but you're already doing that on your Apple Cloud or whatever.
[42:48.120 --> 42:53.280]  Just like be mindful that what you really are doing is constructing a digital library.
[42:54.620 --> 42:55.880]  Weaving books into Wikipedia.
[42:57.520 --> 42:58.280]  Very important.
[42:59.080 --> 43:04.000]  If you find a broken link, you can fix it yourself or tell us.
[43:04.300 --> 43:06.840]  Oh, and you have books that you think should make it.
[43:06.980 --> 43:11.380]  If you have a book that you have seen nowhere and you want it to be on the archive, send it to us.
[43:12.120 --> 43:12.840]  Archive.org.
[43:13.000 --> 43:14.180]  You will find the address on there.
[43:14.380 --> 43:15.300]  We take physical donations.
[43:15.640 --> 43:16.440]  Send it to us.
[43:18.820 --> 43:24.340]  You know, after this experience of COVID, we are all homeschoolers, right?
[43:24.400 --> 43:30.440]  We all had the experience of being like stuck at home and just have like no access to the outside world.
[43:30.660 --> 43:34.360]  That's a reminder of like how this library may be useful to you.
[43:34.460 --> 43:36.000]  All of the stuff at your fingertips.
[43:36.120 --> 43:37.640]  It's not just the caprice.
[43:37.840 --> 43:38.620]  It really matters.
[43:38.800 --> 43:41.380]  Think of students who couldn't complete their classes.
[43:42.800 --> 43:44.740]  And, you know, you could travel the world.
[43:45.100 --> 43:46.960]  Access all of these libraries around the world.
[43:47.060 --> 43:50.160]  There is libraries everywhere and everybody is building digital libraries.
[43:50.460 --> 43:50.740]  It's not...
[43:50.740 --> 43:51.360]  I'm not just like...
[43:51.360 --> 43:52.560]  This is not the spot for the archive.
[43:52.780 --> 43:56.800]  This is really a love letter to digital libraries.
[43:59.740 --> 44:01.200]  And I think...
[44:01.200 --> 44:02.780]  I just want to conclude by saying this.
[44:02.780 --> 44:03.460]  I think...
[44:05.060 --> 44:08.740]  From the dawn of time, you know, people have built libraries.
[44:09.040 --> 44:11.040]  Like they are a fundamental part of...
[44:11.040 --> 44:11.520]  Sorry.
[44:11.600 --> 44:15.380]  Knowledge accumulation is a fundamental part of like how a society functions.
[44:15.660 --> 44:20.360]  And if we are to have a functional society, we have to have functional libraries.
[44:20.600 --> 44:21.540]  And if we have...
[44:21.540 --> 44:26.320]  If we want to have a functional Internet, we have to have functional digital libraries.
[44:26.320 --> 44:36.320]  The dream of the Internet was to have this open library, this open conversation, and this free exchange of knowledge.
[44:38.140 --> 44:43.040]  Boy, has it turned out to be harder than we thought, but it was the right idea, and it still is.
[44:43.520 --> 44:44.060]  Thank you.
[44:53.470 --> 44:54.750]  I think we've got a couple of minutes.
[44:55.230 --> 44:56.510]  Yes, so we have questions.
[44:56.570 --> 44:58.130]  If you have questions, there's a microphone here.
[44:58.730 --> 45:00.170]  I swear to God if you ask me about OCR.
[45:00.170 --> 45:01.230]  No, no.
[45:01.890 --> 45:05.590]  What was your biggest achievement and toughest challenge in your journey?
[45:08.050 --> 45:09.050]  Wow, that's a good one.
[45:09.310 --> 45:10.330]  That's like an interview question.
[45:10.890 --> 45:13.050]  In five years, where will you see yourself now?
[45:17.030 --> 45:23.330]  Okay, not to pat myself on the back, but I think my biggest achievement, nobody cares about this, thankfully.
[45:23.830 --> 45:26.830]  But I rewrote that whole app that you saw.
[45:27.810 --> 45:29.850]  When I picked it up, it was kind of a mess.
[45:30.290 --> 45:34.830]  And then, you know, I made it an even bigger mess, and then it went into production.
[45:35.090 --> 45:39.570]  And so I had to refactor the whole thing, and I managed to do it.
[45:39.730 --> 45:44.050]  I could talk for hours about what that entailed, but it was like...
[45:44.050 --> 45:48.110]  There is this famous metaphor of changing the wheels where you're going 100 miles an hour down the highway.
[45:48.110 --> 45:50.470]  It was exactly that, except the car was on fire.
[45:54.330 --> 45:56.310]  That was my biggest... What was the hardest thing?
[45:56.430 --> 45:57.330]  Oh, definitely the corrections.
[45:57.630 --> 46:03.110]  Definitely, like, that rigamarole of sending back stuff to the original station and, like, track where the book is.
[46:03.190 --> 46:04.030]  That was a nightmare.
[46:04.310 --> 46:05.130]  That was a nightmare.
[46:05.450 --> 46:07.610]  The other online question we have is...
[46:07.610 --> 46:12.890]  And this kind of goes to, like, libraries can have books that are still in print, are still being published.
[46:12.890 --> 46:13.310]  Sure.
[46:13.630 --> 46:18.670]  But how does this digitalization affect copyrights for books still being published?
[46:20.170 --> 46:21.590]  How does it affect copyright?
[46:21.750 --> 46:22.950]  Copyright, so that way...
[46:22.950 --> 46:26.850]  Oh, that is a much bigger conversation that I'm really not qualified to talk about.
[46:30.120 --> 46:30.720]  Thank you.
[46:30.820 --> 46:31.380]  Thank you very much.
[46:31.580 --> 46:32.160]  Very interesting.
[46:32.580 --> 46:34.080]  I have a question for...
[46:34.080 --> 46:34.860]  Well, I have a question, rather.
[46:36.350 --> 46:44.940]  If, hypothetically, I have a partner who looks at me wrong whenever I bring a new book in the house, how can I do this at home so I can keep all my books?
[46:46.720 --> 46:48.380]  How you can digitize stuff at home?
[46:48.700 --> 46:49.060]  Yes.
[46:49.400 --> 46:50.740]  Oh, that's a really tough one.
[46:51.820 --> 46:56.500]  I guess it's a function of how much time you have and how much effort you want to put into it.
[47:00.240 --> 47:02.140]  There is a great little project.
[47:02.140 --> 47:04.660]  It's called the DIY Book Scanner.
[47:05.200 --> 47:10.280]  It's something that we actually drew from in designing our own scanner.
[47:11.080 --> 47:17.220]  It's an open source hardware project that will help you build a little rig.
[47:17.460 --> 47:18.740]  It's made of, like, wood.
[47:18.860 --> 47:19.820]  It's very simple to build.
[47:21.140 --> 47:25.380]  It creates a cradle with Plexiglass and some wood.
[47:25.380 --> 47:32.380]  And you can, like, with two cameras, like very cheap cameras, you can actually build a pretty functional books digitization system.
[47:33.200 --> 47:39.800]  As far as software is concerned, there is a ton of open source software to do book scanning.
[47:40.200 --> 47:44.840]  But if you want to know more about that, I can talk to you at length about software for sure.
[47:46.540 --> 47:49.300]  Davide, I think people here might be interested in the lawsuit.
[47:50.140 --> 47:51.940]  Are you going to mention that?
[47:52.500 --> 47:53.700]  I don't know what you're talking about.
[47:54.220 --> 47:54.620]  No.
[47:54.860 --> 47:55.020]  Aha.
[47:55.700 --> 47:57.080]  I know what to say.
[47:57.320 --> 47:58.480]  I've watched so many movies.
[47:58.640 --> 47:59.080]  Oh, my God.
[47:59.160 --> 48:00.180]  My entire life led up to this.
[48:00.580 --> 48:06.740]  Under the provisions afforded to me under the Fifth Amendment of this constitution, I respectfully decline to answer the question.
[48:09.160 --> 48:10.420]  Do you mind...
[48:10.420 --> 48:13.800]  Finally, all of the time watching the Engram trial is paying back.
[48:14.100 --> 48:15.400]  Do you mind if I mention it?
[48:16.300 --> 48:17.020]  Kind of.
[48:17.440 --> 48:17.580]  Okay.
[48:19.260 --> 48:33.700]  Well, let me just say, four publishers have sued the Internet Archive for copyright infringement, for lending out too many books when all the physical libraries in the world were closed, and setting up the emergency lending library.
[48:34.800 --> 48:44.320]  Before that, we would lend as many books as we had out at a time as we had physical copies of, and we exceeded those limits when all the libraries were closed.
[48:44.520 --> 48:50.900]  So you might look at archive.org in the blog section to see the details about that.
[48:56.260 --> 49:10.820]  I think my question may have almost just been answered, but I wanted to expand on the previous copyright question by narrowing it to... I did notice the feature on the screen about borrowing the book, and I haven't... I probably should have, but I haven't taken advantage of that.
[49:11.100 --> 49:16.540]  Can you tell me more about how that works, and does that only apply to copyrighted work, so the rest you can just see it without borrowing it?
[49:16.560 --> 49:16.900]  Oh, sure.
[49:17.180 --> 49:21.580]  The way that works is that you can...
[49:21.580 --> 49:30.780]  Like, only one person at the time can access the book, and you make an account on archive.org, and you request access to the book, much like you would do in a normal library.
[49:30.780 --> 49:34.640]  You have it for an amount of time, and then you return it, and the next person in line gets it.
[49:35.240 --> 49:35.680]  It's free.
[49:38.340 --> 49:44.880]  Is there anything that you guys, like, either refused to publish or refused to archive that you were given or submitted, or...?
[49:46.560 --> 49:46.880]  Oh...
[49:49.120 --> 49:49.760]  You mean...
[49:49.760 --> 49:52.020]  Like, we looked at it, we're like, uh-huh, not this?
[49:52.060 --> 49:54.120]  Yeah, like it was a super obscene or something?
[49:54.440 --> 49:55.180]  Oh, yeah, not right.
[49:55.380 --> 49:57.080]  No, I think...
[49:57.080 --> 49:57.980]  No, not really.
[49:58.260 --> 49:59.280]  I mean, there is like...
[49:59.280 --> 50:05.280]  I mean, ISIS at some point was like really big on uploading videos on the archive, and so we tried to like put a lid on that.
[50:05.420 --> 50:13.900]  But we didn't delete it, we just like, you know, like tried to dampen the amount of propaganda that would go through our channels, trying not to serve them that way.
[50:15.360 --> 50:18.540]  This is actually a related question.
[50:19.680 --> 50:20.360]  Do...
[50:20.360 --> 50:30.720]  To what extent do digital libraries see it as their responsibility, or to what extent do you think it's their responsibility to contextualize this sort of information?
[50:31.140 --> 50:34.980]  Like take for example phrenology from your presentation.
[50:36.200 --> 50:37.640]  That's a really profound question.
[50:39.800 --> 50:43.100]  Because libraries are typically not just a place.
[50:43.320 --> 50:45.620]  They are, you know, organizations, they're made of people.
[50:45.960 --> 50:49.980]  And the figure of a librarian is really a key part of that.
[50:50.180 --> 50:52.200]  I don't have an answer to that yet.
[50:52.420 --> 50:53.680]  I think there should like...
[50:53.680 --> 50:55.900]  The answer probably there should be some responsibility.
[50:58.040 --> 51:01.760]  How you do that at scale, kind of hard to answer.
[51:01.760 --> 51:11.420]  I know that for smaller, more scoped libraries, there is like a lot of curation that goes on into, you know, contextualizing artifacts with metadata and such.
[51:11.760 --> 51:14.660]  It definitely can be done in that way at scale.
[51:15.880 --> 51:17.220]  An open problem, I guess.
[51:17.380 --> 51:18.860]  But a very important one.
[51:19.060 --> 51:19.840]  Very important one.
[51:21.900 --> 51:22.620]  Great talk.
[51:22.840 --> 51:23.520]  Thank you very much.
[51:23.660 --> 51:23.920]  Thank you.
[51:24.460 --> 51:28.360]  I saw you're a fan of Borges in your biography.
[51:28.920 --> 51:34.560]  I was wondering how, if you could talk about what in his work inspired you to do this.
[51:34.600 --> 51:37.980]  And how do you think Borges would have viewed the Internet Archive?
[51:38.140 --> 51:38.840]  You said Borges?
[51:39.060 --> 51:39.160]  Yeah.
[51:39.900 --> 51:42.240]  I actually forgot to mention.
[51:42.340 --> 51:42.640]  Thank you.
[51:42.740 --> 51:43.700]  Thank you very much for that.
[51:44.040 --> 51:45.240]  So Borges is my favorite writer.
[51:45.800 --> 51:51.440]  And he famously called the universe a library.
[51:52.260 --> 51:57.180]  I think that may be a little ambitious, but certainly society depends on libraries.
[51:57.960 --> 51:58.280]  There's...
[51:58.900 --> 52:00.560]  Don't get me started on Borges, man.
[52:00.680 --> 52:01.740]  Like, we're gonna be here all night.
[52:01.800 --> 52:02.960]  I only have, I think, one minute.
[52:03.060 --> 52:03.560]  Not even that.
[52:03.960 --> 52:04.680]  I think we're done.
[52:05.080 --> 52:05.580]  We're done?
[52:05.860 --> 52:06.180]  We're done.
[52:06.440 --> 52:06.640]  Woo!
[52:06.760 --> 52:07.220]  We did it!
[52:07.820 --> 52:08.480]  Thank you, guys.
[52:11.790 --> 52:12.350]  Thank you, sir.
[52:12.650 --> 52:12.970]  Thank you.
[52:16.610 --> 52:17.810]  Don't forget your power cord.
[52:22.220 --> 52:26.880]  If you have any questions for Davide, he'll be right outside the hall signing autographs.
[52:27.260 --> 52:28.900]  Digital autographs.
[52:29.160 --> 52:30.040]  Thank you.