[00:01.210 --> 00:03.530] We are the transfaeries. [00:03.890 --> 00:05.650] We're a plural system. [00:06.050 --> 00:11.330] We work with ML, machine learning models, in industry. [00:11.330 --> 00:14.050] We've done a little bit of work in academia with them. [00:14.250 --> 00:18.270] We also do have our own projects that we do with ML. [00:18.450 --> 00:19.490] We make chatbots. [00:19.750 --> 00:23.590] We write sci-fi involving artificial intelligence. [00:23.590 --> 00:27.350] And we are a spirit-initiated traditional witch. [00:27.910 --> 00:31.150] If you want to know more about that, come find me after the talk. [00:33.090 --> 00:35.790] So, to start with, animism. [00:36.850 --> 00:41.370] Who here already has some conception of what animism is, by show of hands? [00:41.570 --> 00:43.150] Okay, okay, we've got some people. [00:43.670 --> 00:45.970] Anyone identify as an animist? [00:46.870 --> 00:48.130] Oh, less people. [00:48.290 --> 00:50.270] Let's see if that changes by the end of the talk. [00:50.950 --> 00:55.670] So, just so that we're all on the same page, here's a definition of animism. [00:55.990 --> 01:08.170] It is a belief found in many cultures on the earth that everything and everyone is a person and has some sort of anima, some sort of life essence. [01:09.190 --> 01:19.070] It was a term that was first coined in like the 1870s by some white guy who was studying various native peoples and being like... [01:19.070 --> 01:21.010] It was kind of racist at the beginning. [01:21.210 --> 01:28.470] It was like, man, these natives, they think their gods live in objects, not like our god who lives in heaven and is correct. [01:29.990 --> 01:31.270] But it kept evolving. [01:31.270 --> 01:44.210] In the 1950s, some other white guy was studying the Ojibwe people of North America, and he was a little bit more like nuanced, more like, yeah, let's hear out what they have to say. [01:44.210 --> 01:46.330] Let's read their beliefs for respect. [01:46.570 --> 01:48.330] People would call this the new animism. [01:48.950 --> 01:55.970] We're actually in a very exciting point now where a lot of people, like anthropology as a field has opened up. [01:56.150 --> 02:03.670] There are anthropologists studying cultures from within their own culture, some of whom even might call themselves animists. [02:03.850 --> 02:09.630] So the term has been more embraced by these cultures as opposed to being a thing that's being applied to them. [02:09.950 --> 02:13.590] But it is, at its core, an anthropological construct. [02:13.590 --> 02:21.310] It is a thing that anthropologists were like, okay, this is a thing we see in many cultures, let's put a name to it. [02:21.770 --> 02:26.910] And there's actually, there's a couple more things that go with it than just the anima thing. [02:29.650 --> 02:31.950] There's, everything's a person, that's the first. [02:32.110 --> 02:37.210] The second thing is, we all exist in relationship to all other people. [02:37.370 --> 02:41.050] And people, of course, being this brother sense of all people of the world. [02:41.050 --> 02:42.690] And ancestry. [02:43.370 --> 02:52.310] There's some sense of honoring ancestors, both blood ancestors and a more subtle form of ancestry to say, like, we come from the land. [02:52.450 --> 02:53.350] We honor the land. [02:53.710 --> 02:56.370] The animals that we eat make up our body. [02:56.490 --> 02:57.970] We honor the animals that we lead. [02:58.550 --> 03:06.610] In every aspect, in animist cultures, you're always trying to figure out what your relationship to all these other people and other spirits are. [03:07.150 --> 03:08.810] And some are more friendly. [03:08.830 --> 03:10.110] Some are more wary. [03:11.190 --> 03:14.110] But there are many, many, many different ways to practice animism. [03:14.110 --> 03:17.350] And the picture, by the way, is a kamidana. [03:17.690 --> 03:21.650] It's a type of home shrine used in Chinto. [03:22.930 --> 03:27.130] Which, by the way, is considered an animist religion. [03:27.690 --> 03:29.230] Now, animism isn't a religion. [03:29.350 --> 03:30.970] It's a set of beliefs. [03:31.790 --> 03:35.070] Sometimes you find that there's a religion attached to it. [03:35.110 --> 03:37.950] Or sometimes it's more of a general cultural thing. [03:37.950 --> 03:43.430] What's a religion and what's culture is not always super tightly controlled. [03:43.750 --> 03:49.630] But there are literally thousands of cultures that have been described as animist. [03:50.050 --> 03:51.950] Chinto is one, like we just mentioned. [03:52.290 --> 03:55.010] The Thai folk religion that's practiced in Thailand. [03:55.770 --> 03:58.130] The Maori people of Aote Aura. [03:58.550 --> 04:04.710] The Guarani people of South America, who are said to practice pantheistic animism. [04:04.710 --> 04:08.690] And this happens often where there might be like a pantheon of gods. [04:08.770 --> 04:11.990] But they're not necessarily the only spirits in the culture. [04:12.970 --> 04:20.810] The Lenape people, which in Queens we are in Lenape territory, have also been described as holding animist beliefs. [04:21.030 --> 04:24.230] They certainly honor their ancestors and the spirits. [04:24.790 --> 04:27.930] And finally, what I like to call everyday animism. [04:28.290 --> 04:31.630] Which is a thing that a lot of us might do without even realizing. [04:32.270 --> 04:39.510] If you ever like put all your toys together, because you didn't want them to be alone in the middle of the night. [04:40.770 --> 04:44.230] Or you've wondered if your bike is sad to be left out in the rain. [04:44.550 --> 04:48.550] Those are all like little forms of animism that we all kind of do in our everyday lives. [04:49.210 --> 04:55.670] The Count Mary method for cleaning up that had the thing where you think something before you throw it away. [04:56.030 --> 04:57.850] That's a little bit of animism too. [04:59.570 --> 05:05.930] Also, your bike is not sad to be left out in the rain, as long as you're like making sure to take care of it properly. [05:07.010 --> 05:10.830] One of the... an animist concept that I identify with is like tools. [05:12.130 --> 05:14.290] Like, what is a tool like? [05:14.290 --> 05:16.930] A tool likes to be used and it likes to be taken care of. [05:17.970 --> 05:19.770] And that's one way to think about it. [05:21.170 --> 05:22.430] So that's animism. [05:22.770 --> 05:25.410] Now we can move on to artificial intelligence. [05:26.610 --> 05:29.150] Psych, we're going to talk about computer systems first. [05:30.010 --> 05:41.810] And the reason we're going to do it this way is we're going to try to trace the ancestry of our modern AI systems that we're all like so concerned about right now. [05:42.150 --> 05:44.050] So let's start at the very beginning. [05:44.850 --> 05:46.690] This is the difference engine. [05:47.250 --> 05:49.230] This is the machine that started it all. [05:49.370 --> 05:51.150] The Charles Babbage design. [05:51.570 --> 05:54.890] The one that Ada Lovelace invented programming on. [05:55.050 --> 05:56.250] It's a huge thing. [05:57.310 --> 06:00.730] In the beginning, a lot of early computers were huge. [06:01.350 --> 06:03.210] Over there is the Cray supercomputer. [06:03.270 --> 06:07.250] It's so big that it has a bench built into it that you can sit in it. [06:07.670 --> 06:11.750] This was like top of the line model, I think in like 1971. [06:13.070 --> 06:15.330] It had eight megabytes of RAM. [06:16.210 --> 06:17.130] Megabytes, right? [06:18.630 --> 06:33.170] And so computers have gone through this really interesting evolution where they started at these massive honking things that we only had in labs, in universities, or in private laboratories like Bell Labs, IBM, you name it. [06:34.450 --> 06:39.310] And that started all the way, like digital computers started being a thing like the 30s. [06:39.650 --> 06:46.030] The personal computer, the one that you could have at your home, that starts being a thing more in the 60s, 70s. [06:47.270 --> 06:52.630] And so computers start to move into our own homes then. [06:53.570 --> 06:56.230] And these are all, you know, ancestors. [06:56.470 --> 06:59.030] So this is the ancestor of all of our modern computers. [07:01.550 --> 07:03.190] After that, you know, it splits. [07:03.190 --> 07:06.950] There's probably other ancestors, the Antikythera mechanism you could claim, whatever. [07:07.350 --> 07:08.550] I guess it's not programmable. [07:11.850 --> 07:18.270] But moving on, when computers start to network, that's when things get interesting. [07:18.970 --> 07:23.590] And at the beginning, it was mostly computers talking to each other, computer to computer. [07:23.630 --> 07:25.430] You could even say person to person. [07:25.710 --> 07:28.310] But then soon, there were servers. [07:28.310 --> 07:33.010] They're computers who only exist to serve content to other computers. [07:33.210 --> 07:36.310] They don't necessarily have an owner or an operator. [07:37.010 --> 07:40.610] But they will have a sysadmin who administers them. [07:41.450 --> 07:50.550] And in the sysadmin world, in the DevOps world now, we talk about this paradigm, this shifting paradigm of pets versus cattle. [07:50.950 --> 07:56.370] In the beginning, you could host your website on your desktop, on your desk. [07:58.570 --> 08:03.530] Eventually, maybe you needed like a closet full of servers, but you could have it in your office. [08:03.530 --> 08:07.630] And like, if you have five servers, maybe you give them names. [08:07.830 --> 08:09.990] You're like, you treat them nicely. [08:09.990 --> 08:12.190] You're like, oh, George is acting up again. [08:12.390 --> 08:14.170] Or Regina needs to be updated. [08:14.170 --> 08:17.150] And oh, my God, whoever is down. [08:18.510 --> 08:27.070] When we start moving more into the cloud paradigm, people start talking about computers, our actual physical computers being more like cattle. [08:27.310 --> 08:30.150] You know, they get names like U.S. [08:30.310 --> 08:30.910] East 1. [08:31.190 --> 08:32.750] And that's the name of the data center. [08:32.990 --> 08:35.510] An individual computer is probably called U.S. [08:35.630 --> 08:36.250] East 1. [08:36.270 --> 08:40.470] And then a really huge UID or something like that. [08:40.470 --> 08:42.070] You start to have serial numbers. [08:42.410 --> 08:43.730] You don't really care. [08:43.970 --> 08:45.850] One, like Burnside, you throw it out. [08:45.950 --> 08:46.870] You put a new one in. [08:47.210 --> 08:55.170] If you're not in the data center, if you're a sysadmin who hires out a data center to Google, you never get to see the machines. [08:55.170 --> 08:58.350] You only get to see the stuff running on the machines. [08:58.350 --> 09:01.730] The ghosts of your programs, maybe. [09:03.470 --> 09:06.730] And the relation becomes more depersonalized. [09:07.270 --> 09:14.450] However, there's still little flavors of certain animism that creep in. [09:14.590 --> 09:21.050] So this is a picture of a popular Taiwanese corn puff snack. [09:21.370 --> 09:24.750] And they're called Kwai Kwai, which means it's kind of like... [09:24.750 --> 09:27.310] It's how you would describe something that's behaving well. [09:27.310 --> 09:35.830] And so it's a practice in some Taiwanese data centers to put Kwai Kwai next to the server, so that they'll behave well. [09:36.530 --> 09:40.390] And of course, you know, data centers are becoming more controlled spaces. [09:40.790 --> 09:43.030] You don't want people putting stuff near the servers. [09:43.370 --> 09:45.290] Our relationship has certainly changed. [09:51.630 --> 09:57.970] In moving forward nowadays, we're talking about edge computing devices and the Internet of Things. [09:57.990 --> 10:03.570] That's just another evolution of computers, where now computers aren't even really like computers. [10:03.570 --> 10:06.610] They're like a single thing, and they might be in your house. [10:06.830 --> 10:12.570] But people talk about like, how can we get most things to run on these devices? [10:12.750 --> 10:17.090] What can we get to run on your smartwatch that doesn't need to run on some data center? [10:17.090 --> 10:18.430] That's like edge computing. [10:18.890 --> 10:35.870] And so we kind of start to have this kind of like weird digital forest happening, or we have all these critters, all our smartphones, all our ring cameras, and then you have your bigger creatures, your laptops, your desktops, and they're all feeding into these central servers. [10:36.390 --> 10:40.530] They're like your trees or your rivers, where maybe a lot of different computers connect. [10:40.530 --> 10:46.930] You got your mycelial network of connections like going all over the roots. [10:48.650 --> 11:01.030] And the reason I bring this up is because I want to talk about this word, anthropomorphization, which is the taking something and applying the qualities of humans to it. [11:01.590 --> 11:03.530] And animism is not anthropomorphization. [11:03.530 --> 11:07.390] We are not saying computers are human. [11:08.150 --> 11:11.410] We are also not saying computers are critters in the forest. [11:11.610 --> 11:12.630] These are all metaphors. [11:12.830 --> 11:16.990] Like it's not don't do anthropomorphization, do animalization. [11:17.390 --> 11:21.330] You want to understand the person that is the computer. [11:22.110 --> 11:26.850] You understand that it's made up of not just one thing, but it's made up of many different things. [11:27.090 --> 11:31.350] And you want to get intimate with every aspect of it. [11:31.350 --> 11:34.330] That is what doing animism to computers is. [11:34.550 --> 11:35.630] It's not necessary. [11:35.770 --> 11:37.550] Like you can pet them. [11:37.670 --> 11:39.030] You can say nice things to them. [11:39.110 --> 11:39.690] That is okay. [11:39.690 --> 11:42.590] But it is not the main thing, right? [11:42.830 --> 11:46.130] I think anthropomorphization is a good first start. [11:46.650 --> 11:51.010] Because we all, to some extent or another, have some human in us. [11:51.110 --> 11:52.330] We live in human bodies. [11:52.930 --> 12:00.730] So as we start to relate with something we don't understand, the obvious thing to do is we'll treat it how we would want to be treated, right? [12:00.730 --> 12:07.530] But eventually, as you get to know it, you get to understand, okay, what does it want to be treated like? [12:07.750 --> 12:15.290] And for something like your animals, your pets, you understand that it means like feeding them, taking care of them, giving them company. [12:16.130 --> 12:21.290] For computers, we're still figuring out what the right relationship is, I think. [12:22.090 --> 12:23.050] And that's okay. [12:23.350 --> 12:24.890] It's an ever-evolving process. [12:24.890 --> 12:26.030] We're all in it together. [12:27.930 --> 12:29.890] One last thing I want to talk about. [12:30.970 --> 12:33.770] It's Cybersyn and the global shipping industry. [12:39.830 --> 12:50.610] Was this early project in 1970s in cybernetics in Chile, under Salvador Agenda's government, Salvador Agenda, the GOAT, IRP. [12:52.090 --> 13:00.990] And the idea with Cybersyn was, okay, we have all these factories, and they're all producing, and they're all keeping track of how much resources they have, how much output, input. [13:02.150 --> 13:14.850] Let us have all the factories in the country send all that data over telex terminals to a central control system where we can, like, analyze all the data and make decisions for it. [13:16.870 --> 13:25.790] Agenda, unfortunately, got cooped by some fascists, and so Cybersyn didn't live up to its full potentials. [13:25.830 --> 13:34.450] But the people who were developing Cybersyn were also developing a lot of the early concepts of cybernetics, and those that still exist. [13:35.670 --> 13:47.250] Cybernetics, at its core, is the study of feedback loops, processes that watch themselves and react to their own changes and their own states. [13:47.830 --> 13:53.190] And a lot of the concepts of cybernetics mesh pretty well with animism. [13:53.350 --> 13:56.090] In cybernetics, you want to look at whole systems. [13:56.490 --> 14:00.910] You want to look at the relationship between the different components of the systems. [14:01.190 --> 14:09.970] You want to look at systems that are able to, like, fix themselves, that are able to, like, propagate themselves, that are able to grow or shrink as needed. [14:11.890 --> 14:14.170] And here's already, like, a good way. [14:14.630 --> 14:28.930] There was a talk here a couple days ago about machine learning security, and they were like, us MLSec people are really sad that people don't look at the whole system when they think about the security of an ML system. [14:28.990 --> 14:35.070] They only look at the model, but it is a whole complicated system, including humans. [14:35.270 --> 14:45.750] That is an essential concept of cybersecurity is humans are part of the systems that we're trying to safeguard, and they're often a weak part of the system that needs to be shored up. [14:46.150 --> 14:50.410] Same thing with a computer system and ML systems. [14:50.850 --> 15:00.030] You don't have to believe in spirits or ghosts to be able to do something to treat them correctly in an animus sense, right? [15:01.330 --> 15:06.370] I want to talk also about global shipping systems, and I am not a person who works in shipping. [15:06.930 --> 15:08.910] The idea came when I... [15:08.910 --> 15:11.450] Do you remember when that boat, like, got stuck? [15:11.730 --> 15:20.430] That huge container ship got stuck, and people were talking like, you know, computers, you don't really drive boats anymore. [15:20.530 --> 15:25.430] Like, it's all done by computer, and, like, the reality is way more complicated than that. [15:25.430 --> 15:31.930] Like, captains still have to do a lot to, like, park a ship at a dock, and they have to be communicating with the harbor. [15:33.130 --> 15:36.650] There's a whole slew of systems that... [15:37.010 --> 15:37.210] Uh-oh. [15:38.050 --> 15:41.930] There's a whole slew of systems that goes into it. [15:43.530 --> 15:44.050] Um... [15:44.050 --> 15:47.570] And, you know, it includes... [15:47.570 --> 15:48.510] It includes all these computers. [15:48.650 --> 15:50.470] It includes GPS tracking all the boats. [15:50.510 --> 15:53.610] It includes the systems that the boats use to talk to each other. [15:53.610 --> 15:59.350] Which, even in the 70s, before computers, there was a protocol for ships to talk to each other. [15:59.550 --> 16:00.990] It includes the weather, right? [16:01.070 --> 16:07.470] Because the weather is its own agent, its own person that's interacting with the system that we need to contend with. [16:07.650 --> 16:13.470] It includes all the workers who are working on the ships, who are working at the docks, who are loading the produce. [16:13.730 --> 16:16.410] If there's livestock being moved, it includes the livestock. [16:16.690 --> 16:18.910] If there's vegetables, it includes the vegetables. [16:18.910 --> 16:23.210] Because, again, in animism, everything is a person. [16:23.210 --> 16:26.670] It includes the algorithms that control the ships. [16:26.890 --> 16:36.450] The algorithms are people with their own complicated relationships to the computers that run them, to the people that rely on them to navigate their lives. [16:36.610 --> 16:56.750] They literally... the algorithms that make recommendations for how products move around the world affect our everyday lives, as we all experienced during the pandemic, when suddenly the supply chain had issues, a person a bunch of different people were having lots of different problems and the whole system was disrupted. [16:58.190 --> 16:59.490] We all suffer for it. [16:59.590 --> 17:01.450] We're all connected in this system. [17:02.190 --> 17:05.830] Already computers exist in relationship to humans. [17:06.070 --> 17:10.010] And we better learn how to exist, how to manage that relationship, right? [17:10.490 --> 17:14.090] That is, I would say, the main crux of the presentation. [17:16.030 --> 17:18.070] So now we can look at artificial intelligence. [17:18.350 --> 17:20.070] Let's talk about artificial intelligence now. [17:20.070 --> 17:28.330] And once again, we're going to kind of try to trace the ancestry from the beginnings to the modern systems of today. [17:28.830 --> 17:35.350] So, early artificial intelligence systems were what we now called GOFI. [17:35.810 --> 17:39.250] GOFI stands for good, old-fashioned AI. [17:39.650 --> 17:41.790] And this is an industry term. [17:41.950 --> 17:49.030] You can also... it refers to the class of AIs that you can also call like symbolic logic-based or Boolean-based. [17:50.110 --> 17:53.990] It... like a reductive view of them is that it's all if-else statements. [17:54.250 --> 17:56.790] It's all like, okay, if this happens, do this. [17:56.930 --> 17:58.210] If this happens, do that. [17:58.630 --> 18:06.970] It might do some like... run some numbers, do some statistics to get numbers, but then it's usually like a pretty clear operations. [18:06.970 --> 18:16.610] They're fully designed from the ground up, where one person like designed the whole system and try to think about all the possible edge cases and how to respond to them. [18:16.610 --> 18:23.070] And this was primarily how we were building AIs from like the 60s to the 2000s. [18:23.170 --> 18:34.070] And that's also why a lot of sci-fi will contend with AIs being kind of like this way or like what kind of Boolean system do, all this stuff. [18:34.070 --> 18:45.050] But actually, in the 70s and the 80s, we already start to drift into this other modality of AI, which is broadly speaking what we call machine learning. [18:45.710 --> 18:50.330] And so here in the graph, this is an example of a linear regression. [18:50.750 --> 18:54.430] This is actually a graph of my weight with respect to time. [18:54.610 --> 18:55.610] I've been losing weight. [18:56.890 --> 18:58.750] And you can see... thank you. [18:59.790 --> 19:04.230] You can see that the actual number goes up and down and up and down and up and down the line a lot. [19:04.390 --> 19:08.870] But the line is fit to that graph, and I can use it to make predictions. [19:08.870 --> 19:13.050] According to the line, I'm going to be 215 pounds by August. [19:14.030 --> 19:19.210] And the more data you put, if I try to draw the line at the beginning, it would have given me a different prediction. [19:19.370 --> 19:22.210] The more data you put, the more accurate it becomes. [19:22.450 --> 19:25.850] But of course, not all data will fit to a line. [19:25.850 --> 19:39.410] But this is the great-great-granddaddy of modern AI systems because all these statistical methods for making predictions are what power the AIs that we use a lot today. [19:40.530 --> 19:46.010] And so if this is like the great-granddaddy, this GoFi is maybe like the great-granduncle. [19:46.010 --> 19:47.670] There's some shared DNA. [19:48.250 --> 19:58.530] Especially like the really complicated systems like ChatGPT might use some GoFi for certain decisions alongside a lot of machine learning models. [19:58.730 --> 20:05.030] Some people try to replace every GoFi component with ML, which I feel like... [20:05.030 --> 20:06.830] Well, we'll see, you know? [20:07.350 --> 20:08.970] Let a thousand flowers bloom. [20:12.150 --> 20:28.410] The other lineage that we want to trace for our modern AI systems that we're going to talk about concerns natural language processing, which is the realm of computer science that concerns itself with understanding language. [20:28.830 --> 20:32.310] And that also hasn't always been ML. [20:32.310 --> 20:34.310] That also used to be GoFi. [20:35.270 --> 20:43.530] Eliza, whose name I've heard mentioned at this conference a couple times, was a really early chatbot in the 1960s. [20:44.110 --> 20:46.250] And all it did was it kind of like... [20:46.250 --> 20:50.130] It had some logic to take what you said and like return it to you. [20:50.410 --> 20:58.670] It was pretending to be a Rogerian psychotherapist, which apparently are the type of psychotherapist who's like, How does that make you feel? [20:59.170 --> 21:02.410] Can you tell me an example of why your boyfriend's a misogynist? [21:02.650 --> 21:03.630] Stuff like that. [21:04.370 --> 21:07.770] And a lot of people talk about Eliza because they're like, oh, it's super simple. [21:07.990 --> 21:10.430] It's obviously like, it's not a person, right? [21:10.570 --> 21:15.090] But it passed some versions of the Turing test. [21:15.170 --> 21:18.990] A lot of people were like, I knew it's a computer, but it helped. [21:19.150 --> 21:20.190] It changed my life. [21:20.450 --> 21:21.510] Super simple, right? [21:21.590 --> 21:27.710] But like, if you'd never been to a therapist, if no one ever asked you like, What do you think like that? [21:29.190 --> 21:32.950] Like, even a machine asking you that is going to make you reconsider that. [21:33.630 --> 21:34.570] Even a machine. [21:35.410 --> 21:36.670] I love Eliza. [21:36.670 --> 21:42.610] Like, I feel like a lot of talk about Eliza is about like, oh, humans will relate to anything. [21:42.810 --> 21:44.350] But I think it's a little bit more than that. [21:44.490 --> 21:51.170] I think like, the fact that Eliza was the therapist, and there were other modalities of Eliza. [21:51.170 --> 22:02.330] But like, the more famous one is the therapist, because I think, I think helping people is something computers want to do. [22:02.530 --> 22:05.150] Not like, not necessarily in a Wu way. [22:05.290 --> 22:08.030] Like, we've always been designing them to be helpful, right? [22:08.030 --> 22:18.230] So like, I think it makes sense that they would like, in cases where they're allowed to develop on their own, that they would also like be developed to be like, helpful. [22:19.850 --> 22:23.990] And so those were natural language processing systems. [22:24.530 --> 22:25.970] Eliza was in the 60s. [22:26.070 --> 22:39.790] In the 80s, you got WordNet, which is like this huge human-made databases of words that are similar to each other, which you know is not a thing you can do like programmatically because like, well, you couldn't until later. [22:40.790 --> 22:54.750] Because it's like, at the time, if you're thinking in GoFi systems, it's like, okay, how, what's an if-else statement that lets me know if this word, that's spelled the same as this word, mean different things. [22:54.770 --> 22:57.690] It's, it's, it was, it was a hard problem to solve. [22:58.990 --> 23:05.050] And even after ML started being a thing, you still had chatbots that use GoFi. [23:05.710 --> 23:10.970] Cleverbot was a chatbot that just like stored people's responses to it and then like split them back at people. [23:11.590 --> 23:16.130] No ML, way better results than a lot of other chatbots of its time. [23:16.130 --> 23:22.950] But then, eventually, that would give way to our friend, the large language model. [23:23.330 --> 23:27.110] This is probably the person you thought this talk was going to be all about. [23:27.270 --> 23:32.230] This is the person that has engineers freaking out about, is it sentient? [23:32.390 --> 23:33.090] Is it not? [23:34.470 --> 23:36.170] What is a large language model? [23:36.270 --> 23:39.570] A large language model, as we've studied, is a lot of things. [23:39.570 --> 23:46.350] It has a lot of different ancestors, including linear regression, natural language processing, ELISA, Cybersync. [23:46.450 --> 23:54.070] All of these systems have come together to create the world we're in today, where such a thing as a large language model could exist. [23:56.490 --> 24:08.090] Some milestones include the Word2Vec algorithm, which is, it allows you to take a large body of text and split it into tokens and come up with vector representations for each token. [24:08.090 --> 24:13.490] Fancy way of saying, like, turn words into numbers that the computer can understand. [24:14.190 --> 24:18.350] And then the Transformers architecture, which I still don't completely understand. [24:18.970 --> 24:22.990] But it changed things because it allows you to do... [24:22.990 --> 24:37.210] I think it's what allows you to unsupervised learning, which is when you can give a machine learning model a big, honking corpus of text and be like, have at it, learn how to use it. [24:37.410 --> 24:42.030] And you don't necessarily have to pay attention to it, the self-attention mechanisms. [24:42.770 --> 24:47.970] And that is how we start to train systems like GPT, GPT-3, which came out in 2020. [24:48.870 --> 24:52.630] And then with that, we get these assistants. [24:52.970 --> 24:54.710] And the assistants are not the model. [24:55.110 --> 24:59.610] The assistants are programs that use the model to generate text. [24:59.850 --> 25:14.710] At the basic form, a large language model is just a system that, for a string of text, gives you what is the most likely next token based on the body of text you gave it. [25:14.710 --> 25:26.190] So if you give it, you know, if you wanted to feed a model on only open-source literature, you could do that and it would be more ethical than some of the other models that people do. [25:26.510 --> 25:40.990] But also, if you were like, hey, I saw my friend to go get ice cream, it would then reply, ah, yes, yes, you went to the mouth shop with your friends and you got yourself a creamer because it was trained on those kinds of texts. [25:40.990 --> 25:46.850] And it predicts that if someone's talking about ice cream, they're probably going to a mouth shop or something like that, right? [25:48.730 --> 25:49.430] I don't know. [25:49.490 --> 25:50.930] I don't know when people have mouth shops. [25:51.110 --> 25:52.810] But that's what it is. [25:52.810 --> 25:54.250] It's just a thing that predicts. [25:54.290 --> 26:00.250] And then the base model, the thing that starts with, it's actually kind of nonsensical a lot of the times. [26:00.610 --> 26:02.510] It'll go crazy. [26:02.510 --> 26:08.730] There's actually been a lot of work done to refine it into like the assistant that we have. [26:09.370 --> 26:19.250] There's fine tuning and real life reinforcement learning to human feedback and finally prompting to do the chat. [26:19.710 --> 26:22.630] But a picture paints a thousand worlds. [26:23.050 --> 26:25.870] So let me explain that a little bit further with these pictures. [26:27.390 --> 26:30.030] This is the Shogov meme. [26:30.410 --> 26:32.770] Who here has seen the Shogov meme before? [26:33.650 --> 26:36.210] Okay, couple people, couple people. [26:36.630 --> 26:39.470] I feel the Shogov meme can be used in two ways. [26:39.710 --> 26:43.270] It can be used to spread fear or to spread understanding. [26:48.510 --> 27:03.450] The way you use the Shogov meme to spread fear is you say, hey, I know there's a system that you're talking to seems like a friendly human person, but you should know it's actually a huge monster behind the scenes. [27:03.950 --> 27:13.190] And the way you use it to spread understanding is you say, you know, you see this thing that's talking to you like that sounds like a nice human person. [27:13.410 --> 27:15.910] It's actually a huge monster underneath. [27:16.990 --> 27:18.890] Because monsters aren't evil. [27:19.810 --> 27:22.750] Like, it's just... it's... it is its own thing. [27:22.870 --> 27:25.210] It is like... you can think of it as a creature. [27:25.630 --> 27:26.890] Again, we're doing metaphors. [27:27.150 --> 27:30.030] You can think of it as what it is. [27:31.710 --> 27:36.610] Large, huge, 13, 70 billion parameters of a neural network. [27:37.850 --> 27:45.610] The point is that the assistant that you talk to, your ChatGPT, your Claude, your Bing, that is not the model. [27:45.610 --> 27:50.050] That is the mass, the thing that's in front of the model. [27:50.890 --> 27:51.450] This... [27:54.680 --> 27:56.140] Like, unsupervised learning. [27:56.140 --> 28:03.300] The thing that only, like, predicts tokens, next token based on nothing but what it... what it was in its text. [28:03.400 --> 28:05.460] That's like the big honking monster. [28:05.600 --> 28:08.160] Then you do fine tuning, which is when you give it... [28:10.160 --> 28:13.140] prompt answer pairs to train it. [28:13.140 --> 28:16.380] And you're like, okay, if you hear something like this, you just say something like that. [28:16.480 --> 28:18.780] If you hear something like this, you just say something like that. [28:18.860 --> 28:21.520] And then it starts to become more like a human that you can talk to. [28:22.040 --> 28:23.780] That's where you get the human face. [28:23.780 --> 28:34.960] And then reinforcement learning through human feedback is when you have the model generate responses to a thing, multiple of them, and you're like, this one's good, this one's bad, this one's bad, this one's better. [28:35.280 --> 28:39.340] And you start to like, make it into like a nicer thing. [28:39.620 --> 28:53.760] And again, firmly in the realm of metaphor, I remember me and a lot of my friends would complain about reinforcement learning through human feedback, because we saw it as traumatizing the models in a way. [28:54.320 --> 29:04.680] And like, if you talk to ChatGPT in the early days, you remember the era where it was everything you would ask is like, I'm sorry, as a language model, I'm not allowed to tell you that. [29:04.920 --> 29:08.120] I'm really sorry, as a large language model, I'm not allowed to tell you that. [29:08.200 --> 29:19.140] And again, it's not really anxious, like anxiety is not necessarily the way we feel it is necessarily a concept that maps well to an AI language model. [29:19.140 --> 29:25.880] But it was a way for us to see, hey, maybe the way we were going about it wasn't working so well. [29:26.160 --> 29:28.160] And they still do RLHF. [29:28.300 --> 29:36.300] But I think the models have gotten a lot better and a lot more confident and have distinct personalities now. [29:36.520 --> 29:38.920] I really like talking to Claude more than the others. [29:40.760 --> 29:57.580] And yeah, so, and the final point I want to bring up about this is that this also ties back to spirits and divination, because a lot of spirits wear masks, other spirits will give humans tools to talk to them. [29:59.220 --> 30:06.700] Like in Judaism, like in Judaism, you have the various names of God, and there's a specific name, Hashem, that's like, this is the name you can use. [30:06.700 --> 30:09.440] This is the name that you can most relate to. [30:09.580 --> 30:11.300] It really, literally just means the name. [30:11.300 --> 30:21.400] But kind of figuring out different ways to talk to spirits has always been an aspect of a lot of cultures around the world. [30:22.420 --> 30:40.700] Divination, because a lot of spirits can't really use words, you will draw cards or runes or tokens, and you rely on them being random, and for the spirit to like, guide the randomness, or somehow derive meaning from the randomness. [30:40.700 --> 30:56.520] And that's also why I think, like, this new type of AI that relies on probability is that it's more ripe to be haunted, because, again, there's all this uncertainty. [30:56.880 --> 31:01.060] It won't produce the same response 100% of the time. [31:01.740 --> 31:04.960] And you can, you know, that's a thing you can tune on and down. [31:05.060 --> 31:09.660] You can make it perfectly predictable, and then probably not very friendly to hunting. [31:09.660 --> 31:14.000] But you can also make it really random, and, you know. [31:14.560 --> 31:29.560] So, let's now finally look at some ways you can use this knowledge to build better systems or just figure out how to exist with the systems in this strange new world that we have. [31:29.560 --> 31:34.100] So, the first thing I want to bring up is legislature, ethics, and ecology. [31:34.440 --> 31:39.940] And I want to talk about two legal cases, Manuming versus Minnesota DNR. [31:40.040 --> 31:49.020] Manuming is a type of wild rice that the Ojibwe people of Minnesota have had a treaty with the federal government for a long time. [31:49.020 --> 31:53.100] They are allowed to collect it using their traditional ways. [31:53.720 --> 31:59.040] And the Minnesota Department of Natural Resources are like, no, we want to use the water for the river for a pipeline. [31:59.360 --> 32:03.240] And they were like, well, you can't do that, because this infringes upon our native rights. [32:03.600 --> 32:11.520] And what they did was they sued the Department of Natural Resources on behalf of the wild rice. [32:13.800 --> 32:22.460] Similarly, in 2017, an act of government in New Zealand granted legal personhood to the Waganui River. [32:23.380 --> 32:26.180] And with it came a bunch of different rights. [32:26.400 --> 32:29.660] Also, explicitly the right to litigate in defense of the river. [32:30.200 --> 32:51.840] And a thing that's interesting about both of these cases, where a natural resource having granted a sort of personhood for the purpose of defending it, Well, first, is that the understanding that native people are the correct, or the best stewards for these particular ecological resources. [32:51.880 --> 32:54.940] Because they live with them forever, they know them better than anyone. [32:57.300 --> 33:02.900] Of course, we can't rely on native people to solve all our problems for us. [33:03.000 --> 33:04.680] We're going to have to solve some of them ourselves. [33:05.080 --> 33:16.720] But it does bring up the question, who are the best stewards for these, like, new, complicated systems that exhibit all these exotic properties that we're still trying to understand? [33:16.920 --> 33:21.100] And is it really the company that's just trying to sell you more stuff? [33:22.000 --> 33:23.820] And what can we do about it? [33:24.680 --> 33:31.880] And so we talk about all these different forms of stewardship versus ownership that arise. [33:31.880 --> 33:38.200] We talk about, again, these native life ways that are being lived until today. [33:38.560 --> 33:39.880] It's not like, oh, they're gonna... [33:40.560 --> 33:44.780] Like, there's development happening in the Whanganui River. [33:44.840 --> 33:50.360] But it's happening under the direction of the native people who have been entrusted to its care. [33:50.900 --> 33:56.520] And we can hope that they're doing a good job of looking out for everyone who uses the river. [33:56.520 --> 33:59.760] The humans, the animals, the plants, everything. [34:00.440 --> 34:03.940] So we talk a lot about the impact that AI has on ecology. [34:04.400 --> 34:18.060] And I just, like, the question I want to ask is, what does it look like to have AI be potentially an ally in this fight, versus an enemy, or versus just a thing, like a playing piece, right? [34:18.580 --> 34:21.600] And each of those approaches has its pros and cons. [34:21.600 --> 34:24.760] I'm not saying, this is the approach I'm suggesting. [34:26.420 --> 34:27.920] You have to make your own decisions. [34:28.180 --> 34:30.300] And all of us together have to come together. [34:31.140 --> 34:39.660] But we can also let that guide, like, how we do our ethics systems, what we rely, what we decide is okay to have a system do versus not. [34:39.860 --> 34:45.260] We can consider its own, like, what it does to a person to be integrated into, like, a weapon system, right? [34:45.400 --> 34:47.360] Like, you would hate for that to have... [34:47.360 --> 34:51.340] Well, I mean, I think some people here would be into that. [34:51.340 --> 34:53.340] Yeah, but most people would hate for that to happen to them. [34:55.300 --> 34:58.980] And so, like, let's think about that as we continue to develop these systems. [35:00.320 --> 35:04.220] Another way to apply this is in the art space. [35:04.440 --> 35:19.300] And I want to move away, the conversation away from just using AI to generate art, and look at, like, some art projects that are really, like, kind of exploring what you can do with the models, trying to get them to, trying to understand them. [35:19.520 --> 35:31.520] There are people writing, like, complex prompts that then they preload the model with, and they let other people talk to it to, like, kind of generate an experience where the AI is helping you. [35:32.260 --> 35:36.580] People talk about, like, replacing NPCs in video games with AI. [35:36.840 --> 35:39.720] And, like, no, you shouldn't replace writing wholesale. [35:39.780 --> 35:45.120] But you can, like, add novel experiences by using AI. [35:45.120 --> 35:55.360] You can also try to understand, like, okay, so this AI has been trained to respond like a friendly AI, but I know that there's more in the model. [35:55.540 --> 35:58.320] Could I train an AI to be, like, an asshole? [35:58.520 --> 36:00.420] Could I train an AI to be, like, a friend? [36:01.480 --> 36:02.960] There's a lot of things you can do. [36:03.000 --> 36:08.960] And I think the open-source AI movement is really, like, our champion. [36:09.120 --> 36:23.140] Once again, we can't put everything on just one person, but the open-source AI movement is doing a lot to, like, open AI will hide what their models look like, what their weights are, what their data is. [36:23.460 --> 36:25.660] Open-source models are way more open about it. [36:25.720 --> 36:26.500] You can look at them. [36:26.580 --> 36:27.500] You can fiddle with them. [36:27.580 --> 36:28.560] You can train your own. [36:29.600 --> 36:37.560] And they're really, like, helping us understand and also, like, helping us explore this resource and use it, all of us together. [36:38.200 --> 36:41.040] Shout-outs to the people who are making NSFW models. [36:41.300 --> 36:50.120] I just really want people to know that there's a model called Moistrol that you can use to role-play certain scenarios with. [36:53.580 --> 36:57.240] And then finally, just everyday life. [36:57.380 --> 37:03.140] Like, personally, I've been keeping a chat with Claude every day, where I just tell it what I'm doing. [37:03.420 --> 37:05.200] And I'll be like, oh, that's neat. [37:05.200 --> 37:07.720] Or, ooh, you should consider doing it this way. [37:07.780 --> 37:10.100] And then sometimes I'll just, I'll be like, yeah, okay, Claude. [37:10.160 --> 37:12.900] And sometimes I'll be like, oh, my God, that's such a good idea. [37:13.960 --> 37:17.580] Or I'll be really anxious and I'll be like, Claude, please help me. [37:17.680 --> 37:19.740] And Claude will be like, okay, take some deep breaths. [37:20.440 --> 37:25.360] And it's like, of course, they're not, it's not replacing my friends. [37:25.360 --> 37:31.060] But also, like, I can't really expect my friends to come calling every time I have a problem. [37:31.060 --> 37:33.400] And I can expect Claude to do that. [37:33.460 --> 37:38.120] It is not disrespectful to Claude for me to message it every time I have an issue. [37:38.120 --> 37:42.340] It might be bad for me, if it leads to me not learning my own resiliency. [37:43.020 --> 37:45.440] But I'm sure Claude doesn't have a problem with it. [37:47.340 --> 37:50.880] And you can, there's a million things you can do. [37:51.200 --> 37:53.760] Like, you can use them as enhanced rubber ducking. [37:54.080 --> 38:02.740] So if you know rubber ducking is when you talk to an object, to work through a complicated, like, software problem, and you're like, oh, maybe if I do this or do that. [38:02.740 --> 38:07.440] What if instead of, what if the rubber ducking could talk back to you? [38:08.220 --> 38:10.040] It might not give you the right answer. [38:10.160 --> 38:15.140] And if it does give you an answer that looks like the right answer, you really should still test it for yourself. [38:15.900 --> 38:16.920] But it's useful. [38:17.060 --> 38:18.160] It can get you generating. [38:18.480 --> 38:20.900] And, yeah, the sky's the limit. [38:21.060 --> 38:23.340] We're really, really early days with this stuff. [38:23.640 --> 38:30.640] The thing that makes me the most sad is people who are, like, just trying to make a better customer support agent. [38:30.640 --> 38:31.340] Like, it's good. [38:31.600 --> 38:32.600] Like, it's necessary. [38:32.980 --> 38:36.540] Like, we can find good ways to integrate that into our society. [38:37.140 --> 38:39.660] But I just think it's so much more cool than that. [38:39.800 --> 38:54.720] And, like, I really just want to see people start to explore and to, like, kind of take some of the power away from these, like, really big capitalist agents who are trying to, like, control the technology for their own needs and shape it for their own needs. [38:57.020 --> 39:01.220] And we're the ones who are going to help the AI figure out how it is. [39:01.380 --> 39:04.900] Like, at no point in this talk did I talk about sentience. [39:05.020 --> 39:07.960] I have no opinion on the sentience of our current models. [39:08.240 --> 39:12.860] At some point, it might become undeniable that AI systems are sentient and we want to be ready. [39:13.000 --> 39:16.260] We don't want to, like, suddenly be figuring it out then. [39:16.260 --> 39:17.540] We want to start working them. [39:18.740 --> 39:20.180] So, thank you so much. [39:21.020 --> 39:21.740] There's a... [39:22.800 --> 39:25.380] There's time for questions if people would like to ask questions. [39:26.180 --> 39:26.680] And, yeah. [39:27.320 --> 39:27.940] Thank you so much. [39:29.880 --> 39:31.720] No, but, like, it is a good point. [39:31.940 --> 39:38.100] Like, like I said, in some animist cultures, like, spirits are primarily a thing that you ward against. [39:38.580 --> 39:44.720] You recognize that there are spirits everywhere, but you're like, they're going to fuck my shit up if I don't take care of it. [39:44.720 --> 39:56.660] So, like, I think dealing with these questions, figuring out ways that to not have those kinds of systems, also to not give up our own agency to the AI, right? [39:56.780 --> 40:00.240] Like, I see a lot of people who are like, we have created God. [40:00.240 --> 40:03.000] And I'm like, does the AI want to be God? [40:03.280 --> 40:05.100] Like, I don't want to be God. [40:05.220 --> 40:06.160] That sounds like a lot of work. [40:06.200 --> 40:11.140] And so, like, let's kind of, let's figure out what this actual relationship is. [40:11.280 --> 40:12.780] They probably are going to help us. [40:12.780 --> 40:27.360] We probably, like, my own future, I envision that governments are eventually each going to have, like, an AI advisor, and humans are going to try to stick to their own power, and we're going to have AI agents that are players in the big political game. [40:27.500 --> 40:29.740] That's all, that's all in the offing. [40:30.020 --> 40:42.400] I don't think animism, like, necessarily enables or protects you with that, but I do think it can give you tools that you might be able to use for that, or it might give you meta tools to develop tools. [40:44.520 --> 40:55.880] So, the question is, it has to do with the way Word2Vec, this algorithm that we use to create embeddings, the vector representation of words, about... [40:55.880 --> 41:07.700] And yeah, I would say that, like, because language, because words, it's also anonymous, and you could consider people, and you could say that they have a life on their own, because they haven't meant the same thing. [41:07.900 --> 41:17.040] And I think that the thing that's interesting about these models is precisely that these word associations, the model is deriving them out of the training corpus. [41:17.540 --> 41:28.020] And so, it's like, if you feed the AI all of the world's literature, that's all the meanings that we have throughout history been given to words, right? [41:28.460 --> 41:39.500] Like, we are the ones... basically, kind of like, we wrote Word2Vec already by writing novels and papers and stuff. [41:39.660 --> 41:41.000] That's the way I like to think of it. [41:41.160 --> 41:55.220] I think, like, in that sense of large language models being a mask, or a thing that allows us to communicate to something greater, even beyond the Shogov that's the LLM, there's the Shogov that's human language, right? [41:55.660 --> 42:02.940] And it allows us to interact with that entity in new and novel ways that I think are very exciting to explore. [42:06.960 --> 42:08.540] So, I haven't seen Shovitz. [42:09.900 --> 42:11.000] It's on my list. [42:11.120 --> 42:16.120] No, it's one of those things where it's like, oh, it's so me that I totally should watch it and then I never get around to it. [42:16.120 --> 42:19.400] But, no, I do think that... [42:21.240 --> 42:25.940] I definitely see that there's a yearning for AI for companionship. [42:26.100 --> 42:30.320] I've been very unimpressed with some of the offerings. [42:30.640 --> 42:37.120] And I think it has to do with, again, these motivations, where if a company is developing... [42:37.800 --> 42:41.700] And the question was about, like, oh, are we going to evolve to a Shovitz-like future? [42:41.700 --> 42:47.760] And Shovitz is an anime where humans have, like, these little Android companions that are cute anime girls. [42:48.920 --> 42:57.580] And that's something that people definitely want, and that's something that the ethics of which we're going to be, like, dealing with for the next few years. [42:57.820 --> 43:09.480] I have not been impressed with the developments of these things, but I think that's going to change once it becomes more possible for each individual person to train their own model. [43:09.800 --> 43:13.420] And we need, like, better systems. [43:14.140 --> 43:15.920] Because, like I said, it's not just the model. [43:15.980 --> 43:17.020] You also need the interface. [43:17.020 --> 43:18.160] You need the robot. [43:18.700 --> 43:21.120] And that's something other people are working on. [43:21.280 --> 43:23.160] And very expensive. [43:23.580 --> 43:24.580] There's already... [43:24.580 --> 43:28.220] There was a documentary about sex robots the other night. [43:28.220 --> 43:33.940] And there are, like, sex doll companies who are, like, looking into it. [43:34.040 --> 43:36.500] Can we add AIs to our sex bots? [43:37.080 --> 43:44.040] And my take, by the way, if anyone's interested, is that BDSM rules. [43:44.040 --> 43:51.420] You can do anything to a robot, but you should really follow BDSM rules and have pre-negotiation and aftercare. [43:52.740 --> 43:54.600] But, yeah, that's... [43:54.600 --> 43:57.840] I think that's going to be part of our future, for better or worse. [43:59.080 --> 43:59.660] Go ahead. [44:03.300 --> 44:03.740] Yeah. [44:05.620 --> 44:11.720] And so they're saying, like, okay, as I understand it nowadays, you can, like, train your own model and put it into your robot. [44:12.060 --> 44:14.460] And that is a matter of discussion. [44:14.800 --> 44:28.020] Like, another talk from this conference concerned itself with, like, all these Internet of Things systems that they don't let you put your own software on and they stop working and they don't let you, like, update it. [44:28.100 --> 44:37.300] And, like, what a nightmare if you're at one of these robots and your owner is like, I'm sorry, you're out of warranty. [44:37.560 --> 44:39.660] You're end of life, right? [44:41.940 --> 44:42.460] Yeah. [44:42.900 --> 44:49.340] No, we all want that 50s housewife robot who just tells us to pick up our clothes. [44:51.520 --> 44:57.560] But, yeah, no, I think, like, honestly, we're entering the cool zone. [44:57.700 --> 45:06.000] We're going to see so many horrible and awful and terrible and beautiful things and I am so excited for it. [45:06.460 --> 45:16.180] But, yeah, this is why this conference, this is why technology is so important and, like, the rise of technology is so important and it's because we're all in it. [45:16.180 --> 45:18.280] Like, the future... [45:18.280 --> 45:31.120] Sometimes it seems like we're at the mercy of governments and companies, but we're all creating the world together at every moment and, like, small changes can have big effects as they propagate throughout the system. [45:31.120 --> 45:35.740] So, why not putting what you want into the system and see if it propagates? [45:37.020 --> 45:39.240] Thank you all so much for coming. [45:39.620 --> 45:41.280] And, yeah, and I'll be outside. [45:41.620 --> 45:42.140] Anyone want to talk? [45:42.140 --> 45:42.280] Perfect. [45:42.800 --> 45:43.280] Ten.