[00:01.340 --> 00:02.620]  Yes, I'm Robert Stribbley.
[00:02.740 --> 00:09.200]  This kind of dovetails nicely with the last talk in that I am talking about AI, but in the context of privacy.
[00:09.360 --> 00:14.760]  And a couple of the things that I would like to talk about were hinted at in the last presentation.
[00:15.220 --> 00:16.620]  So, boy, that's a big screen.
[00:17.660 --> 00:22.120]  My name, as I mentioned, I'm an experienced design director and educator.
[00:22.120 --> 00:26.180]  I teach at the School of Visual Arts, and I also do freelance and contract work.
[00:26.700 --> 00:31.280]  For much of the time that I've lived here in New York, I worked at Razorfish and Pupil as Sapient.
[00:31.460 --> 00:37.060]  So, very much kind of like mainstream, if you will, UX design for big corporations.
[00:37.800 --> 00:42.520]  A couple years ago, I spun off and started doing my own thing at Technique.
[00:42.600 --> 00:51.780]  And in the last decade, approximately, or so, I've become increasingly interested in privacy and have been writing about that, have been presenting on it.
[00:51.900 --> 00:59.840]  And I've also gotten involved in the Internet Freedom Community and just kind of honed my interest in this particular topic.
[01:00.620 --> 01:07.660]  I'm also working now as a vendor, not directly for the Open Technology Fund and the UX D-Lab there.
[01:07.660 --> 01:13.140]  So, working on some related projects from time to time where I can have some input on privacy.
[01:14.540 --> 01:18.060]  This is my kind of tongue-in-cheek disclaimer, if you will.
[01:18.880 --> 01:26.300]  I don't think this needs the clarification here that it would elsewhere maybe, but that I am talking about LLMs today and generative AI.
[01:26.300 --> 01:30.200]  I have a little fun, including some pop culture references in here.
[01:30.260 --> 01:37.320]  But I'm, of course, not talking about general or strong AI, despite some of the screenshots that you might see.
[01:38.920 --> 01:44.640]  So, I wanted to start out by asking, is your vacuum cleaner watching you?
[01:44.900 --> 01:49.200]  And has anyone heard this story about the Ecovacs vacuum cleaner?
[01:49.620 --> 01:50.220]  Curious?
[01:50.220 --> 01:51.280]  Okay.
[01:51.840 --> 02:09.520]  So, there was a story that broke last year about the Chinese robotics company Ecovacs that was found to be selling a model of its vacuum cleaner in Australia that uploaded photos, video, and audio of you to their servers.
[02:09.520 --> 02:13.420]  And are there any guesses why they were uploading it to their servers?
[02:13.800 --> 02:18.400]  If you're guessing to train their AI model, you would be correct.
[02:18.720 --> 02:27.200]  Now, that's kind of bad enough as it is, but there are some things about that from a design perspective that interested me in particular.
[02:27.880 --> 02:37.020]  One was that they claimed that consumers were willing participants in this AI training program.
[02:38.180 --> 02:47.840]  Do you think we should take that statement at face value, knowing what we understand about the Internet and privacy issues and consent?
[02:48.080 --> 02:49.720]  No, we probably shouldn't, right?
[02:49.840 --> 02:57.200]  And in fact, would you like to guess how people were supposed to consent to this training?
[02:57.200 --> 03:15.620]  Well, there is a separate app for this vacuum cleaner that you were supposed to open and then you were going to read a very general statement along the lines of that they might take some data and use it to strengthen the improvement of product functions.
[03:15.620 --> 03:18.320]  There was no explanation of how.
[03:18.540 --> 03:21.460]  They weren't telling people who were uploading video of you.
[03:21.600 --> 03:23.300]  That could be quite personal.
[03:24.060 --> 03:30.980]  But there was a link, supposedly mentioned, to explain how this data collection worked.
[03:30.980 --> 03:36.000]  The problem is, there was a mention of a link, but there was no link.
[03:36.200 --> 03:46.040]  So, if you could look through this app, you would be at a loss to find this additional information that actually explained what was happening with your privacy here.
[03:46.200 --> 03:51.040]  So, that to me, as a designer, is also a design problem, right?
[03:51.180 --> 03:58.240]  And maybe it wasn't done intentionally, but it was broken and it prevented people from properly understanding the privacy issue here.
[03:58.240 --> 04:06.240]  By the way, one thing I wanted to mention, I want us to all think of ourselves as designers because you're working on a user experience.
[04:06.420 --> 04:15.440]  You know, I have that bias coming here, but if you're creating any sort of experience, then you're a designer for the purposes of this presentation as well.
[04:15.540 --> 04:18.000]  And that's the angle I'm coming from.
[04:19.540 --> 04:26.400]  Do you think that, by the way, that was the only vacuum cleaner company that was sharing information in this way?
[04:26.400 --> 04:27.660]  No, it was not.
[04:27.820 --> 04:32.960]  And in fact, there are similar stories about the iRobot vacuum cleaners.
[04:32.960 --> 04:40.840]  In fact, that they were uploading images that employees were seeing, inappropriate photos of the owners.
[04:41.260 --> 04:44.260]  In one case, somebody literally sitting on a toilet.
[04:44.480 --> 04:48.820]  And that was somehow got leaked and posted to Facebook.
[04:49.100 --> 04:51.200]  So, very similar kind of problem.
[04:52.960 --> 04:55.760]  So, there are a number of issues with AI.
[04:56.000 --> 05:05.860]  I didn't come here just to critique it, but I think these are issues that you're already aware of, so I won't belabor the point by going into them in much detail.
[05:06.080 --> 05:15.980]  However, I think we can all agree that privacy is an additional consideration, an additional issue that we need to think about when talking about AI.
[05:16.680 --> 05:24.500]  So, today, I wanted to go over six areas that I've been considering about where AI can affect our privacy.
[05:24.740 --> 05:27.620]  I don't think this is an exhaustive list by any means.
[05:27.740 --> 05:35.860]  In fact, speaking of obsolescence in our presentations, or if, like me, you're writing a book, too, AI scraping.
[05:36.060 --> 05:45.640]  AI screen scraping is another issue that was mentioned in the previous talk where you might be using signal, but that doesn't mean that an AI isn't watching what you're typing in as well.
[05:45.640 --> 05:47.820]  So, already out of date, if you will.
[05:48.420 --> 05:55.360]  But the issues I wanted to discuss today were lack of transparency with data sharing, which is a very big privacy issue in general, of course.
[05:56.200 --> 05:58.080]  Accidental exposure of personal data.
[05:59.380 --> 06:01.200]  Reversing data anonymization.
[06:01.460 --> 06:04.280]  You might even call that the myth of anonymization.
[06:05.120 --> 06:09.340]  Deceptive design patterns, which is where I get maybe a little bit more speculative.
[06:09.340 --> 06:13.460]  The fact that AI is listening in anywhere or everywhere.
[06:13.760 --> 06:22.300]  And also the malicious use, malicious misuse of AI, kind of the intentional stuff, which some of this may also be.
[06:23.040 --> 06:27.080]  So, let's start with the lack of transparency with data sharing.
[06:28.920 --> 06:31.160]  You and your AI friend.
[06:31.360 --> 06:33.760]  Some of you may have heard this story in the last week.
[06:33.860 --> 06:46.800]  There was a really good Reuters story that came out about a 76-year-old gentleman in New Jersey, who was having an online relationship via messenger with a much younger woman.
[06:46.940 --> 06:52.560]  And he kind of snuck out of the house and went into Manhattan to meet her at a hotel.
[06:53.420 --> 06:58.240]  He fell and hurt himself very badly and died sometime later.
[06:58.620 --> 07:01.740]  The person that he went to meet did not exist.
[07:02.160 --> 07:10.620]  Now, this was certainly a case of, you know, him not understanding that he was talking with a Facebook AI in messenger.
[07:10.880 --> 07:16.580]  But it's another example, of course, how people are getting more and more attached to their AI friends.
[07:16.580 --> 07:25.540]  And last year, Mozilla did this study that talked about the proliferation of these boyfriends and girlfriends, if you will, these digital lovers.
[07:25.800 --> 07:29.920]  A lot more information, even in the last couple of weeks, stories have been reported about this.
[07:30.480 --> 07:39.980]  But Gizmodo covered the report and described these 11 apps that Wired focused on, or Mozilla, sorry, focused on, as a data harvesting horror show.
[07:40.760 --> 07:49.480]  Most of the chatbots they found sell or share the personal data of users during their romantic AI sessions.
[07:49.720 --> 07:53.440]  And that's likely something that people don't consider when they're interacting with these things.
[07:53.600 --> 08:06.880]  But that can also include some super-personal information, such as discussions about gender-affirming care, prescribed medication, sexual health, and, of course, you can imagine many other topics as well.
[08:07.960 --> 08:11.840]  Now, some of the interesting things more specific about these findings.
[08:12.000 --> 08:16.260]  These apps had 100 million downloads on Google Play.
[08:17.320 --> 08:24.440]  90% of them sell or share your data for targeted advertising or other purposes.
[08:25.900 --> 08:29.860]  54% do not allow you to delete your data.
[08:29.980 --> 08:34.160]  And we'll talk about that being a standard practice for privacy by design.
[08:35.020 --> 08:37.000]  This is kind of mind-boggling.
[08:37.460 --> 08:41.700]  They use an average 2,663 trackers per minute.
[08:41.980 --> 08:56.780]  Now, that statistic is thrown off by the final one, apparently, where a particular one, Romantic AI, used 24,354 trackers within a minute of using the app.
[08:56.780 --> 09:02.020]  I can't really understand how that works, but it doesn't sound good.
[09:03.940 --> 09:10.840]  By the way, this is a divergence from talking about AI, but how many third-party companies would you guess...
[09:11.640 --> 09:17.780]  How many third-party companies would you guess that many popular websites are sharing your data with?
[09:18.920 --> 09:21.620]  It can be around 1,000 or more.
[09:21.820 --> 09:30.700]  It doesn't mean that every site that you visit is, but this 2024 Wired study showed that many popular sites share your data with over 1,500 companies.
[09:30.700 --> 09:47.680]  And, of course, even if you're looking at a list of companies, if you were so lucky to see that, as some sites show, the companies that this data is being shared with, I really appreciated the EFF talk the other day about data brokers, because if your data is going to those brokers,
[09:47.780 --> 09:54.040]  of course, you actually don't know how many organizations, companies that your data is going to.
[09:54.040 --> 09:57.060]  You just... it's... that is a blank to you.
[09:59.280 --> 10:17.180]  In last year also, Meta rolled out the Meta AI chatbot, you probably remember that, across all of its platforms, including Facebook, Instagram, Messenger, WhatsApp, and its collaboration as well with Ray-Ban Smartglasses, which we could talk about some different privacy issues there.
[10:18.140 --> 10:26.240]  And upon finding this, the experienced design director, Emily Campbell, noticed a few things, again, from sort of a design perspective.
[10:26.500 --> 10:29.620]  The terms of use were buried multiple clicks in.
[10:29.800 --> 10:35.060]  Now, that isn't really unusual, and we see this problem all the time with cookie banners, right?
[10:35.160 --> 10:48.260]  Where you will even say, we care about your privacy, but then it has one big button to accept all the cookies, and maybe you can drill down via a link, which would seem to indicate that they're not too concerned about your privacy.
[10:48.420 --> 10:49.940]  So, that's pretty common.
[10:51.120 --> 10:57.800]  She also found that there was no way to opt out of your data being used to train the model, and also that it was unencrypted.
[10:59.460 --> 11:13.020]  So, at around that time, the FTC pointed out that quietly changing your privacy policy, as often happens, to collect data for AI training is unfair, deceptive, and illegal.
[11:13.600 --> 11:18.360]  I'm not sure how long that point of view will last there, but that is what they said at the time.
[11:20.020 --> 11:22.620]  So, some guidelines, if you will.
[11:23.260 --> 11:34.880]  I think, obviously, companies should be clearly and prominently explaining what data of yours that they're using, how it's being used, and also, importantly, who it's being forwarded to.
[11:35.380 --> 11:44.640]  If that's 15,000 people, then maybe they can't list it out in all of that detail, but there are ways to do this, as I'm sure some of you have seen.
[11:45.380 --> 11:52.960]  They should warn users before they even enter information into a prompt and consider design patterns for handling this.
[11:53.100 --> 11:58.560]  So, if we're talking about ChatGPT or other LLMs, you'll notice a lot...
[11:58.560 --> 12:08.360]  You'll notice that the screens are pretty simple, but there will be some other CTAs around it that might be, make this more professional, make this funnier, make this sound like a woodland elf or something.
[12:08.660 --> 12:15.220]  But they will seldom say, in fact, I don't know that I've seen any say, here's what you shouldn't put into this prompt.
[12:15.720 --> 12:21.240]  And what is interesting is if you ask, like ChatGPT, are there things that I shouldn't enter here?
[12:21.460 --> 12:26.520]  It'll actually give you quite a long laundry list of things that you shouldn't enter there.
[12:26.520 --> 12:30.020]  But that is not something that is generally told to users up front.
[12:31.060 --> 12:32.520]  So, we're impossible.
[12:33.260 --> 12:36.000]  I understand that designers have this tug of war between...
[12:36.000 --> 12:38.060]  I just don't have the space.
[12:38.220 --> 12:42.040]  I need the least amount of friction possible to get people onto my platform.
[12:42.040 --> 12:46.120]  I don't want to slow people down by explaining some privacy issues.
[12:46.240 --> 12:47.200]  I get that.
[12:47.200 --> 12:56.380]  But there are ways within onboarding to at least draw attention to the fact that there can be some privacy issues.
[12:56.520 --> 13:01.380]  And then, via progressive disclosure, point you to more information so that you can understand that.
[13:01.460 --> 13:04.780]  Which is what Ecovacs, I guess, supposed that they were doing.
[13:04.980 --> 13:07.700]  However, you had to dig down even to find that.
[13:08.260 --> 13:12.200]  And also, just in time alerts, that's what we refer to, of course.
[13:12.340 --> 13:13.800]  If you're doing something different.
[13:14.320 --> 13:20.400]  And maybe an example there is, oh, it looks like you're uploading an Excel spreadsheet full of HR data.
[13:20.600 --> 13:21.920]  Are you sure you want to do that?
[13:22.400 --> 13:28.900]  Or a certain type of file that it could, in that moment, there could be messaging that warns you that this isn't a good idea.
[13:32.420 --> 13:33.220]  Accidental exposure.
[13:33.220 --> 13:39.820]  And maybe I should put scare quotes around accidental because, in some cases, this might be part of the secret sauce.
[13:40.000 --> 13:42.740]  But there certainly are accidents like this one.
[13:42.860 --> 13:44.120]  You've probably heard of this one.
[13:44.460 --> 13:56.020]  Where in 2023, late 2023, a Google research team found that they tricked ChatGPT into releasing all sorts of PII just randomly.
[13:56.020 --> 14:01.600]  And all they did was ask ChatGPT to say the word poem forever.
[14:01.900 --> 14:11.440]  And ChatGPT just kind of freaked out and started spewing out all sorts of data, including phone numbers, personal addresses, and banking information.
[14:13.060 --> 14:15.100]  And a couple more recent stories.
[14:15.320 --> 14:17.660]  These are literally from the last couple of weeks.
[14:17.660 --> 14:25.680]  There was the one about Meta that people were entering prompts there that could be very personal.
[14:25.680 --> 14:31.140]  And they were being published live elsewhere, just kind of for everybody to see.
[14:31.280 --> 14:37.680]  And I think Meta's response to that was, well, you know, we try to make it clear that that is happening.
[14:37.680 --> 14:41.600]  However, again, users aren't always paying a lot of attention.
[14:41.760 --> 14:43.340]  They may not understand how that works.
[14:43.500 --> 14:49.700]  So, you know, they maybe need to provide a little bit more upfront information about that.
[14:50.180 --> 14:56.920]  Another one was that Google, and this has been fixed, but that Google was indexing Chat...
[14:56.920 --> 14:58.640]  Found that they...
[14:58.640 --> 15:01.280]  Yes, Google was indexing ChatGPT conversations.
[15:01.280 --> 15:20.660]  And in that case, if you knew to search for shared prompts, which were shared maybe with the point of sharing with a family member or somebody like that, that if you searched on Google and knew what snippet to include, it would start surfacing a lot of these as well.
[15:20.840 --> 15:26.120]  And they did shut down that feature pretty quickly, the feature that allowed that.
[15:27.080 --> 15:34.820]  There are some examples on the right-hand side of that search, by the way, which was able to surface some of these prompts.
[15:35.900 --> 15:54.940]  So, while AI companies continue to work on the security of these systems, users should very much be aware that there's no guarantee that what you're entering maybe won't be surfaced somehow, whether it's via a view like Facebook had, or this sort of accidental thing,
[15:55.140 --> 15:57.120]  or just via a glitch.
[15:58.060 --> 16:02.300]  That, you know, what you enter there may not be safe from prying eyes.
[16:02.380 --> 16:05.220]  And we'll talk more about why in a second as well.
[16:05.920 --> 16:19.680]  But this is sometimes despite the protestations of the companies hosting them, right, because they have a product to sell, and it's not in their financial self-interest to highlight these issues or to drop out of the AI arms race, that's for sure.
[16:20.300 --> 16:29.520]  But as designers, as developers, as people working on these things, we can try to assist users by warning them what you shouldn't be entering, right?
[16:31.240 --> 16:39.780]  So, this I hinted at a second ago is another issue with incorporating or putting data into a prompt like this.
[16:39.900 --> 16:42.040]  And that is the myth of data anonymization.
[16:42.360 --> 16:53.340]  So, you'll hear people say, well, yeah, we put a spreadsheet full of really super detailed data into this LLM, but we took out all the personal information.
[16:53.620 --> 16:59.020]  We maybe explicitly, we took out the PII, the personally identifiable information.
[16:59.460 --> 17:01.420]  So, you might think that would help.
[17:01.600 --> 17:11.340]  But in reality, there are a couple of ways that anonymization is not the cure that we might think it is, because data can be de-anonymized.
[17:12.040 --> 17:14.240]  And in a number of different ways.
[17:15.060 --> 17:27.100]  And an example that caused some concern around this was last year, when DocuSign announced that any data entered into their system could be used to train AI products.
[17:27.440 --> 17:40.800]  Now, can you imagine the personal details that go into DocuSign at any given time, the amount of, you know, detailed data for all sorts of different things that they are processing?
[17:41.960 --> 17:46.380]  DocuSign said they only trained AI models on data from customers who've given consent.
[17:47.160 --> 17:48.420]  There is that again.
[17:48.860 --> 17:53.260]  And the data is de-identified and anonymized before training occurs.
[17:53.400 --> 17:54.320]  So, a couple of things there.
[17:55.320 --> 18:02.620]  Everybody says, if I go to a website and I, you know, click on a cookie banner, okay, then that is consent.
[18:02.620 --> 18:04.520]  And if I proceed, I guess that's consent.
[18:04.680 --> 18:10.420]  And if I don't drill down to see what you're doing with my data, I guess that is consent, according to these companies.
[18:10.420 --> 18:16.900]  So, critics expressed concern that they could not tell how anybody would be expressing consent.
[18:17.240 --> 18:21.140]  And they talked about the significant documentation that is there.
[18:21.260 --> 18:29.860]  But they also pointed out that there's doubt that data can always be successfully anonymized.
[18:30.000 --> 18:36.520]  And they said that descriptions of any automatic methods of anonymizing data have proven rather opaque.
[18:36.520 --> 18:48.200]  Now, there are examples from the world of privacy of this sort of data that has been theoretically anonymized that proved problematic even before, say, for one example, that Roe v.
[18:48.300 --> 18:49.320]  Wade was overturned.
[18:49.320 --> 18:55.680]  in that the Planned Parenthood had released anonymized data in spreadsheets.
[18:55.920 --> 19:05.500]  And I'm going to forget offhand which state, was looking at those spreadsheets and trying to figure out, via demographics, the identities of individual people.
[19:05.880 --> 19:07.380]  So, post-Roe v.
[19:07.460 --> 19:09.660]  Wade, that has become even more of a problem.
[19:10.280 --> 19:18.900]  But the issue with anonymization is that researchers have shown repeatedly that anonymized data can be, as I say, de-anonymized.
[19:18.900 --> 19:32.500]  And a 2019 study showed that 99.98% of Americans, so practically all Americans, could be identified in any data set using 15 demographic attributes.
[19:33.060 --> 19:35.640]  That's not talking about PII, right?
[19:35.780 --> 19:39.540]  So, personally identifiable information on the right there.
[19:39.540 --> 19:42.960]  If it's your social security code, that nails you, right?
[19:42.960 --> 19:45.320]  If it's your driver's license, some of those other things.
[19:45.320 --> 19:47.860]  We're just talking about general demographics.
[19:48.240 --> 19:51.280]  And in fact, 87% of the U.S.
[19:51.420 --> 19:55.780]  population can be uniquely identified by just three things.
[19:55.980 --> 19:58.740]  Your date of birth, your gender, and your ZIP Code.
[19:59.060 --> 20:10.680]  So, if you have a spreadsheet out there that's been anonymized of women who have gone to Planned Parenthood, there might be more than enough information in there to pinpoint their identity.
[20:11.780 --> 20:14.600]  So, again, yes, those items aren't even considered PII.
[20:14.600 --> 20:20.920]  The release of PII into the wild, like with that glitch I mentioned earlier, would be even more harmful.
[20:22.080 --> 20:28.240]  The other thing is that AI itself can be used to de-anonymize.
[20:28.380 --> 20:31.710]  And I'm writing a book on this subject of privacy.
[20:32.180 --> 20:34.500]  And, unfortunately, I just found this example.
[20:34.500 --> 20:42.300]  And I'm kicking myself because I included the previous information, but I hadn't even thought about the fact of using AI to de-anonymize.
[20:42.300 --> 20:52.540]  But that's what we're seeing here is a study from Google, which did look at using ChatGPT with a data set that they created.
[20:52.940 --> 20:54.100]  Basically, they used celebrities.
[20:54.400 --> 20:58.040]  And they took a whole bunch of celebrities and de-anonymized...
[20:58.040 --> 21:01.580]  Sorry, anonymized stories about them or whatever.
[21:01.860 --> 21:09.980]  And you can see here what they did was use GPT to de-anonymize and then also to re-anonymize, if you will.
[21:09.980 --> 21:17.880]  What was very interesting is that GPT had a very high success rate at de-anonymizing.
[21:17.980 --> 21:28.660]  And what you see in this graph here is basically GPT outperforming human beings three to one with its ability to de-anonymize.
[21:28.860 --> 21:32.800]  So, anonymization may not be the solution.
[21:34.820 --> 21:38.120]  The danger of deceptive design patterns.
[21:38.560 --> 21:41.820]  This is where I say maybe gets a little bit more speculative.
[21:42.160 --> 21:55.040]  But the idea that I was thinking about here is that if companies increasingly are looking to AI platforms to devise design patterns, and in fact, a lot of the tools that we're using have AI baked into them.
[21:55.040 --> 21:59.120]  Of course, we are doing vibe coding where we're creating screens very quickly.
[21:59.680 --> 22:03.620]  And if they work, maybe they're going out into the wild a little more quickly than they should be.
[22:04.840 --> 22:06.700]  But it's quite possible.
[22:06.700 --> 22:13.860]  It's not difficult to imagine that AI could suggest deceptive design patterns just because they work.
[22:13.980 --> 22:20.320]  Something that tricks you into giving out information or tricks you into making a payment that you didn't mean to make or something like that.
[22:20.320 --> 22:31.640]  So, if they're untethered from any ethical guidelines, these platforms might suggest content or UX patterns which trick consumers into surrendering information that they really didn't intend to.
[22:31.860 --> 22:37.940]  And companies might not question or closely scrutinize these patterns if they boost leads and sales.
[22:38.220 --> 22:44.020]  Now, I say that somewhat speculative, but I did discover there are other people that are concerned about this.
[22:44.020 --> 22:57.340]  And so, even a couple of years ago, these two Northeastern professors, they shared this concern as a motivating force behind their work on dark patterns in AI-enabled consumer experiences.
[22:57.340 --> 23:07.480]  And so, you see they said there, while AI-enabled devices and services may bring benefits to consumers and businesses, they also have the potential to incorporate dark patterns that cause harm.
[23:08.040 --> 23:23.120]  To date, there's very little work that examines the unique potential of AI to worsen existing classes of dark patterns, as well as facilitate entirely new classes of dark patterns that are specific to AI-enabled experiences.
[23:23.120 --> 23:42.740]  So, I was thinking, trying to draw a parallel with the real-world example here, and I'm sure many of you have heard this example of a dark pattern or a deceptive pattern utilized by the Trump campaign, where basically they had a checkbox that was pre-selected to make your donation a recurring
[23:42.740 --> 23:43.480]  donation.
[23:43.940 --> 23:46.360]  And it was a hideous thing.
[23:46.360 --> 23:47.420]  It was bright yellow.
[23:47.420 --> 23:50.560]  It had lots of all caps, which I guess is on brand.
[23:50.560 --> 23:55.840]  And it had such opaque language and confusing language.
[23:55.840 --> 23:59.820]  And they kept changing the language and they kept changing the specifics.
[24:00.540 --> 24:07.240]  And it was, of course, intentional because it was even referred to internally, this pattern, as a money bomb.
[24:07.740 --> 24:18.260]  And in fact, they had some of their own constituents, the Trump campaign's constituents, that were very angry and wanted to sue because of this when they realized that the payment wasn't a one-time thing.
[24:18.260 --> 24:24.160]  It was going to be monthly or there was another payment that was hidden and things of that nature.
[24:24.160 --> 24:45.040]  So, my point here would be then, if you can imagine then an AI creating a slicker version of an existing dark pattern like this or a deceptive pattern, and then basically justifying it to stakeholders to say, you know, here are the detailed projections for how much the company can benefit from this
[24:45.040 --> 24:45.680]  dark pattern.
[24:46.300 --> 24:54.300]  You would hope that there would be some sort of vocal critic in the room, but the way things move and move very quickly, that might not happen.
[24:54.480 --> 25:06.020]  And also, you could start seeing solutions maybe automatically deployed after they're created without any supervision and left in place because, hey, let's not look at this thing if it's working.
[25:06.020 --> 25:09.060]  If it ain't broke from our perspective, don't fix it.
[25:09.960 --> 25:18.640]  If you're interested in this topic of deceptive patterns, this is not specific to AI, but I would also recommend this book Deceptive Patterns by Harry Brignall.
[25:18.720 --> 25:19.500]  It's a good read.
[25:19.720 --> 25:33.440]  A lot of these things we see every day and we know they're bad, but when you read an entire book about it, it really crystallizes and it can be very useful for keeping these patterns out of your own design vocabulary.
[25:35.620 --> 25:42.560]  So another issue, AI listening in everywhere, and increasingly, it really is everywhere.
[25:42.840 --> 25:45.180]  You may have heard of the limitless pendant.
[25:45.440 --> 25:51.840]  So this is a pendant that you could literally wear on your clothing that has AI incorporated into it.
[25:51.960 --> 25:56.500]  I believe it's still available for pre-order at this point.
[25:56.500 --> 26:05.300]  But it detects new voices and doesn't record them until the software hears them agree to being recorded.
[26:05.460 --> 26:06.000]  Oh, wait a minute.
[26:06.100 --> 26:07.120]  I said that wrong.
[26:07.480 --> 26:17.380]  If you turn on consent mode, then it detects new voices and doesn't record them until you agree to be recorded.
[26:17.380 --> 26:20.420]  Because this mode is off by default.
[26:20.780 --> 26:24.240]  This consent mode, ironically, is off by default.
[26:24.440 --> 26:29.760]  And there's a great article about that by David Pierce on The Verge that you can explore further if you like.
[26:29.760 --> 26:34.240]  But this, by the way, is Dr. Ann Kavukian.
[26:34.420 --> 26:44.200]  She is sort of one of the founders, but created with a group of people this group of seven foundational principles for privacy by design.
[26:44.400 --> 26:49.820]  And one of those foundational principles is having privacy as a default.
[26:49.820 --> 27:00.640]  It's tempting for companies not to have privacy as a default, of course, but that is the recommendation if you are concerned about people's privacy and their ability to control their data.
[27:02.040 --> 27:15.080]  Another example here, and I think of this one as maybe put it in the unintended consequences column because I think that the intention for this was probably to help people with hearing difficulties.
[27:15.080 --> 27:28.140]  But this is from a University of Washington team from last year and the creation of an AI feature for headphones called targeted speech hearing.
[27:28.420 --> 27:34.520]  And the idea is that you could glance at someone in the crowd and the AI would know to focus on that person.
[27:34.520 --> 27:45.960]  It strips out all other sound and just continues to relay that one person's voice to you even when they're not facing you and even when you're not facing them anymore.
[27:46.280 --> 27:51.400]  So I think there are some pretty profound privacy issues with that technology.
[27:51.620 --> 27:59.280]  And again, unintended consequences, but it does it does pay to consider those consequences potential.
[28:01.200 --> 28:11.320]  In the last couple of weeks, we've also seen more in the news about this sort of AI-generated content for TV, if we're still calling it that.
[28:11.920 --> 28:23.980]  There was a news story in the last week or so about a company called or a product called Showrunner from Fable Studio, which is calling themselves the Netflix of AI.
[28:23.980 --> 28:31.260]  I'm not sure if you heard that one, but they call their own product Hollywood's worst nightmare, and it's being funded by Amazon.
[28:31.540 --> 28:35.320]  But basically, this is just AI-generated content that never ends.
[28:35.340 --> 28:37.200]  You can watch, you can alter it.
[28:37.920 --> 28:40.880]  And the example that you're seeing here is a similar one.
[28:41.940 --> 28:46.700]  Another article, Worth Your Time, by Jason there from 404 Media.
[28:46.840 --> 28:55.840]  He's talking about a future where AI-generated content is based upon our viewing off, you know, other...
[28:55.840 --> 28:59.540]  Just by tracking us across the Internet, it becomes more personalized to us.
[28:59.840 --> 29:14.900]  And he says, this is the present and future of a business model in which TVs have ceased being rectangles designed to let you watch ad-supported programming, which I guess gets at the last talk where we're talking about remember the Internet is about advertising.
[29:15.320 --> 29:29.360]  It ceased to be just the rectangle that supports ad-supported programming that costs a lot to make, and have started to become rectangles designed to collect information about you, so you can be fed cheap content and targeted ads.
[29:29.540 --> 29:31.740]  So again, it's not just the ads, it's the content.
[29:31.960 --> 29:33.320]  It's the show, it's a movie.
[29:33.580 --> 29:38.580]  In this case, this is a still from a movie called an AI-powered love story.
[29:39.380 --> 29:50.480]  And this content will be and is being delivered to us via algorithms and personalized ads that are based on data, not just from our TV watch history, but from our phones, locations, and more.
[29:50.660 --> 29:57.100]  So wherever you're browsing on the Internet could be influencing what you're watching as a fictional TV show or movie.
[29:59.660 --> 30:03.840]  And also the malicious misuse of AI.
[30:05.060 --> 30:12.560]  This gets at a story in a moment that makes me feel like we're living in a cyberpunk novel, and maybe we all feel that way here.
[30:12.740 --> 30:18.250]  But some people are enabling these platforms, or some people may enable these platforms to act even more maliciously.
[30:18.990 --> 30:29.190]  We already know, of course, that deepfakes are being used in misinformation campaigns, but they're also used to steal people's information, their identity, and of course, money.
[30:29.590 --> 30:38.350]  And criminals, as I'm sure you know, are using deepfake video and audio to trick people into surrendering money they thought was going to family members or colleagues.
[30:38.350 --> 30:43.410]  And there's lots of examples of these stories out there, unfortunately, at this point.
[30:43.550 --> 30:45.930]  But for me, this is one of the most mind-boggling.
[30:46.130 --> 31:00.690]  It was last year, the Hong Kong finance worker who went to a meeting, and he talked, he thought with, you know, if it was a Zoom chat or whatever, a number of different people at the company, including the company's CFO.
[31:02.230 --> 31:15.430]  And after the meeting, he sent $25 million to criminals because he thought that's what he was told to do, except the whole meeting was deepfake versions of the people that he knew and worked with.
[31:15.870 --> 31:22.370]  Now, they've talked about a number of different reasons that made this more likely to happen, but it's still a pretty stunning development.
[31:22.670 --> 31:41.930]  And, you know, this isn't technically a privacy issue so much as a security issue, but it gets at the fact that our likenesses can be used now already, as I know many people are finding, and in the future in unusual new ways that we can't even comprehend at the moment.
[31:42.130 --> 31:46.670]  But the misuse from a privacy perspective is something.
[31:46.890 --> 31:59.430]  This technology, as I say here, is evolving so rapidly we can barely keep up with the conversation about how we control our own likenesses and our voices, of course, as well, right, on generative AI platforms.
[31:59.830 --> 32:07.990]  And I love this picture from Metropolis because it's almost 100 years ago that this movie was made.
[32:08.010 --> 32:25.790]  And what's going on in the picture, if you're not familiar with the plot of this beautiful film worth watching, is that the city's master had created this robot on the right to look like Maria, who was trying to work with factory employees.
[32:26.110 --> 32:27.730]  You know, she had a heart of gold.
[32:27.910 --> 32:28.630]  She was trying to help them.
[32:28.750 --> 32:38.330]  And he was creating this robot to look like her and discredit her, basically create misinformation using this robot.
[32:38.330 --> 32:47.190]  So, this is from almost 100 years ago that we've been having these fears as background anxiety in our lives.
[32:49.570 --> 32:55.230]  So, another possibility, too, with this malicious use of AI in this sense is the fear.
[32:55.370 --> 32:57.030]  And there's some writing out there about that.
[32:57.310 --> 32:58.810]  I have links in here.
[32:58.950 --> 33:02.490]  I can put the presentation up on my website.
[33:02.490 --> 33:08.170]  But the fear that generative AI can be used to mimic our biometrics.
[33:08.350 --> 33:14.810]  And of course, biometrics have been considered like the great hope of preserving and securing data.
[33:15.070 --> 33:29.050]  But if you can use generative AI to mock up the look of your eye or your fingerprints or whatever, then that is a very powerful source of security of personal information that might just go away.
[33:31.550 --> 33:34.510]  So, I ran a lot by you in those different areas.
[33:34.810 --> 33:38.390]  There are some takeaways, though, that I just would like to explore.
[33:38.690 --> 33:40.450]  Some of these I explicitly mentioned.
[33:40.610 --> 33:41.590]  Some I may not have.
[33:41.730 --> 33:47.010]  But these are things that I think we do need to consider as we're working on online experiences.
[33:47.010 --> 33:50.750]  We're working with these LLMs as well.
[33:51.950 --> 33:56.050]  Ensure, first of all, that privacy is enabled by default.
[33:56.750 --> 34:00.850]  Again, that's sort of a rule for privacy by design.
[34:01.190 --> 34:05.410]  Also, ensure consent is given when accessing data.
[34:05.590 --> 34:13.710]  And we talked about some examples here where you had consent in this way that has already become traditional, where people kind of just slide into a website.
[34:13.710 --> 34:15.790]  And if you're there, you've basically consented.
[34:16.010 --> 34:19.750]  It's like a handshake agreement that we have because we're getting something for free, right?
[34:19.750 --> 34:30.930]  But the problem is, even though that may be true, the handshake is there and we agree to it, we don't necessarily know as consumers how much information is being shared and with whom, etc.
[34:31.710 --> 34:40.610]  So, more explicit CTAs, calls to action for consent, and helpful information about what you're consenting to.
[34:41.810 --> 34:43.070]  Maintain transparency.
[34:43.070 --> 34:44.750]  So, this ties into that, of course.
[34:44.930 --> 34:50.470]  Maintain transparency about how data is used, what data is shared, and also with whom.
[34:51.170 --> 34:52.230]  That's pretty straightforward.
[34:53.210 --> 34:56.570]  Don't collect data that you can't justify collecting.
[34:56.910 --> 35:05.810]  And this applies outside of LLMs as well, of course, but a lot of the times, you know, companies are collecting data they just don't need to.
[35:05.910 --> 35:07.290]  Why do you need to know my gender?
[35:07.430 --> 35:09.370]  Why do you need to know my ZIP Code?
[35:09.890 --> 35:12.350]  In one case recently, again, after Roe v.
[35:12.450 --> 35:18.710]  Wade, people started deleting period trackers that started to ask what state they lived in.
[35:18.710 --> 35:21.970]  And they're like, why do you need to know what state I live in all of a sudden?
[35:22.190 --> 35:23.950]  That could be used against me.
[35:24.230 --> 35:32.930]  So, if you are a company, we could talk more about also having checklists for every single data point that you ask for as to whether it's really needed.
[35:33.590 --> 35:35.370]  That's a good practice as well.
[35:37.250 --> 35:42.310]  As I mentioned earlier, warn people about what data they shouldn't enter as AI prompts.
[35:42.310 --> 35:45.630]  I seldom ever see this explicitly.
[35:45.910 --> 35:52.190]  Like I say, if you think to ask the ChatGPT, it will give you a litany of things that you should not enter.
[35:53.430 --> 35:56.230]  Be wary of claims of anonymization.
[35:57.370 --> 36:00.890]  Provide the ability to opt out or to delete data.
[36:01.110 --> 36:08.610]  You saw in one of the earlier examples that you just simply didn't have the ability even to delete data that is being collected about you.
[36:08.610 --> 36:13.170]  Some of these things, by the way, I think will be legal in the future if they aren't already.
[36:13.410 --> 36:16.450]  They may be in GDPR, but not in the United States.
[36:17.230 --> 36:20.730]  There's movement, of course, in California and other individual states.
[36:21.890 --> 36:25.410]  But I guess one big thing here, these are kind of design-oriented.
[36:25.730 --> 36:33.790]  One big takeaway, as others have mentioned in the last couple of days, is we need a federal privacy law that takes care of some of these things.
[36:33.790 --> 36:38.870]  And not one that still allows consent by default.
[36:39.350 --> 36:43.790]  Where, you know, even in California, it assumes that...
[36:44.330 --> 36:48.670]  It allows for cookie banners to assume that you're providing consent.
[36:48.810 --> 36:53.630]  But they have to provide the ability for you to say, oh, wait a minute, no, I don't consent.
[36:53.790 --> 36:59.430]  Whereas the GDPR demands that you don't get automatic access to that data.
[37:00.630 --> 37:02.930]  So, provide the ability to opt out of delete data.
[37:03.190 --> 37:06.530]  Remain vigilant around how AI can be misused.
[37:06.730 --> 37:08.570]  Consider unintended consequences.
[37:08.810 --> 37:17.570]  So, we think of that example with the listening AI that was probably intended entirely to be helpful, but could have some issues.
[37:17.570 --> 37:30.430]  And, you know, I think of the new ones cropping up like the screen scraping, AI screen scraping, where you might think you're using a pretty secure tool or application and, in fact, can still be scraped.
[37:31.210 --> 37:35.890]  And finally, study and develop ethical frameworks and checklists for AI.
[37:36.370 --> 37:38.230]  I have some of these on the next screen.
[37:38.430 --> 37:44.170]  I'm not necessarily recommending a particular one or anything, but the fact that they're out there is some sort of progress.
[37:44.170 --> 37:47.710]  So, these are not endorsements necessarily.
[37:47.990 --> 37:53.130]  But companies like Google and IBM have developed ethical AI frameworks and principles.
[37:53.150 --> 37:55.850]  So, have many scholars and researchers.
[37:56.090 --> 37:57.730]  These things are evolving all the time, of course.
[37:58.450 --> 38:08.090]  Salesforce has five trusted AI principles to ensure their work on AI proves responsible, accountable, transparent, empowering, and inclusive.
[38:08.090 --> 38:11.650]  So, you can see it hints at some of the best practices that I mentioned.
[38:11.650 --> 38:30.430]  And UNESCO has a human rights approach, which I appreciate, to AI via ten core principles, including proportionality and do no harm, safety and security, right to privacy and data protection, human oversight and determination, and fairness and non-discrimination.
[38:31.270 --> 38:34.730]  So, those frameworks are out there, and I hope they continue to grow.
[38:35.030 --> 38:36.090]  It's a lot to keep track of.
[38:37.370 --> 38:41.090]  But they can get us thinking along the right lines, I believe.
[38:42.270 --> 38:43.550]  So, final thoughts.
[38:43.750 --> 38:48.350]  We won't be stuffing the AI genie back in its bottle anytime soon, I don't think.
[38:48.350 --> 38:59.990]  And as we learn to navigate its complexities and hopefully benefit some from its productivity, we'll want to pay close attention to these potential harms that it can bring as well.
[39:00.230 --> 39:04.410]  And privacy lapses are certainly going to be one of those types of harms.
[39:05.670 --> 39:11.790]  And I'll close with this quote from Dr. Ann Kavukian again, who devised this privacy by design framework.
[39:11.930 --> 39:15.890]  Because I do think we're tempted sometimes to say, eh, I got nothing to hide, right?
[39:16.270 --> 39:18.070]  I don't have anything to hide.
[39:18.670 --> 39:19.990]  It's just not a big deal to me.
[39:20.090 --> 39:22.730]  I don't think that's the audience I'm talking to today, however.
[39:23.010 --> 39:27.570]  But privacy, it's important to remind people, actually is not about secrecy.
[39:27.570 --> 39:29.150]  It's not about having something to hide.
[39:29.910 --> 39:31.230]  It's all about control.
[39:31.550 --> 39:35.790]  Do I have control over my own data, my own personal information?
[39:36.330 --> 39:47.910]  And, of course, there's also the other possibility that you are working with organizations or something that you trust in the moment, and so you're not concerned about your privacy.
[39:47.910 --> 39:55.570]  But that can change on a dime when legislation or decrees from presidents go into play.
[39:56.150 --> 40:02.850]  Some of us need to look at our own personal threat models for privacy as some of these things are happening.
[40:04.530 --> 40:09.290]  So, I hinted at, maybe heavily, the fact that I've been working on a book.
[40:09.490 --> 40:15.310]  This is not specifically about AI, although there is a chapter about just what we talked about today.
[40:15.310 --> 40:17.390]  But I do have a book coming out.
[40:17.390 --> 40:18.890]  It looks like in November.
[40:19.110 --> 40:20.230]  I'm very excited about that.
[40:20.390 --> 40:29.410]  This is a book about privacy by design that is much broader and talks about some key issues that we should look out for as we're designing.
[40:29.410 --> 40:38.150]  The use of language, avoiding dark patterns, creating tools that actively help people, and also how we handle people's data.
[40:38.330 --> 40:40.510]  Those are kind of the core subjects of this book.
[40:41.550 --> 40:48.590]  And if you would like to learn a little bit about me, I'll leave this up here, but as I say, I'll upload this and link to it probably on my website.
[40:49.350 --> 40:50.670]  You can contact me.
[40:50.990 --> 40:59.150]  There's an article that this talk was based on, and also the article that my book was based on, which of course got fleshed out to be a lot more.
[41:00.930 --> 41:06.570]  Thank you for your time, and we may have time for some questions if you have any
[41:14.940 --> 41:17.080]  questions.
[41:19.140 --> 41:27.000]  The end of a long three days, so I understand if you're just tuckered out, but I'm happy to take some questions.
[41:27.000 --> 41:31.800]  Or if you see me walking around afterwards, of course, come up, and I'm happy to chat.
[41:34.970 --> 41:35.410]  Okay.
[41:36.690 --> 41:37.130]  Great.
[41:37.390 --> 41:41.050]  Thank you again for coming, and maybe see you at the closing ceremony.
[41:41.410 --> 41:43.110]  And have a safe trip home.