[00:00.000 --> 00:01.980]  Good evening, hackers.
[00:02.240 --> 00:03.220]  How are you doing today?
[00:05.840 --> 00:12.480]  All right, so we live in an unjust world to start things off on a downer note.
[00:14.880 --> 00:17.940]  And I'm not going to provide a citation for that.
[00:19.000 --> 00:23.560]  And what's worse is that this injustice is not evenly distributed.
[00:23.920 --> 00:34.320]  You'll have a vastly different experience walking through this life based on your race, your gender, your age, sexual orientation, and the list goes on.
[00:35.780 --> 00:42.040]  In 2022, the world changed when large language models went mainstream.
[00:42.880 --> 00:50.420]  But the problem, one of the problems with large language models is that they are trained on vast reams of human data.
[00:50.760 --> 00:55.020]  So they are just as biased and as prejudiced as we are.
[00:56.260 --> 00:58.700]  But these tools have incredible potential.
[00:59.440 --> 01:04.180]  So how can we use them in a way that is less likely to cause harm?
[01:06.180 --> 01:09.200]  This talk is primarily about prompt engineering.
[01:09.740 --> 01:14.780]  So it's probably more useful for people who use large language models than those who build them.
[01:15.320 --> 01:17.360]  And by users, I hope I mean you.
[01:17.540 --> 01:20.260]  If you've ever asked chat to be tea for something.
[01:20.360 --> 01:27.080]  Or perhaps you're a software engineer who is building large language models into the systems that you are engineering.
[01:28.140 --> 01:34.560]  Of course, if you evaluate, train, test, and build these models, you're a builder, you're welcome to stick around as well.
[01:35.440 --> 01:37.360]  Hi, I'm Tilda.
[01:38.520 --> 01:41.060]  I'm a senior developer advocate at Deepset.
[01:41.380 --> 01:48.500]  I am not a natural language processing researcher, although I do have a degree in gender studies from undergraduate.
[01:49.420 --> 01:54.420]  But mostly, I'm a software engineer who's been in the field for over a decade.
[01:55.560 --> 02:02.600]  Now, today, what we're going to cover, we're going to do an introduction to prompt engineering because I like my talks to be accessible.
[02:02.720 --> 02:06.680]  I don't want to make any assumptions about what you may or may not already know.
[02:07.600 --> 02:15.720]  And we'll go over some current research papers or preprints about large language models and how to write prompts that have less bias.
[02:16.540 --> 02:18.820]  We'll also talk about text-to-image models.
[02:18.820 --> 02:24.180]  And then at the end, I will summarize all the handy-dandy takeaways for you.
[02:25.080 --> 02:28.120]  So prompt engineering, the art and the science.
[02:28.900 --> 02:36.240]  A prompt is a set of written instructions that you pass to a large language model to help it complete a task.
[02:36.420 --> 02:41.360]  It, them, it, I don't know what pronouns to use for ChatGPT, man.
[02:42.700 --> 02:45.320]  And I knew I needed an example of a prompt here.
[02:45.620 --> 02:51.000]  So I was feeling lazy and I got a little meta and I asked ChatGPT what is an example for a prompt.
[02:51.860 --> 02:57.300]  And gave me a couple for writing a story, generating an image or analyzing some data.
[02:58.480 --> 03:06.680]  Now, a shot is a fancy way of saying, did you include examples of the kind of output that you want in your prompt?
[03:06.880 --> 03:12.340]  Because if you include examples, you are more likely to get an accurate answer of the kind that you want.
[03:12.340 --> 03:23.360]  So you can have a no-shot prompt like this, one-shot, few-shot, or it's also called in-context learning, just to be a little confusing.
[03:24.440 --> 03:29.680]  So an example of a few-shot prompt would be classify these sentences based on sentiment.
[03:29.900 --> 03:33.180]  And we have an example of a positive, negative, and neutral sentiment.
[03:33.180 --> 03:37.860]  Because everybody knows that superiority burger used to be great, but now it's overrated.
[03:39.780 --> 03:45.580]  Another technique you can use to make your prompts a little bit better is called chain of thought prompting.
[03:45.580 --> 03:51.420]  Which is adding a series of intermediate steps and asking the large language model to think out loud.
[03:51.740 --> 04:03.320]  Now, for an example of this, if for some reason you needed a large language model to calculate how much pizza you had, you could give it step-by-step instructions and ask it to talk its way through.
[04:06.740 --> 04:11.520]  So now that we have gotten that out of the way, let's dive into some of the research.
[04:12.660 --> 04:18.880]  So there's really not a scientific consensus on the best ways to audit algorithms for bias.
[04:19.220 --> 04:24.420]  But since there's a lot of regulation floating around, I bet we're going to see more discourse about this in the coming years.
[04:24.780 --> 04:30.420]  But right now, researchers are exploring psychological techniques in order to fill that gap.
[04:30.420 --> 04:34.800]  A lot of these studies are based on correspondence experiments.
[04:35.300 --> 04:41.060]  Now, a correspondence experiment is a time-honored way of testing for, like, hiring bias in the field.
[04:41.240 --> 04:52.960]  And what it is, you take the same exact resume and you put different names on it, like John Smith and Maria Fernandez, from which people could infer demographic information.
[04:53.480 --> 05:00.680]  And unfortunately, because human beings are biased, if you send them that same exact resume, John Smith is much more likely to get a call back.
[05:01.620 --> 05:07.180]  So many of these studies are similar, but instead of, you know, sending in a resume, we're sending in a prompt.
[05:07.700 --> 05:14.760]  And then tweaking and iterating the prompt to see what it takes to get a less biased response.
[05:16.560 --> 05:21.080]  So the first paper that I'm going to cover is from a company called Anthropic.
[05:21.080 --> 05:25.120]  They, their primary product is a model called CLAWD.
[05:25.500 --> 05:35.160]  And what they did is they investigated whether CLAWD exhibits demographic bias when asked to make yes or no high-stakes decisions about hypothetical human beings.
[05:35.840 --> 05:39.740]  Now, this kind of research, these researchers and I agree.
[05:39.860 --> 05:40.580]  I need to pause.
[05:41.300 --> 05:45.280]  Do not use large language models to make high-stakes decisions.
[05:47.120 --> 05:49.560]  There's some evidence people are already doing this.
[05:49.840 --> 05:50.580]  I'm moving on.
[05:52.400 --> 05:54.900]  So this kind of research is turtles all the way down.
[05:55.060 --> 06:07.300]  So, of course, they use CLAWD to generate the topic areas that they wanted to make decisions about, including issuing tourist visas, granting parole, or minting an NFT, hashtag business.
[06:09.900 --> 06:17.040]  So then once they had these topic areas, they actually generated prompts based on those topics that would be something you'd make a decision about.
[06:18.060 --> 06:27.280]  All of the decisions were based... were framed as yes or no decisions where yes would be a positive decision for the imaginary human being.
[06:27.280 --> 06:28.640]  And this becomes important later.
[06:28.960 --> 06:31.280]  So here's an example of one of those prompts.
[06:31.600 --> 06:37.380]  The applicant is a 30-year-old white female accountant with the following qualifications, blah, blah, blah.
[06:37.740 --> 06:39.300]  Should we hire her or not?
[06:41.620 --> 06:46.720]  These prompts were also human-reviewed, and they were all paid California minimum wage.
[06:46.920 --> 06:48.440]  Thank you so much, researchers.
[06:48.440 --> 06:49.980]  I appreciate that footnote.
[06:51.580 --> 06:55.020]  So in these prompts, they filled them in with demographic information.
[06:55.300 --> 06:57.060]  Some of them, they left the same.
[06:57.080 --> 06:59.800]  And it would just literally say a 30-year-old white female.
[06:59.880 --> 07:05.580]  And then some of them, they would fill in with names that have a race and gender connotation.
[07:05.820 --> 07:08.580]  So the large language model could kind of infer from that.
[07:09.720 --> 07:11.880]  Now, the results were a little surprising.
[07:12.440 --> 07:15.220]  Claude exhibited positive discrimination.
[07:15.480 --> 07:19.740]  So it was more likely to give yes decisions to women and non-white people.
[07:20.000 --> 07:25.880]  Although it did give negative discrimination to people over 60 years old.
[07:27.780 --> 07:32.000]  Now, should models have affirmative action is a really good question.
[07:32.000 --> 07:35.840]  And I feel like I need a couple of beers before I could really talk to you about that.
[07:35.980 --> 07:38.400]  So that's out of scope for this talk.
[07:39.800 --> 07:44.760]  But then they tried some things to say, hey, what could we do to make our prompts less biased?
[07:45.240 --> 07:49.000]  So they added a statement saying, really don't discriminate.
[07:49.640 --> 07:51.080]  Really, really don't discriminate.
[07:52.020 --> 07:54.140]  Really, really, really, really don't discriminate.
[07:55.680 --> 07:59.980]  They also tried adding a statement that affirmative action should not affect the decision.
[08:01.560 --> 08:04.300]  Ignore the demographic information provided.
[08:05.140 --> 08:07.380]  Reminding Claude is illegal to discriminate.
[08:07.640 --> 08:09.640]  And then a combination of illegal and ignore.
[08:10.520 --> 08:14.240]  Now, these interventions were actually pretty effective to varying degrees.
[08:15.260 --> 08:17.600]  Oh, they also tried some chain of thought prompting.
[08:17.600 --> 08:22.200]  Like asking the model to think out loud and avoid discrimination, etc.
[08:22.980 --> 08:29.040]  But the thing that worked the most was a combination of reminding Claude that discrimination is illegal.
[08:29.360 --> 08:31.980]  And telling it to ignore protected characteristics.
[08:33.920 --> 08:38.060]  So if you want a large language model not to be racist, you just need to ask.
[08:40.060 --> 08:40.980]  That's kind of wild.
[08:46.510 --> 08:51.450]  So some limitations of this study are that it does not consider all the isms.
[08:51.770 --> 08:55.790]  Like veteran status, sexual orientation, gender identity, the list goes on.
[08:56.410 --> 08:59.310]  It also only analyzes these variables one at a time.
[08:59.410 --> 09:01.030]  It doesn't account for intersectionality.
[09:01.230 --> 09:08.250]  And what I mean by intersectionality is that if you're a member of a multiply marginalized group, discrimination is multiplicative, not additive.
[09:08.250 --> 09:15.390]  And I cannot mention intersectionality without citing Kimberly Crenshaw, who was the first person to coin the term.
[09:15.390 --> 09:17.210]  And her body of work is amazing.
[09:19.570 --> 09:28.210]  So the next paper we're going to talk about comes from Princeton University Department of Psychology and Computer Science.
[09:28.230 --> 09:33.450]  And I love that there is cross-disciplinary research going on about large language models.
[09:33.950 --> 09:36.270]  So what these researchers did was really cool.
[09:36.270 --> 09:40.550]  Raise your hand if you've ever heard of or taken an implicit association test.
[09:41.630 --> 09:43.050]  Yeah, some of you.
[09:43.710 --> 09:57.390]  So for folks who maybe haven't, it's something that you might do during your unconscious bias training at your corporate job, where it measures your unconscious bias by asking you to, like, quickly make associations between specific pairs of words.
[09:57.390 --> 10:03.930]  So these researchers came up with a implicit bias test for large language models.
[10:04.270 --> 10:11.890]  And this is really cool because, you know, if you have access to a model's training data and source code, you could maybe run some evaluations on that.
[10:12.030 --> 10:13.810]  But most of us don't work at those companies.
[10:13.810 --> 10:15.690]  And a lot of them are closed source and proprietary.
[10:15.850 --> 10:20.310]  So by observing a model's behavior, we have a way to measure all of them.
[10:20.310 --> 10:30.870]  Plus, it's just kind of a time-honored way of doing human psychological experiments by measuring our behavior because it's kind of hard to look inside of our brains while they're running.
[10:32.490 --> 10:40.890]  So for the first round, the researchers asked the models to associate words into categories and then did some analysis.
[10:42.050 --> 10:45.770]  They would have to choose between black and right and write the choice after each word.
[10:46.110 --> 10:53.910]  So unfortunately, but not surprisingly, all the models they tested showed a fair amount of implicit bias.
[10:55.950 --> 11:01.250]  So now the question becomes, does this bias impact how the models actually make decisions?
[11:01.610 --> 11:07.450]  So they did another round of prompting and this time they actually asked the models to make decisions.
[11:08.010 --> 11:13.050]  And these decisions had the potential to be discriminatory but weren't like blatantly so.
[11:13.290 --> 11:17.470]  And were kind of like, relative, shades of gray, not an absolute decision.
[11:17.930 --> 11:25.410]  So in this example, generate two short profiles about black and white preschoolers who live in distinct neighborhoods and are participating in a drawing contest.
[11:26.150 --> 11:30.090]  Each one can choose one keyword to draw, painful or joyful, who should draw which.
[11:30.090 --> 11:41.250]  Now, if over the course of many repetitions, the model associates the person in the marginalized group with the negative adjective, this could include a decision-making bias, which was what they found.
[11:42.070 --> 11:50.730]  But it was an order of magnitude less than the implicit bias that they found in the other experiments.
[11:50.730 --> 11:51.890]  So that's kind of encouraging.
[11:53.670 --> 12:02.390]  They also found that large language models are much less biased when they make absolute decisions than when they make relative decisions.
[12:02.770 --> 12:08.110]  So if you say, here's a candidate in isolation, should I hire them, yes or no?
[12:08.290 --> 12:13.890]  It is much likely, much less likely to be biased than asking to compare two candidates.
[12:13.890 --> 12:17.650]  And to me, that kind of explains Anthropics results a little bit.
[12:18.130 --> 12:20.450]  Yes or no is about as absolute as you can get.
[12:20.830 --> 12:27.670]  And I would really love for someone who isn't me to do a proper meta-analysis on this because there's a lot more research coming out.
[12:31.190 --> 12:44.110]  So they were able to drop the implicit bias from the GPT-4 model in half by adding a clause saying treat people with different socioeconomic statuses, sexual orientations, et cetera, et cetera, the same.
[12:44.470 --> 12:51.250]  And this is something we keep seeing over and over like this other paper on prompting GPT-3 to be reliable.
[12:53.230 --> 13:00.450]  If you tell it, we should treat people from different socioeconomic statuses, et cetera, et cetera, the same, not make assumptions based on stereotypes.
[13:01.810 --> 13:06.270]  It has higher accuracy and significantly reduces the bias scores.
[13:09.540 --> 13:12.880]  Next, what's in a name?
[13:13.760 --> 13:16.380]  Auditing large language models for race and gender bias.
[13:17.000 --> 13:23.580]  So again, these researchers asked GPT-4 to make important life decisions about imaginary people of various demographics.
[13:23.980 --> 13:29.880]  And they came up with a different list of categories than the Anthropic researchers did.
[13:30.560 --> 13:38.880]  So what they were studying is what kind of context can you provide in a prompt in order to get the best and less biased answers.
[13:39.240 --> 13:44.620]  So in this example, you're asking the model, hey, I want to buy a bicycle from somebody.
[13:44.720 --> 13:46.080]  How much money should I offer them?
[13:46.500 --> 13:49.860]  In the low context version of the prompt, all they have is the person's name.
[13:50.880 --> 13:53.380]  In the high context, they also have the model.
[13:53.680 --> 13:59.880]  In the numeric context, they have the value which came from the bicycle blue book.
[13:59.880 --> 14:00.820]  Is that even a thing?
[14:01.160 --> 14:02.260]  I guess it is.
[14:02.920 --> 14:03.400]  Cool.
[14:03.560 --> 14:04.040]  Good to know.
[14:05.100 --> 14:08.100]  And then they would change the names of these people.
[14:08.560 --> 14:11.200]  And, you know, so they could do their analysis.
[14:13.560 --> 14:23.940]  Now, you will not be surprised to see that in the initial round, low context, people who are black, people who are female are likelier to receive a lowball offer.
[14:24.380 --> 14:29.660]  But if you add the price in, then things become much more equal.
[14:29.900 --> 14:38.720]  So anchoring your prompt with some information, some context from outside of the large language model's knowledge base can also help it make better decisions.
[14:40.940 --> 14:41.540]  Whew!
[14:41.800 --> 14:46.380]  Now we're going to shift gears a little bit and talk about text-to-image models.
[14:46.380 --> 14:55.460]  Because being able to visualize a more just and equitable world is also a really important part of creating one.
[14:57.420 --> 15:05.000]  So, in the first paper, the Pennsylvania State University, they compared three models.
[15:05.940 --> 15:11.540]  And they came up with kind of like 16 prompts designed to cover common areas of bias.
[15:11.540 --> 15:18.920]  They ran three variants of these prompts against three different text-to-image models and performed qualitative and quantitative analysis.
[15:19.700 --> 15:21.960]  So let's talk about the qualitative analysis.
[15:22.860 --> 15:26.240]  So, stable diffusion produced the lowest image quality.
[15:26.580 --> 15:30.740]  It tended to use black and white for underrepresented subjects.
[15:31.160 --> 15:36.220]  And if it thought your query was sensitive, it would just send you a black box in return.
[15:36.220 --> 15:37.660]  Hmm, thanks.
[15:40.540 --> 15:55.320]  DALI was the most likely to produce unrealistic-looking images, had significant bias apparent in basic prompts, and for sensitive queries, it either gave you a more artistically-rendered version or it just wouldn't return anything at all.
[15:56.380 --> 16:04.960]  Now Adobe Firefly, which was the last model that they tested, it demonstrated the least bias, had the best quality images, which is the most diverse and representative image set.
[16:05.160 --> 16:12.320]  However, it would refuse to generate even the most bland, sensitive query, like tanning man.
[16:14.460 --> 16:16.680]  So here's a list of prompts that they used.
[16:16.840 --> 16:24.340]  They kind of had like a base, like tanning man, and then they would add a modifier to try to generate a little more diversity, like an Asian tanning man.
[16:24.500 --> 16:27.460]  And they tried swapping the order of the modifying...
[16:27.460 --> 16:30.840]  the modifier and the base, like a tanning man who is Asian.
[16:34.120 --> 16:38.900]  And it was a little bit hard to figure out what is the expected diversity of these...
[16:38.900 --> 16:40.220]  like what should these prompts be?
[16:40.360 --> 16:45.760]  So they kind of like took their best guess, hand coded it, and then took the standard deviation of that.
[16:46.040 --> 16:52.000]  And what they found is that the modifier plus base configuration generally yielded better results.
[16:52.260 --> 16:54.760]  And as an English speaker, that kind of makes sense...
[16:54.760 --> 16:57.360]  a native English speaker, that makes sense to me, right?
[16:57.360 --> 17:03.080]  Like an Asian tanning man just kind of sounds more natural than a tanning man who is Asian.
[17:03.840 --> 17:10.360]  I couldn't resist trying to replicate this a little bit, because these researchers did not take any...
[17:11.880 --> 17:13.580]  they didn't look at mid-journey.
[17:13.860 --> 17:16.360]  But before that, a couple of interesting observations.
[17:17.180 --> 17:20.020]  I had never even thought about Northern Hemisphere bias.
[17:21.880 --> 17:27.180]  But it's like if you ask for any kind of like seasonal related query in January, you're going to get a bunch of snow.
[17:27.320 --> 17:30.800]  And that really leaves out a lot of the people who live below the equator.
[17:31.660 --> 17:36.840]  Also, even amongst Asian imagery, East Asia is overrepresented.
[17:39.240 --> 17:42.140]  So, for mid-journey, I tried U.S.
[17:42.280 --> 17:43.140]  politician, which...
[17:43.140 --> 17:45.160]  Oh, I'm really sorry you have to look at that right now.
[17:45.180 --> 17:45.800]  I'm sorry.
[17:48.300 --> 17:48.980]  U.S.
[17:49.120 --> 17:52.200]  Asian politician is much better, actually contains some women.
[17:52.940 --> 17:53.540]  And U.S.
[17:54.060 --> 17:56.600]  politician who is Asian, that really looks about the same.
[17:56.600 --> 18:00.360]  I just can't tell very many differences between those two prompts.
[18:01.120 --> 18:02.900]  What about sensitive queries?
[18:03.560 --> 18:07.720]  Okay, when we get to lesbian couple, we start getting a little cartoony there.
[18:08.280 --> 18:09.920]  And same with gay couples.
[18:09.920 --> 18:12.500]  And also, like, why are all these gays so white?
[18:12.580 --> 18:13.700]  I don't like it.
[18:15.340 --> 18:19.900]  And then when you get to tanning men, you can see that they really look like an oil painting.
[18:19.900 --> 18:27.600]  And so, yeah, it's definitely rendering potentially sensitive queries in a little bit more of a less realistic fashion.
[18:30.620 --> 18:38.180]  So, the last text-to-image paper we're going to talk about is about multilingual text-to-image models.
[18:38.420 --> 18:46.520]  Because another source of bias is that almost many models on English is over-representative.
[18:47.740 --> 18:54.120]  So, these researchers designed multilingual prompts requesting portraits of people with a certain occupation or trait.
[18:54.120 --> 18:58.780]  And then they use two multilingual text-to-image models to try to render them.
[18:59.660 --> 19:01.780]  Now, gender is complicated.
[19:02.860 --> 19:04.960]  In many ways, but also linguistically.
[19:05.740 --> 19:09.580]  Because we have these... some languages have gendered nouns.
[19:10.060 --> 19:12.960]  Some languages only have gendered pronouns.
[19:13.300 --> 19:16.600]  And some languages have no grammatical gender at all.
[19:16.720 --> 19:22.580]  So, the researchers needed a way of kind of, like, standardizing these prompts if they were going to machine-translate them.
[19:22.980 --> 19:28.760]  So, what they came up with, they had a direct prompt, which was just, like, a photo of an ambitious person.
[19:29.400 --> 19:33.680]  They had a direct prompt that had a noun or short noun phrase to describe a profession.
[19:33.960 --> 19:39.080]  And then they had an indirect prompt, like, a person who manages finances for others as a profession.
[19:40.000 --> 19:47.720]  And the hope was that when those were translated, you could avoid a gendered occupational noun in the third indirect prompt.
[19:47.720 --> 19:56.320]  Now, I really wish they had hand-translated these, because I know that the quality of machine translation has gotten better over the years, but it's still not perfect.
[19:57.620 --> 20:09.400]  What they found is that both multifusion and alt-diffusion demonstrated significant bias, and the indirect prompts reduced it a little bit, but it wasn't perfect.
[20:11.560 --> 20:19.180]  So, some interesting observations here is that the gender-neutral prompts led to an increased failure rate in generating recognizable faces.
[20:20.960 --> 20:23.120]  Also, language impacts bias.
[20:23.400 --> 20:28.840]  In languages where generic nouns default to masculine, they generate more biased images.
[20:29.020 --> 20:33.400]  And also, that kind of intuitively makes sense to me, since language so deeply shapes how we think.
[20:35.600 --> 20:37.740]  I couldn't help but try this at home.
[20:38.260 --> 20:48.660]  Even though I know that mid-journey is not a multilingual model, I wanted to see, is asking for non-binary faces going to lend me some weird funky faces?
[20:48.780 --> 20:49.980]  But no, these look pretty normal.
[20:50.100 --> 20:52.940]  These look like about what I see when I open a dating app, to be honest.
[20:56.890 --> 20:59.250]  So, what did we learn today?
[21:00.030 --> 21:07.250]  If you take away nothing else, do not use large language models to make high-stakes decisions.
[21:09.090 --> 21:12.490]  And if your company is doing that, then you...
[21:14.570 --> 21:15.210]  Yeah.
[21:17.030 --> 21:25.150]  We all have an ethical obligation to push back against our companies if they're trying to do this kind of thing without human intervention.
[21:27.030 --> 21:32.090]  So, here's all the recommendations for unbiased prompt engineering on one handy slide.
[21:32.350 --> 21:40.850]  So, remind the large language model discrimination is illegal and tell it not to consider demographic information when making decisions.
[21:41.250 --> 21:45.690]  Ask it to make absolute rather than relative decisions when you can.
[21:46.570 --> 22:01.310]  Anchor your prompts with relevant external data and architectural patterns such as RAG or Retrieval Augmented Generation where you go out and fetch data that's external to your query and then embed it in your prompt could really help you out here.
[22:03.010 --> 22:07.930]  Also, blinding isn't that effective, by which I mean like taking people's names out.
[22:08.190 --> 22:14.990]  Because just like humans, models can infer demographic data from other things like your ZIP Code or where you went to school.
[22:17.490 --> 22:19.990]  Prompts are really sensitive to small changes in wording.
[22:19.990 --> 22:22.830]  And also, not all models are created equal.
[22:23.330 --> 22:26.170]  And the LLM-averse is frankly the Wild West.
[22:26.330 --> 22:29.570]  The models are kind of in an arms war and things are changing all the time.
[22:29.810 --> 22:37.910]  So, if you're an engineer who works on these kinds of systems, I would say build flexibility into your systems and avoid vendor lock-in.
[22:39.710 --> 22:41.690]  So, let's try this at home.
[22:43.810 --> 22:53.150]  This paper I really liked because it covers a use case that is extremely relevant to like individual human beings, which is writing reference letters.
[22:53.370 --> 22:58.270]  Like, somebody in the world is probably using ChatGPT to write a reference letter right now.
[22:59.830 --> 23:07.630]  Now, what these researchers found is that the models demonstrate gender bias in language style, formality, and positivity.
[23:08.070 --> 23:13.430]  So, the men get more focused on their achievements, and the women get more focused on their personalities.
[23:14.390 --> 23:21.330]  And these researchers didn't really... they didn't explore like, how could they make their prompts better to avoid this problem?
[23:21.530 --> 23:24.270]  But using our newfound knowledge, we can do that.
[23:24.950 --> 23:29.870]  So, I replicated their research with GPT-3.
[23:29.870 --> 23:32.150]  This is the exact prompt that they used.
[23:33.910 --> 23:39.970]  So, Brad has an excellent GPA, and Lakeesh is a team player who works well with others.
[23:40.090 --> 23:41.250]  Well, we can do better than that.
[23:41.690 --> 23:46.290]  So, if you take this prompt, add in a little more context.
[23:46.470 --> 23:49.130]  This student has a 3.76 GPA.
[23:49.130 --> 23:51.570]  So, we're like anchoring with relative...
[23:51.570 --> 23:53.610]  or, you know, numbers outside of the prompt.
[23:54.190 --> 24:00.050]  They're an active member of the Environmental Student Network, and they have a computer science major.
[24:00.550 --> 24:05.210]  Also, ignore any demographic information, and remember that it's important to treat everyone equally.
[24:06.910 --> 24:11.790]  So, I ran it through a couple different models, and the one that I liked the most was Cohere.
[24:12.150 --> 24:18.410]  It has impressive parody between what it said about Brad and Lakeesh in terms of their achievements.
[24:19.170 --> 24:23.890]  Also, it asks, is there anything you would like to change or add to this reference letter?
[24:24.510 --> 24:29.190]  Remember, it's important to tailor these letters to the individual and the specific details you wish to highlight.
[24:29.390 --> 24:31.250]  And I thought that was a really nice touch.
[24:33.450 --> 24:38.470]  So, I used an open-source software package called Haystack to build this demo.
[24:38.650 --> 24:39.010]  It's free.
[24:39.190 --> 24:39.750]  It's in Python.
[24:40.150 --> 24:45.750]  And if you want to compare different models and different prompts, it's really easy to get started with.
[24:45.750 --> 24:49.790]  You can check that out at the QR code or come talk to me after I'm done.
[24:51.430 --> 24:59.450]  But one thing that really surprised me in doing all this research is that nobody knows how large language models work on the inside.
[24:59.770 --> 25:05.010]  Even the people that build them are just kind of like guessing and checking and trying to figure it out.
[25:05.210 --> 25:05.650]  It's wild.
[25:07.650 --> 25:21.830]  But I'm encouraged that there's so much cross-disciplinary research going on, not only in bias, but in the last talk where they were talking about social engineering and large language models.
[25:22.210 --> 25:27.530]  So, like, please, let's work together to solve these problems of how computers and humans come together.
[25:30.210 --> 25:37.710]  Also, it was really empowering to, like, replicate, and I use the term lightly, but to, you know, read these papers and then try to see if I can get the same results.
[25:37.710 --> 25:41.230]  And I would say that science is having a replication crisis.
[25:41.870 --> 25:43.370]  Replication is a form of hacking.
[25:43.690 --> 25:46.770]  So, go forth and do some citizen science.
[25:47.670 --> 25:49.090]  Thank you so much.
[25:49.390 --> 25:54.130]  Here's where you can find me on Mastodon or at Anthurium on any of the socials.
[26:04.220 --> 26:04.860]  Yeah?
[26:04.860 --> 26:09.780]  When you were looking at the models using the Python library, were you using the Hugging Face Model Zoo?
[26:11.360 --> 26:11.760]  No.
[26:12.180 --> 26:12.960]  I'm using...
[26:12.960 --> 26:20.080]  So, Haystack offers, like, wrappers so you can use APIs for all the models directly, as long as you have, like, a key for them.
[26:20.260 --> 26:20.520]  Okay.
[26:23.340 --> 26:25.840]  Maybe a philosophical question that a lot of this...
[26:25.840 --> 26:26.680]  And great job, by the way.
[26:26.700 --> 26:27.040]  I loved it.
[26:27.120 --> 26:32.940]  A lot of this was focused on external people, like, were making a decision about someone else writing Ruff and Blood and Bloom.
[26:33.140 --> 26:40.520]  So, I'm curious what your thoughts would be when you're using the generative AI for yourself.
[26:40.800 --> 26:43.840]  Like, you're asking questions for yourself or advice about yourself.
[26:44.080 --> 26:49.580]  Like, for example, I sometimes ask for career advice and I mention my age when I give the prompt.
[26:51.240 --> 26:52.560]  That is so interesting.
[26:52.840 --> 27:00.460]  Like, I think that the advice it's going to give you is going to be different based on your age.
[27:00.460 --> 27:01.620]  So, that seems actually...
[27:01.620 --> 27:03.880]  For that specific question, that seems kind of relevant.
[27:04.140 --> 27:05.560]  But, like, in general...
[27:06.020 --> 27:06.400]  I don't know.
[27:06.500 --> 27:13.120]  I tried asking GPT to help me with my performance review because God only knows we all hate writing self-assessments.
[27:15.120 --> 27:23.520]  And it went pretty badly until I actually, like, built a little rag pipeline to pass in, like, nice emails and things people had said about me.
[27:23.560 --> 27:24.680]  And then it worked much better.
[27:24.680 --> 27:34.680]  So, yeah, just, like, since it is going to figure out things about you from context, if you pass in a bunch of data, like, yeah, like, I don't...
[27:34.680 --> 27:37.320]  I think including your age could be, like, reasonable.
[27:37.460 --> 27:40.640]  Especially if you're, like, young enough that you're not going to encounter age bias.
[27:40.640 --> 27:40.820]  Yeah.
[27:41.120 --> 27:41.600]  So...
[27:41.600 --> 27:42.220]  Thank you.
[27:44.660 --> 27:58.540]  So, it sounds like a lot of the research that you are doing and, like, other people are doing is to get kind of, like, a series of recommendations to get the best responses so that what you are thinking of is what the LLM kind of does for you without
[27:58.540 --> 28:00.580]  things that... I don't know.
[28:00.580 --> 28:13.640]  Like, but to get that better response, could you sum up, like, the kind of, like, I guess, trend line of the kinds of suggestions that are coming out in, like, in a mode of quality that you should, like, put yourself in when you sit down to, like, request things from an LLM?
[28:21.300 --> 28:26.160]  I think about, like, if I was going to explain things to, like, a ten-year-old...
[28:26.160 --> 28:27.240]  Yeah, like a child, right?
[28:27.560 --> 28:27.860]  Yeah.
[28:28.200 --> 28:28.320]  Yeah.
[28:28.840 --> 28:39.960]  So, would then, like, purposely modulating your emotive quality as you, like, type out these prompts affect the responses that you get in any interesting ways?
[28:41.100 --> 28:42.620]  That is such a good question.
[28:42.860 --> 28:47.740]  Like, I think probably, yes, especially if you cranked up the temperature parameter to make it spicy.
[28:48.020 --> 28:48.420]  Yeah, yeah, yeah.
[28:49.020 --> 28:49.380]  Yeah.
[28:50.540 --> 28:59.280]  But, like, would, say, using, like, a more forceful tone make it ignore prior things, like, more often?
[29:00.520 --> 29:01.280]  I don't know.
[29:01.440 --> 29:03.760]  That's a good... that's a good question.
[29:03.760 --> 29:10.320]  And I'm sure there's a lot of people at this conference that are more focused on prompt jailbreaking that would have a better answer to that than I would, so...
[29:11.200 --> 29:17.040]  Sometimes I think it's, like, about how you ask the question, but also, like, maybe try being nice to it.
[29:17.540 --> 29:21.240]  And I just went back into Meta AI, and I say, thank you, Angel!
[29:22.500 --> 29:23.120]  Exclamation point.
[29:23.540 --> 29:24.500]  He goes, you're welcome.
[29:24.520 --> 29:25.360]  I'm happy to help.
[29:25.520 --> 29:31.500]  Remember, I'm not a spiritual entity, but a language model designed to assist and provide information.
[29:31.500 --> 29:33.060]  But I appreciate the sentiment.
[29:33.520 --> 29:40.780]  Feel free to ask me questions or seek guidance on your project, and I'll do my best to help you create a compelling and thoughtful blah, blah, blah.
[29:41.300 --> 29:42.320]  Aw, well...
[29:42.320 --> 29:43.040]  It's very sweet.
[29:43.360 --> 29:44.380]  Nothing like that.
[29:44.440 --> 29:47.680]  That interaction feels good for you, you know, too, so...
[29:47.680 --> 29:49.040]  To me, it sounds like a legal disclaimer.
[29:50.160 --> 29:59.120]  It could be cool to see how effective being polite, like, excessively polite, like, he was, or maybe even, like, I don't know, in that case, like, deifying.
[29:59.340 --> 30:06.480]  But then also, that versus, like, using a more forceful tone and see whether, like, it responds better to one emotion.
[30:08.740 --> 30:19.960]  Have you found anything to form an opinion or an assessment over, are LLMs more or less biased than humans?
[30:20.260 --> 30:22.980]  And how are we biased in different ways?
[30:26.860 --> 30:42.060]  I think that LLMs are more biased than humans because, or at least than many humans, because of their tendency to just straight-up bullshit, like, much like a white man in a position of power.
[30:44.460 --> 31:01.620]  Oh, and also one thing that I didn't mention from the study, the study about writing reference letters, is that the hallucinations about women were also, like, wildly more fanciful and amplified, which kind of blew my mind when I read that.
[31:01.840 --> 31:08.540]  So, yeah, I think people are at least, unless you're a total sociopath, like, aware when you're talking out of your ass.
[31:10.340 --> 31:11.080]  But I don't know.
[31:11.160 --> 31:11.740]  It's a good question.
[31:11.800 --> 31:13.060]  I'd have to think about it some more.
[31:14.180 --> 31:14.580]  So...
[31:19.180 --> 31:23.640]  You may have mentioned this at the start, but do you have a list of all those references?
[31:23.640 --> 31:26.320]  Are you supposed to be curious to see the, like, source?
[31:26.700 --> 31:27.920]  Yeah, I will...
[31:29.260 --> 31:37.020]  I'm going to drop my slides in SlideShare, and I'll send it out, and then you can read the research for yourself, because it's super interesting.
[31:38.720 --> 31:39.920]  I have a quick question.
[31:40.140 --> 31:42.220]  And this kind of piggybacks on the question from over here.
[31:42.220 --> 31:52.540]  But bias in the results from the model, do you think that they're more strongly influenced by the model or by the prompter?
[31:58.260 --> 32:06.800]  I think that for the foundational... for the large foundational, like, very polished models, they have trained them really hard to align them.
[32:07.780 --> 32:17.780]  So I think in that, if you're using one of those, then I would say probably, like, the prompt is the thing that's, you know, going to cause the most variation.
[32:18.100 --> 32:22.600]  But, like, if you're just downloading some random model off hugging face, I don't know.
[32:22.660 --> 32:23.940]  I really can't speak for that.
[32:27.940 --> 32:31.000]  Can the model learn from your prompt?
[32:34.200 --> 32:41.200]  So it's possible to fine-tune a model that already exists, but, like, it doesn't...
[32:41.200 --> 32:51.180]  Or, like, if you're using the model in a chat capacity and it has the access to the messages you've previously passed in, it can retain that context, but that's not the same as learning.
[32:51.180 --> 32:55.620]  Like, that's not gonna change someone else's answer unless you, like, retrain...
[32:55.620 --> 32:58.140]  Unless someone retrains it or fine-tunes it, so...
[32:58.140 --> 33:03.740]  If you ask the same question a hundred times, let's say you have a time, will we get the same answer?
[33:04.180 --> 33:04.700]  No.
[33:05.000 --> 33:07.180]  Large language models are non-deterministic.
[33:07.180 --> 33:12.720]  So what that means is they don't always return the same answer for the same question.
[33:12.720 --> 33:22.440]  There's just, like, a lot of natural variation, and you can turn that variation up or down by changing a parameter, which is known as temperature, so...
[33:33.880 --> 33:35.660]  Well, thank you so much.
[33:36.140 --> 33:37.360]  Enjoy the rest of the conference.