[00:00.000 --> 00:12.040]  All right, so we're going to be talking a little bit about social engineering principles, some basic concepts that have been practiced throughout the course of human history since confidence men and before that.
[00:12.320 --> 00:16.600]  And then we're going to talk about why they work in the field, specifically with physical engagements.
[00:17.060 --> 00:24.340]  And then we're going to talk a little bit about what we've done to leverage generative AI to actually create new fish and lure types using those same principles.
[00:25.720 --> 00:28.640]  So as we just discussed, I'm Security Sean.
[00:28.860 --> 00:30.080]  I have my colleague Zale here.
[00:30.780 --> 00:33.720]  So Zale has been red teaming since he was a minor.
[00:34.360 --> 00:39.900]  I don't want to date you, but decades, we'll call it, since before bug bounties existed.
[00:40.220 --> 00:43.460]  And, you know, I've been a social engineer for a long time.
[00:43.640 --> 00:47.220]  I'd like to think I'm a natural-born scumbag, so it just comes naturally.
[00:48.100 --> 00:50.480]  All right, so we're going to go through psychological principles.
[00:50.480 --> 00:57.000]  We're going to talk a little bit about those analog case studies that I talked about, the physical social engineering engagements and why those principles work.
[00:57.200 --> 01:03.880]  Then we'll go through a little bit of AI voice cloning history and what makes sense to actually leverage today for some background there.
[01:04.180 --> 01:13.620]  And then we'll go through our case study that is really what the topic is, which is getting high like planes using the actual generative AI fishing lure that we talked about.
[01:13.780 --> 01:21.340]  And then some prevention and detective capabilities, knowing that this is always going to be a battle and there's always going to be a cat and mouse type of struggle.
[01:22.500 --> 01:25.980]  So a little bit of background, and I apologize for those of you that know some of these things.
[01:25.980 --> 01:28.360]  We'll get through the foundational pieces quickly.
[01:28.600 --> 01:33.300]  So pretexting is just quite simply a character you might play when you're performing social engineering.
[01:33.680 --> 01:41.680]  This might be as simple as a Nigerian prince that has a bunch of money that you need to give somebody and you mail them a letter or send them an email.
[01:41.860 --> 01:49.620]  Or it could be as sophisticated as actually doing research and figuring out the manager's name and assuming a manager and trying to get to the assistant manager.
[01:49.620 --> 01:50.720]  And we'll get into some of that.
[01:50.940 --> 01:52.740]  So really masquerading as somebody else.
[01:53.100 --> 01:54.820]  Next, we talk about social proof.
[01:55.020 --> 02:02.740]  So actually demonstrating that you have more value than someone else in a social setting as a form of trying to garner interest or generate clout.
[02:03.040 --> 02:06.100]  So really, people try and do things that they see other people doing.
[02:06.280 --> 02:09.520]  And so if you can do it first, you might be somebody leading the pack.
[02:09.660 --> 02:11.680]  And we try and trick targets into actually doing that.
[02:12.700 --> 02:17.240]  Authority is obvious, but typically people obey authority figures.
[02:17.240 --> 02:21.760]  They don't necessarily want to follow instructions from somebody that doesn't have authority.
[02:21.760 --> 02:26.440]  And if they think that you have authority, then often you'll be able to convince them to do something.
[02:26.840 --> 02:30.420]  A reciprocity, people often return a favor.
[02:30.640 --> 02:38.420]  So if you can provide somebody a favor, like let's say opening a door, typically they'll open that next door for you, like a man trap.
[02:38.420 --> 02:44.400]  So you can actually give somebody something, like a piece of information, at a con, at a bar.
[02:44.680 --> 02:47.400]  Oh, you know, this is the key card access system that I use.
[02:47.500 --> 02:48.360]  It's a pain in the ass.
[02:49.360 --> 02:50.540]  What do you use?
[02:50.620 --> 02:51.600]  Or you don't even have to ask that.
[02:51.720 --> 02:52.660]  Sometimes they'll offer that.
[02:52.800 --> 02:55.160]  So that reciprocity can work in that way with information.
[02:56.060 --> 02:59.540]  Urgency, this is pretty obvious, but in phishing attacks you see this all the time.
[02:59.840 --> 03:01.060]  Act now or else.
[03:01.220 --> 03:02.940]  Click this link before your FedEx package.
[03:03.120 --> 03:04.440]  Go to somewhere else, right?
[03:04.500 --> 03:05.140]  That type of thing.
[03:05.140 --> 03:13.860]  So urgency is a very important piece of most of these puzzles to trick a victim into disclosing sensitive data or short circuiting that critical thinking process.
[03:14.660 --> 03:15.440]  And liking.
[03:15.640 --> 03:17.920]  Most of the time, you know, you think about sales.
[03:18.100 --> 03:22.160]  People buy things from people they like or from people they know.
[03:22.340 --> 03:29.580]  So if you can generate a likable character in your pretext, oftentimes you'll get somebody to cooperate with you much more easily.
[03:30.660 --> 03:32.420]  And baiting and quid pro quo.
[03:32.620 --> 03:37.700]  So we talked about reciprocity earlier where you actually give somebody something and then you get something in return.
[03:38.060 --> 03:41.220]  Baiting is really the promise of giving them something.
[03:41.240 --> 03:47.600]  So you can entice them with some potential outcome by hinting at that outcome without actually giving them anything.
[03:49.740 --> 03:52.460]  All right, so let's talk a little bit about modes of thinking.
[03:52.800 --> 03:58.580]  So if you're familiar with some core psychology concepts, modes of thinking might not be new to you.
[03:58.680 --> 04:03.460]  But to go over it, everybody is typically a primary thinker in one of these modes.
[04:03.520 --> 04:07.940]  So an auditory thinker listens mostly to what they are thinking about.
[04:07.940 --> 04:13.840]  Those people might be on a phone call and close their eyes oftentimes to try and think through what's going on in the conversation.
[04:14.400 --> 04:19.080]  A visual thinker is obviously somebody that uses visual cues to understand something.
[04:19.240 --> 04:22.720]  People that really like charts and graphs to understand things.
[04:22.900 --> 04:24.400]  And a kinesthetic thinker.
[04:24.540 --> 04:26.060]  Somebody that has their hands on something.
[04:26.260 --> 04:29.680]  Think an engineer or a mechanic or a doctor.
[04:29.680 --> 04:37.700]  So an auditory thinker, typically we find these individuals are in sales or in roles that are on the phone all the time.
[04:38.280 --> 04:44.400]  Not because they got good at what they did on the phone and it made them better at an auditory thinker.
[04:44.460 --> 04:46.540]  They already were good at thinking that way.
[04:46.560 --> 04:50.260]  So they conduct most of their business in an auditory way.
[04:50.480 --> 04:55.460]  And visual thinkers, in my experience, most executives that I've come in contact with are visual thinkers.
[04:55.460 --> 04:58.320]  You might go through the most in-depth explanation.
[04:58.480 --> 05:04.140]  But if you don't have something visual to represent that data, they typically fall asleep or gloss over.
[05:04.500 --> 05:10.020]  So what you can do is, if you're trying to convey a message like, do you understand?
[05:10.320 --> 05:13.020]  You can say, does that sound good to you?
[05:13.160 --> 05:18.260]  Or how does that grab you from a kinesthetic thinker standpoint?
[05:18.260 --> 05:31.920]  And so if you can identify a mode of thinking and you have that same message but you use terminology that aligns to the visual or the auditory or the kinesthetic part of how they might think, you have your message land much more effectively and much, much quicker.
[05:32.800 --> 05:41.340]  And so it's important later, we'll get into why, but if you can establish somebody's primary mode, you can really blow the door off of an engagement.
[05:43.080 --> 05:46.880]  All right, so let's get in a little bit of physical engagement examples.
[05:47.320 --> 05:50.960]  So in this example, we had a financial services company.
[05:51.520 --> 06:00.140]  We went through a series of open-source intelligence or OSINT exercises to understand the footprint of the building, the camera locations.
[06:01.060 --> 06:07.040]  One of the pretext elements was we couldn't use the CIO's name in this example.
[06:07.180 --> 06:09.860]  And the goal was to try and get in the CIO's office.
[06:10.680 --> 06:24.420]  So when we showed up on site, we used a sense of urgency and developed a pretext that we were late for a meeting and waited until about 10 minutes before, I think it was 8 o'clock in the morning, and very passionately were yelling into the phone, but
[06:24.420 --> 06:33.680]  only when somebody was walking up to the door saying, I'm going to be late for this meeting, I need you to come down and let me in, I left my badge on the table, on the desk, whatnot.
[06:33.680 --> 06:42.700]  And so if you time it right, you end up leveraging somebody's emotional well-being, frankly, their ability to help, and they'll open the door for you.
[06:42.740 --> 06:43.440]  And that's what happened.
[06:43.660 --> 06:53.440]  And so in this example, typically what happens when you go into an organization is you're nervous, your adrenaline's up.
[06:54.080 --> 06:58.820]  You might find the success is exciting, but you also might find it gives you a way.
[06:59.020 --> 07:00.600]  You start to get a little more nervous.
[07:00.620 --> 07:04.580]  You wouldn't necessarily stumble over your words, but you do because adrenaline's a hell of a drug.
[07:04.580 --> 07:07.440]  So when you break into a bank, what's the first thing you might do?
[07:07.640 --> 07:09.300]  You all probably do the same thing.
[07:09.500 --> 07:10.600]  You get a free cup of coffee.
[07:10.960 --> 07:11.980]  Calm yourself down.
[07:12.180 --> 07:13.480]  You go to the break room.
[07:13.620 --> 07:14.060]  You relax.
[07:14.320 --> 07:19.100]  You figure out where the cameras might be, and you start to develop rapport to whoever might be in that room.
[07:19.360 --> 07:22.520]  So in this example, we're in the break room.
[07:22.620 --> 07:25.760]  I'm getting a cup of coffee, and I see someone there, an older woman.
[07:26.420 --> 07:27.300]  Let's call her Betty.
[07:27.700 --> 07:28.940]  And so I introduce myself.
[07:28.940 --> 07:31.160]  I start to generate that likable character.
[07:31.160 --> 07:33.700]  And I say I'm Sean.
[07:34.220 --> 07:35.620]  I use his name Steve.
[07:35.900 --> 07:38.180]  And I say I'm working for the CEO this week.
[07:38.340 --> 07:42.240]  I couldn't name her on the CIO, but I did some research, and I used the CEO's name.
[07:42.500 --> 07:45.160]  I'm here for two weeks, and I'm doing a consultative engagement.
[07:45.520 --> 07:47.100]  You know, I'll be here doing X, Y, Z.
[07:47.360 --> 07:48.440]  I make up something.
[07:48.600 --> 07:50.840]  And so we talk for maybe seven, ten minutes.
[07:51.680 --> 07:53.440]  So I start to walk through the environment.
[07:54.940 --> 07:58.400]  And as I'm walking through the environment, I realize everybody has a visible badge.
[07:58.480 --> 07:59.980]  It's like not negotiable.
[07:59.980 --> 08:02.500]  So this is common in a banking environment.
[08:03.020 --> 08:07.420]  I get to this huge open space where there's a cubicle farm and there's very low walls.
[08:07.720 --> 08:10.720]  So as I'm walking through, I see everybody's eyes on me.
[08:10.860 --> 08:14.400]  And this manager stands up, beelines right towards me.
[08:14.680 --> 08:17.040]  And she's got this look on her face like, who are you?
[08:17.160 --> 08:17.600]  What are you doing?
[08:17.620 --> 08:18.960]  She starts to ask me some questions.
[08:19.180 --> 08:23.180]  As I'm thinking about what to say, Betty stands up and says, oh, that's Steve.
[08:23.300 --> 08:24.240]  He's working for the CEO.
[08:24.360 --> 08:25.260]  He'll be here for two weeks.
[08:25.260 --> 08:26.640]  She said, okay, no problem.
[08:26.820 --> 08:27.660]  So she let me go.
[08:27.820 --> 08:31.260]  So immediately, social proof was generated, validation.
[08:31.560 --> 08:35.240]  I leveraged the authority, the extended authority of working for the CEO.
[08:35.820 --> 08:37.560]  And now all of a sudden, I'm one of them.
[08:37.700 --> 08:42.640]  And not only that, Betty looks like a superhero because she knows this secret project that nobody knows about.
[08:42.640 --> 08:47.880]  And they, as I was separating, I said, oh, and by the way, where's John's office, the CEO?
[08:47.880 --> 08:50.060]  Because I figured the CIO's office is next to him.
[08:50.340 --> 08:52.500]  Oh, it's on the third floor, fourth door from the right.
[08:52.620 --> 08:55.900]  So immediately get more information and conclude the engagement.
[08:56.640 --> 09:01.660]  So again, that's something that she was able to leverage to generate that social proof.
[09:01.980 --> 09:04.340]  And it just goes pretty far.
[09:05.600 --> 09:12.880]  So the next one, and this one, it's actually important that you have a clipboard because nothing generates authority like a clipboard.
[09:13.660 --> 09:18.980]  So when you're walking through an environment, having a clipboard, a suit, depending on the pretext, is very important.
[09:18.980 --> 09:28.460]  And this example during OSINT, we found that they had posted some Twitter, some photos on Twitter of Christmastime, but it included the workstation.
[09:28.580 --> 09:32.240]  So you could tell from the workstation images that they were older.
[09:32.500 --> 09:33.880]  They weren't necessarily up to date.
[09:34.700 --> 09:39.740]  So also in the engagement, this credit union had been held up at gunpoint.
[09:39.740 --> 09:41.540]  So we couldn't use any threatening pretext.
[09:41.680 --> 09:45.420]  We couldn't use anything that would cause fear or strike a panic.
[09:45.640 --> 09:47.500]  So that was, you know, it's always do no harm.
[09:47.500 --> 09:51.500]  So that wasn't a problem, but it was very important to understand that that was an issue.
[09:51.660 --> 09:57.660]  So we went through OSINT and discovered that we called the different locations.
[09:57.660 --> 09:59.880]  We got the manager's name from public sources.
[10:00.200 --> 10:06.340]  We asked if the manager was available during lunch hours to be able to calculate when exactly the lunch hour was.
[10:06.460 --> 10:07.360]  We called different branches.
[10:07.800 --> 10:11.900]  Turns out all of the lunch hours were the same for all the managers in all of these locations.
[10:11.900 --> 10:18.800]  And so what we did when we showed up was we asked for the manager's name on purpose during the lunch hour.
[10:18.960 --> 10:24.720]  Because the assistant manager is not equipped to deal with deviations of protocol like a manager is.
[10:24.900 --> 10:30.580]  So the assistant manager is kind of new and she's walking through her questioning.
[10:30.580 --> 10:31.580]  I'm there.
[10:32.520 --> 10:33.680]  Let's call it Steve again.
[10:33.900 --> 10:34.460]  I'm here to...
[10:34.460 --> 10:40.740]  I'm doing a latency exercise from corporate to the branches to verify speed and network and things like that.
[10:40.980 --> 10:41.860]  She said, oh, okay.
[10:41.960 --> 10:43.520]  Let me just call the security office.
[10:44.060 --> 10:45.040]  What do you need to do?
[10:45.240 --> 10:46.980]  I just need to take a look at the computers.
[10:47.220 --> 10:49.480]  That includes the one behind the teller desk.
[10:49.560 --> 10:50.640]  It'll just take a few minutes.
[10:50.860 --> 10:51.760]  I'll be out of your hair.
[10:51.760 --> 10:53.840]  And I also need to hit these five other branches.
[10:53.840 --> 10:56.000]  And I got to be back home by four.
[10:56.160 --> 10:57.820]  So it'll just take a few minutes.
[10:57.940 --> 11:03.700]  So trying to generate that urgency piece, she persisted to call the security officer or tried to.
[11:04.120 --> 11:07.540]  So I gently reach out, touch her arm, ask her her name.
[11:07.700 --> 11:08.380]  Let's call her Mary.
[11:08.960 --> 11:12.120]  Mary, you know, are your computers slow?
[11:12.380 --> 11:14.080]  Do you have this, you know, OptiPlex?
[11:14.200 --> 11:15.640]  I just named the model that she had.
[11:15.860 --> 11:17.760]  She said, yeah, they're God awful.
[11:17.760 --> 11:22.300]  Now, if you've ever been in a financial organization, they're always slow, every time.
[11:22.680 --> 11:24.220]  So she said, yeah, they're God awful.
[11:24.300 --> 11:28.320]  I said, you give me 30 seconds and I'll get out of your hair and you'll have new computers next month.
[11:28.400 --> 11:29.220]  She said, come on back.
[11:29.480 --> 11:31.500]  So now I'm at the teller.
[11:31.840 --> 11:33.340]  I'm surrounded by cash.
[11:33.540 --> 11:36.360]  They have a live teller next to me that's servicing customers.
[11:36.900 --> 11:51.280]  I'm doing some, you know, voodoo with listing directories and random stuff that looks like I'm checking things while also plugging in a ducky and getting every PDF, doc, spreadsheet, you know, text file that they have on that machine in roughly 30 seconds.
[11:51.700 --> 12:04.840]  So in that example, you know, the pretext was very important, understanding the manager's names, the timing, name dropping the manager demonstrated authority, obviously the clipboard, the urgency to try and get out of there and just get out of her
[12:04.840 --> 12:07.660]  hair with some quid pro quo with getting new machines next month.
[12:07.660 --> 12:14.860]  So in that example, you know, obviously it was a combined type of a situation that produced that outcome.
[12:15.060 --> 12:18.780]  But it's also about leveraging two out of the three modes of thinking.
[12:19.280 --> 12:25.100]  A kinesthetic thinker, if you touch their arm, they're more likely to understand what you're saying or get your message.
[12:25.400 --> 12:35.940]  Now, if she was a visual thinker, the one that I didn't touch on in that example because I used her name and repeated it for the auditory thinker, that would have been a bad thing.
[12:36.040 --> 12:40.780]  Touching somebody's arm when they're not into it puts them off, and now all of a sudden you're the enemy, right?
[12:40.960 --> 12:43.020]  So it was a gamble, but it ended up working out.
[12:43.920 --> 12:44.420]  All right.
[12:44.640 --> 12:53.240]  So let me turn it over to my colleague here to talk a little bit about generative AI tooling now that we've gone through some of the principles of social engineering and how they work on physical engagements.
[12:54.740 --> 12:55.320]  Thank you.
[12:56.060 --> 13:02.000]  So let's talk a little bit about generative AI tooling as it pertains to voice cloning.
[13:02.000 --> 13:11.900]  So in 2016, we have Google releasing their WaveNet model, which was their first model for generating human-sounding speech.
[13:13.720 --> 13:15.800]  In 2017, we have Lyrebird.
[13:16.220 --> 13:24.540]  They introduced a model that could take one minute of audio, source audio material, and you could essentially clone that voice.
[13:26.380 --> 13:37.920]  In 2018, we have China's Badu releasing Deep Voice 3, which allowed for just using 3.7 seconds of source material to clone voices.
[13:39.500 --> 13:51.660]  In 2019, we have Google's yummy Tacotron 2, which took advantage of NVIDIA's GPU acceleration to produce clone speech.
[13:53.460 --> 13:56.560]  And in 2020, we have Descript's Overdub.
[13:56.760 --> 14:09.540]  This was interesting because you could feed Overdub a source file of audio, and it would allow you to edit just like you were editing text in a text editor.
[14:09.540 --> 14:14.140]  So you could, you know, change mispronunciations or remove things or add things.
[14:14.500 --> 14:18.720]  And it would retain the original voice used.
[14:20.980 --> 14:28.860]  In 2021, we have Resemble AI, just another one coming on the scene, allowing for voice cloning.
[14:29.620 --> 14:40.300]  And in 2022, Volley, Microsoft's Volley, allowed for using three seconds of source audio to generate voice clones.
[14:41.860 --> 14:44.840]  And last but not least, we have Eleven Labs.
[14:46.060 --> 14:49.940]  They provided emotional nuance to their clones.
[14:49.940 --> 14:51.900]  It was much more realistic sounding.
[14:52.620 --> 14:58.360]  And we ended up using Eleven Labs, but I'll touch on that a little bit later.
[15:00.640 --> 15:05.040]  So what are some use cases today for voice cloning?
[15:05.320 --> 15:12.400]  I know that YouTube creators have admitted to churning out material faster, especially voiceovers.
[15:13.380 --> 15:16.600]  Cloning their own voice just to create content quicker.
[15:17.380 --> 15:32.340]  In education, for audio books, I know that it's being used to take preferred voices or to kind of retain voices used through series of books and things of that nature.
[15:32.600 --> 15:40.760]  In healthcare, if you think of like patients with dementia, they may have voices that they're familiar with for like reminders, things of that nature.
[15:42.540 --> 15:45.600]  In customer service, it's being used in IBR systems.
[15:46.460 --> 15:50.380]  And in gaming, I know that they're using it for both narration and character voices.
[15:51.340 --> 15:54.240]  And in security, I know it's being used for fraud detection.
[15:55.300 --> 15:59.340]  So let's cover current open-source options for voice cloning.
[15:59.720 --> 16:02.680]  These are not the... this isn't the end-all, be-all list.
[16:04.060 --> 16:05.840]  These are just the three that we tried.
[16:05.840 --> 16:11.680]  So we have Tortoise TTS, Koki TTS, and Paddle Speech.
[16:14.260 --> 16:17.780]  So what are some of the issues with open-source solutions?
[16:19.660 --> 16:24.140]  Well, sometimes you need very beefy hardware, GPUs.
[16:24.300 --> 16:25.740]  It's kind of expensive.
[16:27.340 --> 16:34.760]  In comparison to the commercial solutions, I found that the output is kind of robotic or monotone.
[16:35.820 --> 16:41.340]  And it's oftentimes tricky to get the environment configured to even begin doing this.
[16:41.740 --> 16:48.420]  Think of, like, PIP, Python PIP, and, you know, dependency hell, we'll call it.
[16:50.740 --> 16:52.700]  So, current generative AI tooling.
[16:53.080 --> 16:58.600]  These aren't the... you know, these aren't... this isn't the definitive list, but these are the top three that we evaluated.
[16:58.600 --> 17:02.620]  You have 11 labs, which cost $5 a month.
[17:03.400 --> 17:07.080]  And you have Speechify and Resemble, which both offer free trials.
[17:09.980 --> 17:11.800]  So, it's not all great.
[17:12.000 --> 17:16.740]  What are some of the... the issues that we have with the commercial solutions?
[17:17.160 --> 17:18.440]  Well, it costs money.
[17:18.800 --> 17:24.460]  You're transmitting biometric data to someone's server who knows what they're doing with it.
[17:25.220 --> 17:30.680]  And it's not as cool as cloning your own voice on your own hardware on a server that's under your control.
[17:32.780 --> 17:37.780]  So, we ultimately went with a commercial solution, 11 labs.
[17:38.180 --> 17:43.000]  And this is because we simply didn't have the hardware required to produce the voice clones quickly.
[17:44.020 --> 17:46.720]  We found it to be easier to accomplish our goal.
[17:48.340 --> 17:54.580]  And we found the output from the open-source solutions just to not... not be up to snuff.
[17:54.680 --> 17:56.040]  It wasn't good enough to fool anyone.
[17:58.380 --> 18:01.940]  So, I'll go ahead and pass it back to my colleague for some case study.
[18:03.080 --> 18:03.720]  All right.
[18:03.920 --> 18:08.060]  So, let's get into this core content for the case study leveraging generative AI.
[18:10.600 --> 18:13.180]  So, keep in mind, this is a true story.
[18:14.740 --> 18:16.460]  And this was a real engagement.
[18:16.960 --> 18:19.070]  So, this was a sales-based organization.
[18:20.840 --> 18:24.220]  Let's call it an insurance industry organization.
[18:24.640 --> 18:26.880]  And we obtained permission to use the story.
[18:27.120 --> 18:30.160]  And some of the evidence results in conclusions.
[18:30.900 --> 18:32.240]  But not all of the material.
[18:32.440 --> 18:35.880]  And we'll walk through how we made that simulation for you today in a demo.
[18:37.640 --> 18:41.200]  So, the goal was to really extract real-world action from the targets.
[18:41.620 --> 18:44.000]  We had a large user pool that was handed to us.
[18:44.060 --> 18:47.220]  So, we didn't have to do a lot of OSINT in that regard.
[18:47.660 --> 18:50.340]  But we did do some on what the environment was like.
[18:50.500 --> 18:51.760]  What the organization was like.
[18:51.900 --> 18:53.280]  What their habits were like publicly.
[18:54.740 --> 18:56.000]  We were fully remote.
[18:56.260 --> 18:58.760]  And we had very limited time to develop the pretext.
[18:59.340 --> 19:02.860]  Which is another reason, to his point, why we went with a commercial solution.
[19:03.440 --> 19:05.820]  One of the pieces of the abstract you may have noticed.
[19:05.820 --> 19:10.480]  Is when you're generating a red team engagement for a client as a consultant.
[19:10.860 --> 19:13.980]  You often don't have the luxury of doing what an APT might do.
[19:14.160 --> 19:16.020]  Or what other security researchers might do.
[19:16.200 --> 19:17.960]  If they had unlimited time and resources.
[19:18.320 --> 19:22.800]  Which is develop years of pretext and background and OSINT and things like that.
[19:23.160 --> 19:24.300]  So, we had to do it very quickly.
[19:24.540 --> 19:24.900]  And that's...
[19:24.900 --> 19:27.900]  Right now, commercial is the way to go, it seems.
[19:28.800 --> 19:32.440]  We also had cooperation from marketing IT and the CEO.
[19:32.680 --> 19:34.140]  Typically, this is not the case.
[19:34.380 --> 19:36.080]  But the reason we had marketing involved.
[19:36.080 --> 19:39.400]  Is because in order for the onsite piece of the attack to work.
[19:39.620 --> 19:41.940]  We had to have some heads up to them.
[19:42.120 --> 19:45.620]  So that there was some, you know, pieces of the puzzle that they worked on with us.
[19:45.980 --> 19:47.100]  And then IT was aware.
[19:47.300 --> 19:51.200]  So that when people potentially notified them of what was happening.
[19:51.560 --> 19:55.320]  They didn't have a false alarm and a false start from an incident response standpoint.
[19:55.320 --> 19:58.540]  And then the CEO was involved for obvious reasons.
[19:58.540 --> 19:59.900]  And we'll get into in one second.
[20:00.640 --> 20:04.440]  So, as I mentioned, we did have that recon piece.
[20:04.460 --> 20:07.060]  So we were able to get plausible pretext materials.
[20:07.320 --> 20:12.080]  And generate what we think was a very, very compelling lure.
[20:12.840 --> 20:17.800]  So here you'll see what was the fishing lure.
[20:17.960 --> 20:23.320]  But we anonymized some of the pieces based on requests from the client.
[20:23.320 --> 20:27.320]  So we had an onsite contest at the client outing.
[20:28.200 --> 20:29.360]  It was a company outing.
[20:29.760 --> 20:33.140]  And what we did was we took a traditional fishing attack.
[20:33.940 --> 20:40.800]  And we added a video that was simply a photo with a link behind it that went to a video.
[20:41.060 --> 20:43.100]  We toyed with the idea of a voicemail.
[20:44.280 --> 20:47.000]  But nobody really gets a voicemail over email.
[20:47.260 --> 20:49.300]  That's a big red flag typically, right?
[20:49.300 --> 20:55.760]  So, and if you linked a video, it was parsed and sometimes garbled depending on how they looked at it.
[20:55.780 --> 20:56.920]  If it was on their phone or whatnot.
[20:57.160 --> 21:08.580]  So we found the most consistent on every user agent and every device type was an image that looked like a video that then served a link that then actually served a hosted video.
[21:08.580 --> 21:14.300]  And so what we have here is actually a few different pieces of the puzzle.
[21:14.480 --> 21:16.640]  We have the company logo.
[21:16.640 --> 21:19.980]  So in this example, it's a fictitious company called Pineapple.
[21:20.720 --> 21:22.760]  And we used their company logo.
[21:22.760 --> 21:25.500]  We used the CEO's image.
[21:25.500 --> 21:27.320]  So it was a photo of the CEO.
[21:27.720 --> 21:32.440]  It was coming from a spoofed email address that was impersonating the CEO.
[21:33.000 --> 21:35.380]  These are all common fishing techniques.
[21:35.940 --> 21:37.400]  And the external banner was there.
[21:37.520 --> 21:40.000]  So we baked in some red flags that they could see.
[21:40.000 --> 21:41.360]  So it wasn't fish in a barrel.
[21:41.480 --> 21:47.520]  So that when we inevitably did see success, people couldn't complain and say it was too unrealistic or whatnot.
[21:49.000 --> 21:54.440]  And for today's demo, like I said, we obtained permission to be able to use pieces of this.
[21:54.440 --> 21:58.120]  And we used the exact script that we used, minus the name.
[21:58.980 --> 22:03.240]  And we'll have Zale go through the actual demo here.
[22:03.500 --> 22:07.480]  And if the demo gods play nice, we'll hear what this sounds like.
[22:09.620 --> 22:11.000]  Hey, sorry to bother you.
[22:16.790 --> 22:19.090]  Hey, sorry to bother you.
[22:21.470 --> 22:22.450]  This is Steve.
[22:22.750 --> 22:23.390]  Yeah, let me see.
[22:24.710 --> 22:25.570]  Sorry, guys.
[22:26.230 --> 22:27.530]  See what I said about demo gods?
[22:27.530 --> 22:28.170]  I jinxed it.
[22:28.870 --> 22:29.310]  Alright.
[22:32.550 --> 22:33.850]  Hey, sorry to bother you.
[22:34.090 --> 22:34.670]  This is Steve.
[22:34.910 --> 22:39.010]  I wanted to give you a quick call before meeting up in person this year at Sales Kickoff.
[22:39.330 --> 22:42.830]  We are going to be doing a contest with a select group of our top sellers.
[22:42.950 --> 22:44.570]  And I wanted you to be a part of it.
[22:44.970 --> 22:46.530]  I can't share a lot of detail.
[22:46.810 --> 22:49.690]  But what I can say is that it will be fun.
[22:50.950 --> 23:01.790]  What we are asking is for this group to each make a paper airplane, write your name on one of the wings, and drop it at the Pineapple event registration desk.
[23:02.250 --> 23:05.790]  This will be a contest for only a few select sellers.
[23:06.030 --> 23:10.910]  So the sales leadership would really appreciate you keeping this close to the vest for now.
[23:11.490 --> 23:13.730]  Talk soon, and see you in Florida.
[23:14.930 --> 23:15.370]  Alright.
[23:16.290 --> 23:16.730]  Alright.
[23:16.810 --> 23:19.470]  Alright, so let's look at what we just heard, or maybe didn't hear.
[23:20.750 --> 23:26.090]  Okay, so, some of those principles we talked about are baked into this message.
[23:26.350 --> 23:30.870]  So, for those of you that didn't hear it, this is what was just played.
[23:30.870 --> 23:36.330]  So, what we see in red, sorry to bother you, that demonstrates liking immediately.
[23:36.990 --> 23:42.590]  If this was really the CEO, why would they care if they're bothering one of the employees?
[23:42.770 --> 23:43.450]  One example.
[23:44.890 --> 23:48.730]  This is the CEO demonstrating authority in orange.
[23:49.370 --> 23:52.310]  I want to give you a quick call demonstrating urgency.
[23:54.110 --> 23:55.570]  Social proof in pink.
[23:55.570 --> 24:00.790]  So, this was a big one because the individuals targeted were in sales in the departments.
[24:01.210 --> 24:03.030]  It's a contest for select group.
[24:03.370 --> 24:05.510]  There's exclusivity built into this.
[24:05.630 --> 24:09.290]  It's inferring that they're the top sellers that are getting this contest.
[24:09.670 --> 24:12.230]  It's only a few people, and we want to keep it close to the vest.
[24:12.350 --> 24:13.590]  All the pink text there.
[24:13.950 --> 24:16.330]  Then you see yellow with reciprocity.
[24:16.490 --> 24:19.250]  I want you to be a part of it, and in exchange, it'll be fun.
[24:20.410 --> 24:23.110]  And blue, baiting and quid pro quo.
[24:23.110 --> 24:29.030]  You see, I want you to make a paper airplane, write your name on one of the wings, and drop it at the registration booth.
[24:29.590 --> 24:35.070]  So, when we went through this, and also noticed that there are all three components to this.
[24:35.190 --> 24:37.390]  There's the video, the visual thinker component.
[24:37.650 --> 24:39.050]  There's the auditory message.
[24:39.790 --> 24:43.610]  And there's also the kinesthetic part of building the actual paper airplane, right?
[24:44.970 --> 24:46.200]  So, what we noticed...
[24:48.230 --> 24:51.270]  Well, first, the red flags are latent.
[24:51.270 --> 24:53.370]  There's nothing but red flags in this.
[24:53.630 --> 24:59.550]  So, irrespective of the Gen AI voice cloning piece, as a message...
[24:59.550 --> 25:04.530]  And your brain's short circuit when there's another vehicle or medium that the message is sent from.
[25:04.630 --> 25:09.310]  But if this was a text-based email, we wouldn't have the outcome that you're about to see.
[25:09.670 --> 25:10.890]  These are all red flags.
[25:11.050 --> 25:11.950]  Sorry to bother you.
[25:12.130 --> 25:12.650]  Quick call.
[25:12.650 --> 25:19.450]  You know, these are elements that you would see in an email and immediately delete it or forward it to IT or whatever your protocol is, right?
[25:19.910 --> 25:24.230]  And so, in this example, we built in enough red flags.
[25:24.310 --> 25:27.070]  The other big one is, I wanted to give you a quick call.
[25:27.230 --> 25:28.090]  Well, it's not a call.
[25:28.570 --> 25:30.370]  It's a video in an email.
[25:30.770 --> 25:33.070]  So, it's automatically not a call, right?
[25:33.070 --> 25:34.930]  So, we tried to build in some obvious things.
[25:36.230 --> 25:38.930]  So, here's one hour after we sent the message.
[25:39.990 --> 25:42.250]  Here's four hours after we sent the message.
[25:43.710 --> 25:55.590]  My favorite part is that somebody actually built one out of the vomit bag on the airplane because they were so excited that the CEO emailed them directly that they couldn't wait to land and make their paper airplane.
[25:55.810 --> 25:57.070]  So, this person fashioned it.
[25:57.210 --> 26:01.710]  They messaged somebody that they worked with saying, I haven't made a paper airplane in 20 years.
[26:02.130 --> 26:08.570]  Tell me how to make one or what's the best type that'll go the farthest and made it out of this vomit bag in a state of panic.
[26:08.690 --> 26:10.890]  Now, clearly, she was a kinesthetic thinker.
[26:12.970 --> 26:13.470]  All right.
[26:13.610 --> 26:14.590]  So, let's look at some stats.
[26:14.890 --> 26:18.170]  So, roughly 10% of the company received the email.
[26:18.370 --> 26:19.130]  200 people.
[26:19.770 --> 26:21.890]  Out of that, 60 people opened it.
[26:22.550 --> 26:25.610]  85 people viewed the video or clicked the URL.
[26:25.950 --> 26:28.350]  And 43 people were unique viewers.
[26:28.350 --> 26:32.250]  Does that mean people watched it twice or three times or they forwarded it to people?
[26:32.390 --> 26:33.370]  This was malware.
[26:33.510 --> 26:34.910]  This is circulating the environment.
[26:35.970 --> 26:41.310]  And so, total paper airplanes were roughly 11% of the people that we sent actually made a paper airplane.
[26:41.430 --> 26:49.630]  So, that means they had to open the email, click the link, watch the video, short circuit, make the paper airplane, walk to the event registration booth.
[26:49.770 --> 26:53.250]  So, six actions that we got out of each individual.
[26:53.250 --> 26:55.910]  Now, you might say 22 people, that's not that many.
[26:56.110 --> 27:04.570]  But if 11% of people in the environment failed a phishing exercise, or even worse, opened malware, this is a pretty big deal.
[27:04.790 --> 27:06.690]  And so, let's bring it into the real world.
[27:06.870 --> 27:09.550]  This, to me, is very similar to a gift card attack.
[27:09.550 --> 27:16.590]  A CEO is saying, hey, I need you to go to Walgreens and get a gift card and read off the serial number off the back and email it to me.
[27:16.990 --> 27:26.510]  So, let's assume that the average BEC for gift card attacks is $2,500, which it is, and 30% of those attacks are over five grand.
[27:26.510 --> 27:31.930]  So, you do that math, and for this many people, it would be roughly $71,000 in loss.
[27:32.430 --> 27:45.050]  Now, if we targeted 10% of the users in Fortune 600, or the 600 Fortune 2000 companies, the total loss of this exercise would have been $44 million, or our gain in that example.
[27:45.050 --> 28:01.530]  And you take it a step further, and if we targeted the 20,000 companies that are between 500 people and 2,000 people, which means that they're big enough to be a target and small enough to probably have a little bit to clean up from a social security
[28:01.530 --> 28:09.570]  awareness standpoint, we're looking at $1.4 billion in this exercise, just based on extrapolating that out and targeting all of those same companies.
[28:12.030 --> 28:12.790]  All right.
[28:13.370 --> 28:16.510]  So, Zael, you want to talk a little bit about some detection and prevention capability?
[28:16.890 --> 28:17.370]  Sure.
[28:17.510 --> 28:17.710]  Thanks.
[28:19.450 --> 28:28.190]  So, one method of detecting and prevention would be asymmetric crypto or cryptographically signing your media.
[28:29.090 --> 28:33.330]  Some pros of this is, you know, it's reliable and historically sound.
[28:34.070 --> 28:45.970]  However, it requires infrastructure that is trusted and decentralized, and also how do you get Susie in accounting to know that she needs to validate media using cryptography?
[28:47.590 --> 28:52.210]  So, we'll move on to the next one here, which is the rotating keyword scheme.
[28:53.430 --> 28:57.650]  So, if this is properly conducted, it can be done entirely offline.
[28:58.830 --> 29:10.470]  The problem is, especially in a large organization, is how do you maintain a list of keywords that you rotate through that have to be used in media, and how do you keep that list synchronized?
[29:12.730 --> 29:15.950]  And then, one of the others would be using AI to detect AI.
[29:16.650 --> 29:23.510]  And at the time of creating these slides, the only AI method of doing this was using Google Synth ID.
[29:24.770 --> 29:31.030]  With this, you can get instant verification, but it's unreliable currently.
[29:31.030 --> 29:40.370]  The tool's still in its infancy, and there's something of an arms race between attackers and defenders using this method currently.
[29:40.990 --> 29:46.030]  Yeah, and as a professor, I have students submitting papers that are forged all the time.
[29:46.210 --> 29:50.930]  So, GPT zero is great, but it's not accurate all the time.
[29:50.930 --> 30:00.670]  So, you have tools like that that can find sentences that are fabricated, but they're not really super reliable yet in that same way.
[30:00.670 --> 30:09.470]  Another potential method to expand on the keyword scheme would be sharding a past phrase using Shamir's secret sharing.
[30:10.370 --> 30:14.050]  The problem with that is the key has to be created on an endpoint.
[30:14.250 --> 30:17.430]  And it also has to be decrypted or put back together on an endpoint.
[30:17.610 --> 30:29.350]  So, if that endpoint's compromised, the generation of the key before it's split, and then the combining of the key to decrypt are both single points of failure and potentially susceptible to malware or compromise.
[30:30.870 --> 30:32.810]  And so, you know, what else can we do?
[30:32.950 --> 30:34.830]  I think education becomes the catch-all.
[30:35.230 --> 30:39.670]  As a social engineer, I hate sitting in talks where people say, well, you need to train the users more.
[30:39.790 --> 30:41.510]  Well, we need to train the users more.
[30:42.290 --> 30:49.310]  So, I think just conditioning with different things like rewards and a scoreboard, a la Wall of Sheep, but in a positive way.
[30:49.550 --> 30:57.490]  So, getting people, you know, $5 Starbucks cards because they reported a phishing email doesn't cost the business anything, but that'll stick in their head.
[30:57.590 --> 30:59.810]  And now, they're not just looking to protect the business.
[30:59.930 --> 31:01.310]  They're looking to get a latte, you know?
[31:01.470 --> 31:03.390]  So, that's one simple example.
[31:04.470 --> 31:06.570]  Another is shifting right in the kill chain.
[31:06.750 --> 31:20.850]  So, like, we're talking about generative AI phishing lures a bit here, but if somebody were to write that email or if somebody were to come up to you in a mall and say, you just want an iPad, fill out this piece of paper, please, you wouldn't do it.
[31:20.850 --> 31:24.310]  But for some reason, you short circuit when there's different technologies involved.
[31:24.490 --> 31:28.430]  So, if we shift right a little bit, those red flags are real regardless.
[31:28.690 --> 31:31.930]  Let's move past the delivery of the message and listen to the message.
[31:32.110 --> 31:33.730]  Is making a paper airplane ridiculous?
[31:34.090 --> 31:35.770]  Like, that's probably where I would start.
[31:36.970 --> 31:41.110]  And then, cat and mouse, you mentioned, Zale mentioned, you know, the cat and mouse game.
[31:41.150 --> 31:42.570]  There's always going to be somebody smarter.
[31:42.810 --> 31:45.250]  There's somebody smarter in this room than we are.
[31:45.730 --> 31:47.130]  There's somebody smarter out there.
[31:47.130 --> 31:51.110]  There's always going to be more people creating things every single day somebody's born.
[31:51.290 --> 31:54.130]  Every single day somebody finds out about generative AI.
[31:54.530 --> 31:58.390]  So, there are things like that that we can always bank on, that there'll be somebody more creative.
[31:58.390 --> 32:00.630]  So, we just have to continue on the problem together.
[32:00.770 --> 32:05.470]  And that means joining forces and creatively exploring these potential solutions.
[32:06.070 --> 32:11.850]  And happy to work through some of those, just pontificating over beer at the Emerald or wherever.
[32:11.850 --> 32:13.210]  This is a dry campus.
[32:13.430 --> 32:14.610]  We've got to go somewhere, I think.
[32:15.530 --> 32:19.170]  So, with that, I think we'll open it up for any questions.
[32:20.770 --> 32:21.150]  Yes.
[32:36.740 --> 32:40.520]  And I do sometimes get invitations to parties where I click on the reading card.
[32:40.920 --> 32:42.800]  And it takes me somewhere and I get the card.
[32:43.020 --> 32:48.460]  But in this particular email that I got this week, when I clicked on the card, it took me to the card site.
[32:48.700 --> 32:53.500]  And then it redirected to another site and said, to show you the card, just authenticate.
[32:54.220 --> 32:59.300]  Are you going to authenticate your Outlook ID, your Facebook ID, your Twitter ID, your Snapchat ID?
[32:59.640 --> 33:00.620]  Oh my God.
[33:00.920 --> 33:01.120]  Wow.
[33:01.640 --> 33:02.080]  Yeah.
[33:02.300 --> 33:04.700]  That's an aggressive party right there.
[33:04.700 --> 33:07.700]  So, the question was, at what point did they mess up?
[33:07.840 --> 33:13.620]  And it seems like lawyers are getting more sophisticated with authentication built into even party invites, right?
[33:13.840 --> 33:17.140]  Did I mess up by even clicking on it at first?
[33:17.220 --> 33:17.960]  Because I shouldn't...
[33:18.560 --> 33:21.920]  I have a friend in marketing and he would say, that's exactly what you should have done.
[33:22.160 --> 33:24.720]  I mean, some of these tools are not thinking about security.
[33:25.040 --> 33:31.320]  Some of these, you know, marketing tools, frankly, invisible pixels and things like that, to me, are a security threat and a privacy risk.
[33:31.500 --> 33:34.300]  But they're a marketing, you know, godsend, right?
[33:34.300 --> 33:37.020]  So, depending on the tool, it could have been an issue.
[33:37.640 --> 33:39.900]  I just can't tell without knowing more.
[33:40.000 --> 33:44.400]  But the employees are okay to click on the video at first.
[33:44.880 --> 33:45.960]  That was acceptable.
[33:46.240 --> 33:50.580]  I mean, in my opinion, the external banner should have prohibited that, right?
[33:50.580 --> 33:56.220]  So, that was probably the biggest red flag, is the CEO from within the company is sending you a request.
[33:56.580 --> 33:58.160]  The banner is before the message.
[33:58.160 --> 33:59.280]  It's before the video.
[33:59.940 --> 34:01.900]  Maybe they didn't notice the spoofed email.
[34:02.080 --> 34:03.280]  We just use his first name.
[34:03.440 --> 34:08.440]  You know, there's a couple other red flags we had blacked out because of the privacy and redaction.
[34:08.440 --> 34:12.040]  But, yeah, I would say the external banner is probably the first one they should have seen.
[34:13.180 --> 34:13.700]  Yes?
[34:13.900 --> 34:15.520]  I came a little bit late here.
[34:15.680 --> 34:18.680]  Did you guys clone the voice of the CEO in that message?
[34:18.900 --> 34:19.780]  Yes, we did.
[34:20.040 --> 34:20.560]  Obviously.
[34:20.560 --> 34:25.300]  And one thing we didn't mention is we use completely public available audio.
[34:25.500 --> 34:25.680]  Yes.
[34:25.680 --> 34:33.400]  So, we use their earnings announcements calls that are every quarter that every single executive is on for most companies that are public.
[34:33.400 --> 34:37.200]  The CFO's on it, CEO's on it, sometimes other members, right?
[34:37.480 --> 34:38.780]  All the shareholders on it?
[34:39.200 --> 34:39.620]  Sometimes.
[34:39.720 --> 34:41.060]  Any shareholder on earth.
[34:41.060 --> 34:41.100]  Investors.
[34:41.360 --> 34:41.580]  Right?
[34:41.740 --> 34:42.920]  One share of that stock.
[34:43.160 --> 34:43.500]  Exactly.
[34:43.900 --> 34:45.980]  And then you want to take it a step further.
[34:46.100 --> 34:51.260]  You target the C-suite by cloning the investor names and the individuals that they care about.
[34:51.300 --> 34:51.800]  There you go.
[34:51.940 --> 34:54.100]  Because they're going to be on the calls asking questions.
[34:54.220 --> 34:58.900]  And we're talking hours and hours and hours and hours of source material that you can train the model with.
[34:59.580 --> 35:08.040]  The reason why I had a situation, I was helping somebody, an elderly person was targeted thinking it's their son on the phone.
[35:08.560 --> 35:10.440]  And he was about to send them money.
[35:10.720 --> 35:12.760]  But we caught it before that.
[35:12.980 --> 35:14.740]  So, this stuff is happening a lot actually.
[35:14.900 --> 35:19.130]  Just from voicemails, they're cloning the voicemail and then making believe that...
[35:19.460 --> 35:20.340]  That's diabolical.
[35:20.360 --> 35:21.480]  Yeah, it's very scary, yeah.
[35:23.980 --> 35:24.940]  Any other questions?
[35:25.280 --> 35:25.420]  Yeah.
[35:26.220 --> 35:32.300]  So, in the beginning you said that you didn't want to make the email too believable.
[35:32.800 --> 35:35.180]  So, you added in the external banner and everything else.
[35:35.340 --> 35:39.920]  Was that at the request of the target company?
[35:40.520 --> 35:45.860]  Or was that something that you were like, oh, let's kind of give them a chance to do the right thing?
[35:46.100 --> 35:51.020]  Yeah, and the question was, did we make the email have red flags in it at the request of the client?
[35:51.020 --> 35:53.500]  Or because we wanted to build it in?
[35:53.640 --> 36:03.560]  I think in my experience, if you don't have something that can be fixed at the end of the punchline, quote unquote, what are we doing here?
[36:03.720 --> 36:05.240]  Outside of tormenting people, right?
[36:05.980 --> 36:09.860]  So, we try and build in something that makes it a real learning opportunity.
[36:09.860 --> 36:25.640]  And if there isn't something to spot, then the victim is not only disgruntled and upset that they were made fun of or used as an example, but then they might actually clam up the next time and not participate and not actually, you know, not just
[36:25.640 --> 36:29.720]  learn from it, but then not be a defensive capability for the company in the front line.
[36:29.880 --> 36:30.980]  So, that's a great question.
[36:31.780 --> 36:32.220]  Yes?
[36:32.360 --> 36:45.160]  Yeah, to that end, I've heard horror stories of insufficiently bad, quote unquote, phishing emails, promising employees extra bonuses or like straight up, this is how you're going to get your annual raise.
[36:45.160 --> 36:48.980]  And then they shame the employee and it's like, no, that was too real.
[36:49.240 --> 36:50.920]  And you're putting somebody's check on the line.
[36:51.080 --> 36:52.980]  That's both, like, that's just scummy behavior.
[36:53.160 --> 36:58.720]  Yeah, to me, that's somebody that has a huge ego that wants to win at tricking the users to prove a point.
[36:58.860 --> 37:04.640]  Now, to be fair, we've had some clients, CISOs, directors of security, what have you, that can't get a budget whatsoever.
[37:05.240 --> 37:08.400]  And, you know, I think we might even have some people in the audience like that.
[37:09.000 --> 37:14.240]  That's just the nature of the industry, you know, the security teams left on the side, you know, that type of thing.
[37:14.240 --> 37:15.960]  It's a part of IT's budget sometimes.
[37:16.320 --> 37:21.220]  And so in those examples, maybe using it once to prove we are susceptible.
[37:21.640 --> 37:22.760]  How can we fix this?
[37:22.840 --> 37:23.680]  Well, we need to budget.
[37:24.040 --> 37:25.960]  But again, that's playing devil's advocate a bit.
[37:25.960 --> 37:28.000]  I still wouldn't agree with that approach.
[37:28.000 --> 37:28.700]  That's a good point.
[37:30.260 --> 37:31.360]  All right, great questions.
[37:31.480 --> 37:32.300]  Anybody else?
[37:34.440 --> 37:35.520]  All right, thank you.
[37:35.520 --> 37:36.000]  Thank you.
[37:36.000 --> 37:36.120]  Thank you.
[37:36.120 --> 37:38.120]  Thank you.