[00:00.000 --> 00:02.640]  I'm doing my presentation on CV Dazzle.
[00:05.100 --> 00:07.300]  You can see the website there for more information.
[00:07.540 --> 00:09.720]  And I'll mention that again towards the end of the talk.
[00:12.500 --> 00:12.980]  So...
[00:30.200 --> 00:34.780]  You do this already once, right?
[00:34.920 --> 00:35.160]  No.
[00:35.160 --> 00:37.200]  Okay, so, yeah, that's gonna be key.
[00:38.620 --> 00:40.160]  I can't see them.
[00:40.580 --> 00:42.600]  How do you deal with everything so small?
[00:49.900 --> 00:50.680]  Here we go.
[00:50.780 --> 00:51.320]  That should do it.
[00:51.840 --> 00:52.360]  We looking good?
[00:53.160 --> 00:53.680]  Yep.
[00:54.100 --> 00:54.280]  Alright.
[00:56.060 --> 00:56.900]  Mirror, okay.
[00:57.600 --> 00:58.540]  Alright, let's try it again.
[01:05.180 --> 01:11.060]  So, CV Dazzle is face deception or is camouflage from face detection.
[01:11.660 --> 01:14.880]  I'm a recent graduate from ITP at New York University.
[01:16.120 --> 01:18.260]  Before that, I worked as a Flash developer.
[01:19.680 --> 01:20.620]  Made some money.
[01:21.120 --> 01:22.620]  Worked as an interactive designer.
[01:23.180 --> 01:24.700]  Made a little bit less money.
[01:24.800 --> 01:25.720]  Worked as a photographer.
[01:26.600 --> 01:27.780]  Made less money still.
[01:28.840 --> 01:30.640]  Shot nightlife and then some products.
[01:32.480 --> 01:35.600]  I'm on Facebook if you want to reach me at Harvey Wallbanger.
[01:36.020 --> 01:37.660]  That's also the name of a drink.
[01:38.020 --> 01:38.760]  If you didn't know that.
[01:40.120 --> 01:41.860]  And Twitter is Adam Harvey.
[01:42.560 --> 01:44.000]  And that's my email.
[01:44.280 --> 01:46.760]  I also have a small company called AH Projects.
[01:48.720 --> 01:53.860]  And it's called AH Projects because it's me and I do my art projects from there.
[01:54.500 --> 01:55.700]  So, I'm here today.
[01:56.460 --> 01:57.800]  That's me also.
[01:58.620 --> 02:04.740]  I'm here today to share what I did for the last semester during my master's thesis at NYU.
[02:05.760 --> 02:08.100]  And the project is CV Dazzle.
[02:09.160 --> 02:10.980]  CV is for computer vision.
[02:11.700 --> 02:18.120]  Computer vision is just the ability to see within an image or a video using a computer.
[02:18.120 --> 02:22.860]  And with that, you can do all sorts of fun stuff like look at their expressions.
[02:22.860 --> 02:25.780]  And you can even determine their age with some new software.
[02:27.420 --> 02:32.940]  And the Dazzle part of it comes from this kind of camouflage that was used during World War I.
[02:33.660 --> 02:41.520]  Dazzle camouflage is kind of a non-obvious, improbable solution to protecting these warships from attacks from submarines.
[02:43.220 --> 02:55.940]  If you maybe squint your eyes, you can kind of see that what's happening is the gestalt of the ship is being broken up and it's hard to determine if the ship is going to the left or the right or sometimes the size of it.
[02:56.820 --> 03:01.820]  And so the idea with CV Dazzle, the same kind of thing.
[03:01.960 --> 03:03.320]  It's to break up the gestalt.
[03:03.320 --> 03:11.260]  Both the original Dazzle and CV Dazzle is not going to completely conceal you like traditional camouflage might.
[03:12.220 --> 03:14.040]  Because that would be kind of improbable.
[03:14.320 --> 03:21.440]  You figure a computer is going to at least be able to detect you using, if not a regular imaging sensor, thermal.
[03:21.900 --> 03:24.000]  Or it could do a motion detection on you.
[03:24.000 --> 03:32.040]  So completely concealing yourself is a little hard, but what you can do is this CV Dazzle.
[03:32.240 --> 03:40.340]  And the idea here is that you are breaking up the gestalt of the face and you're using an analog technique.
[03:40.340 --> 03:41.520]  So that's hair and makeup.
[03:43.960 --> 03:50.220]  Another example of it here, someone else interpreted it for a magazine called Disc Magazine.
[03:52.260 --> 03:57.700]  And I'll go over those later and what is working in them and why they're CV Dazzle.
[03:58.060 --> 04:01.300]  This is one of the targets of CV Dazzle.
[04:01.400 --> 04:15.220]  Face.com, which some of you might have seen in the news lately, it's an automated face recognition system and it can detect your face or your friend's face face and it's powerful and it's a little creepy.
[04:16.080 --> 04:19.420]  And Facebook knows that and I think Google knows that too.
[04:24.000 --> 04:30.980]  Because both of them have all this great technology, but it's a little difficult to roll it out because people are kind of creeped out about it.
[04:32.000 --> 04:41.080]  The outline for what I'm going to talk about today, first I'd like to start with a little history of photography and talk about deception and why it might be good for you.
[04:41.820 --> 04:46.200]  Face detection and recognition, how they work, what the difference is between those two things.
[04:46.600 --> 04:52.680]  And then how CV Dazzle works and how it can be applied with some of the results and the next steps.
[04:54.200 --> 05:01.900]  Some of the other things that I do in school and outside of school is a project like this, which is a tele-operated arm.
[05:02.440 --> 05:08.140]  And this was sent to Copenhagen during the COP15 climate change conference last winter.
[05:08.600 --> 05:12.600]  And some people here in Union Square were able to hit a device.
[05:12.840 --> 05:16.640]  And that arm was actually sent to Copenhagen and connected in real time.
[05:18.700 --> 05:23.780]  Here are all the people in Union Square hitting a pad and then it would activate the arm over there.
[05:23.780 --> 05:26.680]  So that's an experiment in tele-activism.
[05:27.560 --> 05:33.040]  And later this year I'll be working on a larger version of that in the COP16 in Mexico City.
[05:33.420 --> 05:38.440]  And another project that I'm working on is the Anti-Paparazzi Clutch.
[05:38.440 --> 05:44.020]  This is a device that protects your identity from paparazzi photographers.
[05:44.500 --> 05:47.460]  And it's done using LEDs, high brightness LEDs.
[05:47.940 --> 05:52.620]  And it emits a blinding pulse of light and you can do that as a slave flash.
[05:52.840 --> 05:55.480]  Or more like a flashlight and protect against video.
[05:59.100 --> 06:00.380]  Now it's all visible light.
[06:00.660 --> 06:06.160]  IR is an interesting solution but a lot of cameras actually have filters and they cut that light out.
[06:06.700 --> 06:11.460]  And this is a preview of what it will look like, hopefully towards the end of development.
[06:15.090 --> 06:15.870]  So the history.
[06:16.030 --> 06:18.670]  I find the history interesting because I used to study photography.
[06:19.050 --> 06:23.190]  And I worked as a photographer and I worked as a photojournalist a little bit before that.
[06:24.730 --> 06:28.010]  Cameras, ever since they've been developed, they're basically capture devices.
[06:29.690 --> 06:32.850]  Whether it's chemical, using a plate, a really long process.
[06:33.370 --> 06:35.870]  Or if it's a little hidden camera in your pen.
[06:35.870 --> 06:41.150]  They're all doing the same action for you, which is documenting something or capturing something.
[06:43.450 --> 06:52.110]  But ever since cameras were introduced into society back in about 1830, there's been a kind of complex relationship to cameras.
[06:52.510 --> 07:00.230]  And as Susan Sontag points out in her book On Photography, the relationship has been a little predatorial.
[07:00.370 --> 07:04.750]  And the person with the camera is the one that has more power over the person who doesn't have the camera.
[07:04.750 --> 07:09.830]  And what she says is to photograph people is to violate them by seeing them as they never see themselves.
[07:10.130 --> 07:12.250]  Or by having knowledge of them they can never have.
[07:13.890 --> 07:15.770]  In India, it turns them into objects.
[07:15.910 --> 07:16.710]  It commodifies them.
[07:17.250 --> 07:19.570]  There's a recent story in the New York Times about this.
[07:19.850 --> 07:26.410]  And the decorated Mardi Gras Indians where photographers would come in, snap up a lot of photos, and not give anything back.
[07:26.410 --> 07:31.450]  The reaction from the subjects was that they felt robbed by the photographers.
[07:33.910 --> 07:37.010]  There's a lot of different views people have about privacy.
[07:37.210 --> 07:40.050]  Some people think it's a luxury.
[07:40.330 --> 07:41.730]  Some people think that it's a necessity.
[07:41.730 --> 07:45.810]  And that differs between borders of countries too.
[07:46.530 --> 07:48.150]  Of course, the U.S.
[07:48.410 --> 07:51.410]  maybe doesn't value it as much as other countries.
[07:51.570 --> 07:56.050]  Germany has been in the news recently for Google using the Wi-Fi networks.
[07:56.270 --> 08:03.110]  And they definitely have a stronger view on privacy or stronger reactions to losing it.
[08:03.110 --> 08:11.810]  And the big difference is if someone takes your photo here on the street, they can use that for almost anything they want except commercial purposes.
[08:13.370 --> 08:17.290]  But in these countries, the rules are different, and you can't do that unless you're a public figure.
[08:17.510 --> 08:20.430]  Which I think is actually a huge difference between the cultures.
[08:21.310 --> 08:26.970]  So some countries like to protect the citizens, and some countries like to protect the police.
[08:26.970 --> 08:34.770]  If you're at Josh's talk, he talked about how it's becoming illegal now to take photos of police on the street, which is...
[08:35.810 --> 08:38.570]  Well, it's happening in Russia too, I heard.
[08:40.710 --> 08:42.710]  There's a law there that just needs crap out here.
[08:43.110 --> 08:45.650]  Yeah, maybe that's a sign of things to come.
[08:47.510 --> 08:51.590]  So, you know, what happened to privacy since cameras came out?
[08:51.950 --> 08:55.170]  In the 1830s, right, not really initially because nobody has a camera.
[08:55.170 --> 08:56.990]  Images can't move very fast.
[08:57.790 --> 09:06.350]  You skip ahead a little bit to 1883, and there's this guy, Francis Galton, who began experimenting with cameras and extracting information out of them.
[09:06.590 --> 09:18.870]  And his experiment was to overlay images on top of each other, using a composite process, and then look at the similarity between somebody who had TB and somebody who didn't, and try to predict whether you're going to have TB.
[09:19.350 --> 09:23.050]  Which is a dubious experiment, but I think his intentions were okay.
[09:25.570 --> 09:31.230]  1930s was a big time also because that's when Leica introduced 35mm, and that was a game changer.
[09:31.650 --> 09:33.330]  That meant photography was now portable.
[09:34.410 --> 09:37.610]  35mm film was not nearly as expensive as the other kind.
[09:37.810 --> 09:46.230]  So you could take it anywhere, and this led to street-style photography, observational photography, or in-your-face photography.
[09:47.750 --> 09:51.890]  This is a famous image from Henri Cartier-Bresson, a French photographer.
[09:52.510 --> 10:04.290]  And what is really nice about this image is it captures not only the decisive moment, as he called it, but the lightness of portable cameras, and the ability to snap up that moment.
[10:06.730 --> 10:12.850]  1960s comes along, and the term paparazzi is introduced by Federico Fellini, from the film La Dolce Vita.
[10:13.290 --> 10:16.470]  I think most people are familiar with all this stuff.
[10:16.690 --> 10:23.310]  And this really shows and illustrates that predatorial effect that Susan Sontag was talking about.
[10:25.350 --> 10:28.190]  Even the look of some cameras is kind of predatorial.
[10:28.190 --> 10:35.590]  And there's a feeling you get when you walk in front of things like this, where you don't know what's happening to all that footage.
[10:35.610 --> 10:42.830]  You don't know who's looking at it, how long it's being stored, or if your face is being captured, if it's being linked to other things, too.
[10:45.090 --> 10:56.810]  There's an effect, or a study in psychology by other guys named Zions in the 1960s.
[10:56.810 --> 11:03.710]  And what he found out was that there actually is an effect of being watched and observed by people.
[11:04.030 --> 11:11.330]  And the effect is that you're less likely to engage in new behavior, or take a risk doing something new that you're not comfortable with.
[11:12.690 --> 11:22.630]  However, if you're doing something that is already learned and you've mastered, then being watched at a stadium, for example, athletes, then you tend to perform better.
[11:22.630 --> 11:34.010]  The problem is that, you know, you're out on the street in public, and you're not always doing things that you've mastered, so you end up becoming subject to this decrease in your performance.
[11:34.250 --> 11:39.730]  And that, I think, is maybe a measurable result of surveillance, or living in a surveillance society.
[11:41.970 --> 11:46.850]  Talking about surveillance societies, London is probably the most surveilled city in the world.
[11:47.750 --> 11:54.570]  They have about 500,000 security cameras inside what they call their steel ring, or ring of steel.
[11:57.790 --> 12:03.230]  The number... they have about 5 million cameras, or 4 million cameras in the UK also.
[12:03.830 --> 12:14.730]  The number of households in the U.S. that own at least one digital camera is about 78%, or 234 million people that would equal.
[12:15.130 --> 12:22.310]  And on Facebook, they boast about 2.5 billion photos are uploaded every single month.
[12:22.610 --> 12:30.610]  So that's a lot of photos, and Facebook may have some privacy restrictions on that to protect them.
[12:30.690 --> 12:37.590]  But Flickr is a little bit more public, and they have, as of last October, about 4 billion photos on their site.
[12:38.390 --> 12:45.090]  And what's great about all these photos is that anyone who's doing research on computer vision can use all this information and develop new software.
[12:45.330 --> 12:56.430]  So it's a really rich resource for them, and it's good in some ways, but maybe, you know, it's leading to kinds of software that are used in what people are saying are creepy ways.
[12:57.190 --> 13:01.830]  So now I want to talk a little bit about deception, or at least mention its value.
[13:04.070 --> 13:14.450]  Roy Behrens, who wrote a book on camouflage called Camopedia, he said that deception has always been critical for daily survival, for humans and non-humans alike.
[13:14.710 --> 13:17.690]  And judging from its current ubiquity, there's no end in immediate sight.
[13:17.870 --> 13:22.510]  Which I think is interesting because you don't always think about deception as being a positive thing.
[13:23.370 --> 13:40.690]  And then I listened to a podcast the other day from WNYC, their local radio station, and what they said was that people who lie are smarter, their brain is more active in some ways, and that people who tell the truth all the time expressed more depression.
[13:42.230 --> 13:42.690]  So...
[13:42.690 --> 13:43.450]  Woohoo!
[13:43.650 --> 13:43.910]  We're good!
[13:46.210 --> 13:47.610]  Yeah, that was a good one.
[13:47.790 --> 13:48.450]  Good to know about.
[13:50.430 --> 13:53.290]  Another form of deception is illusions.
[13:53.730 --> 13:56.110]  And this is a friend's Halloween costume from last year.
[13:56.490 --> 13:58.890]  And that's actually painted on her face.
[13:59.390 --> 14:02.290]  And it just blew my mind how real it looked.
[14:02.590 --> 14:10.250]  And I thought that was interesting that it deceived me, but I wondered what it would do to computer vision or machines, and how they would interpret that.
[14:10.250 --> 14:18.650]  And then I remember looking on a site, somebody ported OpenCV computer vision framework from C to Java.
[14:18.990 --> 14:23.230]  For anyone who uses processing, you can get an OpenCV library for that.
[14:23.470 --> 14:25.470]  And it showed that it doesn't really have to be a face.
[14:25.650 --> 14:28.710]  So in this way, this guy was deceiving the machine here.
[14:30.410 --> 14:34.650]  So those were like the two points of inspiration for this project.
[14:34.970 --> 14:40.030]  To kind of fool machines, but not create an illusion for humans.
[14:40.630 --> 14:45.450]  So as I mentioned, OpenCV, briefly, it's an open source computer vision library.
[14:45.750 --> 14:50.310]  I'm not sure how familiar people in this crowd are doing the computer vision kind of stuff.
[14:50.550 --> 14:52.150]  It's got about 2 million downloads.
[14:52.430 --> 14:53.910]  It's not just for face detection.
[14:54.050 --> 14:58.690]  A lot of people use it on robots or in unmanned vehicles.
[15:00.050 --> 15:02.630]  Navigation, camera calibration, all sorts of stuff.
[15:03.190 --> 15:06.850]  But it's really popular because it's free and it's pretty robust.
[15:07.290 --> 15:09.730]  So you see it a lot on mobile applications.
[15:09.990 --> 15:11.730]  A lot of people put it on iPhone applications.
[15:13.290 --> 15:16.250]  And it's available in a ton of different programming languages.
[15:16.950 --> 15:27.810]  I don't know how many people know this guy, but if you're a face detection algorithm, all you'd be able to say is that that's a face, not male, not female, or anything else.
[15:27.810 --> 15:33.490]  And if you had a face recognition application, of course you'd know that that's the tra-la-la-la guy.
[15:34.990 --> 15:37.230]  So there's a large difference between those two.
[15:37.450 --> 15:40.090]  Face detection is just isolating a face from the background.
[15:40.490 --> 15:44.690]  And face detection is a form of object detection.
[15:44.910 --> 15:51.330]  And you can do object detection on anything from cars, money, hammer, any kind of object.
[15:51.330 --> 15:57.130]  So face detection is available in all sorts of different algorithms.
[15:57.130 --> 16:00.830]  And people have been working on this problem for maybe 20 years now.
[16:01.270 --> 16:03.910]  And so the technology has really come around and is mature.
[16:04.450 --> 16:10.630]  Mature enough to give really good results and be integrated into all sorts of different software for commercial purposes.
[16:11.410 --> 16:14.430]  Sorry, this example right here is of a hard cascade.
[16:14.830 --> 16:19.750]  This is part of a... it's like a profile that is fed into the object detector.
[16:20.010 --> 16:21.830]  So you can feed in a profile for anything.
[16:22.730 --> 16:27.110]  This is just an example of what the small components of that profile look like.
[16:27.110 --> 16:33.410]  And if you were to visualize those on top of the face, you can see that they cover all these different regions.
[16:33.850 --> 16:39.810]  And what it's... what they mean is that in the algorithm, it's looking at the white region and the dark region.
[16:40.070 --> 16:43.370]  And it subtracts the values from the light region...
[16:43.890 --> 16:46.870]  of the dark region from the light region, if that makes sense.
[16:47.230 --> 16:50.230]  And then it looks to see if that value is within a certain threshold.
[16:51.530 --> 16:59.710]  So if you were to look at someone else's face or this one, you can see the eye region should be darker than the region right below it, which is the cheek, the upper cheekbones.
[17:00.230 --> 17:05.790]  And these are the kind of correlations in the pixels that the algorithm is looking for.
[17:06.390 --> 17:10.350]  This is a sample of 20 or so.
[17:10.670 --> 17:13.910]  The profiles have about several thousand of these.
[17:14.370 --> 17:16.830]  So it's kind of impossible to address each one.
[17:17.070 --> 17:22.350]  You can't go in there and design a solution for one and then another one and go through.
[17:23.450 --> 17:25.390]  Because in fact, a lot of them overlap.
[17:25.670 --> 17:26.970]  And they all have different values.
[17:27.870 --> 17:32.810]  So I put together an example but misspelled the footer.
[17:33.150 --> 17:34.690]  This is how OpenCV works.
[17:34.770 --> 17:38.510]  And what it's doing is scanning this sub-window across the image.
[17:38.730 --> 17:44.310]  And it's looking for all those patterns, for all the differences between the dark and the light regions.
[17:44.310 --> 17:53.250]  And what you can notice is that it moves faster over the regions where it's plain and where there's not a lot of contrast.
[17:54.110 --> 17:54.670]  Thank you.
[17:55.850 --> 17:58.990]  And as it gets closer to the face, you'll notice that it slows down a lot.
[18:02.410 --> 18:09.750]  It may be a little hard to see but there's text on the bottom of the rectangle and it says stage... it's going through a series of stages.
[18:10.650 --> 18:17.030]  And the profile, which has all these little definitions for the rectangles, has about 22 stages.
[18:17.430 --> 18:20.670]  If it makes it through all those stages, then it'll classify it as a face.
[18:21.050 --> 18:23.990]  And if it doesn't, then it just moves on and keeps scanning the image.
[18:24.990 --> 18:31.670]  So you could think, well, maybe you could block this by blocking the first rectangle and doing something really drastic.
[18:32.010 --> 18:39.210]  But the first rectangle is defined so loosely that it's almost anything qualifies.
[18:39.410 --> 18:44.130]  And you can see most of these go up to at least 7 or 8 in the stage number.
[18:44.450 --> 18:47.050]  And right around the face, you can watch, it goes up to 22.
[18:53.100 --> 18:56.500]  And then you see that the rectangles are deposited over the face.
[18:56.500 --> 19:02.600]  And in order for it to confirm a face, what it needs to do is have enough of those rectangles overlap each other.
[19:02.840 --> 19:05.840]  So if you only get one, it classifies it as a false positive.
[19:06.180 --> 19:08.300]  But if you get... the default is three.
[19:08.380 --> 19:11.000]  If you get three overlapping rectangles, then that qualifies as a face.
[19:12.240 --> 19:13.260]  It just takes a while.
[19:14.240 --> 19:25.980]  So going about trying to find patterns that would work for this, I downloaded all these images from fashion sites, from parties, the boombox scene in London, looking at how people are using makeup.
[19:25.980 --> 19:29.980]  to decorate themselves and as a disguise for themselves also.
[19:30.260 --> 19:34.840]  But wondering what the potential is for makeup patterns that are already out there.
[19:35.060 --> 19:39.500]  Including a lot of body decoration in the Pacific Islands and masks.
[19:40.060 --> 19:42.920]  And you can see, which is kind of crazy, that there are a lot of...
[19:43.460 --> 19:44.740]  I can go back to that.
[19:45.120 --> 19:47.480]  Even these things are classified as faces.
[19:48.600 --> 19:54.240]  And when I saw this, then I thought I was in a lot of trouble for my thesis and things weren't looking good.
[19:54.940 --> 19:57.860]  I didn't... it's going to end up looking like...
[19:58.620 --> 20:00.100]  Not sure which one is next here.
[20:01.680 --> 20:09.240]  But there were actually... some of these were promising and I used those as an inspiration for creating my patterns.
[20:09.240 --> 20:12.860]  So face detection, face recognition are different.
[20:13.660 --> 20:19.860]  Automated face recognition, like that program I mentioned in the beginning, face.com uses automated face recognition.
[20:20.240 --> 20:24.340]  The first step in any automated system is face detection or tracking.
[20:24.940 --> 20:26.960]  After that, it tries to align the face.
[20:27.200 --> 20:29.560]  And after that, it does feature extraction.
[20:30.300 --> 20:31.940]  And then it classifies it.
[20:32.200 --> 20:33.640]  And then it'll give an identification.
[20:34.400 --> 20:39.880]  And the goal here is not to target face recognition, which is much more complex.
[20:40.720 --> 20:42.900]  But just take away face detection.
[20:43.220 --> 20:49.300]  And when you do that, you're cutting the head off the snake and it can't... it doesn't have anything to extract features from.
[20:49.980 --> 20:53.940]  So that's one of the underlying principles of how CV Dazzle works.
[20:54.760 --> 21:00.100]  It doesn't explicitly target face recognition, but it does that by targeting face detection.
[21:02.860 --> 21:10.800]  I guess the best way to summarize what it's doing is changing the contrast and the spatial relationships between key facial features.
[21:11.040 --> 21:15.960]  And what I mean by key facial features are things like your nose, eyes, things that everybody has on their face.
[21:16.140 --> 21:18.300]  And those all have shadows and they're defined.
[21:18.620 --> 21:23.040]  And the ratio, the sizes of those are fairly similar.
[21:23.300 --> 21:27.360]  So you can get a really tight pixel correlation between these objects on the face.
[21:28.440 --> 21:29.540]  Just to show you that again.
[21:31.880 --> 21:35.440]  You can see that a lot of them are looking for things right around the eye region.
[21:36.240 --> 21:38.420]  Not very many are on the outside of the face.
[21:38.880 --> 21:40.780]  Most of them are kind of in this region right here.
[21:42.140 --> 21:47.780]  So one way to go about this would be to take that cascade and then look through all the features on there.
[21:48.020 --> 21:49.980]  And where it wants it to be light, turn it dark.
[21:50.080 --> 21:51.240]  Where it wants it to be dark, turn it light.
[21:51.440 --> 21:55.720]  And keep adjusting those thresholds until you get something that doesn't qualify in the face anymore.
[21:56.820 --> 21:59.020]  There may be more work to be done on that.
[21:59.280 --> 22:02.880]  But it ends up looking like a very rough pattern.
[22:04.260 --> 22:07.900]  Like some kind of regenerative artwork from ten years ago.
[22:09.100 --> 22:10.880]  So that didn't really work very well.
[22:11.420 --> 22:16.500]  And another thing to point out is that if it doesn't look good then people aren't going to wear it and it's already failed.
[22:16.500 --> 22:22.780]  An example of that is this camouflage pattern which was used in the Persian Gulf War.
[22:23.420 --> 22:26.980]  And this was actually an effective camouflage functionally.
[22:27.080 --> 22:27.780]  It worked very well.
[22:27.960 --> 22:30.720]  But it was called chocolate chip camouflage.
[22:30.960 --> 22:34.920]  And the soldiers who wore it thought that it didn't feel military enough.
[22:35.160 --> 22:37.440]  So the camouflage pattern was dropped.
[22:37.760 --> 22:42.260]  And then it was later reissued to Iraqi soldiers in 2003.
[22:43.620 --> 22:45.400]  There's a guy that I mentioned before.
[22:45.620 --> 22:48.900]  He talked about the effects of surveillance.
[22:49.840 --> 22:52.560]  Unmastered tasks and decreasing your likelihood to try new things.
[22:53.620 --> 22:58.040]  This was an image that Joshua showed the other day.
[22:58.760 --> 23:02.520]  And I thought this is an excellent example of how to fail.
[23:06.160 --> 23:07.040]  Awesome stuff.
[23:08.420 --> 23:11.200]  So then going about creating these patterns.
[23:11.700 --> 23:19.720]  Since we're not doing the generative thing now or the reverse generation of those patterns the way I went about it was looking at all these key facial features.
[23:20.020 --> 23:21.300]  Testing each one independently.
[23:21.920 --> 23:22.860]  And then looking at the results.
[23:23.320 --> 23:30.100]  And going through these, classifying them in order to determine which regions I should look at to alter with the makeup or hair.
[23:30.780 --> 23:35.220]  So you can see between these the ones with a lot of green rectangles are being...
[23:35.220 --> 23:36.940]  They have a really strong face signal.
[23:37.180 --> 23:42.900]  And the ones without a lot of rectangles on the bottom row you can see one of the white rectangle over the eye.
[23:43.380 --> 23:44.820]  So the eyes are supposed to be dark.
[23:44.940 --> 23:47.920]  If you have a white rectangle that's the inverse of what's happening.
[23:48.600 --> 23:49.380]  And that one didn't show up.
[23:49.580 --> 23:50.400]  So you can determine...
[23:51.020 --> 23:52.440]  You can gain from this one that...
[23:52.440 --> 23:54.780]  a white rectangle over the left eye might work.
[23:55.580 --> 23:58.640]  Then you can test things like this where you just cover...
[23:58.640 --> 24:01.060]  just to see what the hell is going on in here.
[24:01.460 --> 24:04.220]  Different parts of the face, black and white regions.
[24:05.480 --> 24:08.540]  Now, of course, the large ones didn't work.
[24:09.480 --> 24:11.980]  Covering the entire nose and face region...
[24:11.980 --> 24:15.420]  I mean, it looks weird, but just to show you a little bit of what's going on.
[24:15.640 --> 24:16.520]  That would block it.
[24:16.540 --> 24:18.100]  So if you had something that covered that much.
[24:18.600 --> 24:27.500]  The nose bridge area, which I found to be a sensitive area, is not affected if it's the only place on the face that's altered.
[24:28.500 --> 24:30.700]  And just trying a bunch more of these variations.
[24:30.700 --> 24:34.040]  These are all targeting the basic key features.
[24:34.520 --> 24:36.520]  And you can see here again...
[24:36.520 --> 24:38.380]  You know, what's interesting here...
[24:38.380 --> 24:40.000]  This is the one in the upper right.
[24:40.200 --> 24:42.460]  We have a black rectangle going over the face.
[24:43.100 --> 24:44.060]  I don't know if you've seen...
[24:44.060 --> 24:45.440]  There are sunglasses you can buy.
[24:46.080 --> 24:47.300]  They're called like...
[24:47.300 --> 24:49.100]  I don't know... photo blockers.
[24:49.360 --> 24:51.960]  And it's just a black rectangle that goes over your face.
[24:52.080 --> 24:54.380]  And the idea is to protect your identity.
[24:54.620 --> 24:55.940]  It's a pretty cool, cheap solution.
[24:56.760 --> 25:02.240]  But what you can see happening there is that if it's all black, then it still counts it as a face.
[25:02.520 --> 25:09.300]  So if you had on a pair of sunglasses left over from watching Avatar, and you wore those, they might not work.
[25:11.300 --> 25:16.460]  Basically, what's going on here is that if you take a face, and you take its inverse...
[25:16.460 --> 25:21.640]  The inverse is everything that would work for a pattern that blocks face detection.
[25:23.260 --> 25:24.960]  Everything that's dark is light.
[25:25.040 --> 25:26.080]  Everything that's light is dark.
[25:28.820 --> 25:29.700]  Using the...
[25:29.700 --> 25:32.280]  You know, what I learned from all those different examples.
[25:32.460 --> 25:36.020]  And there are a lot more than what was shown just in that brief example.
[25:36.720 --> 25:40.500]  I started to target these key features in the face.
[25:41.080 --> 25:42.880]  Just by doing it digitally first.
[25:43.340 --> 25:46.080]  I took the face and drew on it and tested each one sequentially.
[25:46.800 --> 25:51.220]  And you can see the number of green rectangles decrease as it goes across the row.
[25:51.220 --> 25:58.560]  All the way on the right side, the dotted rectangles, they represent detections, but they're false detections.
[25:58.920 --> 26:05.060]  Like I was mentioning earlier, it requires a few overlapping and over to confirm a true face detection.
[26:07.540 --> 26:14.120]  So using this pattern here, I worked with a hair stylist and makeup artist, and we created a look on that.
[26:14.120 --> 26:18.220]  This is just the model before to show all of the...
[26:18.220 --> 26:23.320]  Those are all of the face detections that are possible by running that through a face detection.
[26:26.540 --> 26:27.100]  And...
[26:27.700 --> 26:30.320]  Yeah, that's during, just applying, styling the hair.
[26:32.780 --> 26:33.900]  And after...
[26:33.900 --> 26:46.500]  So you probably think that it's impractical makeup to wear out in public, but this one is designed to wear to a party or something like that, not when you're robbing a bank.
[26:49.140 --> 26:50.420]  And some of the results.
[26:50.640 --> 26:52.320]  So this is the first one we did.
[26:52.480 --> 26:54.000]  We tried a lot of different hairstyles.
[26:54.920 --> 26:55.480]  Ah, shit.
[27:00.080 --> 27:02.600]  And you can see all the way on the bottom, it's got a little bit of makeup.
[27:02.800 --> 27:05.760]  The hair is coming down over that nose bridge area, but it's not working.
[27:06.120 --> 27:12.820]  Add the makeup, again on the top row, and it works increasingly better from the right side to the left side.
[27:12.820 --> 27:14.280]  And you can see that nose...
[27:14.280 --> 27:18.340]  The hair is really far down in the nose bridge area, because that's one of the key features.
[27:18.680 --> 27:22.120]  And as I was saying with the inverse thing, what you see here is the cheek.
[27:22.280 --> 27:24.540]  It's supposed to be light, so it's dark on the top.
[27:24.720 --> 27:30.140]  And it's also changing the symmetry by having the patterns reversed a little bit.
[27:30.560 --> 27:33.000]  So, you know, it's a lot of trial and error at this point.
[27:33.820 --> 27:44.680]  What is needed then after this is to take what's learned here and turn it into a program so you can do a lot of this automatically, take your face and generate a pattern, a custom pattern for your face.
[27:45.480 --> 27:52.440]  And then I took a series of these photos with multiple angles, multiple expressions, trying to be scientific.
[27:53.000 --> 27:57.820]  And then I uploaded them to the photo tagger, which is this application on Facebook.
[27:58.320 --> 28:02.400]  And I ran them all through there to see what faces would be detected.
[28:05.650 --> 28:09.030]  And it turned out that it wasn't able to find any of the faces using that.
[28:15.980 --> 28:16.460]  Cool.
[28:17.220 --> 28:24.580]  And the next test was Picasa, which whoever uses it probably knows they have face recognition in there.
[28:25.160 --> 28:26.320]  Picasa is made by Google.
[28:26.760 --> 28:28.240]  Google has face recognition.
[28:28.520 --> 28:30.400]  Apple has face recognition in iPhoto.
[28:31.780 --> 28:32.780]  Facebook has it.
[28:32.860 --> 28:34.040]  Yahoo has it in Flickr.
[28:34.760 --> 28:35.680]  Hewlett Packer has it.
[28:35.680 --> 28:39.860]  Almost all the top software companies have some version of face recognition.
[28:40.120 --> 28:42.700]  But the problem is that they're all slightly different.
[28:42.920 --> 28:48.320]  Everyone has what they think is their own advantage over the market and doing something better than the other ones.
[28:48.800 --> 29:02.080]  But what was interesting here, I think, is that as advanced as they are, in comparison to the OpenCV, an OpenCV or the Viola Jones detector in OpenCV was developed in 2003.
[29:02.420 --> 29:03.820]  And that was kind of...
[29:03.820 --> 29:07.620]  That was the turning point for face detection there.
[29:07.720 --> 29:11.980]  That made it really possible for anyone to do it in real time with low computational power.
[29:14.200 --> 29:17.560]  Another really robust face detection is PitPat.
[29:17.920 --> 29:20.300]  You can also go to their site and check out a demo.
[29:20.560 --> 29:22.420]  There are a lot of really cool demos.
[29:22.420 --> 29:24.340]  And I'll make sure to post these on my site.
[29:24.660 --> 29:25.960]  And you can download them.
[29:27.100 --> 29:29.900]  Like the one in the beginning where you can check your age and your expression.
[29:30.660 --> 29:31.440]  Fun stuff like that.
[29:31.760 --> 29:36.120]  So this is, again, PitPat, which is one of the top face detection softwares.
[29:36.660 --> 29:39.800]  Award-winning years in a row.
[29:40.000 --> 29:43.200]  It's a company that was spun out of Carnegie Mellon.
[29:44.040 --> 29:46.880]  And in my testing, it was the most robust one.
[29:47.120 --> 29:50.360]  And in here, you have the same example that was tested against OpenCV.
[29:51.020 --> 29:55.540]  And what you can see here, the green rectangles are strong confidence.
[29:55.780 --> 29:57.380]  Yellow, medium confidence.
[29:57.640 --> 29:59.780]  And blue, it thinks it might be a face.
[30:00.120 --> 30:04.500]  But the chances of it actually being a face are very low.
[30:04.700 --> 30:08.120]  So I would think that it would declassify that as a face.
[30:08.120 --> 30:14.380]  Because what it also classifies with blue are things like somebody's chest.
[30:19.360 --> 30:25.140]  So CV Dazzle is not like some cryptologist tool.
[30:25.400 --> 30:28.820]  It's actually very much embedded in fashion.
[30:28.820 --> 30:36.500]  And that's what I think is maybe makes it work is that it can hide under the guise of fashion, hair and makeup.
[30:36.940 --> 30:40.940]  And you can get something like this, which I just noticed on the cover of Elle magazine.
[30:41.240 --> 30:45.640]  Where you have hair that's asymmetrically in brightness.
[30:45.640 --> 30:48.160]  And the hair is coming down over the nose bridge kind of thing.
[30:48.380 --> 30:50.420]  So that wasn't detectable in OpenCV.
[30:50.680 --> 30:52.020]  And I thought that was really interesting.
[30:52.640 --> 30:53.740]  I haven't contacted her yet.
[30:53.740 --> 30:56.400]  I don't think... I don't know if she'd be interested.
[30:58.780 --> 31:03.760]  From here, like I was mentioning, what I really want to do is turn this into a project that anyone can use.
[31:03.760 --> 31:07.740]  A kit that someone could take and create a pattern for their face.
[31:07.920 --> 31:08.980]  You know, male or female.
[31:09.500 --> 31:13.280]  Whether you want to use makeup, whether you are cool with that.
[31:13.420 --> 31:14.480]  Or if you want to use hair.
[31:14.860 --> 31:16.400]  Or if you even want to use glasses.
[31:16.760 --> 31:27.320]  And the idea of using something as simple as hair and makeup is that then designers could take that information and turn it into actual objects or fashion.
[31:27.560 --> 31:33.960]  For example, some of the ones I thought of, you could do a collar that kind of pops up a little bit.
[31:34.080 --> 31:35.260]  And it hides an area here.
[31:35.500 --> 31:38.980]  And then you combine that with other things like hair or glasses.
[31:39.600 --> 31:45.540]  And glasses aren't really that great even though they're an obvious solution because they are incorporated.
[31:45.700 --> 31:46.740]  So many people have glasses.
[31:47.040 --> 31:48.460]  They're just part of these algorithms.
[31:49.740 --> 31:55.080]  And another thing is doing little modifications like band-aids or breathe right strips on your face.
[31:55.300 --> 31:58.280]  Or growing facial hair and cutting it a certain way.
[31:59.720 --> 32:01.060]  I'll move on from that slide.
[32:02.440 --> 32:11.520]  And another thing is with that software then creating a rating system where people can compete against each other and see who can develop a stronger look.
[32:12.420 --> 32:20.140]  The way that they'd be rated is by the system called SPF-CV, which is like SPF from sunscreen.
[32:20.900 --> 32:21.340]  Nice.
[32:21.640 --> 32:22.560]  From computer vision.
[32:23.860 --> 32:25.240]  It goes from zero to 50.
[32:25.860 --> 32:26.860]  50 being the best.
[32:27.800 --> 32:31.220]  And what I think I want to do for all you guys is post some of the source code.
[32:31.420 --> 32:38.440]  If people are interested in checking it out on my website, in the next two weeks or so, you can download it.
[32:38.440 --> 32:39.860]  Most of it is in Java.
[32:40.120 --> 32:41.480]  Some of it is being ported to C.
[32:42.140 --> 32:44.020]  And some of it was in ActionScript.
[32:44.220 --> 32:45.460]  And that was just a horrible idea.
[32:46.640 --> 32:47.680]  Which I'll never do again.
[32:49.240 --> 32:51.380]  And so now I'll take any questions you have.
[32:51.500 --> 32:53.400]  I think I left a little extra room for questions.
[32:53.540 --> 32:54.400]  Hopefully you guys have some.
[32:55.440 --> 32:55.840]  Thanks.
[33:05.210 --> 33:06.070]  I have one here.
[33:07.070 --> 33:15.790]  All the facial recognition systems, are they mostly using something based on OpenCV or are they using proprietary or how does it work exactly?
[33:16.370 --> 33:18.550]  Is it based on that or just totally not?
[33:19.370 --> 33:22.370]  I think some of them use a similar detection process.
[33:23.810 --> 33:30.470]  The Viola Jones method is an algorithm for checking the areas in a face.
[33:30.750 --> 33:36.670]  But what most people are doing is similar in the way that it's creating that profile.
[33:37.310 --> 33:43.670]  The profiles are made using these things called AdaBoost, FloatBoost or GentleBoost.
[33:43.930 --> 33:48.790]  And as soon as those came out in around 2003, that's when things really took off.
[33:48.790 --> 33:56.470]  And I think, from what I've read, most people are using the kind of AdaBoost technology that's used in creating those profiles still.
[33:56.690 --> 33:58.570]  I think a lot of commercial people are doing that.
[33:58.710 --> 34:02.730]  Because what you see is this was fairly effective in a lot of the tests.
[34:05.270 --> 34:14.590]  Yeah, so it sounds like most of this is informed by using a hard cascade because it operates on aggregates of pixels, correct?
[34:14.790 --> 34:14.850]  Yeah.
[34:14.850 --> 34:15.430]  For light and dark.
[34:15.430 --> 34:25.870]  As computation gets better, and you don't really need that optimization anymore, or maybe in the far future it might not, how will the CV Dazzle change in order to adapt to that?
[34:27.490 --> 34:44.870]  So what I found in the way that it worked out for developing these patterns is that the best approach is if you can get a copy, a demo of this, or if you can get anything where you can at least see the result, then you can do the same test that I did for this,
[34:44.950 --> 34:46.150]  and you can test all the areas.
[34:46.610 --> 34:57.190]  But I think with the new algorithms, we're also looking at things more fine-grained, like the texture of the face, and more fine-tuned details.
[34:57.270 --> 35:01.050]  Because these are really blocky features that are being defined in the OpenCV.
[35:01.770 --> 35:11.310]  And I've done some tests with the other ones, and I've kind of... what I found is that when you add a little texture or a lot of little detail in certain areas, then you get a better result.
[35:11.470 --> 35:25.230]  So basically, if you can get a demo, or if there's a copy online or something that you can use, you can create these kind of plates, the context sheets with all the tests, run those, and then get the feedback from them and develop stuff from there.
[35:27.030 --> 35:27.650]  Hey, Adam.
[35:27.770 --> 35:28.230]  Good talk, man.
[35:28.310 --> 35:28.730]  Real nice.
[35:29.530 --> 35:30.330]  Got a question here.
[35:30.370 --> 35:31.610]  Anti-Paparazzi Clutch.
[35:31.810 --> 35:31.950]  Yeah.
[35:32.090 --> 35:33.670]  Are you actually developing that as a commercial product?
[35:34.050 --> 35:34.210]  Yeah.
[35:34.550 --> 35:35.110]  No kidding.
[35:35.310 --> 35:35.610]  Awesome.
[35:35.790 --> 35:36.050]  All right.
[35:36.050 --> 35:39.730]  I'm looking forward to seeing this and that together on some rock star somewhere.
[35:40.170 --> 35:40.950]  It'd be awesome, dude.
[35:45.060 --> 35:49.100]  Just a quick question about the hair and makeup.
[35:49.260 --> 35:50.500]  Did you disaggregate those features?
[35:50.620 --> 35:52.820]  Because, you know, you're going for this very disjunctive look.
[35:52.820 --> 36:01.260]  Maybe you went into more detail in your thesis about whether it's the hairline and the makeup, or whether the makeup alone is more effective at preventing detection of a face.
[36:01.500 --> 36:05.680]  And it seems like it's much easier to prevent actual recognition of whose face that is.
[36:05.820 --> 36:09.160]  So did you go into more detail on that later in the project?
[36:10.200 --> 36:23.040]  Well, I think the makeup would be more effective if people are running the face recognition on the face, because it's changing a lot in this area, which is defined just kind of right above my eyebrows in that area.
[36:23.800 --> 36:27.100]  The face detection is looking for a little larger area.
[36:27.240 --> 36:28.700]  It kind of looks for the shape of the head.
[36:28.960 --> 36:30.600]  Some of them look for a border in that.
[36:31.480 --> 36:41.300]  What I like about the hair is that it's really convenient if you have long hair and you can kind of throw it in front of your face and then throw it out for a quick change up.
[36:41.300 --> 36:50.040]  The makeup is... it might work better if you're going to a party and you want your photos to end up in some automated system, whatever you're doing.
[36:50.280 --> 36:52.640]  But it's not as quick of a change.
[36:53.460 --> 36:54.780]  Does that answer...?
[36:54.780 --> 36:55.240]  Sort of.
[36:55.340 --> 37:01.020]  I mean, it seems like from the brief examples you presented that the makeup was more of a crucial thing because you're having the very high contrast.
[37:01.440 --> 37:01.860]  Uh-huh.
[37:01.960 --> 37:07.740]  Did you also look at different skin tones, hair colors, how effective those were in...
[37:08.700 --> 37:14.100]  how much contrast was helpful in blocking the face detection versus blocking the face identification?
[37:15.780 --> 37:23.520]  I tested just with this model and with only slight variations on this lighting condition.
[37:23.900 --> 37:31.320]  Yeah, what really needs to happen to make this a much more serious project is test of all different skin colors.
[37:31.660 --> 37:37.740]  It's not so much the skin color that these algorithms are looking for because they just change the image to black and white.
[37:38.060 --> 37:41.100]  But darker and lighter skins are going to have a much different result.
[37:41.100 --> 37:41.140]  Yeah.
[37:41.480 --> 37:43.840]  So maybe different color palettes for the blocking makeup.
[37:44.260 --> 37:44.340]  Thank you.
[37:44.340 --> 37:48.760]  But then, like with dark skin, you have light makeup to change the areas and that kind of thing.
[37:49.060 --> 37:49.180]  Yeah.
[37:49.320 --> 37:49.480]  Thank you.
[37:49.560 --> 37:50.000]  It's very interesting.
[37:52.240 --> 37:52.680]  Yeah.
[37:52.680 --> 37:52.720]  Yeah.
[37:52.960 --> 38:08.440]  On the PitBat example that you showed, the last one of the model, in the top left hand corner, the blue box, with more frames of reference, do you think that the software could have a higher accuracy of detection?
[38:09.640 --> 38:10.960]  The one that I'm working on?
[38:12.100 --> 38:20.280]  Either that you're working on with the styles that you're working on for camouflage, but also of the software being able to have more accuracy.
[38:20.960 --> 38:21.660]  Yeah, definitely.
[38:21.860 --> 38:30.380]  I think, I mean, theirs could have more accuracy by feeding it more images and they were looking at maybe what people are doing to not be detected.
[38:30.780 --> 38:42.280]  But yeah, the more points you have, the more examples that you can submit to those systems, and the finer tuned those tests are, then the more detailed, the better results you can get back from that.
[38:42.280 --> 38:52.240]  And what I showed before was about 20 different images, but it's more like running about 100 to 200 different variations on that.
[38:52.580 --> 39:05.480]  And you can see in that example in the row where it gradually decreased, you're kind of, you're doing that process where you're like, I don't know, there's a, I forget what it's called, some sorting algorithm, binary tree.
[39:05.480 --> 39:16.480]  Now you're looking at, you take the extremes, then you look in the middle, and if that doesn't work, then you go back and then you cut closer and more, more focus on certain areas.
[39:20.700 --> 39:21.260]  Yeah.
[39:21.670 --> 39:37.720]  So, these patterns and this, this type of feature altercation only really seems to be applicable in a certain type of social situation, at least right now, unless it's, you know, kind of face recognition becomes so pervasive.
[39:37.720 --> 39:43.900]  that a blue collar guy is willing to go to the Marlins game with this kind of stuff on his face to make it stop.
[39:44.050 --> 39:44.340]  He's done it anyway.
[39:44.980 --> 39:45.920]  Yeah, that's true.
[39:48.190 --> 39:58.540]  But this is very applicable for someone like a rock star who wants to go to a club and doesn't, you know, and doesn't want to be recognized all over Facebook when people take pictures of them.
[39:58.670 --> 39:59.190]  I get that.
[39:59.380 --> 39:59.520]  Yeah.
[39:59.520 --> 40:25.050]  But the first thing I'm going to do as someone who thinks in terms of Internet social media sites and developing applications for them is spider through Facebook on a regular basis, which is easy to do, pull out pictures and XF data that matches for this geographic location and match and figure out your open source pattern generation algorithm and match to that.
[40:25.520 --> 40:29.440]  And then just have a human look at that and go, oh, look, there's Beyonce.
[40:29.690 --> 40:47.170]  So in a way, this is creating just an alternative way of flagging someone on the, I mean, you're, you're not, this is really cool and neat, but like there, there are, this isn't really going to solve a lot of problems in reality.
[40:47.170 --> 40:49.400]  Is it, I mean, for anyone?
[40:49.650 --> 40:52.400]  It depends what kind of problems you have.
[40:52.740 --> 40:59.040]  I think this and the anti-paparazzi thing both operate within a, I guess, only to a certain audience.
[40:59.260 --> 41:03.220]  And it's not a widespread, you know, it's not for everybody.
[41:03.960 --> 41:10.570]  The, yeah, celebrities and their images being mined, they're public figures and that's going to happen.
[41:10.570 --> 41:11.540]  And there are even services.
[41:12.150 --> 41:18.380]  Face.com has one celebrity finder where it trolls through photos on Twitter, I think.
[41:18.650 --> 41:20.400]  And it matches up the celebrity faces.
[41:20.610 --> 41:24.650]  And there are so many examples of those out there that it becomes easier and easier to find.
[41:24.920 --> 41:31.040]  But isn't that the problem you're trying to solve for here specifically for celebrities or people like that?
[41:31.040 --> 41:33.690]  I mean, is that the problem you're trying to solve for?
[41:33.980 --> 41:38.050]  Is, what is the problem you're trying to solve for here?
[41:38.380 --> 41:39.760]  It's not celebrities.
[41:40.040 --> 41:49.280]  This is for, I think, yeah, for me, one of the goals here is to create something that not just has a functional value.
[41:49.460 --> 41:53.190]  And the functional value, I guess it's whoever wants to wear it.
[41:53.190 --> 42:00.130]  Like, I think, I was kind of designing it for, like, automated systems like Facebook and going to a party.
[42:00.260 --> 42:03.300]  That's kind of, um, one of the ideas for it.
[42:03.540 --> 42:09.360]  Imagine you're a teacher and you don't want your students to know you went to that party and got caught on lastnightsparty.com or whatever.
[42:12.380 --> 42:19.400]  A quick comment about the Rockstar comment earlier was, I wonder what KISS looks like on the facial recognition software.
[42:19.400 --> 42:36.340]  But the question was, I've heard a lot of reports of software that analyzes people's entire bodies and their posture to distinguish the difference between someone walking to their car in a parking lot and someone assaulting someone in a parking lot or a prisoner playing basketball or climbing up a fence.
[42:36.960 --> 42:42.650]  And I was wondering if there was any thoughts on camouflage applied to those sort of situations.
[42:42.650 --> 42:47.020]  Uh, some of those are really hard to do.
[42:47.500 --> 42:53.760]  Because in those situations you have people looking at your thermal, um, your thermal signature.
[42:54.320 --> 42:55.650]  And gate detection.
[42:56.150 --> 43:00.900]  And I think Josh mentioned that you can change the way you walk by putting different things in your shoes.
[43:01.130 --> 43:03.820]  And you just don't know what people are looking for.
[43:03.820 --> 43:13.400]  But there was, if you're not aware, a program called HID, Human Identification at a Distance, which was rolled out shortly after 9-11.
[43:13.720 --> 43:19.800]  And that looked at, um, the way people walk so they can identify dental tears at up to 150 feet.
[43:20.090 --> 43:20.920]  It was their goal.
[43:21.590 --> 43:23.900]  And I'm not sure where they are with that technology.
[43:23.900 --> 43:26.920]  But it's definitely since then developed.
[43:27.260 --> 43:30.050]  And you have the gate detection, motion detection.
[43:30.570 --> 43:35.400]  Actually, concealing yourself from all these is a huge task.
[43:35.690 --> 43:41.040]  Um, whether you can make camouflage for motion, I don't know.
[43:41.190 --> 43:43.300]  I mean, that's, it'd be really difficult, I think.
[43:43.980 --> 43:50.260]  So, you know, I thought the face detection was probably the one to target with my budget.
[43:53.560 --> 44:01.160]  Um, I just sort of had a sort of, a few tweaky questions about this in terms of, um, I was hoping to maybe see some other things you tried that failed.
[44:01.200 --> 44:02.900]  I don't know if you have any other examples with you.
[44:03.080 --> 44:07.700]  And I was also noticing, I mean, this is, uh, really high contrast paint at the very least.
[44:08.220 --> 44:14.300]  Did you try anything with, like, how, how low you could step down the, um, you know, the opacity of that black in order for it to work?
[44:14.580 --> 44:16.480]  Also sort of wondering, this is still pretty symmetrical.
[44:17.120 --> 44:18.760]  Um, could you just do, like, half a face?
[44:18.880 --> 44:19.960]  Would that, would that still work?
[44:19.960 --> 44:22.960]  I mean, like, you know, would you be able to black out one half, things like that?
[44:23.680 --> 44:24.540]  Yeah, I think so.
[44:24.760 --> 44:26.620]  It's all about trying different things.
[44:26.680 --> 44:35.480]  And some of the examples that I found in that movie that was looking through party photos, uh, there was one in there where it was Lady Gaga as Jesus Christ.
[44:35.780 --> 44:37.680]  And that image seemed to work.
[44:37.800 --> 44:40.600]  I mean, that was one that was, um, asymmetrical.
[44:40.900 --> 44:42.820]  It had lace coming down on one side.
[44:43.460 --> 44:47.740]  And, uh, the other, okay, the other question is the contrast.
[44:48.100 --> 44:53.080]  Yeah, high contrast works better if it's too similar to the skin color.
[44:53.400 --> 44:56.220]  And if it's a color, if it's red or green or something.
[44:56.480 --> 45:01.400]  And when you change that to black and white, then it could be the same tonal value as the skin.
[45:02.310 --> 45:05.920]  So, most of the successful ones that I found, they were all high contrast.
[45:09.020 --> 45:10.340]  I've just got a quick question.
[45:10.720 --> 45:16.080]  Uh, the, most of what you've been talking about and doing is, uh, with regards to face detection.
[45:16.800 --> 45:32.220]  Um, but have you done any sort of extra research on the side on whether there's any less severe modifications to just f*ck with it enough that it can't directly identify you or make you less computationally identifiable.
[45:32.660 --> 45:34.600]  As a face or as someone in the database?
[45:34.780 --> 45:38.140]  No, uh, that it would still pick you up as a face.
[45:38.280 --> 45:43.320]  You pick you up as a person, but it would make it more difficult to identify.
[45:43.320 --> 45:47.780]  So you don't draw a lot of attention, but just enough to make your face not identifiable.
[45:47.800 --> 45:47.980]  Exactly.
[45:48.200 --> 45:57.020]  So whenever somebody's actually looking at it and somebody's, you had the pictures of group photos that, uh, a human operator looks at that and says face, face, face, face, square, square, square, square, square.
[45:57.020 --> 45:57.340]  Oh, really?
[45:57.480 --> 45:58.660]  There's a face without a square.
[45:58.980 --> 45:59.340]  Yeah.
[45:59.780 --> 46:14.260]  Um, I mean, have you had a look at if there's any, um, less drastic, uh, makeup jobs or hairstyles that will still register as a face, but not be as easily identifiable?
[46:14.640 --> 46:20.600]  I think definitely face recognition is not quite as mature yet as face detection.
[46:20.600 --> 46:22.540]  And there are a lot of ways around it.
[46:23.180 --> 46:24.100]  Uh, who was it?
[46:24.240 --> 46:28.020]  HP or Lenovo had a face recognition system on their laptop.
[46:28.520 --> 46:34.360]  And all you had to do to get in was hold up a picture of the person in front of the webcam and you had access to their computer.
[46:34.900 --> 46:38.820]  And then, of course, there was the lovely escapade where they couldn't recognize anyone who wasn't white.
[46:39.120 --> 46:39.380]  Yeah.
[46:40.320 --> 46:43.580]  Uh, so it's got a long, a long ways to go of face recognition.
[46:44.120 --> 46:46.680]  But that's where all the money is right now and that kind of stuff.
[46:48.080 --> 46:53.340]  So you, you said you come from the background of, uh, photography and, like, fashion.
[46:53.880 --> 47:01.080]  Um, when you frame your research in that context, how does it change the perception of your research to, like, hardcore CS people?
[47:02.880 --> 47:05.480]  Yeah, I'm not from computer science background.
[47:06.120 --> 47:09.420]  Uh, how does it change?
[47:09.580 --> 47:14.800]  Well, um, I'm not sure how to present to, or even CS, I'm not sure how to present to CS.
[47:15.920 --> 47:17.300]  And what do you guys want to hear?
[47:19.880 --> 47:20.860]  Who thought this was science?
[47:21.480 --> 47:22.560]  I gave you pretty pictures.
[47:23.360 --> 47:24.480]  Who thought it was good science?
[47:30.690 --> 47:35.490]  I'm talking more, more in the, like, the dissection of the face detection algorithm.
[47:35.650 --> 47:47.730]  It sounds like a lot of what you're doing is, is more of a, uh, um, a trial and error that's, that's based very specifically on a certain subset of algorithms.
[47:48.150 --> 47:48.630]  Yeah.
[47:48.630 --> 47:54.110]  So, um, like, what have you heard among, like, CS colleagues about this?
[47:54.310 --> 47:55.870]  Like, at NYU or, or elsewhere?
[47:55.870 --> 48:02.010]  Um, I heard from the creator of OpenCV, uh, Gary Bradsky.
[48:02.670 --> 48:06.570]  And he said, he found it funny, like a joke.
[48:07.390 --> 48:11.070]  And he thought that it was probably a waste of time or something.
[48:11.510 --> 48:23.990]  But, yeah, you know, I think that presenting this kind of thing to people in more of a technology background can be difficult because, um, and I want to make this mire down in the technology.
[48:23.990 --> 48:28.590]  It's more about the, how you can look with it or how you can have fun with it.
[48:30.010 --> 48:39.910]  And when I present it in, in other terms, like a project all about privacy, then it just doesn't have as much of an appeal.
[48:40.330 --> 48:45.390]  And the computer science thing, you know, email me and I can, I'm happy to talk more.
[48:45.390 --> 48:51.910]  I think if it's a computer science group, then I would change this and focus more on that kind of stuff.
[48:53.090 --> 48:54.110]  That next was it.
[48:57.260 --> 48:57.900]  Thank you.
[48:58.640 --> 48:59.100]  Great.
[48:59.440 --> 48:59.860]  Thanks a lot.