[00:21.030 --> 00:22.990]  And we're rolling on up here.
[00:26.770 --> 00:36.450]  Our next speaker is... Our next speaker is the host of the Data Skeptic podcast.
[00:37.710 --> 00:41.130]  Machine learning has been accelerating at a crazy clip.
[00:41.850 --> 00:51.010]  And yes, speaking of Rule 34 and of the people that are looking for you, thanks to Rule 34, undoubtedly some machine learning is going to happen.
[00:51.030 --> 00:52.010]  We're going to be involved with that.
[00:52.770 --> 00:54.550]  Folks, I want to present our next speaker.
[00:56.730 --> 00:57.650]  Kyle Polich.
[01:03.380 --> 01:04.960]  Thanks everybody for coming out.
[01:06.340 --> 01:09.280]  So yeah, my day job, I'm what's called a data scientist.
[01:09.580 --> 01:11.780]  So I do a lot of machine learning and some other similar stuff.
[01:12.060 --> 01:15.660]  And I want to talk about how hacking can kind of overlap with that.
[01:16.140 --> 01:20.240]  Notice this isn't hacking using machine learning or hacking with machine learning.
[01:20.360 --> 01:22.360]  It's hacking machine learning algorithms.
[01:22.360 --> 01:27.000]  So I want to talk about how you basically attack the algorithms themselves and the models they produce.
[01:28.360 --> 01:29.580]  Sorry, give me one second here.
[01:34.160 --> 01:36.400]  I guess I have to be over here to switch over.
[01:37.200 --> 01:37.500]  Sorry.
[01:38.980 --> 01:39.350]  There we go.
[01:39.680 --> 01:40.780]  So, quick outline.
[01:41.420 --> 01:45.040]  I'm going to go through what is machine learning pretty quick in case someone needs a primer.
[01:45.340 --> 01:46.960]  Why would it be interesting to hack it?
[01:47.040 --> 01:48.820]  And what do I even mean by hacking it?
[01:48.850 --> 01:51.920]  Since it's not sort of a traditional sense of what one does when they hack.
[01:51.920 --> 01:58.720]  This is, as you mentioned, a growing important topic in that machine learning is showing up everywhere in technology.
[01:58.880 --> 02:02.070]  We heard in the last talk that it's being used to de-anonymize Tor.
[02:02.320 --> 02:06.880]  So it's something everyone, especially the hacker community, needs to be paying some attention to, I think.
[02:07.020 --> 02:10.000]  So hopefully I can set a few people down on some paths like that.
[02:10.760 --> 02:12.980]  So what do I mean by hacking?
[02:13.100 --> 02:17.000]  Well, let's first look at what some of my peers have said, other people that do machine learning type stuff.
[02:17.000 --> 02:25.850]  So, Jake Vanderplass, really smart guy, just did a talk at Psycon called Hacking Statistics.
[02:26.120 --> 02:27.070]  So a related area.
[02:27.400 --> 02:29.180]  This was his definition of hacking.
[02:29.480 --> 02:30.620]  Might work for some of you.
[02:30.700 --> 02:32.820]  I think it's in the spirit of what it means to hack.
[02:33.340 --> 02:36.600]  Similarly, there's a great book called Bayesian Methods for Hackers.
[02:36.670 --> 02:37.600]  Highly recommend this.
[02:38.220 --> 02:43.880]  Cameron doesn't really give a formal definition of hacking, but you get the semblance of sort of what he's going at here.
[02:44.460 --> 02:46.940]  Here's two guys, or two more examples I don't know.
[02:47.160 --> 02:50.740]  This book, Machine Learning for Hackers, defines a hacker this way.
[02:50.900 --> 02:53.960]  Someone that likes to solve problems and experiment with new technologies.
[02:54.140 --> 02:55.400]  I think that's certainly very fair.
[02:56.840 --> 02:58.680]  Lastly, this is a guy, I don't know him.
[02:59.040 --> 03:01.340]  He does these short machine learning videos on YouTube.
[03:01.520 --> 03:10.080]  Never defines what he means by hacking, but I think given this quote, you get the sense that what he really means is the ability to do something quick, to bootstrap it.
[03:11.520 --> 03:14.840]  And by the way, this is the definition of machine learning he gives, which is a pretty good one.
[03:14.920 --> 03:19.660]  The study of algorithms that learn from examples and experience instead of hard-coded rules.
[03:19.860 --> 03:23.380]  So, what do I mean when I say hack a machine learning algorithm?
[03:23.780 --> 03:27.340]  Well, I really go to, you know, the Hackers Manifesto and I take pieces out of that.
[03:27.980 --> 03:39.300]  I believe information should, and I added in typically, be free because I think I don't want my medical records to be free, but I like things like Wikipedia and the spirit of that to be free.
[03:39.520 --> 03:42.580]  I like open data, the open data movement being around.
[03:42.840 --> 03:46.900]  By the way, New York City, you guys are now number four on the open data census.
[03:47.110 --> 03:48.900]  I'm from Los Angeles, we're number one.
[03:49.260 --> 03:50.580]  Just saying, talk to your mayor.
[03:50.840 --> 03:55.440]  If there's one arms race I want government involved in, it's being better at open data and transparency.
[03:58.680 --> 04:06.920]  So, these are all things that I think fit in my definition of what hacking is, but ultimately it's getting down to figuring out exactly how things work, possibly to exploit them.
[04:07.020 --> 04:13.200]  And to me, that's the spirit of being a hacker, and we can very much do that with machine learning, and I'm actually going to argue that we should be doing more of that.
[04:13.760 --> 04:21.500]  One could also add in, perhaps, that hacking is synonymous with mistrusting authority, or what's the famous phrase, trust but verify.
[04:22.120 --> 04:28.120]  I'm a big fan of that, so if machine learning algorithms are being put up, as authority, we should have some skepticism of them.
[04:29.140 --> 04:30.420]  So, what is machine learning?
[04:30.620 --> 04:36.480]  It's the learning to explore the study of the construction of algorithms that they can learn from and make predictions about data.
[04:36.700 --> 04:39.680]  So, there's some general categories I can give examples of that do this.
[04:39.980 --> 04:42.600]  Common use case is what we call classification and labeling.
[04:42.860 --> 04:44.420]  So, this is something like fraud detection.
[04:44.660 --> 04:47.100]  A credit card company has lots of transactions coming through.
[04:47.360 --> 04:49.440]  Most of them are legitimate, good, and well-intentioned.
[04:49.560 --> 04:51.440]  Every once in a while, there's one that is fraudulent.
[04:51.860 --> 04:55.700]  It's not being charged, not coming from the person who actually possesses the card.
[04:55.980 --> 05:00.220]  You'd like to use machine learning to find those rare cases and stop those transactions.
[05:00.960 --> 05:02.180]  Clustering is another area.
[05:02.560 --> 05:07.240]  Do a lot of things like social networking, could apply to fraud, but it's basically an unsupervised technique.
[05:07.840 --> 05:14.140]  Unsupervised meaning you don't provide it labels the way you would in a classification example where I say these are some known fraudulent transactions.
[05:14.320 --> 05:18.560]  It just says, here's some data, give it to an algorithm, ask it to put it into logical groups.
[05:19.220 --> 05:21.700]  We could also, I emphasize, decision problems.
[05:22.260 --> 05:28.960]  So, something like, what is the optimal order that a truck should make deliveries in to minimize cost or travel time or things like that.
[05:29.080 --> 05:31.860]  These are common types of things machine learning people work on.
[05:32.960 --> 05:35.460]  So, here's a quick, I'm going to try and do a one-minute crash course.
[05:35.580 --> 05:37.080]  This is not a demo on machine learning.
[05:37.180 --> 05:39.660]  I just want to give everybody a sense of what I mean by that.
[05:39.660 --> 05:41.800]  So, hopefully, you can follow the rest of the talk.
[05:41.940 --> 05:44.720]  There's plenty of great resources for getting really deep into this.
[05:44.820 --> 05:47.740]  If anyone wants to get some recommendations, happy to talk to you afterwards.
[05:47.740 --> 05:53.200]  I would say the basic idea is you first get some amount of data you have into what we're calling the tidy format.
[05:53.380 --> 05:57.680]  That's where you have columns being variables and rows being observations.
[05:57.940 --> 06:00.360]  In this case, let's say these are individual users of a system.
[06:00.600 --> 06:05.260]  And we're doing something in security where we want to detect people who might be using dictionary passwords.
[06:05.700 --> 06:08.880]  So, we have some set of data where we know people are or are not.
[06:09.080 --> 06:10.720]  And we know some metadata about them.
[06:10.820 --> 06:13.080]  Perhaps their age, the domain of their email address.
[06:13.260 --> 06:15.240]  This is all fictitious stuff, just for example.
[06:15.240 --> 06:21.880]  But if we were doing some sort of penetration testing, maybe because we have a limited number of ways we can try and attack a system.
[06:22.180 --> 06:24.260]  We want to strengthen the organization we're working for.
[06:25.680 --> 06:27.460]  This is our objective function.
[06:27.600 --> 06:34.520]  We would like to build a system that's able to find patterns that predict yes or no is someone using a dictionary password instead of a more secure password.
[06:35.780 --> 06:40.380]  So, if it were a trivial problem, you could develop some human-generated rule-based systems.
[06:40.380 --> 06:44.020]  But it's generally not trivial, and these are things that might correlate with it.
[06:44.160 --> 06:49.980]  So, for example, we would expect that someone who is an admin or a root should probably not be using a dictionary password.
[06:50.120 --> 06:52.680]  Someone who's administrating a system should know better.
[06:53.160 --> 06:54.340]  We could look at age.
[06:54.480 --> 06:56.440]  Now, age could be completely independent.
[06:56.600 --> 06:58.860]  Age could have nothing to do with if you have a good password or not.
[06:59.120 --> 07:02.260]  But I could also argue, well, maybe with age you gain wisdom and experience.
[07:02.260 --> 07:03.720]  You know how to use passwords better.
[07:03.720 --> 07:08.360]  Or maybe the youth are more in touch with technology and know how to make passwords better.
[07:08.500 --> 07:09.840]  I could kind of argue it all ways.
[07:10.240 --> 07:14.180]  Machine learning is the right way to find out if there's any information here.
[07:14.400 --> 07:18.440]  The algorithm will find it and will incorporate that in an efficient way.
[07:19.080 --> 07:23.800]  So, an important part of getting started in machine learning is cleaning and transforming your data.
[07:24.000 --> 07:27.920]  So, here's some examples where we can see, for some reason, we're missing information.
[07:28.320 --> 07:31.220]  We also have some ages that don't seem to be possible ages.
[07:31.220 --> 07:33.600]  So, what does a machine learning person do?
[07:33.760 --> 07:36.580]  Generally, we filter this stuff out, for better or for worse.
[07:36.880 --> 07:40.660]  If you'd like to be filtered from a data set, try and have some outlier values.
[07:42.060 --> 07:45.880]  Another thing you might want to do is clean up or aggregate some data.
[07:46.040 --> 07:56.640]  For technical reasons I won't get too deep into, it might be worthwhile to take, you know, just say something is Android, group them all together or by maybe major release if you know there's security issues at different levels.
[07:56.860 --> 07:59.020]  But there's a lot of time spent working on this.
[07:59.020 --> 08:01.540]  Machine learning is a lot like feeding a pet lizard.
[08:01.660 --> 08:02.520]  Anyone's ever had that?
[08:02.940 --> 08:06.080]  They're really finicky and they don't often want to eat what you're giving them.
[08:06.200 --> 08:08.760]  And you kind of have to prepare it just right and hope they're going to eat.
[08:08.860 --> 08:12.160]  And that's what some algorithms are like depending on the data you have available.
[08:13.240 --> 08:16.160]  Similarly, like, this is not a common domain name.
[08:16.700 --> 08:20.440]  Probably you see a lot of Gmail if you're looking at email accounts and Yahoo and stuff like that.
[08:20.440 --> 08:24.600]  As you get into the longer tail, maybe we just mark that as like other or small domain.
[08:24.860 --> 08:28.480]  So this is sort of the cleanup process that data scientists will go through.
[08:28.720 --> 08:30.480]  There's also going to be a lot of dirty data.
[08:30.740 --> 08:32.460]  These are obviously more incorrect ages.
[08:32.620 --> 08:33.810]  These are not email domains.
[08:34.020 --> 08:36.100]  This is not a believable email address to me.
[08:36.500 --> 08:40.760]  For some reason we have what looks like HTTP status codes instead of education.
[08:41.540 --> 08:43.220]  Data always looks like this.
[08:43.220 --> 08:44.380]  It's always garbagey.
[08:44.620 --> 08:46.220]  There's a lot of cleanup to be done.
[08:46.550 --> 08:53.080]  And if you don't do it right, your algorithm could exhibit some bias or could not predict things that are predictable.
[08:53.380 --> 08:57.310]  So poisoning a data set is something that is an effective attack on machine learning.
[08:58.620 --> 09:01.280]  So if you've done all that sort of cleanup, what do you do next?
[09:01.380 --> 09:03.540]  You run an algorithm, something like logistic regression.
[09:04.080 --> 09:09.780]  Basic idea here is if the probability of being insecure is a composition of all the variables.
[09:09.780 --> 09:17.280]  You'll notice here I took out age and I just turned it into this Boolean is young or not, which I guess is maybe one if you're let's say under 25, zero if not.
[09:17.560 --> 09:22.020]  And the objective is to find the optimal weights of these values.
[09:22.240 --> 09:26.200]  So some baseline, maybe 10% of people have insecure passwords.
[09:26.500 --> 09:30.260]  I would expect Linux users to be above average at having good passwords.
[09:30.260 --> 09:35.760]  So this would probably be a negative coefficient, meaning that when this is true, when it's one, you reduce the probability.
[09:35.760 --> 09:43.160]  And given the data you have, find the weights that best give you a model that fits the data and predicts future data.
[09:44.140 --> 09:50.860]  We like to think of this as interpretable in the same way I was saying, well, if this is negative, that means Linux users are reducing the...
[09:51.380 --> 09:52.660]  are less likely to be insecure.
[09:52.780 --> 09:54.180]  Same thing with an administrator.
[09:56.240 --> 09:59.280]  Another type of model or an algorithm is decision tree types.
[09:59.420 --> 10:03.220]  They are a little bit different and they're able to capture some nonlinear type relationships.
[10:03.220 --> 10:06.620]  So here maybe the crux is, is the user young or not?
[10:06.940 --> 10:10.540]  When they are young, the next important variable is what operating system they are on.
[10:10.660 --> 10:15.660]  If they are not qualifying as young, maybe it's are they on a corporate email is the next most important predictive feature.
[10:16.360 --> 10:19.780]  So we like this also because this is very interpretable.
[10:19.880 --> 10:25.820]  I can read this and maybe get some explanatory power about what's the sort of fundamental relationships going on in the data.
[10:26.480 --> 10:37.740]  We celebrate these techniques for being interpretable until we see something like this, which is completely mind-bogglingly uninterpretable, but is a more common output for some of the more complex type problems that you might work on.
[10:38.960 --> 10:43.500]  Last thing you should know is in any machine learning algorithm, you're going to have some training data.
[10:43.680 --> 10:47.300]  You want to find like the optimal weights or the optimal tree given the data you have.
[10:47.480 --> 10:55.960]  Set some data aside for testing because if your model cannot well represent, predict or explain data that it wasn't trained on, then you've built a bad model.
[10:56.080 --> 10:59.280]  You've just memorized your data set or overfitted.
[11:00.440 --> 11:04.360]  The last thing on my crash course is how we look at the value of these.
[11:04.480 --> 11:05.960]  This is something called a confusion matrix.
[11:06.280 --> 11:12.180]  So the idea is if you're trying to predict something, you predict let's say yes or no, whether or not the user is a security risk.
[11:12.520 --> 11:19.080]  There's an actual observable value and then you have true positives and true negatives, which are good cases where your model adequately described the data.
[11:19.420 --> 11:22.760]  False positives, false negatives, which are cases you'd like to improve upon.
[11:23.380 --> 11:25.540]  Rarely do you get anything that's a perfect match.
[11:26.240 --> 11:28.880]  If you did, it's probably not something you should be using machine learning on.
[11:29.680 --> 11:35.000]  But ideally you have this sort of diagonal of a very good model and then you can study where is it breaking down.
[11:35.640 --> 11:38.780]  I can study that as a researcher saying how do I improve my model.
[11:38.960 --> 11:42.840]  You can study that as someone who wants to attack it saying where are the weaknesses of this model.
[11:44.240 --> 11:47.220]  Remember some of this, we'll get into a little bit of it more later in the talk.
[11:47.220 --> 11:50.640]  So at the end of the day, this is what ML is really about.
[11:50.920 --> 11:53.340]  Make predictions for some reason on a data set you've got.
[11:54.580 --> 11:56.260]  So what is the process like?
[11:56.440 --> 11:57.740]  It's generally these stages.
[11:57.860 --> 11:59.420]  This is something called CRISP-DM.
[11:59.680 --> 12:01.420]  It's not real popular, but I like the idea.
[12:01.540 --> 12:02.660]  So I include it in a lot of talks.
[12:02.940 --> 12:04.360]  It starts from feature engineering.
[12:04.520 --> 12:08.140]  You remember I said like, well, let's see if age is relevant to a problem.
[12:08.240 --> 12:09.180]  It may or may not be.
[12:09.380 --> 12:12.680]  And then I chose to represent it later as that is young type variable.
[12:12.680 --> 12:19.900]  Those are choices I make about what's the data I have available and how do I choose to represent it in a way that the algorithm will find it most beneficial.
[12:20.660 --> 12:27.280]  Algorithm selection I won't cover because I think it gets into more of the technical details, but one has to pick the algorithm they want to use to build a model.
[12:27.820 --> 12:33.680]  Validation, of course, is a back and forth process of checking your data, looking at the model, finding weaknesses, improving it.
[12:34.180 --> 12:41.800]  Tuning the algorithm, for example, you can tune something to have less false positives at the expense of more false negatives if that suits your situation.
[12:41.800 --> 12:43.440]  Something like medical diagnosis.
[12:43.660 --> 12:45.680]  You want to think very carefully about how you tune that model.
[12:46.480 --> 12:50.340]  And then ultimately you deploy your model into some production system where it starts interacting with the real world.
[12:51.640 --> 12:53.320]  So a little bit more on feature engineering.
[12:53.520 --> 12:55.260]  What might we do in terms of security?
[12:55.260 --> 12:56.240]  What would be a cool feature?
[12:56.380 --> 13:01.740]  Well, I could look and say, what percentage of a user's files are set to 777 permissions?
[13:02.180 --> 13:06.920]  That might be an interesting indicator that would help the algorithm latch on to something predictive about a user.
[13:07.100 --> 13:14.360]  And machine learning people in general spend a lot of time thinking about domain-specific problems and how to represent what they have in a useful way.
[13:15.660 --> 13:20.100]  Another way to look at it is to think of every machine learning problem as essentially a system.
[13:20.660 --> 13:27.020]  The objective is to generate some model where you put in these observed features, the things that I talked about constructing, and out comes some result.
[13:27.220 --> 13:30.240]  Whether that's that the user is a security risk or not.
[13:30.380 --> 13:33.380]  The user is worthy of getting a loan or not.
[13:33.380 --> 13:36.840]  This is the general framework for what machine learning kind of does.
[13:37.740 --> 13:40.180]  So if you have a system, what do we do with systems?
[13:40.640 --> 13:42.600]  We take them apart and we see how they work.
[13:42.820 --> 13:48.160]  So that's why I think the hacker spirit is something that the field of ML can get a little bit more benefit from.
[13:48.700 --> 13:52.160]  So why would we hack machine learning algorithms and the models they produce?
[13:52.600 --> 13:53.840]  Well, one obviously just to learn.
[13:54.160 --> 13:56.920]  That's very much in line with coming to a conference like this.
[13:56.920 --> 14:03.080]  I think, secondly, it can advance the field if we're identifying common mistakes people make in machine learning.
[14:03.760 --> 14:07.600]  Showing how they're exploitable is a good way of teaching people how to make better systems.
[14:08.280 --> 14:14.380]  There's some amount of personal gain, whether that's for, you know, you want to build a system that benefits you or you're working on some project.
[14:14.720 --> 14:25.640]  And lastly, as a watchdog, and I think this is one of the critical points I want to bring up and why I wanted to give this talk today, I think the hacker community as a whole, one of the services that provides society is to be a watchdog.
[14:25.640 --> 14:30.000]  I myself have never looked at the source code for the Linux kernel.
[14:30.300 --> 14:36.580]  But I trust that many people, some of them in this room have, and because of that I can be reasonably sure that there's no back doors in it.
[14:36.780 --> 14:44.260]  I can't say the same about closed source operating systems, but even though I haven't done those audits myself, there are watchdogs out there doing that.
[14:44.860 --> 14:52.500]  As machine learning creeps into more and more places in our culture and our society and the technology that we rely on, there need to be people watching it.
[14:52.500 --> 14:56.900]  And I think there are people in the hacker community that could add a great deal to that conversation.
[14:57.480 --> 15:00.160]  For example, predictive policing is a big topic.
[15:00.620 --> 15:05.100]  I recently interviewed a woman named Christiane Lum who works for this organization, HR DAG.
[15:05.440 --> 15:09.380]  They found errors in the DOJ's reporting of homicides by police.
[15:09.380 --> 15:11.720]  The DOJ was under-reporting those.
[15:13.180 --> 15:15.340]  More in general, they're looking at predictive policing.
[15:15.480 --> 15:22.100]  When people are trying to use machine learning to discover maybe where police should be deployed or who is a good suspect or not.
[15:22.620 --> 15:26.560]  These are things that, frankly, they're going to happen whether we like them or not.
[15:26.600 --> 15:33.680]  So why don't we try and make sure that the systems that are doing that are as useful as possible and maybe restricted in cases where they need to be.
[15:33.900 --> 15:39.080]  I am very much against the idea of future crime, but there are some fine lines.
[15:39.240 --> 15:43.260]  You know, I also interviewed some researchers from Arizona State University that are looking at...
[15:43.260 --> 15:45.940]  working with the Chicago Police Department, looking at social networks.
[15:46.080 --> 15:59.900]  Their objective is in the event that there's a gang violence event, let's say like a murder, they want to predict who might retaliate and for the police to do outreach and engage with those people, maybe cool off the situation, prevent it from escalating.
[16:00.880 --> 16:02.940]  Now, I think that's a very fine endeavor.
[16:03.140 --> 16:04.040]  I like their work.
[16:04.160 --> 16:04.900]  I like what they're doing.
[16:05.260 --> 16:07.500]  But like any tool, it's how you use that.
[16:07.620 --> 16:14.460]  If you use that as an excuse to go exploit someone, say, well, the algorithm told us you're guilty, so we're here to question you or search your house.
[16:14.620 --> 16:15.700]  I don't agree with that.
[16:15.840 --> 16:19.460]  But if you genuinely went out to someone and say, hey, it looks like there's something going on.
[16:19.540 --> 16:20.220]  How can we help?
[16:20.320 --> 16:21.420]  That's a public service.
[16:21.740 --> 16:22.920]  Those sorts of things are good.
[16:23.360 --> 16:25.000]  There's going to be predictive policing.
[16:25.740 --> 16:32.700]  So we should be kind of watching out and making sure that it's benefiting our society and innocent civilians and whatnot.
[16:33.840 --> 16:38.540]  There's also the potential for algorithms to become racist, even unintentionally racist.
[16:39.060 --> 16:43.160]  So let's think of a case that maybe you work for a real estate company, they have a website.
[16:43.500 --> 16:47.980]  Your objective is to provide really good recommendations for people searching on property.
[16:48.140 --> 16:52.260]  So there's some obvious stuff like distance, maybe I should have said difference between the max price.
[16:52.260 --> 16:58.840]  If I go on a website for real estate and I search for houses under $400,000, there's probably a house selling for $90,000.
[16:59.200 --> 17:02.380]  That's not what I'm looking for, even though it qualifies for my search criteria.
[17:02.640 --> 17:06.990]  So they're trying to give me the user experience they think I want by optimizing those things.
[17:06.990 --> 17:10.990]  They can do that in some of the clear ways, you know, like looking at percentage...
[17:11.660 --> 17:17.230]  If I'm always looking at houses with pools, they might be able to figure out, hey, this guy likes pools, show him more houses with pools, things like that.
[17:17.880 --> 17:20.460]  There are people trying to use machine learning to detect patterns.
[17:20.710 --> 17:22.860]  That's what it's really good at, and exploit those patterns.
[17:23.420 --> 17:26.620]  Overall, this sounds like a good idea, but here's an interesting project.
[17:26.620 --> 17:35.180]  What if we looked at your current zip code, which we could infer from your IP address, and we looked at, well, people who used to live here, where do they live next, by and large?
[17:35.920 --> 17:38.530]  There are common cases where this seems really good, right?
[17:38.570 --> 17:45.970]  I also included, like, the number of bedrooms and bathrooms, because maybe there are certain neighborhoods, let's say Manhattan, where people are making family choices.
[17:46.140 --> 17:48.960]  They want to have kids, so they want to have a bigger house, yard.
[17:49.700 --> 17:54.770]  If they're going to move somewhere from one bedroom to a three bedroom, there are certain zip codes where that's good for.
[17:54.770 --> 17:56.330]  There's also cultural trends.
[17:56.470 --> 18:03.810]  You know, different neighborhoods have different charms, and I think each of us can think of where we're from and say that, oh, it makes sense, people from this area commonly move to certain other areas.
[18:03.960 --> 18:06.810]  If you find those patterns, you can help make good recommendations.
[18:07.470 --> 18:10.230]  But you can also reinforce a redlining process.
[18:10.770 --> 18:23.200]  If, by whatever means, it happens to be the case that people were being pushed away or restricted from living in certain areas, the algorithm, if that's in the training data set, it will be a self-fulfilling prophecy, and it will learn that pattern because it's there.
[18:23.910 --> 18:27.730]  And the person who built that algorithm doesn't have to be a racist person.
[18:27.880 --> 18:29.120]  They could be very well-intentioned.
[18:29.330 --> 18:32.900]  It's sort of embedded in the data, and it's something we need watchdogs looking for.
[18:34.210 --> 18:37.120]  I think everybody knows this story, Microsoft's Little Blunder.
[18:37.710 --> 18:40.960]  I think... I won't cover it in too much detail because I think most people know it.
[18:40.960 --> 18:45.530]  But as far as I know, basically Tay was sort of a... was hacked.
[18:45.700 --> 18:50.400]  I mean, people probably used direct messaging to send this thing, really awful stuff like this.
[18:51.450 --> 18:57.290]  And that if it's in the background taking in with no filter any data and trying to build up...
[18:57.290 --> 18:59.530]  I don't know if it's Markov models or some other system.
[19:00.270 --> 19:02.270]  It's... this thing was basically hacked.
[19:02.460 --> 19:04.680]  So we have an example of this stuff going on in the wild.
[19:04.680 --> 19:07.470]  I think it's good because it... well, not good that they did it this way.
[19:07.570 --> 19:09.710]  I would rather have said like nonsense stuff.
[19:09.920 --> 19:12.490]  But it exposes the fact that this kind of thing could happen.
[19:12.710 --> 19:14.200]  I've actually got a hilarious example too.
[19:15.780 --> 19:17.310]  A friend of mine had a camera.
[19:17.360 --> 19:19.290]  It was trying to have all these fancy features added.
[19:19.440 --> 19:22.490]  And one of them was it was going to highlight the faces of anyone who had their eyes closed.
[19:22.790 --> 19:23.420]  Sounds good, right?
[19:23.530 --> 19:25.090]  You know, quick snap of the moment.
[19:25.380 --> 19:26.140]  Somebody's eyes are closed.
[19:26.230 --> 19:26.920]  Retake the photo.
[19:27.530 --> 19:30.210]  Well, this camera... we had a diverse group together.
[19:30.210 --> 19:34.490]  And it highlighted all the Asian people and said, they all have their eyes closed.
[19:34.510 --> 19:35.420]  Which they didn't.
[19:36.090 --> 19:40.530]  So, I can laugh at it the way you laugh at like a toddler who makes a mistake.
[19:40.730 --> 19:41.440]  Like, no, no, no.
[19:41.530 --> 19:42.120]  Come on, we got a T.
[19:42.200 --> 19:42.810]  That's not right.
[19:42.970 --> 19:51.620]  But I think the moral here is people who work on these systems should be aware and think harder about how they're going to affect the real world and affect other people.
[19:52.200 --> 19:59.900]  While that example is silly and kind of didn't really hurt anybody, you know, loan servicing financial stuff can hurt people.
[19:59.900 --> 20:03.140]  So, there needs to be a lot more, I think, thought and responsibility.
[20:03.290 --> 20:07.590]  Not to say that there isn't, but just more of that and more people looking at it is what I want to encourage.
[20:08.910 --> 20:12.640]  PageRank algorithm is not technically machine learning or at least not in its regular form.
[20:12.710 --> 20:14.210]  This is something Google introduced.
[20:14.400 --> 20:18.920]  The idea of how links on the web can convey authority of a site.
[20:19.090 --> 20:21.490]  So, these bigger sites have a lot of incoming links.
[20:21.590 --> 20:26.290]  These must be something like, you know, hope.net or 2600, probably authority sites.
[20:26.290 --> 20:28.970]  And if they link somewhere, that conveys a lot of weight.
[20:29.120 --> 20:32.470]  Whereas these might be like little, like WordPress blogs that were just started yesterday.
[20:33.270 --> 20:41.160]  A common hack for the PageRank algorithm was set up a whole link farm of small nodes and point them at your actual website to build that up.
[20:41.360 --> 20:42.970]  Now, of course, Google is fighting that.
[20:43.180 --> 20:44.120]  It's a form of spam.
[20:44.310 --> 20:46.970]  But there's a cat and mouse game going on here all the time.
[20:47.660 --> 20:51.530]  Interestingly enough, Google Search for a long time was famous for not including much machine learning.
[20:51.530 --> 20:53.330]  That has all changed.
[20:53.490 --> 21:00.960]  In October 2015, Google announced that RankBrain, which is an ML-based system, is the number three ranking factor in how things go on the web.
[21:01.090 --> 21:05.680]  So, what we search for and what we find is influenced by machine learning, which let's hope is unbiased.
[21:07.330 --> 21:08.880]  Anybody familiar with this book?
[21:09.070 --> 21:13.490]  Awfully valuable book, The Making of a Fly selling here for $1.7 million and $2.1 million?
[21:14.140 --> 21:15.250]  How did this happen?
[21:15.830 --> 21:18.680]  Well, very simply, not really a deliberate hack.
[21:18.680 --> 21:23.470]  This was an accidental attack, also not technically machine learning, but a good demonstration, I think.
[21:23.880 --> 21:27.270]  Somebody came along and said, hey, I have this book, presumably this guy.
[21:27.660 --> 21:29.810]  I just want to move a lot of volume.
[21:29.900 --> 21:36.230]  So, what I'll do is I'll find the cheapest price and I'll just set mine slightly below that at like .9983 times the highest price.
[21:36.420 --> 21:36.570]  Cool.
[21:36.700 --> 21:37.680]  I'll always underbid everybody.
[21:38.140 --> 21:42.400]  Someone else came along and said, I'm more of a shyster type person.
[21:42.420 --> 21:52.360]  I don't actually have this book, but if I think I can get someone to overpay for it, I'll buy it from the other guy and then I'll ship it to me and reship it to you and I'll just make a little margin on that.
[21:53.200 --> 21:54.920]  Legal, I guess, even if it's a little shady.
[21:55.140 --> 22:01.310]  So, he set up his algorithm to find whatever price is the average price that was available and add 27% to that.
[22:02.040 --> 22:03.520]  So, these are the two algorithms.
[22:03.660 --> 22:06.440]  When there are two in place, this is the ultimate path.
[22:07.080 --> 22:07.920]  So, in what?
[22:08.340 --> 22:10.860]  Six lines of code, we have this runaway system.
[22:11.580 --> 22:12.330]  What did it do?
[22:12.330 --> 22:14.360]  It created a silly situation.
[22:14.590 --> 22:17.260]  I mean, if you need this book, I guess it's really bad for you.
[22:17.440 --> 22:22.020]  But in general, I mean, this didn't really hurt anybody.
[22:22.120 --> 22:24.590]  But it's easy for this kind of thing to happen.
[22:25.120 --> 22:26.640]  These are simple rule-based systems.
[22:26.830 --> 22:32.700]  Machine learning tends to kick out stuff that can potentially be described as rules, but is very complex and not often interpretable.
[22:34.420 --> 22:36.680]  There's an interesting situation called the Hathaway effect.
[22:36.680 --> 22:44.380]  The claim is that social media and news and even search volume about Anne Hathaway has affected the trading price of Berkshire Hathaway stocks.
[22:44.660 --> 22:47.120]  Now, this is interesting because it's intuitively plausible.
[22:47.380 --> 22:54.550]  It seems reasonable that someone working on Wall Street would say, hey, we're going to scan social media and sentiment and tie that into some bidding strategy.
[22:54.860 --> 23:04.860]  So, until one day when, at the premiere of Les Miserables, there was a rumor that Anne, how do I put this lightly, had neglected to wear a particular garment.
[23:04.860 --> 23:08.940]  And this generated a lot of search traffic, particularly for images.
[23:10.400 --> 23:13.540]  And at the same time, the Berkshire Hathaway stock went way up.
[23:13.740 --> 23:15.900]  So, that's the Hathaway effect.
[23:16.220 --> 23:18.220]  Yet, let's be a little skeptical.
[23:18.220 --> 23:29.620]  The part that is not often quoted is that the same day Berkshire Hathaway announced that they were going to buy back some stock, which is generally a sign that they think it's undervalued and also they're going to go buy it so it's worth more.
[23:30.050 --> 23:32.360]  Perhaps it was Anne's faux pas.
[23:32.360 --> 23:36.460]  I don't know if that was a rumor or if it was true, but rumor alone is maybe enough.
[23:36.590 --> 23:40.140]  Perhaps we could flood social media and confuse these trading algorithms.
[23:40.260 --> 23:41.280]  That would be an interesting hack.
[23:42.440 --> 23:51.760]  But I don't want to take for granted that it's as powerful of a hack as one might have you believe, because this is probably the real reason why the stock shot up at the same time.
[23:51.980 --> 23:53.200]  But plausible, interesting.
[23:54.100 --> 23:55.360]  Spam filter is a good example.
[23:55.440 --> 23:56.500]  And this is sort of a loose timeline.
[23:56.860 --> 24:00.920]  I'm sure the first people who worked on spam just said, well, let's make a blacklist of words.
[24:01.020 --> 24:04.420]  Let's say if the word Viagra or free shipping comes up, label it as spam.
[24:04.660 --> 24:07.220]  If the word breast appears, let's label it as spam.
[24:07.420 --> 24:08.940]  And unfortunately, Susan G.
[24:09.050 --> 24:13.380]  Komen won't get her messages through, but I guess it'll filter out a lot of spam.
[24:13.570 --> 24:14.620]  But how do you hack that?
[24:14.720 --> 24:16.480]  You misspell words, you adjust the content.
[24:16.880 --> 24:18.360]  Okay, well, can we IP blacklist?
[24:18.480 --> 24:18.700]  Yes.
[24:18.900 --> 24:20.380]  But now we can turn in botnets.
[24:21.380 --> 24:22.680]  Let's introduce machine learning.
[24:22.790 --> 24:24.310]  In my opinion, the correct way to do this.
[24:24.310 --> 24:34.280]  Early on, the naive Bayes classifier, which we'll look at momentarily, was used to great success to filter spam, promoting a new technique called Bayesian poisoning that we'll also look at.
[24:34.620 --> 24:40.180]  Going back the other way, you could hide your spammy message in an image, and then the text doesn't work, so we go to OCR.
[24:41.090 --> 24:42.830]  Similarly, we can look at behavioral features.
[24:42.920 --> 24:45.460]  Do people open it, click on it, but we can hack the callback.
[24:45.590 --> 24:55.040]  So there's this constant cat-and-mouse game, and I think it's healthy that people should be maybe trying to look for exploits, and in so doing, strengthening the systems.
[24:55.810 --> 24:57.440]  So where is ML at work?
[24:57.440 --> 24:58.330]  Here's a long list.
[24:58.440 --> 25:00.810]  I'm not going to read through for time, but it's everywhere.
[25:01.000 --> 25:03.700]  Machine learning is in literally every part of technology now.
[25:04.020 --> 25:08.050]  So my call to action is, please, try and hack these models.
[25:08.220 --> 25:09.680]  I think that's overall healthy.
[25:09.920 --> 25:11.040]  We need watchdogs.
[25:11.420 --> 25:23.440]  And hackers I've come to know really enjoy the joy of finding things out, and there's a lot to be found out in ML, and a lot to be learned by studying these models and how they work, and the underlying patterns they're actually discovering and exploiting.
[25:23.700 --> 25:28.360]  So if you get into this, you can learn a useful skill, but you can also improve the machine learning community as a whole.
[25:30.240 --> 25:31.590]  So what does it mean to hack?
[25:31.760 --> 25:35.760]  Well, to me, I would say it's when you go in trying to manipulate that system.
[25:35.960 --> 25:38.020]  So can you get it to apply the wrong label?
[25:38.160 --> 25:41.790]  Maybe you were disqualified for a loan, and you get that mark disqualified.
[25:42.980 --> 25:47.160]  You could put a person in a different cluster than where they might otherwise be categorized.
[25:47.260 --> 25:49.260]  You could bias the output of a regression.
[25:49.500 --> 25:57.310]  You could make some sort of inefficient, force the system to make inefficient or suboptimal choices, maybe by introducing a lot of noise so that the patterns are obscured.
[25:57.570 --> 26:05.460]  And you can just overall weaken the usefulness of certain observable variables by polluting them, if you can contribute to the data it's trained on.
[26:05.660 --> 26:09.260]  So how would hacking machine learning be similar to other forms of hacking?
[26:09.260 --> 26:14.660]  Well, at its fundamental level, yeah, I think hacking is about taking stuff apart, figuring out how it works.
[26:14.760 --> 26:16.000]  There's a lot of brute force in it.
[26:16.740 --> 26:20.880]  You may not have direct access to a system, which is true also in a lot of situations.
[26:21.550 --> 26:26.160]  But for me, it boils down to looking for oversights or mistakes made by model creators.
[26:26.220 --> 26:30.940]  Because as you'll see in some of my examples, there's nothing intrinsically wrong with machine learning.
[26:30.940 --> 26:39.360]  It's actually really robust approaches and pretty mature techniques, so long as one applies them all well and is really responsible with how they're done.
[26:39.810 --> 26:45.360]  And it's someone with the skills of a hacker who might be able to find ways around that and exploit it.
[26:46.790 --> 26:51.050]  So that's not even to include the fact that machine learning could be a tool for a hacker.
[26:52.170 --> 26:57.090]  So other similarities, I would say, is that ultimately we're asking, how did the creator build the system?
[26:57.220 --> 26:58.900]  What design choices did they make?
[26:59.020 --> 27:04.960]  And then thinking through what those might mean about the way the system operates and how can we get the system to reveal its own weaknesses?
[27:05.740 --> 27:08.790]  So differences from other things we would typically consider hacking.
[27:10.460 --> 27:12.810]  Inspecting the model parameters, not the source code.
[27:12.980 --> 27:19.920]  So if you can get source, you can comb through it and say, hey, is there a chance for some sort of buffer overload attack or different types of things you might find.
[27:20.870 --> 27:24.050]  The output of a machine learning algorithm is generally a model.
[27:24.240 --> 27:29.700]  So it's sort of something that's processed and is encoding a set of rules or steps or weights.
[27:29.960 --> 27:32.160]  And it's not always clear how they were arrived at.
[27:32.420 --> 27:36.160]  So inspecting those model parameters and saying, how can I manipulate the system?
[27:36.280 --> 27:39.260]  It's just a little bit different from sort of looking through source code for exploits.
[27:40.160 --> 27:44.020]  Also, machine learning systems frequently use something called online learning.
[27:44.140 --> 27:44.780]  So they're adaptive.
[27:44.780 --> 27:48.380]  So even as you find an exploit, the system could be self-correcting.
[27:48.540 --> 27:49.920]  This is not for the faint of heart.
[27:50.040 --> 27:51.480]  These are challenging problems to solve.
[27:51.620 --> 27:53.640]  And I hope people will take some of them up.
[27:54.480 --> 27:56.820]  Also, the inner workings are a little bit more obfuscated.
[27:57.380 --> 27:59.680]  It's that sort of interpretability example I gave earlier.
[27:59.920 --> 28:01.720]  It's not always clear how these systems work.
[28:01.810 --> 28:10.180]  And in fact, we're entering an era when a lot of the really advanced stuff, especially in image recognition, the people that build those systems can't tell you precisely how they work.
[28:10.260 --> 28:11.070]  They just know that they do.
[28:12.460 --> 28:14.400]  There's a lot of other people in this space.
[28:14.400 --> 28:22.460]  I had the opportunity to interview the first author here, Christian Sandvig, on a paper, this paper, under the general topic of auditing algorithms.
[28:22.740 --> 28:29.180]  These are five proposals this team is putting forward for how we should be auditing algorithms.
[28:29.180 --> 28:31.360]  I think their work is very important and should be looked at.
[28:31.360 --> 28:39.330]  The two I specifically want to look at in this talk are around the code audits and the sock puppet audits.
[28:39.500 --> 28:48.120]  Because while scraping is interesting and a cool topic that I think a lot of people would hope would like, it's a little bit outside of what I consider actually hacking of a machine learning algorithm.
[28:48.280 --> 28:49.700]  Just interesting techniques nonetheless, though.
[28:50.680 --> 28:52.540]  So I want to discuss it in two pieces.
[28:52.760 --> 28:56.260]  First, when you have the ability to directly interact with the model.
[28:56.360 --> 28:59.590]  Maybe you have it in hand or you have some instance of it built up.
[29:00.380 --> 29:03.420]  It's actually a lot easier then versus when you have limited access.
[29:03.570 --> 29:05.220]  So I'll talk to these two things in separate parts.
[29:05.480 --> 29:13.260]  So given that, let's say, there's some algorithm, you're able to get a hold of it, a copy of it, or, you know, there's an existing software package you can stand up an instance of.
[29:13.460 --> 29:14.780]  You can do a code audit.
[29:15.420 --> 29:19.550]  But again, it's the source code and the model code are not necessarily the same thing.
[29:19.810 --> 29:24.360]  What generally is probably going to be the most effective thing to do is assume you have a black box.
[29:24.460 --> 29:27.680]  You have a machine learning system that outputs some result.
[29:27.900 --> 29:31.070]  And what you want to consider is the input features that go into it.
[29:31.200 --> 29:32.260]  What is it looking at?
[29:32.330 --> 29:33.020]  What is it monitoring?
[29:33.160 --> 29:36.760]  Whether that's network traffic or logs or different types of data going in.
[29:36.760 --> 29:40.830]  And seeing how changes here can affect this output.
[29:41.090 --> 29:44.520]  And maybe you can learn what's called a decision boundary that I'll talk about more in a second.
[29:45.200 --> 29:46.540]  Basically, the idea is this, you know.
[29:46.700 --> 29:51.860]  This guy, he's denied, changes his shirt, and apparently coordinates the back of his chair as well.
[29:52.090 --> 29:53.020]  Still getting denied.
[29:53.830 --> 29:56.180]  Give up on that, changes his hair, suddenly he's approved.
[29:56.400 --> 30:01.880]  So that's, in some sense, the moral of what I think in terms of trying to hack through the feature space.
[30:02.430 --> 30:05.570]  So most algorithms are built with this concept of a decision boundary.
[30:05.570 --> 30:10.440]  So, if we have black cases and white cases as two classes.
[30:10.640 --> 30:13.570]  Maybe this is fraud and not fraud, or whatever you're studying.
[30:14.460 --> 30:17.070]  And generally, you don't have a problem that's this clean.
[30:17.200 --> 30:18.380]  These are generally very mixed up.
[30:18.480 --> 30:23.290]  But what you'd like to define is some sort of decision boundary, some separation between these two.
[30:23.480 --> 30:26.280]  So, for example, H3 here is one decision boundary.
[30:26.420 --> 30:30.830]  That everything to the left is probably considered in class A, everything to the right in class B.
[30:30.830 --> 30:35.590]  But, of course, this is making a lot of mistakes because it's putting some white and some black in the same group.
[30:35.720 --> 30:36.680]  So we can rule this one out.
[30:37.220 --> 30:38.720]  H1 actually looks pretty good.
[30:38.920 --> 30:40.400]  It perfectly separates the data.
[30:40.590 --> 30:43.830]  But there's something intrinsically unpleasant about it.
[30:43.900 --> 30:49.400]  Because what ultimately we want is not just to get a system that describes the data we know about.
[30:49.400 --> 30:55.940]  But when we have a new data point come in, let's say one that's right here, is that going to be considered a black point or a white point?
[30:56.020 --> 30:56.720]  What would we predict?
[30:57.090 --> 30:58.780]  Well, by this model, we'd predict white.
[30:58.980 --> 31:00.380]  By this model, we'd predict black.
[31:00.740 --> 31:02.980]  This one seems to separate the data better.
[31:03.640 --> 31:05.460]  I won't get into the technicalities of why.
[31:05.570 --> 31:09.790]  I would just say appeal to your visual sense that this is probably the best separation of the data at hand.
[31:11.500 --> 31:12.880]  It doesn't always look this clean.
[31:12.880 --> 31:16.120]  Here's a more common example where data is very much commingled.
[31:16.660 --> 31:25.310]  And a more sophisticated algorithm, in this case something called a support vector machine, has to draw that boundary in a much more complex thing, which is actually drawing it in a higher dimensional space.
[31:25.500 --> 31:27.550]  And this is just sort of a two-dimensional representation of it.
[31:27.780 --> 31:31.480]  So inside of most machine learning algorithms, it's all these decision boundaries.
[31:31.680 --> 31:32.980]  So where they are, we might not know.
[31:33.070 --> 31:36.200]  But can we find them is the interesting question in terms of trying to hack them.
[31:36.290 --> 31:42.330]  So if you can manipulate the feature vectors, that's the things that go into it, the inputs.
[31:42.330 --> 31:45.790]  We can maybe explore and find out where are the decision points.
[31:46.040 --> 31:53.960]  So, you know, if I want to get a loan and they're going to look at my savings account balance, what's the exact minimum dollar amount I need to have in there to get the result I want.
[31:54.500 --> 31:56.570]  Now there's something we call the curse of dimensionality.
[31:56.980 --> 32:06.400]  Essentially it means that if you're looking at a wide enough data set that has a wide enough cardinality, you cannot brute force attack and explore the whole space.
[32:06.520 --> 32:07.260]  So you have to be smart.
[32:07.460 --> 32:10.310]  You have to look for ways that you can explore that space intuitively.
[32:10.310 --> 32:15.260]  I think a lot of that is figuring out the way that the machine learning people think.
[32:15.440 --> 32:16.960]  So I want to share some ideas about that.
[32:17.500 --> 32:25.640]  But you can also apply techniques like gradient descent or think about the prior information and domain knowledge that the person who built that system brings to the table.
[32:25.790 --> 32:32.660]  So what if we were managing an online forum, let's say, and we wanted to enforce a private community.
[32:32.660 --> 32:39.020]  So not censorship exactly, but not let people harass each other or not let people dox each other in the forum.
[32:39.180 --> 32:41.900]  And if someone tried to dox, take that post down as soon as possible.
[32:42.360 --> 32:50.260]  So you would look at every post as soon as you can and label it as potentially violating your terms or not.
[32:50.830 --> 32:53.880]  So it's either in class A or B, acceptable or unacceptable.
[32:54.140 --> 32:55.550]  That doesn't make it 50-50.
[32:55.740 --> 32:58.360]  In fact, I would assume most of your posts are probably a good one.
[32:58.460 --> 33:01.000]  It's the minority cases that you're trying to track down.
[33:02.590 --> 33:04.280]  So a little bit more on feature space.
[33:04.550 --> 33:08.000]  I tend to think that there's this spectrum of what we work on.
[33:08.360 --> 33:12.380]  In things like fraud detection, you have to be very clever about what can we look at?
[33:12.480 --> 33:15.570]  What are the indicators that might be there?
[33:15.570 --> 33:19.620]  You know about some of the stuff that goes into this, like have you shopped at that merchant before?
[33:19.810 --> 33:21.160]  Is it a certain distance from your home?
[33:21.460 --> 33:26.440]  These are things that are sort of intuitive and have to be well-crafted in a hand-picked fashion.
[33:26.940 --> 33:29.830]  There's the other extreme that I call the kitchen sink feature set.
[33:29.960 --> 33:36.160]  Best example being in computer vision where you take every pixel, often it's grayscale value, and that's your feature.
[33:36.380 --> 33:40.290]  Sort of meaningless in a vacuum, but can these machines train on that data?
[33:40.420 --> 33:42.440]  And surprisingly, they can do that very, very well.
[33:43.050 --> 33:45.260]  Much of that is due to advancements in deep learning.
[33:46.790 --> 33:47.440]  Let's see.
[33:48.130 --> 33:53.550]  So there's another concept I want to introduce called adversarial machine learning.
[33:54.140 --> 33:54.600]  This is a...
[33:55.110 --> 34:00.660]  I guess it's an old idea, but it's becoming much more popular, especially as we try and build up the more massive systems.
[34:01.310 --> 34:03.570]  It's that you have one system that's the discriminator.
[34:03.700 --> 34:06.540]  It's trying to pick out the classes and label the data.
[34:06.740 --> 34:08.280]  Another system that's the generator.
[34:08.280 --> 34:13.440]  It's trying to create or find new examples that are most likely to confuse the discriminator.
[34:14.040 --> 34:18.020]  So discriminator is optimizing to tell the difference between real and fake.
[34:18.240 --> 34:21.540]  And the generative model is optimizing to better fool the discriminator.
[34:21.640 --> 34:23.980]  And it's sort of the arms race of adversarial machine learning.
[34:24.320 --> 34:27.340]  And you end up with kind of fun situations like this.
[34:28.050 --> 34:29.300]  Is it a dog or not?
[34:29.400 --> 34:30.040]  That's the objective.
[34:30.380 --> 34:32.140]  What will most confuse your system?
[34:32.140 --> 34:33.640]  Well, I guess maybe bagels.
[34:36.980 --> 34:40.900]  Similarly, a lot of this, I guess, starts from this great paper that came out a little while back.
[34:41.000 --> 34:42.690]  Intriguing properties of neural networks.
[34:42.860 --> 34:46.540]  They built an image recognition system that recognizes this is a car.
[34:46.540 --> 34:47.110]  I agree.
[34:47.420 --> 34:49.150]  It labels this is not a car.
[34:49.300 --> 34:49.880]  I disagree.
[34:50.110 --> 34:51.020]  It labels this is a car.
[34:51.210 --> 34:52.150]  I don't agree.
[34:52.590 --> 35:04.740]  They did this by essentially looking at the way the machine learning system was running in the background and saying, how can we tweak and fine tune and slightly change our inputs to trick this system?
[35:05.900 --> 35:07.800]  Here's a maybe more fun example of that.
[35:08.170 --> 35:08.630]  Panda, right?
[35:08.710 --> 35:09.240]  We all agree.
[35:09.780 --> 35:11.420]  This is just some random noise.
[35:11.550 --> 35:14.980]  Now, if we put that down at a low threshold, we can still see the panda.
[35:15.090 --> 35:16.400]  Maybe not even see the noise.
[35:16.670 --> 35:18.590]  But what does the machine learning system see?
[35:18.820 --> 35:19.500]  A gibbon.
[35:22.020 --> 35:26.960]  Now, you could also argue, as some people have, well, this is just sort of a nonsense like corner case.
[35:26.960 --> 35:27.780]  This isn't real.
[35:28.040 --> 35:29.650]  This is just some white noise.
[35:30.420 --> 35:36.260]  Could you do this with sort of introducing brand new images that confuse the system?
[35:36.540 --> 35:37.540]  And indeed you can.
[35:38.020 --> 35:43.280]  A follow-up paper, deep neural networks are easily fooled by a good team.
[35:43.440 --> 35:45.740]  I had the pleasure of interviewing Ang Nguyen about this work.
[35:46.280 --> 35:52.190]  These are two images that their system creates and they get labeled as an African gray parrot and a guitar, respectively.
[35:53.090 --> 35:54.130]  And like, yeah.
[35:54.360 --> 35:56.820]  Like, that's art to me, actually.
[35:57.000 --> 36:01.260]  I mean, I don't know if you guys are familiar with Lady Lovelace's objection, but I think it's out the window now.
[36:01.460 --> 36:03.380]  Like, it's not just white noise.
[36:03.400 --> 36:04.960]  They're creating something really interesting here.
[36:06.340 --> 36:12.480]  So, let's move on to the case where you don't, as probably as more often, you don't have access to the model.
[36:12.630 --> 36:19.130]  The police are using some machine learning and they're not going to share their research, probably for good reason, or at least they're well-intentioned in doing that.
[36:19.260 --> 36:21.710]  But can you interact with that system from the outside?
[36:22.320 --> 36:26.670]  Can you reverse engineer its decision boundaries, maybe change the inputs to it in some way?
[36:26.840 --> 36:28.340]  More or less a sock puppet attack.
[36:28.540 --> 36:29.900]  So, these are some of my favorite tools.
[36:30.300 --> 36:31.400]  This would be a great workshop.
[36:31.860 --> 36:34.300]  I'd love to show people how to do some of this stuff if you see me around.
[36:35.050 --> 36:36.610]  Most of these are Python libraries.
[36:36.840 --> 36:37.820]  Scrapey is for crawling.
[36:37.980 --> 36:40.110]  Request is a simpler version to get single pages.
[36:40.610 --> 36:44.880]  BeautifulSoup is this great library for easily extracting data from HTML.
[36:46.170 --> 36:47.380]  Selenium automates your browser.
[36:48.020 --> 36:52.260]  I think most of you know about in Chrome, you can get the developer console to see what the network traffic was.
[36:52.460 --> 36:57.040]  Oftentimes, you can see that there's some API returning a JSON format that has the data that's going on the page.
[36:57.480 --> 37:04.590]  These are great tools for if you're interacting with a website, finding out what data is there, and maybe learning something about what's going on behind the scenes.
[37:05.860 --> 37:10.590]  If you want to hack machine learning, you're going to have to think like a machine learning researcher, and this is our process.
[37:11.280 --> 37:13.170]  So, first, we start with data exploration.
[37:13.480 --> 37:17.280]  I won't get into too much of this for time, but basically, we try and look through that whole data set.
[37:17.440 --> 37:20.260]  It's generally impossible, because there's too many variables and observations.
[37:20.460 --> 37:23.880]  But we try and look at summary statistics, get a high-level understanding of what's there.
[37:23.920 --> 37:25.610]  We look at outliers, things like that.
[37:26.240 --> 37:28.820]  We go through and do a lot of data cleaning, as I mentioned before.
[37:30.130 --> 37:30.740]  This is an example...
[37:30.740 --> 37:31.280]  Did I skip?
[37:31.480 --> 37:31.590]  Yeah.
[37:31.740 --> 37:31.960]  Oh, yeah.
[37:32.110 --> 37:35.070]  So, yeah, you guys, you did that loyalty card swap-up stuff.
[37:35.170 --> 37:38.840]  Is there one of those here at HOPE today, where you don't want to be tracked, so you switch loyalty cards?
[37:38.840 --> 37:40.710]  Here's what happens when you do that.
[37:41.550 --> 37:48.050]  If your buying habits are significantly different from what appears normal, you look weird in the data set.
[37:48.400 --> 37:51.240]  If I was working on that problem, I'd throw your data out.
[37:51.420 --> 37:54.880]  So, if you'd like to be anonymized, congratulations, the swap will do that.
[37:55.040 --> 37:56.190]  If you want to bias...
[37:56.190 --> 37:57.050]  We'll likely do that.
[37:57.170 --> 38:03.800]  If you want to bias my data set, it's probably not going to happen, because if you're an outlier, I want to throw you out of my data set.
[38:03.880 --> 38:06.860]  I actually might go look at you to specifically figure out why you're weird.
[38:07.440 --> 38:08.420]  Here's a fun story.
[38:09.130 --> 38:10.020]  I was...
[38:10.020 --> 38:11.570]  How do I make this simple and quick?
[38:12.240 --> 38:20.800]  Looking at the relationship at a company I was working at between the amount of users we had and what you'd expect by population in the corresponding counties.
[38:21.070 --> 38:22.480]  So, I sorted by counties.
[38:22.480 --> 38:23.940]  That's why you see this increasing.
[38:24.520 --> 38:27.460]  And our numbers were sort of noisy around that.
[38:27.590 --> 38:28.190]  Makes sense, right?
[38:28.300 --> 38:29.360]  You wouldn't perfectly match.
[38:29.480 --> 38:30.760]  We're a little bit higher or lower in some places.
[38:30.760 --> 38:33.440]  But I noticed these two outliers, so I thought, what the heck is that?
[38:33.710 --> 38:34.880]  Why are we so...
[38:34.880 --> 38:39.210]  I know it's sort of upside down, but it meant we were over-representative in...
[38:39.210 --> 38:42.240]  One of these is Beverly Hills, one is Poughkeepsie, New York.
[38:42.480 --> 38:46.380]  I said, why on earth would we be different in Beverly Hills and Poughkeepsie?
[38:46.520 --> 38:50.550]  Until I found out that we got this county-level data by aggregating zip code data.
[38:50.940 --> 38:57.920]  Beverly Hills, of course, having the famous zip code 90210, and Poughkeepsie being the area having the zip code 12345.
[38:59.750 --> 39:03.880]  So, I removed those two studies from my analysis.
[39:04.240 --> 39:07.170]  They were not included, and I found my result in my model without them.
[39:08.570 --> 39:10.690]  This is a more simplified version of that.
[39:11.190 --> 39:15.920]  Clearly, the data set here, the relationship between some x and some y, is this linear fashion.
[39:16.070 --> 39:21.040]  If I include this data point, this is the model that a typical linear regression will find.
[39:21.190 --> 39:26.300]  I don't like that because it doesn't properly recommend the pattern I see in the data, so I remove this.
[39:26.300 --> 39:28.740]  So, if you're too weird, you're going to get yourself thrown out.
[39:28.900 --> 39:30.130]  Maybe that's what you want, maybe it's not.
[39:30.570 --> 39:32.380]  Similar story with this type of plot.
[39:32.880 --> 39:34.300]  We worry about overfitting.
[39:34.550 --> 39:44.900]  If the black dots are the actual data, we would generally think this is a linear system, that the relationship between x and y is just standard sort of y equals mx plus b type math.
[39:45.150 --> 39:52.400]  The reason we want to not overfit is because if we get a new data point, say something right in here, we want to predict the y value from the new x.
[39:52.980 --> 39:56.540]  And it seems like this line is a better predictor than this line.
[39:56.800 --> 40:01.240]  Even though it has made no mistakes fitting this data, it's not going to generalize very well.
[40:01.360 --> 40:02.650]  So, we worry about this sort of stuff.
[40:02.820 --> 40:05.320]  That's something to consider if you want to try and hack our models.
[40:06.460 --> 40:07.960]  I'm going to skip this one for time.
[40:08.480 --> 40:11.150]  And lastly, just return to the idea that we have an objective function.
[40:11.150 --> 40:13.780]  We're always trying to get some high level accuracy.
[40:14.050 --> 40:16.190]  And there are compromises to be made along the way.
[40:16.400 --> 40:19.260]  So, this is the way I'm thinking about a system I'm developing.
[40:19.570 --> 40:22.280]  If you want to outsmart me, you should probably think about it in the same way.
[40:23.170 --> 40:25.070]  So, Bayesian poisoning that I mentioned earlier.
[40:25.300 --> 40:30.260]  The way in which you might try and attack a spam filter using the Naive Bayes algorithm.
[40:30.690 --> 40:33.150]  So, the word prince can appear in an email.
[40:33.540 --> 40:37.650]  If it's preceded by Nigerian prince, it seems like it might be spam.
[40:37.880 --> 40:39.540]  But aren't there actually Nigerian princes?
[40:39.540 --> 40:42.260]  That must be really hard for them to ever communicate with anybody.
[40:43.790 --> 40:47.110]  But also, you know, we recently lost a great artist named Prince.
[40:47.360 --> 40:50.360]  And it shouldn't necessarily be spam to write an email about that.
[40:50.550 --> 40:53.940]  So, we look at this statistically with... this is Bayes' theorem.
[40:54.110 --> 40:58.670]  So, the prior likelihood that something is spam, probably pretty low.
[40:58.900 --> 41:05.590]  The likelihood that it's spam given that the word prince appears, divided by the likelihood of the word prince appearing, gives us this, which is what we want.
[41:05.690 --> 41:09.820]  That's the basic statistics behind the sort of old school way of doing this sort of filtering.
[41:10.050 --> 41:11.840]  Much more sophisticated techniques in place now.
[41:12.040 --> 41:15.900]  But Bayesian poisoning... oh yeah, and then we'd kind of just take the product of all those.
[41:16.000 --> 41:19.150]  This is essentially watered down the Naive Bayes algorithm.
[41:20.300 --> 41:21.920]  This is the idea of Bayesian poisoning.
[41:22.110 --> 41:27.130]  Take an otherwise good email, add in some garbage stuff to it that will make...
[41:27.130 --> 41:35.880]  If this is labeled as an email that's not spam, adding something that makes it look a little bit like spam makes it harder to detect using the frequency of these words in the future.
[41:35.880 --> 41:38.340]  Or similarly, this is something I considered spam.
[41:38.480 --> 41:40.690]  It was a snippet from an email I got that I didn't want.
[41:41.050 --> 41:46.240]  And this I appended a nice email I got from my nephew, which I definitely want and I don't want labeled as spam.
[41:46.400 --> 41:52.000]  So, if you were to append this and provide that as the data set, that's essentially, in a nutshell, Bayesian poisoning.
[41:52.670 --> 41:54.020]  So, along these lines, real quick...
[41:54.610 --> 41:56.860]  Actually, no, I'm going to skip ahead because I'm running a little low on time.
[41:57.420 --> 41:58.460]  Let's talk about regression.
[41:58.690 --> 42:02.170]  So, the basic idea is you have some data, some relationship you want to predict.
[42:02.170 --> 42:03.980]  What's the x values?
[42:04.070 --> 42:05.570]  What's their relationship to the y values?
[42:06.090 --> 42:08.380]  That is a model, not a very good one.
[42:08.500 --> 42:08.960]  Why not?
[42:09.240 --> 42:13.820]  Because it has a large amount of error from the actual data I have available.
[42:14.070 --> 42:19.630]  This is a much better model because when I look at the difference between what the model would predict and what I actually found is very small.
[42:19.860 --> 42:21.920]  This is the core idea of most machine learning.
[42:22.400 --> 42:24.070]  And this is how we express it mathematically.
[42:24.400 --> 42:25.500]  We go through each point.
[42:25.610 --> 42:26.170]  We sum over them.
[42:26.260 --> 42:29.300]  We say, what is it actually minus what does our model predict?
[42:29.300 --> 42:33.760]  We want to find the values, the coefficients that minimize this.
[42:34.610 --> 42:37.780]  So, and just to be clear, the i is sort of like the i-th prediction.
[42:39.380 --> 42:46.760]  Yeah, so then this is a fancy way of saying find the coefficients that satisfy this the most, which is probably something like this.
[42:46.960 --> 42:49.480]  If you want to know how to do that, learn a little bit about machine learning.
[42:49.540 --> 42:50.280]  I can give you some pointers.
[42:50.780 --> 42:52.940]  Just trust that you could do that without brute forcing.
[42:53.050 --> 42:54.040]  It's actually pretty easy to do.
[42:55.110 --> 42:56.650]  Here's a more real-world data set.
[42:56.650 --> 43:00.540]  This is some data I had that looks at information about cars.
[43:00.740 --> 43:03.340]  So what's the relationship between horsepower and the weight of the car?
[43:03.540 --> 43:05.380]  It seems to have this linear relationship.
[43:05.670 --> 43:11.920]  And these are outliers I should have thrown out because they don't seem to be relevant that there would be a zero horsepower car.
[43:12.020 --> 43:15.920]  I guess that's easy to build, but not exactly the relationship I'm trying to look for.
[43:15.980 --> 43:16.960]  I should have thrown those out.
[43:17.050 --> 43:18.500]  So they're biasing my model.
[43:19.240 --> 43:20.610]  Okay, so they bias it.
[43:20.800 --> 43:23.760]  Could I poison this data set if I could somehow contribute something?
[43:23.760 --> 43:24.520]  Well, sure.
[43:24.900 --> 43:26.960]  I added about a hundred new points.
[43:27.130 --> 43:30.240]  The old relationship, this one, is these are the optimal values.
[43:30.420 --> 43:36.070]  When I add in all these green values, I get a new fit because I've poisoned it and that I've changed the output.
[43:36.300 --> 43:40.650]  But to do that, I had to add an extra hundred points to put in that much of a skew.
[43:40.940 --> 43:44.480]  What if I added just two points very high above?
[43:44.740 --> 43:48.840]  I still can skew it by only adding two new interesting data points to the data set.
[43:48.840 --> 43:51.380]  But of course, these are pretty extreme.
[43:51.780 --> 43:59.210]  A good machine learning researcher would have noticed and removed these probably, except that often we're working in very high dimensional space, not two dimensional space like this.
[43:59.340 --> 44:00.640]  So we could miss something.
[44:00.840 --> 44:02.920]  If we've missed something, there's a chance to exploit it.
[44:04.120 --> 44:05.060]  Same data here.
[44:05.640 --> 44:07.140]  Some pretty extreme values.
[44:07.360 --> 44:09.440]  I gave you this model so you could see where they are.
[44:09.560 --> 44:12.020]  I trimmed them off because they'd be somewhere up, you know, in the sky.
[44:12.560 --> 44:14.210]  This is how much I can bias it.
[44:14.360 --> 44:16.160]  This is the correct model I'd like.
[44:16.160 --> 44:20.200]  This is the one I was able to generate by just adding in two fake points.
[44:20.900 --> 44:26.680]  Now, generally, this is not going to be a successful attack because they're such extreme outliers, I would have removed them.
[44:26.960 --> 44:39.310]  But also there's a process called regularization, where more or less you add in some penalty that says, not only do you want to minimize the residuals, the error here, but you also want to penalize for any values that are too extreme.
[44:40.020 --> 44:41.940]  And there's different types of regularization.
[44:41.940 --> 44:53.500]  One is called Ridge, which says, essentially, bring me a model that uses the least number of parameters, tries to not rely on too many types of observations, or types of variables.
[44:53.780 --> 44:58.080]  And another called Lasso, which says, try and make them not too extreme value.
[44:58.600 --> 45:03.140]  There are many types of regularizations, but these are the two probably most common ones.
[45:04.040 --> 45:06.140]  Most people are using these and they're in place.
[45:06.240 --> 45:10.480]  So even if we missed a situation like this, regularization might catch it.
[45:10.480 --> 45:13.180]  So machine learning people generally do a really good job.
[45:13.360 --> 45:14.740]  It's not going to be an easy thing to hack.
[45:14.920 --> 45:19.260]  But let's talk about a quick demo I put together that might include one.
[45:19.500 --> 45:23.540]  I made these short and simple because I knew I was going to run long.
[45:23.900 --> 45:25.160]  Hopefully they'll be interesting enough.
[45:25.400 --> 45:29.100]  I'm going to use the Jupyter Notebook, which is a great way to interact with code.
[45:29.160 --> 45:31.420]  And I'm going to use a site called Algorithmia.
[45:31.640 --> 45:33.120]  I like Algorithmia a lot.
[45:33.210 --> 45:33.700]  It's really cool.
[45:33.820 --> 45:35.710]  It's a place where anybody can upload an algorithm.
[45:35.710 --> 45:38.420]  They've designed a model and then we can consume it.
[45:38.420 --> 45:40.140]  You can charge for it if you'd like.
[45:40.500 --> 45:42.800]  And it's kind of this sort of like models as a service.
[45:42.940 --> 45:46.500]  So if I come up with some cool thing and I want to share it with people, this is a great way to do it.
[45:46.580 --> 45:48.700]  So I'm not trying to pick on them or say their site's not good.
[45:48.880 --> 45:54.000]  I really like what they're up to, but I'm going to pick on one of their algorithms just to say like I hacked it here today.
[45:54.160 --> 45:57.400]  Alright, so just some first loads.
[45:57.620 --> 46:00.100]  I'm going to grab my creds from my file.
[46:00.380 --> 46:01.300]  So here's an example.
[46:01.380 --> 46:03.180]  This is a sentiment analysis algorithm.
[46:03.180 --> 46:04.160]  Here are three inputs.
[46:04.160 --> 46:06.040]  I absolutely love the HOPE conference.
[46:06.180 --> 46:07.620]  It's the most fun I've had all in months.
[46:07.880 --> 46:10.560]  That we assume would be given a positive sentiment score.
[46:10.880 --> 46:15.660]  Radio Statler broadcasts from the Hotel Pennsylvania doing HOPE, maybe neutral, sort of positive.
[46:16.260 --> 46:18.480]  I hate the government because they spy on my email.
[46:18.600 --> 46:19.060]  It's the worst.
[46:19.200 --> 46:23.420]  We assume that would have a negative sentiment, at least if this is a good sentiment algorithm.
[46:23.420 --> 46:30.440]  So I'm going to run those through the Algorithmia system, and now let's plot those scores.
[46:30.900 --> 46:35.180]  So here we have, I hate the government, it's the worst, as a very negative sentiment.
[46:36.000 --> 46:38.600]  Radio Statler, this statement, very neutral, which I agree with.
[46:38.710 --> 46:39.520]  And this is very positive.
[46:39.700 --> 46:43.840]  So this is the objective of that algorithm, to take free text and tell you the sentiment of it.
[46:43.940 --> 46:44.580]  It seems to be working.
[46:46.100 --> 46:46.900]  Alright, so...
[46:46.900 --> 46:47.820]  Three minutes, sorry.
[46:47.820 --> 46:47.920]  Alright.
[46:49.210 --> 46:54.060]  So, let's just see if we can break down and understand what exactly is going on.
[46:54.160 --> 46:58.360]  I would do that, in this case, by looking at some specific examples.
[46:58.560 --> 47:00.360]  So, what do I get if I say, I hate this?
[47:01.230 --> 47:02.040]  Negative .75.
[47:02.280 --> 47:03.140]  How about, I like you.
[47:03.240 --> 47:04.300]  Seems like that should be positive.
[47:04.520 --> 47:04.710]  Zero.
[47:04.940 --> 47:05.400]  Huh, weird.
[47:05.840 --> 47:08.300]  Seems like the algorithm could use some more training data.
[47:08.640 --> 47:10.480]  What if I have an expletive?
[47:11.120 --> 47:12.560]  Oh, it's positive.
[47:12.580 --> 47:13.520]  That's a little weird.
[47:14.100 --> 47:14.500]  Alright.
[47:15.640 --> 47:17.120]  Hi, credit card provider.
[47:17.460 --> 47:18.960]  Here's something I'm complaining about.
[47:21.140 --> 47:22.120]  I run that.
[47:22.260 --> 47:22.540]  Negative.
[47:22.780 --> 47:24.100]  Okay, so my complaint is negative.
[47:24.420 --> 47:25.540]  Ah, so here's an interesting use case.
[47:25.640 --> 47:31.440]  What if I thought my credit card processor was going to prioritize my complaint based on how angry I was?
[47:31.560 --> 47:33.900]  Might I want to politely make it look more...
[47:33.900 --> 47:34.840]  Because, yeah, I'm a polite person.
[47:34.900 --> 47:36.260]  I don't want to just type the F word in there.
[47:36.360 --> 47:39.820]  But maybe make it seem like it's a higher sentiment if I think they're prioritizing.
[47:40.060 --> 47:44.540]  So, let's just say I have my email there, which I think is fairly, you know, cordial.
[47:44.780 --> 47:46.260]  And then I say, I hate this at the end.
[47:46.260 --> 47:50.560]  I went from having a slightly negative to a very negative example.
[47:50.920 --> 47:51.660]  Okay, that makes sense.
[47:51.880 --> 47:54.440]  What if I now put on this expletive?
[47:54.820 --> 48:02.860]  Well, I went from slightly negative to actually slightly positive, suspiciously in the amount we saw earlier.
[48:02.860 --> 48:06.260]  So, what happens if I take the difference of those two?
[48:06.710 --> 48:10.740]  I get exactly the score of the term alone.
[48:10.900 --> 48:11.620]  You see what's going on here?
[48:11.760 --> 48:17.640]  So, it's thinking that this phrase, whether it's this word or the word you, is positive by this amount.
[48:18.000 --> 48:25.200]  My overall score for that email, when I include this word, minus the score when I don't, is equal to it.
[48:25.200 --> 48:29.980]  So, it seems to me I figured out that this is looking in a very insensitive way to context.
[48:30.200 --> 48:31.210]  It's looking at just words.
[48:31.480 --> 48:33.800]  So, I've just kind of reverse engineered how they're doing this.
[48:33.900 --> 48:37.240]  They're looking at individual words and I guess summing up their, each score.
[48:37.420 --> 48:39.440]  So, room for improvement on that algorithm.
[48:39.860 --> 48:41.540]  But that's what's cool about algorithmia.
[48:41.680 --> 48:44.380]  I could follow up with the creator of that, maybe make a suggestion.
[48:44.380 --> 48:46.280]  If it's on GitHub, send a pull request.
[48:46.460 --> 48:46.780]  Who knows?
[48:47.560 --> 48:51.840]  Getting back to it, since I've got to wind up here, I'll skip my last example.
[48:51.960 --> 48:54.040]  But it's on GitHub, if anyone wants to check it out.
[48:57.990 --> 48:58.350]  Sure.
[49:00.630 --> 49:03.010]  So, this is going to be for my other example I had to skip.
[49:03.110 --> 49:03.990]  So, just some quick reminders.
[49:04.230 --> 49:06.970]  If you've got direct access to a model, maybe you can reverse engineer.
[49:07.210 --> 49:12.690]  If you have partial access, you can try and, through trial and error, figure out something about the system like we did there.
[49:12.690 --> 49:17.210]  If you want to directly attack it, maybe you can introduce bias, especially if you add noise to the data set.
[49:17.810 --> 49:22.150]  Why, again, discriminatory, not the nice kind, is going on.
[49:22.410 --> 49:24.450]  The machine learning community can definitely benefit from challenges.
[49:24.850 --> 49:26.370]  Same reasons why everybody hacks.
[49:26.670 --> 49:28.550]  I don't know how to do bug bounties with this stuff.
[49:28.650 --> 49:29.530]  I'd love to hear feedback.
[49:30.310 --> 49:30.950]  Hack responsibly.
[49:31.010 --> 49:31.750]  Don't do anything illegal.
[49:32.010 --> 49:34.290]  But I want to start a discussion on this.
[49:34.390 --> 49:34.910]  So, that's me.
[49:35.050 --> 49:36.190]  If you want to check out my podcast.
[49:36.410 --> 49:40.270]  Also, if you're into these hex stickers that are showing up on laptops, I'd love to give you one for that.
[49:40.430 --> 49:41.150]  So, thanks.
[49:41.250 --> 49:41.670]  This is my time.