[00:01.210 --> 00:04.450]  All right, so this presentation is near and dear to me.
[00:05.130 --> 00:09.870]  Software testing is a big part of software development, and this is focused entirely on software testing.
[00:11.070 --> 00:15.570]  It's oriented towards college students, but I know there's some seasoned professionals in here.
[00:15.710 --> 00:21.970]  So what I did was I tried to sprinkle a few things for an intermediate, handful of things that advanced.
[00:22.190 --> 00:35.730]  So my goal is that everybody in this room and online will hopefully take away something, even if you've probably seen 80% of this content, because this is basically best practice on software testing.
[00:36.330 --> 00:38.130]  Inside here, I have little things sprinkled.
[00:38.250 --> 00:47.090]  I call them code breakers, where it's basically, if you're a software tester and you add this to your test set, you're basically going to guarantee to break the code.
[00:47.290 --> 00:50.390]  Then you can take that and complain to developers and say, hey, what's up?
[00:51.010 --> 00:52.170]  So that's code breakers.
[00:52.310 --> 00:57.030]  We'll start with what I call the classical approach to software testing and development.
[00:57.610 --> 01:00.310]  It's probably what most of y'all do in college.
[01:00.310 --> 01:04.170]  until you learn, you learn, you know, the ways of the pipeline and you get inspired.
[01:04.510 --> 01:09.790]  So then I'll go into the pipeline development and then I'll show you a few little tricks that you can do.
[01:10.810 --> 01:14.110]  Who here has ever committed a syntax error?
[01:15.110 --> 01:17.750]  When I'm done with this presentation, you'll never do that again.
[01:18.210 --> 01:19.230]  I'll show you how.
[01:20.050 --> 01:22.050]  All right, let's get this going.
[01:23.170 --> 01:24.090]  It's about me.
[01:25.230 --> 01:26.550]  I'm just going to breeze through this.
[01:26.670 --> 01:28.890]  If you want to find out more about me, you can look online.
[01:29.190 --> 01:32.810]  HOPE has a nice little blurb, but I've been doing this for 20 years.
[01:32.990 --> 01:34.250]  I've written two patents.
[01:34.490 --> 01:36.170]  I've presented 30 conferences.
[01:36.490 --> 01:41.810]  I invented a semi-famous open-source project that gets used everywhere.
[01:42.350 --> 01:44.570]  So I've done a lot of stuff.
[01:44.890 --> 01:45.250]  Look me up.
[01:45.370 --> 01:46.510]  I'm mostly on GitHub.
[01:46.930 --> 01:51.670]  I don't do much social networking, but I am on LinkedIn because in my industry, you basically have to be.
[01:51.830 --> 01:54.170]  So just go there if you want to look up more about me.
[01:54.170 --> 01:58.710]  And we are at the first code breaking section.
[01:59.170 --> 02:00.570]  Near and dear to my heart.
[02:00.770 --> 02:02.710]  Now, question for you all.
[02:03.890 --> 02:04.910]  What time is this?
[02:08.360 --> 02:11.880]  So I admit I struggle with 24 hour time.
[02:13.040 --> 02:17.080]  It's a thing like once you take calculus, you forget how to do arithmetic.
[02:17.480 --> 02:22.180]  So I have to kind of do a little minus 12 to get into my mind about what's going on here.
[02:22.340 --> 02:24.640]  But it's a 24 hour time, military time.
[02:24.780 --> 02:25.980]  That is hours.
[02:26.120 --> 02:28.520]  Minutes, seconds, minutes, seconds, 23, 59, 60.
[02:28.680 --> 02:29.680]  I want to know what time that is.
[02:31.680 --> 02:32.120]  Okay.
[02:32.420 --> 02:33.580]  So a couple of choices.
[02:35.540 --> 02:37.060]  This is midnight, right?
[02:37.540 --> 02:45.180]  So the 60, you know, that set that to zero, take, carry the one that becomes 60 or becomes a 24.
[02:45.480 --> 02:48.600]  It's 2,400 hours, which is actually zero hours, which is midnight.
[02:49.220 --> 02:49.600]  Midnight.
[02:49.780 --> 02:50.600]  It says it's midnight.
[02:50.760 --> 02:51.340]  This is just midnight.
[02:51.540 --> 02:53.180]  It's midnight with extra steps.
[02:53.480 --> 02:54.520]  Basically, that's what it is.
[02:54.680 --> 02:55.460]  So we got here.
[02:56.120 --> 02:57.000]  Midnight with extra steps.
[02:57.260 --> 02:57.580]  All right.
[02:58.400 --> 02:58.820]  Other choice.
[02:59.340 --> 03:00.220]  It's invalid.
[03:00.600 --> 03:02.340]  Like this timestamp does not exist.
[03:02.980 --> 03:04.760]  That 60 should have never happened.
[03:05.080 --> 03:07.820]  Like it should have gone 59 to zero.
[03:08.740 --> 03:11.240]  And it's an about timestamp.
[03:11.540 --> 03:23.140]  So whatever system this was fed to should have flagged as an error and basically gone into whatever failure, recovery, retry mechanism that's appropriate, whatever system.
[03:23.140 --> 03:23.820]  I don't know.
[03:23.960 --> 03:25.580]  Maybe, maybe you toss it out completely.
[03:26.020 --> 03:29.580]  Maybe you go and request a refetch or something.
[03:29.940 --> 03:30.440]  Invalid.
[03:30.500 --> 03:31.020]  This is invalid.
[03:31.220 --> 03:32.980]  We go into some kind of failure mode.
[03:33.540 --> 03:34.080]  Votes for that.
[03:34.500 --> 03:35.200]  Votes for that.
[03:38.160 --> 03:39.540]  Both answers are wrong.
[03:41.280 --> 03:47.240]  What if I told you this is a 100% valid timestamp?
[03:47.440 --> 03:48.320]  Would you believe me?
[03:49.960 --> 03:50.760]  It's true.
[03:52.280 --> 04:01.440]  So 10 years ago, the NTP servers network time protocol, that's what keeps our clocks in sync.
[04:02.260 --> 04:07.820]  The governing body for it, I forget the name of it, declared a leap second.
[04:08.040 --> 04:08.880]  That's what this is.
[04:09.020 --> 04:09.880]  It is a leap second.
[04:10.580 --> 04:11.040]  Yes.
[04:11.800 --> 04:13.820]  It happened 10 years ago about this time.
[04:14.140 --> 04:20.560]  The NTP servers fed that down to the Internet, this timestamp, and broke like 2,000 websites.
[04:21.660 --> 04:22.740]  I remember that.
[04:23.060 --> 04:25.200]  Because they didn't know what to do with that.
[04:25.400 --> 04:28.140]  So either they threw it out or they didn't know what to do.
[04:28.320 --> 04:31.300]  So this is an official timestamp because it came from the NTP server.
[04:31.520 --> 04:33.920]  That's the source of truth for all our time.
[04:33.920 --> 04:45.300]  Now, since then, Google has now stood up their own NTP service, and they promise, this is their promise, if there's ever a leap second, they won't do this.
[04:45.660 --> 04:50.080]  They'll instead take that extra second and smear it across 12 hours.
[04:50.260 --> 04:57.320]  So the bonus second will basically be two milliseconds off until it all catches up onto the next day.
[04:57.320 --> 04:57.320]  Okay.
[04:57.700 --> 05:10.560]  So if you're a test engineer or a developer, or you're building your test set, and you're working with time, which happens all the time, take that, put it into your test set, and see what happens.
[05:10.800 --> 05:15.540]  And when the system breaks, which it will, you can go to the developers and say, hey, what's up?
[05:15.680 --> 05:17.840]  And they'll say, that timestamp's invalid.
[05:18.040 --> 05:19.900]  And you'll say, no, it isn't.
[05:20.160 --> 05:23.520]  Or you can say, it depends what NTP service they're using.
[05:23.520 --> 05:29.540]  So anyway, that's the first code breaker to put into your back pocket, if you want to watch everything break.
[05:29.880 --> 05:30.860]  So get that in mind.
[05:31.180 --> 05:35.760]  Let's get to the classical approach of software development.
[05:37.280 --> 05:40.860]  I used to call this the wrong way, but that's not right.
[05:41.020 --> 05:49.600]  Because, I mean, millions and tons of very successful software projects use this technique very successfully.
[05:49.600 --> 05:54.760]  And, but there's problems with it, as we'll see in a minute.
[05:55.140 --> 06:03.760]  So, the way it's typically done without, you know, if you're not inspired, and the ideas of the pipeline, is that you write the code.
[06:04.660 --> 06:09.200]  You test the code, write the code, test code, and it's good.
[06:09.400 --> 06:17.560]  So like a bug comes in, you replicate it on your desk, write, test, write, test, the bug is now gone.
[06:17.560 --> 06:23.480]  You fixed it, you create a patch, write, and then you toss it over, and the QA department catches it.
[06:23.720 --> 06:27.800]  And what they usually do is, they have the previous version running.
[06:28.660 --> 06:36.060]  And the bug is this, they install your patch, the bug goes away, everything's good, finished, ship it.
[06:38.440 --> 06:40.060]  But, there's a problem here.
[06:41.580 --> 06:42.460]  What's the problem?
[06:43.800 --> 06:45.380]  The bug comes back.
[06:46.920 --> 06:55.760]  How many, I know, I know you're a seasoned professional, and there's a few of you in here, whoever has ever heard the phrase, I thought we fixed that.
[06:57.560 --> 06:59.480]  Right, right, right, right.
[06:59.480 --> 07:03.900]  So, my friends, there, there is a better way.
[07:04.760 --> 07:06.200]  We call it unit testing.
[07:08.300 --> 07:14.440]  The way unit testing works, the ideas behind this, you, you flip the script, basically.
[07:14.440 --> 07:17.960]  So, what you do is, a bug comes in.
[07:18.160 --> 07:28.660]  Instead of just replicating the bug, you write a test, a unit test, that stimulates the features in your code to replicate the bug in code.
[07:29.240 --> 07:35.100]  And then once your unit test is failing, because the bug's still there, you iterate on your code.
[07:35.100 --> 07:42.060]  So, you code, test against unit test, code test, code test, unit test, it's passing, the code is fixed.
[07:42.420 --> 07:49.100]  And then what you do is, you check in both the code and the test, the two go together, into the repo.
[07:49.380 --> 07:56.820]  And the idea is, is that once the bug is fixed, because the test exists alongside of it, it stays fixed.
[07:57.440 --> 08:04.740]  And then that now gets hooked in to all your regression testing, versus before, you would have to do all that manually.
[08:05.100 --> 08:06.700]  So, that's the idea, to find it.
[08:07.220 --> 08:08.420]  Fix the bug once.
[08:08.640 --> 08:10.440]  And with the test, it stays fixed.
[08:10.620 --> 08:13.240]  Now, I'm going to show you an example here, real quick.
[08:14.220 --> 08:17.420]  This is a very simple example that works with Python.
[08:17.800 --> 08:20.380]  You can start using this right now.
[08:21.340 --> 08:26.120]  So, fans of Python, I love Python.
[08:26.940 --> 08:30.160]  Throughout this demonstration, I'll be using Python.
[08:30.460 --> 08:33.980]  And it's more because it's just a very nice, easy to use language.
[08:35.100 --> 08:41.500]  All right, so, see here, no, you can't see this.
[08:42.660 --> 08:44.280]  You absolutely cannot see this.
[08:46.880 --> 08:48.280]  Well, this is going to be tricky.
[08:49.840 --> 08:52.840]  Okay, I did not anticipate looking over here.
[08:53.140 --> 08:58.520]  All right, so, what we have over here is, this is the way I like to develop the code.
[08:58.960 --> 09:03.060]  Particularly with Python, because it's unique quality, it supports this kind of development.
[09:03.060 --> 09:09.340]  And what you basically do is, up at the top, you write whatever your function that you're working on.
[09:09.520 --> 09:10.900]  Python, we call these modules.
[09:11.460 --> 09:18.180]  If you're familiar with Java, classes, C++ classes, Python's modules, it's all basically the same stuff.
[09:19.200 --> 09:23.060]  And up at the top, you have whatever module function you're working on.
[09:23.060 --> 09:25.460]  So here, I'm summing up a list, right?
[09:26.740 --> 09:31.500]  A list goes in, add them all up, and then I'll put the result.
[09:31.760 --> 09:32.600]  Very simple.
[09:33.280 --> 09:36.340]  Now, below that, I have your unit test.
[09:37.180 --> 09:44.320]  I have a test set, 1, 2, 3, 4, that's my answer, and I sum it all up, and I output the answer.
[09:44.720 --> 09:56.740]  So, effectively, what you're doing for every single module, or every single class, if it's Java, C++, or what have you, up at the top, you define it, down at the bottom, you test it.
[09:56.980 --> 09:58.840]  And eventually, you'll get a collection of these.
[09:58.840 --> 10:00.740]  So, let's run this and see what it looks like.
[10:05.420 --> 10:10.220]  I'm just going to duplicate my displays, because this will be a whole lot easier.
[10:18.450 --> 10:18.950]  All right.
[10:19.050 --> 10:19.830]  Am I on both now?
[10:20.130 --> 10:20.590]  All right.
[10:20.630 --> 10:20.970]  Very good.
[10:21.230 --> 10:21.950]  This will be a lot easier.
[10:22.570 --> 10:22.890]  All right.
[10:23.250 --> 10:29.330]  So, Python, I'm going to test to you, nope, wrong directory.
[10:34.740 --> 10:42.800]  So, as we expect, you know, now, every module, go to the top, test to the bottom.
[10:43.000 --> 10:44.300]  Now, I can see what you're getting at.
[10:44.480 --> 10:46.040]  I can feel it.
[10:48.080 --> 10:52.180]  You're like, but Dan, I've got like 200 classes and 200 modules.
[10:52.380 --> 10:56.700]  I can't simply just go through and run, you know, the main routine on every single one of these.
[10:56.880 --> 11:00.040]  That's highly inefficient, and you're absolutely correct.
[11:00.760 --> 11:05.920]  That's why we have a test framework, and for Python, the one I like to use is called PyTest.
[11:06.620 --> 11:07.980]  I love PyTest.
[11:07.980 --> 11:09.400]  And that looks like this.
[11:10.660 --> 11:20.320]  So, once you outgrow code at the top, main routine at the bottom that you run, you start hooking it all together with a framework.
[11:20.640 --> 11:22.240]  And for this, I use PyTest.
[11:22.240 --> 11:23.580]  And here's what it looks like.
[11:24.300 --> 11:27.860]  So, you can see here, I imported unit testing.
[11:28.020 --> 11:29.020]  That's this file right here.
[11:29.240 --> 11:31.420]  And I've imported some list.
[11:33.240 --> 11:37.760]  And then, if I had more stuff, like maybe I'm building a client and server.
[11:37.760 --> 11:42.500]  I would do from my class, import client, import server.
[11:43.140 --> 11:50.780]  And then, instantialize all of these and just start banging them together to find all the flaws and test it out.
[11:50.960 --> 11:52.240]  So, here's my unit test.
[11:52.420 --> 11:56.060]  These are all the various tests that you might be interested in if you're going to sum up a list.
[11:56.660 --> 11:58.100]  Test all the positive integers.
[11:58.660 --> 12:00.100]  Test all the negative integers.
[12:00.100 --> 12:01.700]  Test an empty set.
[12:01.860 --> 12:02.400]  Right here.
[12:02.820 --> 12:04.120]  Always test the empty set.
[12:05.020 --> 12:06.000]  Test a single limit.
[12:06.360 --> 12:09.960]  Test max set, which I have commented out because time.
[12:11.880 --> 12:17.200]  But you would have that in there because this would be running and you wouldn't be too concerned about time.
[12:17.760 --> 12:19.260]  I'm going to throw garbage into it.
[12:19.580 --> 12:20.940]  Be sure you test garbage.
[12:21.440 --> 12:29.180]  See, the problem is, it's like maybe I don't support inputting garbage, but there's a workflow to where the user might be able to submit garbage.
[12:29.740 --> 12:33.620]  And you at least want to know that you will handle that in the proper way.
[12:33.860 --> 12:40.080]  And it is possible, like with all these tests, that you'll say, hey, I broke the system.
[12:40.900 --> 12:44.000]  And the response would be, we don't support that.
[12:44.180 --> 12:46.820]  And maybe that's okay.
[12:47.140 --> 12:50.780]  Like, it's possible that this is just something you're not going to support.
[12:50.780 --> 12:58.600]  And if it pops up, you'll handle it as is because it's just too much resources, too much rework, too much refactoring to support it.
[12:58.600 --> 13:01.620]  But you at least need to know that the problem exists.
[13:01.920 --> 13:02.960]  And that's what this would be doing.
[13:04.460 --> 13:07.120]  Also, make sure you test none or nil.
[13:07.500 --> 13:08.860]  That will always break you.
[13:09.060 --> 13:10.520]  And I have that test in here.
[13:13.040 --> 13:15.740]  For you Python fans, who have used PyTest before?
[13:17.800 --> 13:19.440]  Yeah, get very familiar with PyTest.
[13:19.560 --> 13:21.840]  It has other unique qualities, which I'll show you in a bit.
[13:23.680 --> 13:25.320]  So we'll see what this looks like.
[13:29.630 --> 13:34.150]  All right, it ran all my tests, had a very nice report.
[13:34.570 --> 13:37.950]  And they can just scoop it all up and let you know something fails.
[13:37.950 --> 13:39.290]  It's beautiful.
[13:42.600 --> 13:48.460]  Now, I got to do Command Shift F5.
[13:50.120 --> 13:51.120]  There we go.
[13:52.080 --> 13:52.460]  All right.
[13:52.940 --> 13:53.880]  Next code breaker.
[13:56.340 --> 13:58.040]  Y'all struggled with the previous one.
[14:02.220 --> 14:03.460]  How often do we have a leap here?
[14:10.140 --> 14:13.800]  At least a handful of you are on the right track.
[14:14.540 --> 14:19.500]  So, it's not every four years.
[14:20.580 --> 14:32.460]  So, if you come across a developer who is writing their own calendar system, first, just go ahead and smack them right there.
[14:36.880 --> 14:40.360]  And then feed this data set into them.
[14:40.400 --> 14:41.500]  I'm going to show you something.
[14:42.360 --> 14:43.860]  It's not every four years.
[14:47.080 --> 14:49.560]  So, hopefully you can see my calendar down here.
[14:51.020 --> 14:52.980]  You can sort of see it.
[14:53.260 --> 14:54.160]  I'll narrow it.
[14:54.520 --> 15:01.040]  So, let's scroll to the year 2100 and see what we got right here.
[15:01.760 --> 15:02.320]  All right.
[15:02.660 --> 15:03.580]  Go to February.
[15:04.680 --> 15:04.820]  Oh!
[15:05.080 --> 15:05.200]  Oh!
[15:05.900 --> 15:06.820]  My leap year is gone.
[15:07.280 --> 15:07.920]  That's weird.
[15:09.160 --> 15:10.860]  Like I already said, I'm bad at arithmetic.
[15:11.160 --> 15:12.540]  Like, so let's add four years.
[15:13.220 --> 15:15.220]  Like, I mean, I can't do arithmetic anymore.
[15:16.120 --> 15:17.140]  I just can't.
[15:17.340 --> 15:18.180]  So, 2104.
[15:18.300 --> 15:19.240]  Pretty sure that's plus four.
[15:21.700 --> 15:22.140]  Oh!
[15:22.340 --> 15:23.280]  Leap year came back.
[15:23.440 --> 15:24.080]  Oh my God.
[15:24.500 --> 15:25.280]  That's amazing.
[15:26.720 --> 15:28.660]  The actual math is thus so.
[15:29.490 --> 15:33.500]  It's every four years, unless it's divisible by 100.
[15:34.200 --> 15:37.480]  Unless that is divisible by 400.
[15:38.720 --> 15:42.780]  So, year 1900, no leap year.
[15:43.000 --> 15:45.400]  Year 2000, yes, leap year.
[15:45.840 --> 15:49.900]  That was a once every 400 year event, by the way.
[15:50.620 --> 15:54.040]  And year 2100, no leap year.
[15:54.040 --> 15:59.640]  Because the problem is, is that we don't revolve around the sun every 365 and a quarter days.
[15:59.640 --> 16:03.600]  It's more like 365.224.
[16:03.600 --> 16:05.460]  I think that's pretty close.
[16:05.860 --> 16:09.200]  So, every 100 years, we're off by a day.
[16:09.700 --> 16:11.200]  I mean, that's the Julian calendar.
[16:12.000 --> 16:13.280]  Thanks to Julius Caesar.
[16:13.560 --> 16:14.700]  Yes, that's Julius Caesar.
[16:15.040 --> 16:20.120]  And then about 1500 years later, we're now off by like two weeks.
[16:20.320 --> 16:24.780]  And then Pope, I think it was Greg, the Gregorian calendar made the leap year tweak.
[16:24.780 --> 16:28.060]  So, every 100 years, we drop a day.
[16:28.200 --> 16:33.400]  But every 400 years, you know, to get that 0.4, you know, 0.24, we add a leap year.
[16:33.560 --> 16:35.340]  So, our calendar should be good.
[16:35.340 --> 16:39.160]  We'll be off by a day after 3000 years with that one tweak.
[16:39.680 --> 16:41.480]  But that's all nice.
[16:41.780 --> 16:47.300]  But feed the year 2100 February into your data set.
[16:47.420 --> 16:51.040]  If you notice, an engineer has attempted their own calendar.
[16:51.760 --> 17:00.200]  Now, with that said, it is 100% accurate response that they will reply.
[17:01.100 --> 17:02.860]  You know, Dan, that's great.
[17:03.200 --> 17:05.880]  But I don't care about 75 years from now.
[17:06.960 --> 17:08.840]  And that may be a valid response.
[17:09.000 --> 17:14.700]  Like, this system, you don't expect, you know, to care 75 years from now.
[17:14.940 --> 17:18.600]  You're like, okay, modulo four it is.
[17:18.780 --> 17:20.380]  And you're good to go.
[17:20.380 --> 17:21.760]  So, that could be a response.
[17:22.400 --> 17:26.100]  Typically, what this will break is, like, finance systems.
[17:26.340 --> 17:29.700]  You know, like, I think 529 college funds or things.
[17:29.840 --> 17:32.380]  Like, things that extend out, you know, pretty far.
[17:32.820 --> 17:38.100]  Already, the Y2K 38 year 2038 problem.
[17:38.320 --> 17:39.580]  Y'all can look that up.
[17:39.940 --> 17:42.180]  That's already breaking some mortgage calculators.
[17:42.260 --> 17:44.580]  Because that's now, you know, less than 30 years from now.
[17:44.580 --> 17:52.280]  But this is a fun one to shove in there because, like, you don't really think about that kind of stuff.
[17:52.460 --> 18:01.020]  And since the year 2000 was the surprise bonus leap year, then it's not, you know, people assume that it's always going to be every four years.
[18:01.020 --> 18:04.520]  So, that's the code breaker number two.
[18:04.960 --> 18:14.820]  Now, if you're upset that I had two time-based code breakers in here, I'm not going to apologize.
[18:16.260 --> 18:22.460]  We could have an entire conference dedicated just to problems with time.
[18:24.580 --> 18:25.560]  Another example.
[18:27.140 --> 18:28.480]  How many months are in a year?
[18:29.820 --> 18:31.660]  I like how everybody's quiet.
[18:34.860 --> 18:36.300]  It's not always 12.
[18:36.660 --> 18:37.460]  It's not.
[18:38.580 --> 18:41.200]  Any of you ever worked in finance or accounting?
[18:42.040 --> 18:42.980]  A little bit?
[18:42.980 --> 18:47.420]  Okay, so it's rare, but it exists.
[18:47.680 --> 18:51.320]  But over there, sometimes it's a 13-month calendar.
[18:52.400 --> 19:03.260]  So, what it is, like, accounting and finance people, they like everything to be nice and clean and even and make sense and no oddities and bounce between 28, 30, and 31.
[19:03.620 --> 19:04.360]  They hate that.
[19:04.520 --> 19:08.940]  So, like, so they'll take a 13-month calendar with 28 days, weeks.
[19:08.940 --> 19:24.460]  So, that's four weeks, you know, all the months and days and weeks are the same because, I mean, 13 times, I forgot your math, 13 times 28, that's 364, you know, very clean and even.
[19:24.620 --> 19:32.180]  And then they have one bonus day, you know, for reconciliation or what have you, or maybe they actually toss in caffeine with their half-calf.
[19:32.340 --> 19:33.400]  I don't know what they do on that day.
[19:33.640 --> 19:34.480]  Probably pretty wild.
[19:35.600 --> 19:39.840]  Now, you all, as a developer, what are you going to do with that bonus day?
[19:40.800 --> 19:41.700]  Where are you going to put it?
[19:42.740 --> 19:45.640]  That's, that's, it's all fun stuff.
[19:46.220 --> 19:46.560]  So, yeah.
[19:47.460 --> 19:49.560]  13-month calendar and finance, it's pretty wild.
[19:50.940 --> 19:51.360]  All right.
[19:54.080 --> 19:55.280]  Other frameworks.
[20:00.630 --> 20:02.370]  Oh, let's see, I did it wrong.
[20:03.330 --> 20:04.130]  Here we go.
[20:05.210 --> 20:06.650]  We saw a pie test.
[20:06.850 --> 20:07.330]  There are others.
[20:07.590 --> 20:14.470]  What I recommend is that if you, whatever language you're working in, that is the framework.
[20:14.470 --> 20:17.610]  I recommend that you use to at least get started.
[20:18.350 --> 20:26.490]  They, every major language, every, you know, big environment typically has a framework associated with it to help your unit testing.
[20:28.070 --> 20:29.990]  JUnit, that's for, for Java.
[20:30.290 --> 20:34.330]  And I think Kotlin, it's been a minute since I've worked on Android.
[20:34.710 --> 20:38.530]  But I think Kotlin and Android student are still using a variant of JUnit.
[20:38.890 --> 20:41.150]  Mocha is for all you Node.js people.
[20:41.530 --> 20:43.870]  And they recently added support to TypeScript.
[20:44.450 --> 20:51.270]  If you're still using JavaScript instead of TypeScript, I recommend jumping to TypeScript as soon as possible.
[20:51.470 --> 20:54.050]  It'll make development a whole lot happier.
[20:55.070 --> 20:56.430]  PHP unit.
[20:56.790 --> 21:02.290]  I know PHP gets a blowback, but they've dramatically improved in like just the past five years.
[21:02.490 --> 21:09.610]  If you're working on a WordPress site or Drupal or Laravel or something of that type, PHP units, really nice.
[21:10.290 --> 21:21.770]  Google test is what I use for plain C or C++ for anything that typically doesn't have, you know, a, you know, a C or C++ framework library to go with it.
[21:21.770 --> 21:26.290]  That's my go-to XC test for iOS development.
[21:26.610 --> 21:34.950]  I think as of recently, Swift test has now completely supplanted XC test.
[21:34.950 --> 21:39.070]  I'm not entirely sure, but that one's been around further.
[21:39.130 --> 21:40.950]  And we saw PyTest.
[21:41.130 --> 21:42.450]  So just keep those in mind.
[21:42.710 --> 21:56.110]  What I really like about PyTest is that it hooks into all the other stuff because I referenced this earlier, but what's better than a full end-to-end system test?
[21:57.250 --> 22:01.450]  A full end-to-end system test that is automated.
[22:03.490 --> 22:10.930]  Now, it is painful to go through and just manually run your test.
[22:10.930 --> 22:15.770]  Instead, what you want to do is you have another computer running it.
[22:15.970 --> 22:23.270]  So you write your code, you commit it, you push it, and then all those series of tests just run automatically.
[22:25.150 --> 22:27.990]  These are the big ones that I want you to keep in mind.
[22:28.810 --> 22:34.570]  Azure DevOps, if you're in a Microsoft shop, it hooks in really well with that.
[22:35.690 --> 22:37.970]  Jenkins, I see Jenkins all over the place.
[22:38.190 --> 22:39.270]  It is really nice.
[22:39.270 --> 22:40.110]  I love Jenkins.
[22:40.370 --> 22:41.790]  Good general purpose one.
[22:42.310 --> 22:43.570]  CICD for GitLab.
[22:43.830 --> 22:45.970]  That's the one that I'm most familiar with.
[22:46.130 --> 22:49.310]  I work in a lot of secure environments.
[22:49.390 --> 22:51.750]  So we can't host our stuff in the cloud.
[22:51.950 --> 22:55.590]  So we love standing up our own private GitLab and Jenkins.
[22:55.870 --> 22:56.290]  That's how we do.
[22:57.210 --> 22:58.510]  Atlassian has Bamboo.
[22:59.010 --> 23:07.050]  Weirdly, I don't know why this is, but I see JIRA and Bitbucket all over the place, but they don't use Bamboo.
[23:07.790 --> 23:10.130]  Bamboo has been there for like a decade.
[23:13.100 --> 23:13.500]  Yeah.
[23:13.800 --> 23:14.900]  I mean, but it's really snug.
[23:15.380 --> 23:15.560]  I mean...
[23:15.560 --> 23:15.920]  Really?
[23:16.240 --> 23:17.500]  Well, that would explain it.
[23:18.800 --> 23:22.300]  I just wonder why, you know, we got JIRA, but not Bamboo.
[23:22.400 --> 23:22.920]  Well, there you go.
[23:24.680 --> 23:25.660]  That's what they call it.
[23:25.760 --> 23:26.240]  That's what they call it.
[23:26.240 --> 23:26.580]  Okay.
[23:27.400 --> 23:28.200]  GitHub Actions.
[23:28.760 --> 23:29.800]  Now, oddly, it's...
[23:29.800 --> 23:30.520]  I know it's odd.
[23:30.660 --> 23:32.440]  Like, Azure DevOps owned by Microsoft.
[23:32.680 --> 23:34.080]  GitHub Actions owned by Microsoft.
[23:34.260 --> 23:35.620]  And you can now connect it to.
[23:35.720 --> 23:37.680]  You can have your GitHub call Azure DevOps.
[23:37.900 --> 23:38.840]  You know, it's all wild.
[23:39.020 --> 23:40.360]  But anyway, it all works fine.
[23:40.600 --> 23:43.180]  You're missing different in the city, which...
[23:43.880 --> 23:45.700]  There's only so much room on the slide.
[23:50.820 --> 23:59.320]  Now, for all you aspiring developers and test engineers, since like, y'all like testing now, because I told you you actually can earn money in testing.
[23:59.600 --> 24:09.960]  If you're all aspiring, in your next interview, find a way to casually name drop one of these, and you'll get a job offer.
[24:11.400 --> 24:11.980]  It's true.
[24:12.360 --> 24:14.680]  They love hearing you talk about this kind of stuff.
[24:15.800 --> 24:16.760]  Let's see an example.
[24:17.080 --> 24:23.240]  I'll be using GitLab on top of Ubuntu, using PyTest, all free stuff that you can use right now.
[24:23.340 --> 24:29.060]  If you want to start adding pipelines to your own stuff, and you don't want to sign up for an account, you can do exactly what I'm doing right here.
[24:29.620 --> 24:30.400]  Let's see it.
[24:41.500 --> 24:42.680]  So, come on now.
[24:42.680 --> 24:43.280]  All right.
[24:43.460 --> 24:45.320]  So, I have my runner running.
[24:45.520 --> 24:47.840]  I have GitLab running in the background.
[24:48.840 --> 24:51.600]  This is a VMware Workstation Pro.
[24:52.420 --> 24:55.380]  I have a paid version because I actually use commercial development.
[24:55.660 --> 25:01.500]  But if you're a student or a hobbyist, they've now made this free.
[25:01.600 --> 25:03.580]  So, you can just download and start using it.
[25:03.680 --> 25:05.520]  I've been using it for like 15 years.
[25:05.660 --> 25:06.360]  It works perfectly fine.
[25:06.620 --> 25:09.040]  There's also VirtualBox and all kinds of other tools.
[25:09.640 --> 25:11.120]  And of course, Ubuntu is free.
[25:11.120 --> 25:13.080]  You just download, install, GitLab.
[25:13.280 --> 25:16.320]  They have a little helpful guide to get you going with the official version.
[25:16.560 --> 25:19.580]  You can download, you know, app git and all that good stuff.
[25:19.680 --> 25:21.160]  And you'll be up and running in no time.
[25:21.940 --> 25:24.640]  Start to finish, maybe an hour.
[25:24.980 --> 25:26.520]  And then you can get your pipeline going.
[25:30.640 --> 25:32.880]  So, I'm not going to submit the code.
[25:33.060 --> 25:35.800]  I'm just going to instead show you what it looks like.
[25:35.880 --> 25:36.960]  Because there's really not much to see.
[25:36.960 --> 25:39.220]  It's like it just appears and then done.
[25:39.420 --> 25:40.820]  And it takes like five seconds.
[25:40.960 --> 25:43.380]  If I had a more involved test to be more useful.
[25:44.300 --> 25:47.720]  So, here are code commits that I pushed when I was working on this.
[25:49.520 --> 25:50.560]  This one passed.
[25:53.780 --> 25:54.540]  There it is.
[25:57.220 --> 25:58.180]  Does that look familiar?
[25:58.920 --> 26:00.260]  That's what I ran on the command line.
[26:00.520 --> 26:03.040]  So, I pushed the code and it just did it all for me.
[26:03.040 --> 26:05.580]  Let's see what this, let's see what this looks like.
[26:09.660 --> 26:12.620]  And let's go back a couple.
[26:14.380 --> 26:16.520]  And it's called a git lab.
[26:17.620 --> 26:18.540]  There it is.
[26:20.800 --> 26:25.960]  So, with pipelines, what you do is you create stages.
[26:26.760 --> 26:30.300]  And so, this is my, you saw it earlier over here.
[26:30.540 --> 26:32.080]  This is my test stage.
[26:32.080 --> 26:33.260]  I called it Pied Test.
[26:33.380 --> 26:35.080]  And that's where that name came from.
[26:35.260 --> 26:40.460]  And you can just keep making more and more stages to get you going.
[26:40.920 --> 26:47.220]  So, you'll probably have like a compile stage to build everything.
[26:47.580 --> 26:50.780]  You'll have a test stage to test everything.
[26:50.980 --> 26:53.620]  They all flow into each other.
[26:53.760 --> 27:00.360]  And then if the test works, then you'll have like an integration or make an installer stage.
[27:00.360 --> 27:03.100]  You know, all the tests pass, make an installer.
[27:03.360 --> 27:09.500]  Then you'll have a deployment stage where you take it and you shove it to whatever service that you may deploy.
[27:09.700 --> 27:13.340]  Like me, I use Artifactory a lot to hold all my artifacts.
[27:13.640 --> 27:18.800]  But you may shove it up to some other cloud service to where the developers can go get it.
[27:19.040 --> 27:21.380]  So, and it's all done automatically.
[27:21.380 --> 27:27.540]  You just write the code, hit submit, and then computer in the background, just turns through it.
[27:27.720 --> 27:29.140]  And then it's ready.
[27:29.300 --> 27:30.400]  And you go and grab it.
[27:30.600 --> 27:37.520]  And the really nice part about this is that I don't recommend it because they get mad.
[27:37.520 --> 27:44.200]  You can basically just deploy thin clients to your developers because the backend is doing all the heavy lifting.
[27:44.420 --> 27:46.980]  They don't have to personally know how to compile the code.
[27:47.160 --> 27:48.440]  All they need is an editor.
[27:48.640 --> 27:49.800]  And they can just work.
[27:49.980 --> 27:52.560]  And the backend handles everything for you.
[27:52.740 --> 27:54.380]  So, I really like it.
[27:57.690 --> 27:58.670]  Next code breaker.
[27:58.870 --> 27:59.650]  It's not about time.
[27:59.770 --> 28:00.610]  I promise.
[28:00.810 --> 28:02.590]  I promise there's only two of those.
[28:06.270 --> 28:07.890]  I still went back to the beginning.
[28:08.950 --> 28:10.310]  My shift key's not going.
[28:12.690 --> 28:14.810]  Did you know you can have emoji file names?
[28:17.570 --> 28:18.690]  Because I didn't.
[28:20.710 --> 28:21.950]  What a horrible idea.
[28:25.590 --> 28:28.290]  Like a thing I didn't know I needed and I still don't need.
[28:31.690 --> 28:32.850]  I'm going to show you how to make one.
[28:42.680 --> 28:43.640]  I think so.
[28:43.780 --> 28:44.500]  I'm not entirely sure.
[28:45.320 --> 28:46.480]  I mean there's all kinds of problems.
[28:46.640 --> 28:49.600]  Like first of all, like it renders differently on every operating system.
[28:49.740 --> 28:51.580]  You know, because they all have their own implementation.
[28:51.960 --> 29:01.280]  But the way you make one is you create your new file, then you hold the Win key, the Windows key, and you hit dot.
[29:03.880 --> 29:05.220]  And there you go.
[29:06.800 --> 29:10.920]  And now we'll get a bunch of emojis going on over here.
[29:14.040 --> 29:17.060]  And you can already see that it renders kind of odd right here.
[29:28.090 --> 29:30.490]  So yeah, there we go.
[29:31.190 --> 29:32.090]  Emoji file name.
[29:32.350 --> 29:38.530]  Now, this already breaks some systems, as you can imagine.
[29:40.590 --> 29:42.070]  So, I'm going to show you.
[29:42.750 --> 29:44.270]  Let's open up my Git Bash up here.
[29:46.550 --> 29:48.890]  Oh, look at that pile of nastiness.
[29:50.630 --> 29:52.490]  Git Bash is already struggling with it.
[29:52.490 --> 29:55.650]  Now, to be fair to Git Bash, I think they did fix this.
[29:55.870 --> 29:59.350]  I mean, I'm talking fixed it as recently as maybe last year.
[29:59.590 --> 30:04.530]  But that's basically how it's encoded, you know, that pile of stuff.
[30:04.830 --> 30:17.010]  So, if you're at a system that accepts file uploads, which is everywhere, and you're a tester, create you an emoji file name and upload it and watch the system show.
[30:17.690 --> 30:22.310]  And you can go to the developer and say, hey, upload an emoji file name.
[30:22.490 --> 30:24.470]  They're like, why did you do that?
[30:24.570 --> 30:25.970]  You're like, I don't know, Gen Z.
[30:26.570 --> 30:27.130]  They'll probably do.
[30:30.410 --> 30:32.930]  You're like, upload your test, like, peace.
[30:34.850 --> 30:42.150]  The variation in that is, go into the product file and add some Hebrew or some other things.
[30:42.310 --> 30:47.790]  And then just save that to your material and find out which one of your warehouses are using older software.
[30:48.130 --> 30:49.210]  Yeah, there you go.
[30:49.490 --> 30:49.970]  Yeah.
[30:52.050 --> 30:57.490]  So, the way I typically handle this is I sanitize the inputs.
[30:57.710 --> 31:02.650]  Like, if it's like a character set that I don't want to support, I'll just change it to an underscore or a dash.
[31:02.830 --> 31:04.550]  Like, I've seen a lot of those happening.
[31:05.770 --> 31:10.910]  If you uploaded this to a system I've written a while back, probably the entire thing would be underscore.
[31:10.910 --> 31:14.830]  But, you need to be able to handle this.
[31:15.010 --> 31:20.190]  And it's quite possible that whatever system you're working on is not going to be able to handle it.
[31:20.470 --> 31:21.070]  So, yes.
[31:21.430 --> 31:22.810]  If you just normalize everything,
[31:26.460 --> 31:26.480]  collisions?
[31:27.080 --> 31:27.560]  Absolutely.
[31:28.400 --> 31:29.360]  Absolutely collisions.
[31:29.660 --> 31:34.840]  That's why you usually have a unique identifier associated with the file name that you map it to.
[31:35.620 --> 31:37.460]  So, good question.
[31:40.140 --> 31:41.680]  Now, emoji file names.
[31:42.120 --> 31:42.880]  Horrible idea.
[31:47.940 --> 31:51.840]  I'm not going to bother maximizing anymore because I messed that up three times.
[31:53.380 --> 31:55.580]  So, you can do more than just test.
[31:55.840 --> 31:59.900]  You can build, deploy, already went through all this.
[31:59.900 --> 32:08.680]  But, the other point I want to make a sense is like you can have like one main pipeline and it spits off, you know, jobs to other pipelines.
[32:08.940 --> 32:15.540]  So, you can have your Windows, Mac, Linux, iOS, all building in parallel on different runners.
[32:16.140 --> 32:23.480]  So, you can have like this build up, this full up system of compiling and testing all going on once with just a single push.
[32:23.660 --> 32:24.760]  So, keep that in mind.
[32:24.860 --> 32:29.020]  You just keep expanding your pipeline to all kinds of systems and save a lot of time.
[32:33.510 --> 32:34.730]  What's better than a pipeline?
[32:35.030 --> 32:35.970]  Or, not better than playing.
[32:36.050 --> 32:38.350]  What's a good way to help a pipeline?
[32:40.650 --> 32:42.450]  Not having to run the pipeline.
[32:42.890 --> 32:44.090]  I hinted at this earlier.
[32:44.430 --> 32:46.290]  Who's committed a syntax error before?
[32:47.670 --> 32:48.450]  Not anymore.
[32:48.790 --> 32:49.290]  Let me show you.
[32:51.870 --> 32:53.950]  You'll never have to commit a syntax error again.
[32:56.890 --> 32:59.530]  It's called a git hook.
[33:02.430 --> 33:04.970]  It's in your git directory under hooks.
[33:05.890 --> 33:07.590]  And I'm running low on time.
[33:07.770 --> 33:09.290]  So, I don't have too much time to explain it.
[33:09.370 --> 33:14.350]  But I want you to look up pre-commit hook whenever you get a chance.
[33:15.170 --> 33:24.110]  And what this does is every time I commit Python code, it checks to see which files have been modified.
[33:24.110 --> 33:28.010]  Then it runs the PyLint syntax checker against it.
[33:28.190 --> 33:30.990]  And if that passes, I'll let my commit in.
[33:31.110 --> 33:33.110]  If it doesn't, I'll reject it.
[33:33.350 --> 33:38.130]  So, I never personally have to commit a Python syntax error.
[33:38.910 --> 33:40.630]  So, let's see how it works.
[33:55.700 --> 33:56.400]  Right, right.
[33:56.480 --> 33:59.240]  There's all kinds of stuff you can shove into your hooks.
[33:59.880 --> 34:01.780]  This particular example is...
[34:01.780 --> 34:04.000]  All right, that looks like a syntax error to me.
[34:06.740 --> 34:09.120]  And let me try to commit it.
[34:09.660 --> 34:21.780]  So, git status, git commit, call this oops, running PyLint, rejected.
[34:22.840 --> 34:27.220]  I just saved a pipeline run because it would not pass syntax errors.
[34:27.520 --> 34:35.020]  Now, you can add more stuff in there, but I like to just stick with syntax errors because you don't want to do git commit and basically waiting for a whole Python run.
[34:35.120 --> 34:36.160]  That kind of defeats the purpose.
[34:36.160 --> 34:40.240]  But a simple thing like, is this at least syntactically correct?
[34:40.440 --> 34:47.200]  You can do that ahead of time and not, you know, consume a runner to make sure you're all good to go there.
[34:51.830 --> 34:54.370]  So, that's PyLint syntax errors.
[34:57.780 --> 35:08.040]  I'm going to skip the network code breaker issues and instead shamelessly plug my workshop that's coming up.
[35:09.460 --> 35:12.080]  Where I'll be talking a lot about...
[35:12.740 --> 35:14.160]  Well, I'll hint at it.
[35:19.520 --> 35:28.400]  So, I have a workshop coming up in, I guess, an hour and a half or so, at 1230, right around the corner.
[35:28.520 --> 35:31.240]  Well, I'll be doing some network troubleshooting steps.
[35:31.380 --> 35:33.800]  It's called Hacking Network APIs.
[35:33.800 --> 35:34.900]  It's free.
[35:35.220 --> 35:39.600]  What I'll be doing is I've got a Raspberry Pi network audio player, okay?
[35:40.040 --> 35:52.700]  And it talks to it and I'll be reverse engineering the protocols to build up a separate system that controls it using my open-source project called Packet Sender.
[35:52.960 --> 35:58.360]  So, Packet Sender will be over here, got Wireshark over here, reverse engineering, recreating the protocols.
[35:58.600 --> 36:00.000]  Then that's part one.
[36:00.180 --> 36:06.740]  Part two is there is malware on the Reservoir Pi, which I'll be unearthing in part two.
[36:07.120 --> 36:09.500]  And then we can discuss that kind of problem.
[36:09.500 --> 36:18.100]  And the next one space will be part three, where I'll show latency versus throughput tests by trashing my network.
[36:18.320 --> 36:20.060]  So, and that should be a lot of fun.
[36:20.260 --> 36:24.920]  So, it's a two-hour time block because, like, I know what you're all thinking.
[36:25.140 --> 36:31.040]  You're like, Dan, Dan, you know, I've been listening to Blab for 40, 50 minutes.
[36:31.360 --> 36:33.520]  You know what would make hope really special?
[36:34.340 --> 36:38.680]  Two more hours of listening to you, Blab.
[36:38.680 --> 36:41.740]  Like, I know that's what you're all thinking, and I'm here to give.
[36:43.480 --> 36:47.340]  So, find me over there, and we'll do all that.
[36:49.440 --> 36:54.640]  And it looks like we might get out a little early, which I don't think is anybody complaining about.
[36:54.920 --> 36:56.400]  But my name is Dan Nagel.
[36:57.040 --> 36:59.040]  I do conference presentations.
[36:59.040 --> 37:00.540]  I do open-source development.
[37:00.540 --> 37:05.040]  If you have a conference and you think my presentation will be helpful to you, you come see me.
[37:05.040 --> 37:08.900]  I'm here all day today and at least half a day tomorrow.
[37:09.380 --> 37:14.680]  I hope all this was useful to you and that all your testing and software demo goes well.
[37:14.840 --> 37:15.780]  Are there any questions?
[37:16.040 --> 37:16.940]  We do have a few minutes.
[37:17.780 --> 37:18.260]  Yes.
[37:18.840 --> 37:19.900]  Do you have any opinion?
[37:20.040 --> 37:21.140]  So, you talked about PyTest.
[37:21.320 --> 37:21.500]  Yes.
[37:21.520 --> 37:25.300]  Do you have any opinion on just using the built-in unit test?
[37:25.300 --> 37:27.760]  That was slide number four or five.
[37:27.880 --> 37:29.720]  But yes, that is absolutely valid.
[37:29.880 --> 37:31.200]  I love the built-in unit test.
[37:31.920 --> 37:39.040]  What I use PyTest for is when I want to string them all together into like one full up system integrated test.
[37:39.360 --> 37:46.420]  And for me personally, PyTest works really well for that because it hooks into all the other tests to build up a full complete test system.
[37:49.890 --> 37:57.330]  Do you have any opinions on methods for antibiotic test-based synthesis like testing or property-based testing?
[37:57.550 --> 38:09.970]  The idea of getting to, I guess, is that a certain quality of it, but a certain quality of it can be given specific inputs and outputs of a certain relations on how there's an input set.
[38:10.470 --> 38:24.190]  I found that they could be a little like, I mean, I write that it would be pretty cool, but like it's sometimes interesting input cases, but sometimes we just run into like input space too big to generate or do something.
[38:24.770 --> 38:25.330]  All right.
[38:25.530 --> 38:29.110]  So, the question is, is building out the test sets.
[38:29.570 --> 38:31.270]  Essentially, good ways to do that.
[38:31.270 --> 38:42.330]  And there are, what I typically do, there are a lot of resources online, where you can basically say, I have this scenario, what's a good test set for it?
[38:42.730 --> 38:47.330]  And there's just like actual, just Git repos full of this kind of stuff.
[38:47.470 --> 38:49.190]  So that's usually my starter set.
[38:49.450 --> 38:53.790]  Like you saw my like positive, negative, empty test sets right there.
[38:53.970 --> 39:00.830]  I actually just picked that up, like actually did a search for like, what's a good test strategy for integers?
[39:01.270 --> 39:05.990]  And like, and I actually mentally forgot like, oh yeah, like the empty set.
[39:06.230 --> 39:07.790]  I mean, even I forget the stuff like that.
[39:07.930 --> 39:10.630]  So I basically just grab a set and start with that.
[39:10.750 --> 39:13.350]  And then as problems arise, to start adding to it.
[39:14.330 --> 39:15.730]  We have a couple of questions.
[39:16.090 --> 39:16.530]  Yes.
[39:20.100 --> 39:34.880]  One, how do you see the role of AI driven testing evolving in mission critical systems, where false positives or missed defects can have major financial or legal implications?
[39:34.880 --> 39:39.880]  Do you think we're ready to trust AI for compliance-heavy environments?
[39:40.220 --> 39:40.960]  All right.
[39:41.300 --> 39:42.940]  Well, I'll answer that.
[39:43.000 --> 39:49.840]  And by saying that I personally work in a, this very secure environment.
[39:49.840 --> 39:52.380]  So AI is just simply not allowed.
[39:52.680 --> 39:55.780]  So that's, that's the stance I have for that.
[39:55.920 --> 40:02.380]  But as far as like AI driven development or other things, I use it to help brainstorm.
[40:03.460 --> 40:09.300]  Like for, for example, your, um, right, where'd it go?
[40:10.160 --> 40:15.360]  Your pre-commit this thing right here, like AI actually helped me write some of this.
[40:15.840 --> 40:18.460]  So like, and this is not going to my codes.
[40:18.620 --> 40:19.820]  Like, I'm not worried about it.
[40:19.960 --> 40:24.820]  It's there's like no danger in my pre-commit, you know, thing, uh, causing issues.
[40:25.000 --> 40:27.240]  And I can personally prove free this.
[40:27.380 --> 40:29.920]  And it's like, yes, this, this does what I think it's going to do.
[40:29.920 --> 40:40.340]  So we're like little snippets of help, like the pre-commit, like, uh, Mr. AI chatbot, I need a pre-commit hash to syntax check my, my Python, you know, boom, it'll help you with this.
[40:40.460 --> 40:43.800]  Or a syntax check my C, C plus plus, boom, I'll get through this.
[40:44.060 --> 40:48.920]  Or Mr. AI, like, uh, I need, I'm trying to build a data set to sum up integers.
[40:49.320 --> 40:59.540]  And like, so basically brainstorming and ideas, but I, I would never like just copy this big, huge chunk of code and just blop it over there without, uh, giving it a thorough read, read through.
[41:00.380 --> 41:03.020]  Oh, and please repeat the questions for everyone.
[41:03.140 --> 41:06.080]  And when they ask them in here, thank you.
[41:06.300 --> 41:06.420]  Yes.
[41:06.620 --> 41:11.200]  On that same point, are you looking at integrating tools like Static Graph at any part of the pipeline?
[41:12.320 --> 41:13.600]  I'm not familiar with that.
[41:13.700 --> 41:14.480]  It was the question.
[41:14.820 --> 41:16.160]  Static software testing tools.
[41:16.580 --> 41:18.180]  Oh, static analysis tools.
[41:18.500 --> 41:20.180]  Uh, the question was integrating static analysis tools.
[41:20.320 --> 41:22.780]  And yes, that happens absolutely all the time.
[41:22.920 --> 41:26.300]  Like tools like, uh, Fortify is a, is a big one.
[41:26.300 --> 41:31.180]  Um, I've seen these, these things hook into like, uh, Nessus network scanners.
[41:31.420 --> 41:35.240]  So yes, you basically just keep dumping tools and tools and tools into this.
[41:35.360 --> 41:41.760]  And eventually what's going to happen is your, your, uh, your pipeline could end up taking like six or seven or eight hours or longer.
[41:42.180 --> 41:45.140]  And what you do is you basically set up a timer.
[41:45.420 --> 41:45.720]  Okay.
[41:46.240 --> 41:54.180]  Run the full up Python overnight and the developer push pipeline would be like an abbreviated pipeline that might take an hour.
[41:54.320 --> 42:02.520]  So you have like three or four different sets of pipeline where the big full up one with the static analysis, the Fortify and all that good stuff, that'll be an overnight pipeline job.
[42:02.680 --> 42:03.740]  That's how you set that up.
[42:03.880 --> 42:13.940]  But yes, you know, where you can, those things can all, you know, these days you can stand up multiple instances of your system under tests.
[42:14.400 --> 42:18.600]  And so just go ahead and run all the different tests in parallel.
[42:19.200 --> 42:28.420]  So the other person was speaking about, uh, standing up multiple instances to handle this stuff, to, to spread out all the jobs and, and parallel to, to save time.
[42:28.560 --> 42:36.680]  So you can, so you can have like one, uh, like VM over here running your Fortify scan and VM over here running your static analysis, another one over here.
[42:37.080 --> 42:40.480]  Like just, just spread it all out to run in parallel versus me.
[42:40.620 --> 42:44.000]  When I said the overnight test, that was mostly if you're going to do it all in series.
[42:45.840 --> 42:47.400]  One more question online.
[42:47.700 --> 42:48.060]  Yes.
[42:48.300 --> 43:00.640]  Uh, it's does writing tests, does writing unit tests before the code as TDD purists advocate, make any difference than writing after?
[43:00.640 --> 43:00.760]  Yeah.
[43:01.880 --> 43:03.420]  Test before the code.
[43:03.600 --> 43:08.800]  And I would say the answer is yes, but I'm not a purist at all.
[43:10.280 --> 43:14.860]  So what I do is I basically write both at the same time.
[43:14.860 --> 43:30.680]  And as I'm writing the code, I try to keep in mind, like, um, the idea of how am I going to, to unit tests where I don't code myself to an whole, a whole, where I have to like back out and refactor a bunch of stuff to be able to get the unit test first.
[43:31.040 --> 43:34.100]  So no, I, I don't write, you know, test and then code.
[43:34.300 --> 43:37.140]  Cause like, I'm not a curious, I don't really care.
[43:37.500 --> 43:42.020]  It's more like make sure I write the code in a way that can be tested.
[43:42.260 --> 43:45.620]  And basically the two just get developed together in my mind.
[43:46.080 --> 44:04.520]  So now the, the, the example I put in earlier, where I mentioned about, you know, a bug coming in, that will probably be code first, you know, code, the code, the bug replication first, and then go make the, I mean, code the test for the bug first and then fix the bug code.
[44:04.800 --> 44:08.180]  That's, that's be where the, that idea would come from.
[44:08.980 --> 44:12.680]  It can also help if you've written a detailed list of the requirements.
[44:13.340 --> 44:13.460]  Right.
[44:13.760 --> 44:18.260]  So, you know, if you have that, then you have a means to take that and turn it into tests.
[44:19.100 --> 44:22.980]  So, so his idea was, um, detail list requirements also helps.
[44:25.100 --> 44:25.620]  Yes.
[44:26.720 --> 44:33.680]  So it can be challenging if you're trying to connect to a database or something like that, when you're doing a database.
[44:34.020 --> 44:37.180]  So the typical way is using like a mock, right?
[44:37.300 --> 44:38.500]  So you're mocking out the database.
[44:38.800 --> 44:39.120]  Yes.
[44:39.380 --> 44:41.360]  I'm curious if that's how you do it or if you have...
[44:41.360 --> 44:43.520]  I, I 100% use mock.
[44:43.740 --> 44:44.020]  Okay.
[44:44.020 --> 44:45.500]  I use mock all the time.
[44:45.940 --> 44:54.660]  Uh, Python has a, um, an actual mock library and it is called mock.
[44:55.940 --> 45:03.440]  And then there's, uh, there's even variants of it called like, I think, uh, magic mock is another one.
[45:03.580 --> 45:07.100]  So anyway, yes, that is exactly how you do it.
[45:07.240 --> 45:11.560]  You would mock the database to where, and you can get really fancy.
[45:11.560 --> 45:19.620]  Like you can mock the database and like, if you get this query, then just, uh, respond with this canned data set that you then parse.
[45:19.820 --> 45:26.980]  But so yes, that is absolutely how you do it is you mock out all the different network connection points.
[45:27.040 --> 45:33.920]  So you can basically what you want to do is you don't want to have to stand up a server when you're doing unit tests.
[45:34.060 --> 45:35.520]  Now there's two kinds of tests.
[45:35.680 --> 45:39.260]  There's one where there's a pure unit test where it's basically entirely command line.
[45:39.260 --> 45:49.420]  Then there is like the integration test, what I call integration test, where you actually would stand up a real server, a real this, and your mocks go away.
[45:49.640 --> 45:53.140]  And now you're simulating real connections talking to each other.
[45:53.300 --> 45:54.200]  So there's two approaches.
[45:54.400 --> 45:56.340]  One is the one, your idea where you mock everything.
[45:56.620 --> 46:00.680]  The second test where you don't mock everything and you run a real integration test.
[46:01.260 --> 46:03.720]  And I, I think we are out of time now.
[46:05.360 --> 46:05.960]  Are we?
[46:06.100 --> 46:06.340]  Yes.
[46:06.340 --> 46:07.340]  Yeah, we're out of time.
[46:07.460 --> 46:07.900]  We're out of time.
[46:07.980 --> 46:09.100]  Actually, can I ask one question?
[46:09.280 --> 46:09.900]  You want to ask a question?
[46:09.900 --> 46:10.120]  Yes.
[46:10.140 --> 46:11.020]  I just want to ask one question.
[46:11.180 --> 46:11.440]  You're allowed.
[46:11.540 --> 46:18.340]  My question is about screenshot tests and what your, how you feel about them.
[46:18.440 --> 46:22.520]  Have you run into them and any challenges with them?
[46:22.720 --> 46:25.060]  I am 90% back in.
[46:25.180 --> 46:27.120]  So I've not had to deal with screenshots.
[46:27.200 --> 46:29.440]  I can't give you a good answer to that.
[46:29.580 --> 46:29.820]  I'm sorry.
[46:29.820 --> 46:30.500]  Okay, no worries.
[46:31.700 --> 46:36.160]  I used to do it for, when I worked for a small shoe company called Nike.
[46:36.780 --> 46:47.220]  And what we had to do on screenshots, and we'd forget it all the time, is there must be a date and time and system ID somewhere on the screen.
[46:47.460 --> 46:53.080]  Because people would take the correct answer and they're like, anyone can make that, you know?
[46:53.080 --> 47:01.420]  So his, his suggestion was, is that if you do a screenshot, make sure you capture the version, the time and the date, along with your screenshot inside of it.
[47:01.820 --> 47:03.740]  And system ID, make sure you get all the...
[47:03.740 --> 47:04.720]  Yes, that's a good idea.
[47:05.640 --> 47:08.860]  Otherwise your account, your screen group can tell you it's a little bit.
[47:09.160 --> 47:09.560]  Okay.
[47:10.400 --> 47:11.380]  All right, we're done.
[47:11.580 --> 47:11.960]  Thank you.