[00:01.880 --> 00:04.000]  My name is Devon McCormick.
[00:04.140 --> 00:10.320]  I'm a long-term coder, mostly in finance, mostly using array languages.
[00:10.580 --> 00:23.720]  And I'd like to talk to you today about one of my favorite ones, called J, and give you a little bit of background on the language and why it's important in general.
[00:23.880 --> 00:30.420]  And then show you some specific little hacks I've been working on in the past couple months.
[00:32.680 --> 00:38.220]  So, first of all, I'd like to... this is a graph in the grand old tradition, my axes are not labeled.
[00:39.500 --> 00:42.160]  And it's basically the axes represent my feelings.
[00:43.980 --> 00:50.000]  So... and it's kind of a rough graph of how languages relate to each other.
[00:50.420 --> 00:54.280]  And so you see in that upper... oh great, I got my screensaver on.
[00:55.020 --> 01:00.100]  In the upper right-hand corner there, you see J, and you notice it's next to APL and K.
[01:00.100 --> 01:02.220]  And you notice that J goes to 11.
[01:04.140 --> 01:04.760]  Okay.
[01:06.420 --> 01:07.660]  But it's...
[01:07.660 --> 01:12.240]  this group of languages is very different than any you've probably encountered before.
[01:12.740 --> 01:17.460]  So I'm going to give a little...
[01:18.520 --> 01:19.920]  kind of leave this here.
[01:20.700 --> 01:24.220]  just as this is what's called NuVoke on the J website.
[01:24.240 --> 01:27.780]  This is the entire language in one kind of long page.
[01:28.420 --> 01:35.920]  And it's a very different language because if you'll notice, we don't use words for the most part.
[01:36.060 --> 01:36.880]  We use symbols.
[01:37.160 --> 01:44.740]  And the symbols are one or two or occasionally more characters that represent different operations.
[01:45.980 --> 01:48.640]  So I'm not going to go into detail on that.
[01:48.640 --> 01:52.780]  I'm going to mention some as I go through my talk on particular items.
[01:53.080 --> 01:55.900]  But this is just to give you an idea of how very different it is.
[01:56.000 --> 01:58.960]  You'll notice how there's a color coding there.
[01:59.860 --> 02:01.780]  So the pink ones are nouns.
[02:02.720 --> 02:05.180]  Noun is Jspeak for data.
[02:06.260 --> 02:14.360]  We use a different vocabulary because other languages, they talk about higher-level functions, but it's this blurry thing.
[02:14.720 --> 02:18.400]  In J, we have at least three higher-level functions, and they have names.
[02:18.540 --> 02:21.620]  There's an adverb, which is when you modify a verb.
[02:22.320 --> 02:23.480]  So to...
[02:23.480 --> 02:29.480]  For instance, if you want to turn addition into summation, you use an adverb.
[02:43.110 --> 02:46.790]  There's conjunctions, which are when you combine two verbs together.
[02:47.210 --> 02:54.970]  So if you want to do, say, a matrix multiply, you're combining summation and multiplication, and use the dot in between them to join them together.
[02:55.890 --> 03:00.110]  But that's just to give you a flavor of how very different this language is.
[03:00.710 --> 03:04.570]  The nice thing about it, in spite of all these differences, is it's extremely consistent.
[03:06.130 --> 03:07.890]  And things all work together.
[03:08.110 --> 03:12.270]  There's a lot of very careful choices in the design of the symbol set.
[03:12.430 --> 03:14.430]  And so things tend to be related.
[03:15.130 --> 03:19.470]  The bad part about using a symbol set is you can't look stuff up alphabetically.
[03:19.930 --> 03:22.610]  But, like I say, the whole thing kind of fits on one long page.
[03:24.150 --> 03:32.150]  I'm going to digress a little bit here to talk about why this sort of thing is important.
[03:32.290 --> 03:38.270]  So Jay is part of a group of languages called the array languages, which started with APL back in the 60s.
[03:38.270 --> 03:43.470]  And this is one of these old ideas that keeps getting rediscovered.
[03:44.270 --> 03:52.730]  As someone said that Google invented MapReduce in 2006, and Ken Iverson cleverly reinvented it in 1966.
[03:55.150 --> 03:58.330]  And this is a part of a...
[03:58.330 --> 03:59.910]  Excuse me, I'm going to just put on my notes here.
[04:02.150 --> 04:06.910]  There's this tendency in computers because everyone wants everything that's new and shiny.
[04:07.130 --> 04:11.850]  And they forget about the stuff that's old and not so shiny and still works well.
[04:12.610 --> 04:13.930]  And this is...
[04:14.970 --> 04:16.590]  So these are just some more of the symbols.
[04:16.850 --> 04:18.250]  You can look at them while I'm talking.
[04:18.650 --> 04:28.190]  And as an example of how people forget things, is if anyone has subscribed to the news group Risks in Computing, which has been around since the mid-80s.
[04:28.190 --> 04:35.210]  And every now and then you get an issue with a dozen or so massive failures in computing.
[04:35.850 --> 04:43.590]  And the depressing thing is if you've read this for a long time, the same things keep coming up again and again and again and again.
[04:43.710 --> 04:48.230]  It's like, you know, pushing something out before it's tested, not allowing enough time.
[04:49.230 --> 04:52.470]  You know, squirrels knocking out electrical supplies is one that...
[04:52.470 --> 04:53.350]  You can't do anything about that.
[04:55.270 --> 04:59.250]  But, for instance, Microsoft had a leap year bug not many years ago.
[04:59.710 --> 05:00.810]  You mean a leap year bug?
[05:00.890 --> 05:01.150]  Seriously?
[05:01.890 --> 05:02.290]  Okay.
[05:03.390 --> 05:06.150]  So it seems like there's a problem with...
[05:06.690 --> 05:09.290]  People don't learn from the past in the field of computing.
[05:09.390 --> 05:12.590]  And so I'm gonna show you these things from the past, which...
[05:12.590 --> 05:15.710]  Jay is a center from APL, which goes back to the 60s.
[05:15.710 --> 05:25.030]  And Scott Lachlan, the blogger, put it that one of the problems today is that people don't pay attention to the greats of the past.
[05:25.230 --> 05:31.630]  And so there are Mozarts and Beethovens of programming, but most programmers are more familiar with Lady Gaga.
[05:32.290 --> 05:33.970]  Not to disparage Lady Gaga.
[05:37.310 --> 05:44.450]  But one of the themes that we've talked about in these array languages is the idea of a language as a tool of thought.
[05:45.690 --> 05:49.250]  And this is an idea that goes...
[05:50.890 --> 05:52.970]  There's Mozart, Beethoven, Lady Gaga.
[05:57.400 --> 06:02.200]  So you have tools, which are handy things to get things done, and you have tools of thought.
[06:02.340 --> 06:05.140]  So here we see music notation as a tool of thought.
[06:05.780 --> 06:06.820]  Calculus notation.
[06:07.660 --> 06:09.020]  You know, Leibniz won.
[06:09.380 --> 06:12.680]  Newton had this poorer notation that did not take off.
[06:13.380 --> 06:19.320]  Things like math, international phonetic alphabet, circuit diagrams, architectural diagrams.
[06:19.440 --> 06:20.420]  These are all tools of thought.
[06:20.520 --> 06:21.920]  These are all things that help you think better.
[06:22.200 --> 06:29.340]  And we'd like to think that this sort of thing, like the J language, among others, helps you do that.
[06:29.620 --> 06:31.880]  The other part is it's an interpreted language.
[06:31.920 --> 06:33.160]  It's very interactive.
[06:33.480 --> 06:35.220]  You can get things done very quickly.
[06:35.220 --> 06:37.780]  So everything I've done here, I just did in the past couple months.
[06:38.540 --> 06:40.520]  And it really didn't take a whole lot of time.
[06:40.660 --> 06:43.300]  I spent a lot of time playing around with things.
[06:44.200 --> 06:46.760]  So anyway, how I got started on this is...
[06:47.180 --> 06:48.620]  I had this image.
[06:48.720 --> 06:49.500]  I don't think it was this one.
[06:49.620 --> 06:52.460]  But there's this image, and it has...
[06:54.920 --> 06:57.640]  And there's a certain amount of text associated with it.
[06:57.720 --> 06:58.680]  And it wasn't this image.
[06:58.760 --> 06:59.180]  It was some other.
[06:59.200 --> 07:00.080]  It was more technical image.
[07:00.120 --> 07:02.760]  It had text, and you couldn't understand the image without the text.
[07:02.820 --> 07:04.080]  And I thought, I want to keep these together.
[07:04.480 --> 07:06.520]  But I don't want to go pasting text onto an image.
[07:06.900 --> 07:09.220]  And if I keep in a separate file, it's going to get lost.
[07:09.600 --> 07:11.360]  So my brilliant idea is...
[07:11.360 --> 07:15.000]  Well, what would happen if I just appended text to the end of the image?
[07:15.820 --> 07:17.800]  So here, I'm defining the text.
[07:18.080 --> 07:20.160]  So that text equals...
[07:20.160 --> 07:21.420]  So I mentioned one...
[07:21.420 --> 07:22.200]  Oh, I didn't mention.
[07:22.640 --> 07:26.300]  One of the weird things about J is evaluation is from right to left.
[07:27.120 --> 07:30.160]  And this is something people have trouble with because everyone's learned PEMDAS.
[07:30.780 --> 07:33.300]  And like, oh, exponentiation before multiple.
[07:33.500 --> 07:36.220]  And that PEMDAS is just a very bad rule.
[07:36.620 --> 07:38.840]  It's been around for a long time and people accept it.
[07:38.840 --> 07:39.880]  But it doesn't...
[07:39.880 --> 07:43.300]  It falls apart very quickly once you have more than about five things.
[07:43.700 --> 07:46.020]  And J has a hundred or so things.
[07:46.240 --> 07:49.020]  And even something like C has dozens of things.
[07:49.460 --> 07:53.180]  And so an implicit hierarchy just does not work well.
[07:53.180 --> 08:01.640]  And also, even in these notations, when you get things at the same level of the hierarchy, you evaluate them positionally anyway.
[08:02.020 --> 08:06.220]  So J just gets rid of all those rules, just evaluate things positionally, and go from right to left.
[08:06.320 --> 08:08.920]  Which sounds weird, but it actually works very well.
[08:09.020 --> 08:12.180]  So for instance here, I'm assigning text to this text.
[08:13.000 --> 08:14.320]  The 0 colon 0 says...
[08:14.320 --> 08:17.500]  You know, this is input up until the last paren there.
[08:17.840 --> 08:20.200]  And I also stuck a little hash sign in front of it.
[08:20.260 --> 08:21.380]  Hash is called tally.
[08:21.380 --> 08:23.480]  You know, count how many things are here and tell me.
[08:24.500 --> 08:28.980]  So, as I'm defining this, I just want to make sure that, you know, I got as many things as I want.
[08:28.980 --> 08:33.300]  So I stick the hash on there and at the end it tells me it's 502 characters.
[08:34.500 --> 08:38.560]  So the next thing I did is I took that text and I appended it to the JPEG.
[08:39.320 --> 08:40.320]  Turns out that works.
[08:40.440 --> 08:40.800]  It's fine.
[08:40.940 --> 08:42.740]  The JPEG knows how big it is.
[08:43.100 --> 08:45.900]  If you put extra stuff at the end of the file, it just ignores it.
[08:46.620 --> 08:51.020]  And so if we go down here into Emacs, we see...
[08:51.020 --> 08:52.180]  A bunch of gobbledygook.
[08:52.320 --> 08:53.220]  I don't know if you can see that.
[08:53.400 --> 08:55.600]  But down there at the bottom is the text.
[08:56.620 --> 08:57.700]  It's just stuck in there.
[08:57.900 --> 08:59.380]  And this works fine.
[08:59.460 --> 09:01.280]  So this got me thinking about steganography.
[09:01.460 --> 09:04.120]  The hiding of text in images, for instance.
[09:04.760 --> 09:06.220]  And essentially I've achieved it here.
[09:06.460 --> 09:07.060]  That's steganography.
[09:07.800 --> 09:12.100]  But it's pretty poor because if anyone actually looked at the file, they would see what I was trying to hide.
[09:12.100 --> 09:18.860]  So it occurred to me that we could do something clever.
[09:19.520 --> 09:25.140]  So what I did is I took this famous photo and I reduced it.
[09:25.320 --> 09:32.140]  So now it's a 226 by 30 version of this famous great wave off Kanagawa.
[09:32.580 --> 09:37.160]  And the reason I did that is I wanted to pick a good test case, something that's difficult.
[09:37.160 --> 09:44.140]  I wanted a small image and a fair amount of text and see if I can successfully hide text in a small image like this.
[09:44.220 --> 09:46.620]  Because any problem should be much more evident.
[09:50.320 --> 09:51.420]  So I did that.
[09:51.500 --> 09:54.520]  So the text I chose is this, the Gettysburg Address.
[09:55.400 --> 09:56.740]  You see it's 1500.
[09:56.980 --> 10:00.580]  And I did the same thing there where I checked the size of it.
[10:00.820 --> 10:02.160]  Okay, I used dollar this time.
[10:02.260 --> 10:04.160]  Dollar gives you the shape, not the count.
[10:04.160 --> 10:07.100]  This happens to be a vector, so the shape and the count are the same.
[10:09.800 --> 10:10.180]  But...
[10:10.740 --> 10:12.160]  So I did stuff like this.
[10:12.340 --> 10:14.100]  So here I'm reading the image in.
[10:15.280 --> 10:16.140]  Checking the size.
[10:17.440 --> 10:18.260]  Oh, this...
[10:18.260 --> 10:19.040]  Oh yeah, this is...
[10:19.040 --> 10:22.560]  I was comparing different formats, because some formats are lossy and some aren't.
[10:23.480 --> 10:28.980]  So it occurred to me, gee, if you embedded it in a lossy format, you could lose stuff, possibly.
[10:29.480 --> 10:32.840]  As it turns out, it works just fine as long as you don't mess around with the image.
[10:32.840 --> 10:40.940]  But anyway, this is just reading in different versions of that same picture, BMP, JPEG, and ping, and comparing them.
[10:41.080 --> 10:43.540]  You see BMP and JPEG are different.
[10:44.180 --> 10:48.100]  But BMP and PNG are the same, because they're both lossless.
[10:49.260 --> 10:49.760]  And...
[10:50.660 --> 10:52.260]  Well, we'll get to this later.
[10:52.420 --> 11:01.360]  But when you're playing around with things in RGB space, you could actually flip the red and the blue planes and get purple pumpkins like that.
[11:01.360 --> 11:01.480]  But...
[11:01.920 --> 11:02.940]  We'll get to that later.
[11:03.400 --> 11:06.980]  So my first try was just, well, let's just stick the text on the front of the image.
[11:07.840 --> 11:08.760]  So I did that.
[11:13.140 --> 11:14.940]  Basically, here, I'm...
[11:16.420 --> 11:19.100]  I'm doing A dot I dot text.
[11:19.260 --> 11:20.620]  A is the alphabet.
[11:20.840 --> 11:24.840]  It's all 256 8-bit character combinations.
[11:24.840 --> 11:29.360]  So I'm essentially turning the text into numbers, and then sticking that onto the file.
[11:30.440 --> 11:31.440]  And write it out.
[11:31.600 --> 11:32.580]  And then we take a look.
[11:32.580 --> 11:34.420]  But we see that that doesn't work very well.
[11:34.520 --> 11:36.680]  Because this is the one with the text.
[11:36.960 --> 11:39.900]  And you can see it's just blank stuff at the top.
[11:39.900 --> 11:41.380]  So that's a bad idea.
[11:41.900 --> 11:44.760]  But, you know, it took me a minute to do that.
[11:44.880 --> 11:46.040]  So I said, let's try another one.
[11:46.200 --> 11:48.060]  Let's try to distribute the text evenly.
[11:49.040 --> 11:51.120]  So here, we read in the text.
[11:51.320 --> 11:52.100]  We figure out...
[11:52.100 --> 11:56.160]  I take the tally of the image divided by the tally of the text.
[11:56.180 --> 12:00.920]  That tells me I have about 49 pixels per each letter in my text.
[12:00.920 --> 12:06.620]  So if I just distribute it every 49 pixels or so, which is what I'm doing here.
[12:09.040 --> 12:12.440]  And, again, at the end, I assigned index as iXS.
[12:13.620 --> 12:16.920]  You know, 0.5, I'm rounding, because it's 49 point something.
[12:17.960 --> 12:20.980]  And just to make sure it looks okay, I'm taking the max reduce.
[12:21.700 --> 12:23.440]  So that's telling what's my biggest index.
[12:23.460 --> 12:26.720]  Just to make sure that I did it right, and I'm not trying to index past the end of the array.
[12:26.940 --> 12:27.580]  It's fine.
[12:28.500 --> 12:33.020]  So I insert the text in there evenly, and then we look at the image.
[12:33.120 --> 12:36.540]  And we see, okay, it's better, but you can still see it.
[12:36.620 --> 12:38.880]  And, in fact, you see these diagonal lines here.
[12:39.120 --> 12:40.240]  The white spots.
[12:41.680 --> 12:45.940]  Because you put it at even intervals, and they don't match up with the shape of the image.
[12:45.940 --> 12:49.300]  So you get that characteristic staggering thing.
[12:49.760 --> 12:51.780]  So, okay, well, it's a little better, but it's still not there.
[12:52.480 --> 12:54.720]  So I thought, well, how about if we distribute it randomly?
[12:55.160 --> 12:56.340]  So I'm doing the same thing.
[12:56.960 --> 12:58.640]  I've taken the text.
[12:58.820 --> 13:00.180]  I'm generating some...
[13:00.880 --> 13:08.420]  At first, the indexes, I'm just generating them randomly, based on the number of items in the text and the length of the image.
[13:08.840 --> 13:13.520]  So I'm picking random numbers from zero to one less than the length of the image.
[13:13.600 --> 13:16.700]  And I'm picking the number, you know, 1,512 of them.
[13:17.340 --> 13:18.740]  And I distribute those randomly.
[13:18.880 --> 13:21.680]  And that, eh, it might even be worse than the previous one.
[13:21.680 --> 13:24.320]  You can see you still got little white dots and they're scattered randomly.
[13:24.600 --> 13:26.020]  So maybe not quite as obvious.
[13:27.300 --> 13:33.660]  Well, okay, this is not obviously not working because each letter is blunking out an entire pixel.
[13:34.100 --> 13:38.620]  So what if we look at the image in 3D with three RGB planes?
[13:39.380 --> 13:45.620]  So I have something called RGB 3, which actually, it's its own inverse.
[13:45.880 --> 13:51.840]  So you see the first line, I apply RGB 3 to the great wave of Kanagawa picture.
[13:53.060 --> 13:58.680]  And it gives me the 226 by 330 by 3, where the 3 is the last dimension of the RGB planes.
[13:59.560 --> 14:03.360]  And just to show you it's its own inverse, I apply it twice and I get the original one back.
[14:03.680 --> 14:10.920]  So it's kind of cool to be able to write something in its own inverse because it can tell from, you know, what you're trying to do by looking at what you're feeding it.
[14:11.840 --> 14:17.540]  And so I look at the total size of the ravel of the 3D one.
[14:17.940 --> 14:23.780]  Okay, monadic comma turns anything to the right into a vector, no matter what its dimensionality.
[14:23.780 --> 14:25.580]  And I take the tally.
[14:25.740 --> 14:27.920]  So I've got 223,000.
[14:28.960 --> 14:31.540]  And they're not quite pixels now, they're parts of pixels.
[14:32.860 --> 14:35.760]  And I do the distributed evenly thing again.
[14:35.760 --> 14:39.180]  So I divide that into the, you know, the size of the text into that.
[14:39.340 --> 14:44.400]  So every 147, whatever we call those sub-pixel things, I'll put one in.
[14:45.320 --> 14:47.860]  And so I do that and I write it out.
[14:48.040 --> 14:54.260]  And here we see, okay, maybe a little better, but now you still got the diagonal thing because I distributed them evenly.
[14:54.900 --> 15:00.800]  But the little diagonal pixels are different colors depending on which plane I happen to hit with my letter.
[15:01.240 --> 15:03.060]  So that's kind of starting to get there.
[15:03.140 --> 15:06.140]  If you saw that, you would just think it was kind of a poor res image.
[15:06.260 --> 15:09.500]  You might not suspect there was something hidden in it unless, of course, you're looking for it.
[15:10.560 --> 15:12.220]  So here's, this is my fifth try.
[15:13.040 --> 15:17.780]  And the thing to understand here is each of these tries took me minutes to do.
[15:18.580 --> 15:22.200]  Okay, so you're talking like 10 minutes of work up to this point.
[15:22.720 --> 15:25.220]  And so this one, now we're going to get a little more sophisticated.
[15:25.220 --> 15:34.740]  And I said, okay, what if I turn the text into Boolean instead of, you know, integers, and I distribute those bits?
[15:35.320 --> 15:43.400]  And just to minimize the effect, I'm going to put those bits into the low order bits of each element on the RGB plane.
[15:44.360 --> 15:51.960]  So there's the hash colon is how you turn things into another base.
[15:52.380 --> 15:58.460]  If you don't give it a right argument, if you only give it a right argument, it assumes you want base two.
[15:59.180 --> 16:03.820]  And so you see, I do the a dot, idot to turn the text into numbers.
[16:03.820 --> 16:08.560]  I apply the base thing to turn it into Boolean, and it gives me this table of numbers.
[16:08.980 --> 16:13.240]  And we see the table is, okay, it's not 1512 anymore because I did this a month later.
[16:13.240 --> 16:14.100]  It was slightly different.
[16:14.240 --> 16:16.560]  But it's the size of the text by seven.
[16:17.340 --> 16:23.300]  Because if you don't give it a left argument, it takes a look at what's the highest number here, and it gives you that many bits.
[16:23.900 --> 16:28.620]  So if I didn't have any values above 63, it would give me a table that would be six wide.
[16:29.340 --> 16:34.020]  So, well, we actually want to force it to eight bits just for, you know, neatness.
[16:34.360 --> 16:38.120]  So as it turns out, you can give an optional left argument.
[16:38.240 --> 16:39.780]  And this is the way many things work in J.
[16:39.780 --> 16:46.160]  You have a default monadic case where it does something useful like minus negates things if you don't have a left argument.
[16:46.200 --> 16:48.160]  If you have a left argument, it subtracts things.
[16:49.260 --> 16:49.680]  Simple.
[16:49.940 --> 16:52.020]  And a lot of the things are related like this.
[16:52.580 --> 16:55.960]  So I'm just going to show you briefly the RGB3 thing I used before.
[16:56.160 --> 16:57.340]  This one that's its own inverse.
[16:57.360 --> 16:59.240]  That's the entire definition of it right there.
[16:59.780 --> 17:05.600]  And I know it's unreadable, but once you learn the language, this is very readable.
[17:05.740 --> 17:06.860]  So you see the 3s in there.
[17:07.120 --> 17:21.560]  I'm checking the shape of the last dimension and essentially making sure that the shape has three things in it.
[17:21.560 --> 17:24.000]  So it's a three-dimensional thing with three as the last dimension.
[17:24.160 --> 17:29.840]  That means you want to turn it back into collapse the RGB planes.
[17:30.020 --> 17:32.940]  And if that's not true, then you're going to expand it to RGB planes.
[17:33.380 --> 17:35.140]  And there's 256, 256.
[17:35.780 --> 17:38.680]  You notice it's that same hash colon that I used up there.
[17:39.160 --> 17:42.340]  But now it's a... I give it a left argument.
[17:42.340 --> 17:47.020]  I'm saying I want to do this in base 256, not base 2, which is the default.
[17:47.700 --> 17:52.340]  And so 256 turns it into values from 0 to 255, as you would expect.
[17:52.860 --> 17:57.040]  And then the one after the little tick mark is the inverse operation where it collapses.
[17:57.240 --> 17:59.780]  It assumes it's a base 256 number and it turns it back.
[18:01.240 --> 18:08.720]  So I force... by the same method, I force eight digits by specifying a left argument of eight 2s.
[18:08.760 --> 18:09.800]  So this is base 2.
[18:10.460 --> 18:14.580]  By the way, you don't always have to use... there is such a thing as mixed bases.
[18:15.440 --> 18:17.280]  You don't always convert to one thing.
[18:17.400 --> 18:18.520]  Think of time, for instance.
[18:18.740 --> 18:19.880]  Time is a mixed base.
[18:20.660 --> 18:20.880]  Right?
[18:21.500 --> 18:23.700]  And so you can use that as your left argument.
[18:23.860 --> 18:26.760]  Like if you have a number of seconds and you want to turn it into hours, minutes, and seconds.
[18:27.100 --> 18:27.980]  It's easy to do.
[18:29.300 --> 18:31.080]  And so I take a look.
[18:31.260 --> 18:34.280]  I check the sizes of them just to make sure I've got enough.
[18:34.580 --> 18:38.020]  So I have 12,088 bits.
[18:38.020 --> 18:43.700]  The image is still 223,000.
[18:44.040 --> 18:49.420]  When I turn it into RGB, it's this four-dimensional array.
[18:50.380 --> 18:51.380]  So just to say this...
[18:51.380 --> 18:53.680]  J naturally handles high-dimensional arrays.
[18:53.820 --> 18:56.800]  And this has, to me, been one of the marvelous things about these array languages.
[18:57.380 --> 19:01.120]  Everyone else, you're always doing things on one item at a time.
[19:01.360 --> 19:04.300]  And when you want to do something bigger than that, oh, you've got to have a loop.
[19:04.360 --> 19:06.040]  And it's much more complicated.
[19:06.180 --> 19:08.600]  And you say, oh, this extends to multiple dimensions.
[19:08.980 --> 19:10.820]  And people ask... and people argue.
[19:10.960 --> 19:13.800]  Why would you ever need more than one dimension or something?
[19:14.340 --> 19:16.900]  Well, it turns out you could accidentally get...
[19:16.900 --> 19:18.120]  So this is clearly...
[19:18.120 --> 19:19.900]  I need four dimensions here.
[19:20.040 --> 19:21.540]  Because I've got a two-dimensional object.
[19:21.540 --> 19:23.100]  I split it into three planes.
[19:23.320 --> 19:27.160]  And each item in the plane, I gave it an eight-bit representation.
[19:27.640 --> 19:32.320]  That's why I have a four-dimensional object, which is perfectly easy to handle.
[19:32.720 --> 19:35.000]  Now, as it turns out, I don't really care about the first few dimensions.
[19:35.000 --> 19:37.120]  I just want a something-by-eight table.
[19:37.340 --> 19:40.400]  Because I'm going to put my bits into that last eighth column.
[19:40.680 --> 19:43.180]  So what I do is I collapse the first two dimensions.
[19:43.800 --> 19:44.880]  This comma reduce.
[19:45.880 --> 19:48.540]  So reduction is this adverb I mentioned.
[19:48.920 --> 19:52.300]  So you give it a verb on the left and your data on the right.
[19:52.520 --> 19:57.340]  So if that was a plus instead of a concatenation, it would add things up.
[19:57.520 --> 19:59.560]  If it was minus, it would subtract them.
[19:59.620 --> 20:00.680]  If it was divide, and so on.
[20:00.840 --> 20:02.820]  So it's very general in that sense.
[20:02.820 --> 20:06.680]  If you have this adverb, you can stick any verb you want on the other side.
[20:06.800 --> 20:07.880]  It could be your own verb.
[20:07.960 --> 20:09.160]  It could be a user-defined verb.
[20:09.500 --> 20:11.780]  And then it applies it across the array.
[20:12.860 --> 20:14.500]  And so in this one, I do it.
[20:14.640 --> 20:20.440]  And lo and behold, we have an image that looks quite like the original image.
[20:21.100 --> 20:23.840]  And in fact, I do a lot of photography stuff.
[20:23.860 --> 20:32.160]  And the way I tell if two images are identical or where they differ, I'll take the two images and I exclusively wore them together in the paint program.
[20:32.820 --> 20:34.960]  And if they're identical, you get this black square.
[20:35.720 --> 20:39.120]  And everywhere things are different in all three pixels, you get a white pixel.
[20:39.400 --> 20:43.900]  Where they're different in just like the R or the G or the B, you get a red or green or blue pixel.
[20:44.260 --> 20:47.920]  In this case, I did that for this in the original image.
[20:48.520 --> 20:49.900]  And it looked black.
[20:50.020 --> 20:51.180]  Okay, it wasn't black.
[20:51.460 --> 20:55.820]  There are actually little gray squares in there, but you can't really see them.
[20:57.060 --> 20:59.440]  The last bit might have been a zero in some cases.
[20:59.760 --> 21:06.720]  So basically, this is indistinguishable by sort of naked eye from the original image.
[21:06.820 --> 21:07.940]  So I've got the text hidden in there.
[21:08.460 --> 21:11.420]  Now, of course, hiding it is no good if you can't pull it out.
[21:11.420 --> 21:14.780]  So the next thing we want to do is extract it.
[21:15.200 --> 21:17.880]  So I read in the new one I created.
[21:18.400 --> 21:20.720]  I turned it into a three-plane thing.
[21:21.320 --> 21:22.380]  Then I...
[21:23.000 --> 21:27.220]  I know this is pretty ridiculous there, but I'm taking that.
[21:27.340 --> 21:28.420]  I'm applying the...
[21:28.420 --> 21:31.080]  I'm making eight bits out of each number.
[21:31.620 --> 21:35.780]  And I'm reducing the first two dimensions so it's a something by eight.
[21:35.780 --> 21:38.100]  And then I'm extracting...
[21:38.100 --> 21:43.400]  And I have the magic numbers in there, seven and 12088, because 12088 is the length of the text.
[21:43.520 --> 21:43.940]  I knew that.
[21:44.280 --> 21:46.920]  And seven is the seventh column, starting at zero.
[21:47.100 --> 21:51.840]  So I'm pulling out the last column, the first 12,088 items.
[21:52.520 --> 21:55.940]  And then this line here turns those...
[21:55.940 --> 21:58.340]  this vector of bits.
[21:58.560 --> 22:06.680]  It's basically the negative eight bracket there with the scan sign means, give me groups of eight that aren't overlapping.
[22:06.880 --> 22:10.060]  If I'd use a positive eight, it would give me groups of eight that are overlapping.
[22:10.740 --> 22:13.220]  And then do the inverse...
[22:13.220 --> 22:17.240]  the base thing to convert them from base two into numbers.
[22:17.600 --> 22:20.780]  And then look up those numbers in the character set and turn them into characters.
[22:21.520 --> 22:25.560]  And lo and behold, we've got our original text.
[22:26.660 --> 22:27.860]  So that basically works.
[22:27.920 --> 22:31.220]  But it's a little complicated and we're using magic numbers in there.
[22:32.320 --> 22:34.240]  So, let's try to generalize this.
[22:36.440 --> 22:36.880]  So...
[22:39.540 --> 22:40.420]  Oh, yeah.
[22:42.200 --> 22:45.420]  One way to generalize it would be to say, OK, I don't know how long the message is.
[22:45.540 --> 22:48.840]  I'm going to just take all those terminal bits and turn them into characters.
[22:49.160 --> 22:49.800]  You could do that.
[22:50.020 --> 22:55.660]  It's a much more expensive operation, because I'm doing 223,000 things instead of 12,000.
[22:56.580 --> 22:58.440]  And you get all this garbage at the end.
[22:59.020 --> 23:01.560]  But you have to know, OK, that's the actual end of the text.
[23:01.680 --> 23:03.820]  So it's still a little clunky in that respect.
[23:04.020 --> 23:09.420]  And you might have text with high bits, too, where you couldn't tell that that's, you know, what you don't want.
[23:09.580 --> 23:13.560]  Because, I mean, you could embed an executable in an image if you wanted to.
[23:15.140 --> 23:21.120]  So, what I do is I come up with an expression to do the extraction.
[23:21.380 --> 23:22.920]  So this is a pretty doozy.
[23:23.180 --> 23:27.000]  Part of the reason it's a doozy is, you see those 256s in there?
[23:27.440 --> 23:34.160]  I took the RGB3 program and I just put it in line, so that I can make it a single line of code with fewer dependencies.
[23:34.380 --> 23:36.300]  So I don't have to have RGB3 sitting around.
[23:36.500 --> 23:38.080]  It's actually embedded in the code.
[23:38.360 --> 23:42.000]  So this is a standalone piece of code to do the entire extraction.
[23:42.000 --> 23:44.820]  So you recognize the little parts of it from before.
[23:45.620 --> 23:48.560]  And we still do have the magic numbers in those, but we'll get to those in a sec.
[23:50.740 --> 23:54.120]  So, the length of this expression is 112.
[23:55.720 --> 23:58.120]  We know this because I wrote it to the end of the file.
[23:58.420 --> 24:01.940]  And when you write something, it tells you how many bytes you wrote.
[24:02.320 --> 24:06.060]  So that extraction expression is 112 bytes long.
[24:06.520 --> 24:13.720]  So what I can do is I can say, well, let's assume that I'm always going to use the last four bytes to make an integer, which tells me how long this thing is.
[24:14.800 --> 24:18.420]  So 112 in four bytes is that in Big Endian.
[24:21.680 --> 24:23.940]  And I take...
[24:24.980 --> 24:27.620]  NN is the...
[24:31.680 --> 24:32.460]  Oh yeah, okay.
[24:33.020 --> 24:34.260]  I used fread before.
[24:34.360 --> 24:35.380]  fread has different formats.
[24:35.480 --> 24:37.560]  You can say fread file name, it just reads the file.
[24:37.700 --> 24:43.900]  If you say you want to read a piece of the file, in this case, I want to start four from the end and read four bytes.
[24:44.060 --> 24:45.260]  So that's that bit at the end.
[24:45.380 --> 24:48.140]  So he has a different form to specify different kinds of reading.
[24:48.780 --> 24:51.440]  And a lot of J things are overloaded like this.
[24:51.900 --> 24:54.380]  And so I grab the last four.
[24:55.380 --> 24:57.000]  I mean, I grab the bytes.
[24:57.140 --> 24:59.920]  I grab the last 116 bytes.
[25:00.120 --> 25:03.280]  I throw away the last four because that was the number that told me how long it was.
[25:04.920 --> 25:09.660]  And then I get that text, that line of text there, which I originally put on the end of the file.
[25:11.540 --> 25:21.400]  And in J, to execute a line of text or turn text into numbers, you use the double quote dot that's do.
[25:21.760 --> 25:27.440]  And so that basically reads this thing, throws the last four off, and then executes it.
[25:27.440 --> 25:32.600]  And so it has the, does it happen in a minute?
[25:32.780 --> 25:36.180]  Oh yeah, I do it based on the file there.
[25:36.360 --> 25:38.020]  So basically you get the text again.
[25:38.240 --> 25:43.660]  So now you have something that is quite general, that you put the extraction expression at the end.
[25:43.880 --> 25:45.760]  So this kind of goes back to my original ideas.
[25:46.060 --> 25:49.440]  Gee, you can append text onto a JPEG and it doesn't care.
[25:49.740 --> 25:55.100]  So now the text I append on the end of the JPEG is actually the text to extract the secret message.
[25:55.100 --> 26:00.420]  And of course you could go further, you could obfuscate that somehow so it doesn't look like a line of text.
[26:00.940 --> 26:04.540]  But that was sort of my little stagonography exercise.
[26:06.180 --> 26:09.640]  So any questions or comments on that?
[26:12.600 --> 26:13.060]  Okay.
[26:14.040 --> 26:16.640]  The other big thing, how am I doing on time here?
[26:18.080 --> 26:19.080]  Oh good, okay.
[26:20.100 --> 26:21.180]  Yeah, terrific.
[26:21.360 --> 26:21.860]  That should be right.
[26:22.040 --> 26:29.080]  So the next exercise I entered, and I want to emphasize that doing this whole exercise, took me maybe a couple hours altogether.
[26:29.360 --> 26:33.080]  It took much longer to write it up than it did to actually do it.
[26:33.760 --> 26:39.620]  And this is just very typical of these languages because you have so much power embedded in each symbol.
[26:39.820 --> 26:42.200]  You don't have to like, oh, I need to sort something.
[26:42.320 --> 26:43.600]  Where's the sort library again?
[26:43.680 --> 26:43.920]  No, no.
[26:44.300 --> 26:44.920]  Sorts are primitive.
[26:46.500 --> 26:52.260]  You know, so a lot of your basic things you do that most languages need libraries for, they're built into the language.
[26:52.260 --> 26:54.320]  So you don't have to go looking around for stuff.
[26:54.720 --> 26:58.040]  In fact, there's a common expression for averaging things in J.
[26:58.680 --> 27:10.760]  And there is an average function in the statistic library that no one ever uses, because it's so easy to write the expression that it's not worth the time to remember where it lives, and what the name of it is.
[27:10.760 --> 27:11.620]  Is it AVG?
[27:11.740 --> 27:12.400]  Is it average?
[27:12.740 --> 27:17.100]  It's faster to write the expression yourself than to use the library.
[27:18.220 --> 27:28.460]  So this next one is, I had this idea, well, since one of the nice things about J is I can read in these files, and they're just a vector of bytes.
[27:28.980 --> 27:31.240]  And I can do whatever I want with that vector of bytes.
[27:31.560 --> 27:37.100]  So in this case, what I did is I took a simple C program with a for loop.
[27:38.320 --> 27:40.620]  I compiled it, I ran it, it works.
[27:41.200 --> 27:42.800]  And then I wrote another version.
[27:43.360 --> 27:47.340]  It's the same program except the loop counter is five instead of four.
[27:48.580 --> 27:49.380]  Compile that.
[27:50.300 --> 27:53.480]  Now the thing as I do here is I read in both those files.
[27:54.560 --> 27:56.780]  So that expression, I'm doing an F read.
[27:57.160 --> 28:02.820]  That little gobbledygook there at the amper.unbox means each.
[28:03.020 --> 28:06.360]  So it means apply the unbox operation to things on the right.
[28:07.040 --> 28:08.880]  Do the thing on the left, which is read.
[28:09.020 --> 28:12.700]  And then once you're done, apply the inverse of that so you box them back up.
[28:12.920 --> 28:17.880]  So that reads in the two files as a two element boxed array.
[28:18.420 --> 28:20.280]  And then this is multiple assignments here.
[28:20.280 --> 28:22.540]  I'm assigning them to file one and file two.
[28:23.600 --> 28:24.960]  And so I've got these two.
[28:25.040 --> 28:27.420]  I look at their shapes of both of them.
[28:27.600 --> 28:28.440]  They're the same.
[28:28.520 --> 28:29.380]  They're the same length.
[28:29.600 --> 28:30.620]  The tally is the same.
[28:31.920 --> 28:33.960]  Is file one equivalent to file two?
[28:34.580 --> 28:40.580]  Equivalence is different than equality because equivalence takes into account the shape and the type and all that stuff.
[28:40.620 --> 28:43.500]  Whereas equality is a more fine-grained operation.
[28:43.520 --> 28:44.880]  So they're not equivalent.
[28:46.520 --> 28:49.120]  I add up the number of places where they differ.
[28:49.120 --> 28:51.340]  So the little tilde colon is not equals.
[28:51.840 --> 28:53.960]  So where does file one not equal file two?
[28:54.040 --> 28:54.900]  And how many of those are there?
[28:55.200 --> 28:58.720]  Turns out there's only 22 bytes that differ between the two files.
[28:59.200 --> 29:00.320]  So what are these bytes?
[29:00.540 --> 29:01.560]  Well, I looked them up here.
[29:02.480 --> 29:04.520]  IDOT takes a Boolean and gives you a number.
[29:04.620 --> 29:09.160]  So if you have like 0, 1, 0, it gives you a 1 because there's a 1 in the 1th place.
[29:09.600 --> 29:15.060]  So it takes a Boolean and turns it into a vector of integers where you have a 1 in your Boolean.
[29:15.060 --> 29:19.720]  So this gives me the list of indexes where these files differ.
[29:20.500 --> 29:23.960]  So I can take those indexes.
[29:24.060 --> 29:28.820]  And if you notice, just looking at the numbers, you've got a little number.
[29:29.580 --> 29:30.880]  You've got a kind of bigger number.
[29:30.980 --> 29:32.480]  Then you've got a much bigger number.
[29:32.640 --> 29:35.060]  And then it goes into this 37196.
[29:35.640 --> 29:36.640]  It turns into a sequence.
[29:37.420 --> 29:39.480]  So there's some sequence at the end there.
[29:40.300 --> 29:41.560]  So I thought, what is that sequence?
[29:41.760 --> 29:43.920]  Now, in this case, I've been using...
[29:43.920 --> 29:46.300]  These are all compiled with Visual Studio 19.
[29:46.880 --> 29:49.720]  I also have examples where I was using MinGW.
[29:50.020 --> 29:52.040]  And it gives you obviously different binaries.
[29:52.520 --> 29:56.820]  So in this case, it gives you gobbledygook for whatever that sequence is at the end.
[29:58.480 --> 30:02.640]  Depending on your compile options, it might actually have the names of the file with the path in them.
[30:02.740 --> 30:03.480]  I've noticed depending...
[30:04.060 --> 30:05.660]  I don't know the exact circumstances.
[30:05.940 --> 30:08.140]  So that's not really that interesting for what we're doing here.
[30:08.640 --> 30:11.140]  But what we want to do is take a look at the...
[30:12.340 --> 30:12.880]  Oh, yeah.
[30:13.020 --> 30:13.880]  So that's gobbledygook.
[30:13.980 --> 30:15.400]  So, well, maybe the values mean something.
[30:15.500 --> 30:17.100]  So I turn those characters into numbers.
[30:17.620 --> 30:19.500]  They don't look particularly compelling.
[30:20.060 --> 30:23.320]  But now we look at those first four, which I wasn't looking at before.
[30:23.880 --> 30:28.060]  And you notice, in those first four differences, there's a four and a five.
[30:28.680 --> 30:28.940]  Huh.
[30:29.440 --> 30:30.940]  That's the limit on our loop counter.
[30:31.580 --> 30:36.460]  Gee, I wonder what would happen if I just reached into that file and changed one of those.
[30:38.100 --> 30:43.520]  So that's the 6330 is that position where I had the four or the five, depending on which file.
[30:43.740 --> 30:45.660]  In this case, I'm going to put a nine in there.
[30:46.200 --> 30:48.760]  I'm doing a nine take of the alphabet, turn it into a character.
[30:49.140 --> 30:52.120]  At this, you know, that position, I'm sticking it into file one.
[30:52.980 --> 30:55.440]  We're naming that result file, you know, FL3.
[30:55.800 --> 30:56.980]  Then I'm writing that out.
[30:57.680 --> 30:58.580]  And then I run it.
[31:08.100 --> 31:10.220]  Now, I've used this trick on a number of things.
[31:10.280 --> 31:13.100]  I tried it with some other opaque binary formats.
[31:13.240 --> 31:15.160]  So here, I look at some image files.
[31:15.820 --> 31:22.680]  So here I created 127 by 127 white square and a 255 by 255 white square.
[31:22.860 --> 31:28.520]  I did white squares because otherwise you'd have all these differences that I'm not interested.
[31:28.520 --> 31:32.820]  What I'm trying to do is find out where JPEG stores the size information.
[31:33.200 --> 31:35.760]  So I want it to be completely the same except for the size.
[31:36.200 --> 31:37.160]  And I do that.
[31:37.300 --> 31:39.620]  I find there's...
[31:39.620 --> 31:42.600]  No, they happen to be different sizes because they're obviously different sizes.
[31:42.600 --> 31:44.600]  So I just look at the first...
[31:44.600 --> 31:46.820]  The smaller ones, 467 bytes.
[31:46.980 --> 31:49.500]  So I just look at the 467 characters.
[31:49.800 --> 31:53.940]  I find they actually have a lot of differences, 317 differences.
[31:54.680 --> 31:58.720]  So again, I compare the indexes and I see this bit about...
[31:58.720 --> 32:01.780]  You've got some small numbers, they go up to bigger numbers and so on.
[32:02.980 --> 32:08.400]  And so I think, well, let's just look at the first 17 numbers because that's the amount that will fit on the screen comfortably.
[32:09.100 --> 32:09.940]  And guess what?
[32:09.940 --> 32:16.420]  In the first 17 numbers, you've got 127, 127 in one file and 255, 255 in the other file.
[32:18.220 --> 32:19.100]  So that's...
[32:19.100 --> 32:24.580]  And then those two indexes, 19 and 23, are where those bytes reside.
[32:24.960 --> 32:32.880]  So now I can go into that file and I can tell the JPEG, I can mislead the JPEG by giving it a different number or the PNG, whatever I'm using here.
[32:33.400 --> 32:39.400]  And I do that with the PNG and it gives me garbage, as you would expect, but it doesn't break, which is interesting.
[32:39.980 --> 32:44.760]  I try it with the bigger file, I get bigger garbage.
[32:45.520 --> 32:46.980]  Then I try it with the JPEGs.
[32:48.820 --> 32:49.200]  And...
[32:50.240 --> 32:52.260]  Let's see, why did I repeat that there?
[32:52.440 --> 32:53.500]  Oh, because they're different sizes.
[32:53.680 --> 33:02.860]  And so I'm doing the same exercise with JPEG files, where I get different indexes of the sizes, but it's the same exercise of finding where the sizes are and changing them.
[33:03.060 --> 33:04.800]  And now this is a JPEG and not a PNG.
[33:05.900 --> 33:15.100]  And so I do the same thing as I take my 127 by 127, and I put 63s in there, misleading the JPEG.
[33:15.820 --> 33:17.120]  And I write it out.
[33:17.720 --> 33:19.020]  Turns out JPEG is tolerant.
[33:19.300 --> 33:22.680]  So you can't see it, but there's a white square there.
[33:23.940 --> 33:25.200]  A 63 by 63.
[33:25.440 --> 33:26.440]  So I do it again, I make it...
[33:26.440 --> 33:27.240]  Okay, that was too small.
[33:27.320 --> 33:28.160]  What if I make it too big?
[33:28.160 --> 33:29.640]  So I make it a 200 by 200.
[33:30.540 --> 33:32.080]  Well, there's another white square there.
[33:32.520 --> 33:33.920]  You have to take my word for it.
[33:34.700 --> 33:36.560]  But the JPEG is tolerant that way.
[33:36.640 --> 33:38.400]  It doesn't seem to care that...
[33:38.400 --> 33:45.220]  You know, the file ends before it reached it, but it reuses... I don't know what it does, but that tells you a little about the behavior.
[33:45.480 --> 33:46.700]  So I tried some other experiments.
[33:46.760 --> 33:48.500]  What if I just stick a random byte in there?
[33:49.180 --> 33:50.180]  The problem with...
[33:51.200 --> 33:55.000]  Even a PNG, even though it's a lossless format, it has some kind of internal format to it.
[33:55.000 --> 34:06.140]  So arbitrarily changing four bytes, I just picked, you know, 100 to 103rd byte, and I changed them to themselves plus one, whatever...
[34:06.140 --> 34:07.500]  Those are the three values.
[34:07.860 --> 34:10.200]  This greater than colon is...
[34:10.200 --> 34:11.420]  used monatically as increment.
[34:12.220 --> 34:18.000]  So I read them in, I incremented them by one, turned them back into characters, and wrote them back out to the same location.
[34:18.920 --> 34:20.120]  Now it does...
[34:20.120 --> 34:20.820]  Now it complains.
[34:20.960 --> 34:22.260]  So this is in PaintShop Pro.
[34:22.760 --> 34:24.520]  It knows it's not a valid PNG.
[34:25.680 --> 34:28.080]  And I do the same thing with the JPEG.
[34:28.500 --> 34:36.420]  And it's even better, because it completely kills PaintShop Pro, because it's not smart enough to, you know, do some checking or whatever.
[34:36.980 --> 34:38.180]  Or at least on a JPEG.
[34:39.300 --> 34:39.720]  So...
[34:39.720 --> 34:41.360]  I thought that was interesting.
[34:41.800 --> 34:49.900]  But the real reason I did the initial exercise with the for loops, I thought, gee, if I could figure out the bytes that correspond to the for loop...
[34:49.900 --> 34:50.800]  I mean, I found the limit.
[34:51.240 --> 35:00.720]  But if I could figure out, oh, these set of bytes mean I'm in a for loop, I could go into someone else's code, I could find their for loops, I could find their limit, and change that limit.
[35:01.220 --> 35:05.080]  So if I wanted to force a buffer overflow, I could raise that.
[35:05.200 --> 35:06.980]  Or a buffer underrun, I could lower it.
[35:07.480 --> 35:09.960]  The problem is, it turns out, this was harder than I thought.
[35:10.160 --> 35:12.460]  And up until last night, I thought this was going to...
[35:12.460 --> 35:15.780]  But I would show you how, you know, the exercise I did to try and figure this out.
[35:17.480 --> 35:19.220]  And, you know, it's still a work in progress.
[35:19.400 --> 35:21.340]  But it turned out, I did actually get a good result here.
[35:24.160 --> 35:28.880]  So, just because I have different C compilers on my different machines, I used a different one here.
[35:29.100 --> 35:30.940]  Because the technique is very general.
[35:31.160 --> 35:34.800]  I'm saying because I've got a, sort of a captive audience here.
[35:34.880 --> 35:38.220]  I can compile whatever thing I want and compare it to whatever I want.
[35:38.680 --> 35:42.740]  And then, trying to extend that to something I did not compile is a different exercise.
[35:42.980 --> 35:44.640]  It's more of a forensic exercise.
[35:45.180 --> 35:46.480]  So, here I have...
[35:46.480 --> 35:48.260]  Well, let's put multiple for loops in.
[35:48.520 --> 35:53.480]  And just to make it interesting, we'll use a different starting count and ending count.
[35:53.820 --> 35:56.200]  And we'll do two of these with different numbers.
[35:57.040 --> 36:01.600]  So, 1, you're 2, 3, 4, 5 for the start, 13, 16, 17, 19.
[36:02.420 --> 36:08.620]  So, I do the same exercise, reading them both in, finding where they differ, the indexes of where they differ.
[36:09.540 --> 36:11.380]  And then, I look at the values in those places.
[36:11.400 --> 36:12.020]  And what do you know?
[36:12.120 --> 36:13.200]  There's all the loop counters.
[36:13.620 --> 36:18.520]  Now, I don't know if it's minGW or if it's the fact that I specified a non-zero start.
[36:18.960 --> 36:23.460]  But you notice that the ending portion is one less.
[36:24.280 --> 36:27.840]  So, like the first one there, it goes from 13 to 15.
[36:28.140 --> 36:29.940]  Down here, it shows you 13 to 14.
[36:31.120 --> 36:33.900]  And so, I don't know, does it...
[36:33.900 --> 36:35.980]  Because I didn't check this with Visual Studio 19.
[36:35.980 --> 36:40.400]  I don't know if it's a difference in the compilers or the difference of specifying a non-zero starting point.
[36:40.480 --> 36:41.680]  Anyway, but I don't care.
[36:41.680 --> 36:43.120]  Because I know what they are and I know where they are.
[36:43.180 --> 36:43.880]  And I can change them.
[36:44.980 --> 36:46.080]  But the exercise here...
[36:46.500 --> 36:46.620]  Oh, yeah.
[36:46.700 --> 36:49.580]  And also, we notice we have...
[36:51.320 --> 36:54.980]  Notice that the starting counter is at 2656.
[36:55.480 --> 36:58.680]  And the ending counter is at position 2707.
[36:59.460 --> 37:00.140]  So, it's weird.
[37:00.140 --> 37:03.480]  It's putting the start in one place and the end over here.
[37:03.600 --> 37:04.540]  It's not keeping them together.
[37:06.040 --> 37:07.860]  And that's true of the other ones as well.
[37:07.940 --> 37:09.980]  So, what I do is say, Well, I wonder how far apart those are.
[37:10.140 --> 37:14.400]  So, I say, Why don't I take the successive differences of that chunk of indexes?
[37:14.440 --> 37:15.400]  So, that's what I do here.
[37:16.480 --> 37:18.380]  And I find the first difference is 51.
[37:18.620 --> 37:20.380]  And the second difference is 6.
[37:20.500 --> 37:21.900]  So, that's the first loop.
[37:21.900 --> 37:23.520]  It has...
[37:23.520 --> 37:26.920]  The starting counter is 51 before the ending limit.
[37:27.200 --> 37:28.520]  And then the next...
[37:28.520 --> 37:32.060]  And then the start of the next one is 6 away.
[37:32.460 --> 37:35.860]  So, it looks like each for loop takes up 57 bytes.
[37:36.620 --> 37:40.620]  And that the relevant numbers are in these positions.
[37:49.490 --> 37:58.690]  So, I'm just reiterating those numbers correspond to, you know, where the 2 is and the 4 is in that first loop and so on.
[38:00.090 --> 38:07.550]  So, what I'm going to do here is I'm going to attempt to figure out what defines a for loop by looking at the bytes around that.
[38:07.550 --> 38:15.790]  So, what I do is I take the positions that are associated with the counters in the for loop.
[38:17.210 --> 38:19.990]  I back it up a little because I don't know if that's the actual start of it.
[38:20.150 --> 38:21.350]  I subtract 6.
[38:21.790 --> 38:29.970]  And then I construct a set of indexes for each for loop, which should be grabbing the bytes around where the counters are.
[38:30.450 --> 38:31.850]  And then I look at those bytes.
[38:32.230 --> 38:33.170]  And what I see...
[38:33.170 --> 38:41.070]  So, this is the first one, the first file going to 2, the second one going to 13.
[38:41.990 --> 38:46.410]  And there is this little string of bytes before it, 199.69.
[38:46.810 --> 38:51.710]  The reason I need to put the 252 in is you look at the next set of indexes.
[38:52.170 --> 38:56.530]  You've got the 199.69, but you've got a 248 there instead of a 252.
[38:56.710 --> 38:58.910]  Look at the next loop, you've got a 244 there.
[38:58.910 --> 39:03.070]  So, that's something the compiler is doing to keep track of the different loops.
[39:04.350 --> 39:06.830]  But the thing is the same as that 199.69.
[39:07.250 --> 39:10.530]  And there's other things that you can find that are like that.
[39:10.650 --> 39:11.070]  They're the same.
[39:11.190 --> 39:16.910]  In fact, that 252 reoccurs a couple times as the 248 does and 244.
[39:17.650 --> 39:19.970]  So, that's some internal bookkeeping thing it's doing.
[39:20.170 --> 39:22.890]  But the nice thing about this is I don't actually have to understand all that.
[39:23.750 --> 39:26.090]  Because I'm just sort of hacking away blindly here.
[39:26.510 --> 39:33.070]  So, the next thing I do is I take these byte sequences, which look interesting, and I count them in one of the files.
[39:34.210 --> 39:37.150]  So, that 199.69, it occurs four times.
[39:37.330 --> 39:40.310]  I have four for loops, four for loops.
[39:40.850 --> 39:41.670]  So, that's good.
[39:41.870 --> 39:44.310]  And the other one also occurs four times.
[39:44.430 --> 39:46.390]  So, that's probably also associated with the for loop.
[39:46.810 --> 39:49.830]  In fact, all the ones that I checked, there were four occurrences.
[39:51.210 --> 39:54.930]  So, maybe, okay, maybe one of those indicates a for loop.
[39:55.130 --> 39:58.050]  So, now, I'm going to grab an executable that someone else wrote.
[39:58.690 --> 40:04.430]  So, I like to play Bejeweled, which for those of you younger people in the audience, that's what Candy Crush used to be.
[40:06.350 --> 40:09.330]  And so, I had this Bejeweled executable, which I did not write.
[40:09.970 --> 40:13.190]  And I read it in as a string of bytes.
[40:13.670 --> 40:16.050]  And I look for those sequences that I found up above.
[40:16.470 --> 40:21.250]  Turns out there's zero, zero, oh, there's almost 2,000 of that 199.69.
[40:21.990 --> 40:24.190]  So, maybe that's a... I don't know if that's a for loop.
[40:24.370 --> 40:26.610]  Or, again, it could be dependent on the compiler.
[40:26.810 --> 40:28.510]  It could be dependent on compiler switches.
[40:28.530 --> 40:30.070]  There's a lot of variation there.
[40:30.890 --> 40:32.250]  So, I was just playing around.
[40:32.350 --> 40:33.590]  I said, okay, let's just poke around here.
[40:33.710 --> 40:38.090]  Where do I find the 199.69 out of these almost 2,000 instances?
[40:38.650 --> 40:40.510]  So, I see, I get that bunch of numbers there.
[40:41.110 --> 40:43.910]  And so, I just start grabbing the numbers, looking.
[40:44.090 --> 40:45.870]  And so, I grab that second one.
[40:46.170 --> 40:48.010]  And I'm like, oh, there's the 199.69.
[40:48.230 --> 40:48.670]  Oh, look.
[40:49.130 --> 40:49.990]  There's a 240.
[40:50.390 --> 40:54.030]  Remember how we saw the 252, the 248, the 244?
[40:54.410 --> 40:56.450]  And then a number indicating the loop counter.
[40:56.830 --> 41:00.170]  So, that looks suspiciously like it might be a for loop.
[41:00.950 --> 41:03.150]  And so, I just, okay, I'm going to change that 8 to a 9.
[41:03.670 --> 41:04.490]  And I do that.
[41:04.770 --> 41:05.530]  See if it runs.
[41:05.710 --> 41:06.430]  Okay, that runs.
[41:06.910 --> 41:08.150]  I do it for the next one.
[41:08.370 --> 41:09.030]  Okay, it runs.
[41:09.410 --> 41:13.670]  And I was doing this late last night, finishing up this talk.
[41:13.850 --> 41:15.350]  And I figured, okay, this is where I'm going to end it.
[41:15.490 --> 41:17.270]  This is, you know, a project in progress.
[41:17.890 --> 41:18.570]  Then I got one.
[41:18.690 --> 41:18.890]  Okay.
[41:19.390 --> 41:23.690]  That one, the game started to run, but it froze when I changed that one.
[41:24.450 --> 41:26.550]  And then, finally, I found this one.
[41:27.310 --> 41:30.290]  Which you can't tell, because what happened here is I started to play the game.
[41:30.830 --> 41:33.010]  And I did a little cascade there.
[41:33.190 --> 41:34.730]  And it showed me the 210 points.
[41:34.890 --> 41:36.730]  And then it froze at that point.
[41:37.350 --> 41:38.870]  And the counter just started going up.
[41:39.830 --> 41:44.290]  So, basically, if I wanted to get the high score, I've kind of figured out a way to do it here.
[41:45.430 --> 41:48.250]  And so, as I left today, that counter was still running.
[41:48.490 --> 41:49.730]  It was at, this is at a million.
[41:50.090 --> 41:52.590]  The most I've ever done in this game is like 150,000.
[41:52.950 --> 41:53.930]  So, that's way better.
[41:54.230 --> 41:55.770]  And I left it running when I left today.
[41:55.930 --> 41:58.190]  So, I'm going to see what happens when it hits 10 million.
[41:58.650 --> 42:01.730]  If there's any kind of, you know, if they have accounted for that or not.
[42:02.990 --> 42:04.050]  But it'll take a while.
[42:05.110 --> 42:09.050]  And just to show you again the power of J, as I said, okay.
[42:09.390 --> 42:11.610]  I wonder how long that's going to take to hit the 10 million.
[42:11.790 --> 42:15.370]  Well, I look at the counter, and I note the time.
[42:15.610 --> 42:20.710]  So, QTS stands for timestamp, as an empty argument.
[42:21.050 --> 42:26.370]  So, that shows me at the counter, you know, 1,210,904 was at that time.
[42:26.550 --> 42:27.310]  I wait a while.
[42:27.530 --> 42:29.690]  I look at the counter again, do the timestamp.
[42:29.690 --> 42:37.490]  And so, I can tell, because I have date-time utilities, I've written to take differences of timestamps.
[42:37.750 --> 42:42.330]  So, the difference between those two points was 364 point-whatever seconds.
[42:43.310 --> 42:55.850]  This returns it in both seconds and years, months, days, you know, the whole timestamp, which is problematic because you can't really... months have different number of days.
[42:56.070 --> 42:57.930]  So, it's a sort of problematic measure.
[42:57.930 --> 42:59.090]  The number of seconds is much more useful.
[42:59.910 --> 43:02.210]  And so, I assign the number of seconds.
[43:03.530 --> 43:06.370]  I count how many points left we have to hit 10 million.
[43:06.570 --> 43:07.330]  It's that number.
[43:07.790 --> 43:09.810]  And I divide that.
[43:10.110 --> 43:14.730]  I take the difference of the...
[43:14.730 --> 43:16.370]  I should have points to doing there.
[43:17.790 --> 43:19.470]  I take the points per second.
[43:19.510 --> 43:24.930]  So, that's the difference between those two counters and the time divided by the number of seconds.
[43:25.150 --> 43:26.430]  So, I'm doing...
[43:26.430 --> 43:28.650]  My counter's going up 289 every second.
[43:29.230 --> 43:31.610]  And then I divide that into how many points I have left.
[43:31.610 --> 43:33.870]  And it's going to take 30,000 seconds.
[43:34.210 --> 43:34.650]  Okay.
[43:34.750 --> 43:35.550]  Well, that's not too useful.
[43:35.830 --> 43:37.170]  What's 30,000 seconds?
[43:37.390 --> 43:45.290]  Well, it turns out, using the anti-base, which we've seen before, and giving it a mixed base, 0, 60, 60.
[43:45.410 --> 43:46.950]  Zero means, okay, we're done.
[43:47.050 --> 43:48.170]  Just dump the remainder here.
[43:48.170 --> 43:51.930]  So, that's hours, minutes, and seconds.
[43:52.410 --> 43:56.690]  So, it's eight hours, 20 minutes, and 34 seconds is that 30,000 seconds.
[43:57.370 --> 43:58.670]  So, this was last night.
[43:58.830 --> 43:59.870]  I think it wasn't going to finish in the morning.
[43:59.890 --> 44:01.770]  So, I left it running.
[44:01.790 --> 44:03.210]  And we'll see if it hits it.
[44:03.870 --> 44:05.390]  Anyway, that's sort of my...
[44:05.390 --> 44:08.210]  And this is all just, as I said, you know, minutes to do this.
[44:08.450 --> 44:10.290]  Fairly complex things, I think.
[44:10.850 --> 44:13.430]  And that's what I see as the real power of this.
[44:13.450 --> 44:17.790]  And I'd really like to see more people using this, because it's an extremely useful language.
[44:17.790 --> 44:18.290]  Which is free.
[44:18.530 --> 44:19.130]  It's open-source.
[44:19.450 --> 44:22.650]  It runs on Linux, and Mac, and Windows, and Pi.
[44:23.130 --> 44:26.870]  It has a very active user community that is very helpful.
[44:27.190 --> 44:30.430]  I run my meetup every month.
[44:32.930 --> 44:35.790]  So, the main site is jsoftware.com.
[44:36.090 --> 44:37.950]  And that's my meetup there in the middle.
[44:38.750 --> 44:41.210]  We've been meeting monthly for almost 20 years now.
[44:42.750 --> 44:49.330]  And the pandemic was actually a big blessing, because we used to meet in person at some place where you could drink beer.
[44:50.230 --> 44:51.710]  And I got a certain turnout.
[44:51.850 --> 44:54.770]  But now that I'm doing it all on Zoom, two things.
[44:54.870 --> 44:57.450]  It forces me to get everything up online before the meeting.
[44:57.610 --> 44:59.990]  So, I don't have gaps like I do in my previous meetings.
[45:00.150 --> 45:06.370]  And plus, I'm getting a nice, solid group of people from, you know, Australia and the west coast of the U.S. and such in my meetings who normally wouldn't be able to attend.
[45:09.710 --> 45:11.790]  So, it's been kind of a boon for me.
[45:12.730 --> 45:13.850]  And there's an interesting...
[45:14.370 --> 45:21.010]  The essays on j are very sophisticated essays on what people have done with the language.
[45:22.890 --> 45:34.530]  And the minimal j, since the language is so daunting, I tried to put together a minimal subset of some of the most basic things so you could learn a few little symbols first and kind of work your way up.
[45:34.530 --> 45:44.070]  I like to say, on that whole big page of things that I showed you at the beginning there, I used to... for years, I was only maybe three-quarters of the way down the page.
[45:44.550 --> 45:46.450]  And even now, there's still stuff on that page.
[45:46.530 --> 45:47.510]  I don't really know what they do.
[45:48.130 --> 45:50.850]  But I'm pretty much considered an expert J programmer.
[45:51.070 --> 45:53.370]  So, you know, you shouldn't let that stop you.
[45:54.130 --> 45:58.550]  And then this is a real Zippy the Pinhead cartoon, which for some reason is apropos.
[46:01.230 --> 46:03.170]  And any questions or comments?
[46:04.750 --> 46:05.250]  Yes?
[46:05.950 --> 46:06.630]  Thank you.
[46:13.300 --> 46:17.120]  Maintainable are, like, large APL-style codebases, like...
[46:17.680 --> 46:19.700]  Or do large codebases like that exist?
[46:19.740 --> 46:22.360]  Or is it mostly, like, how, like, one-liners to do, like...
[46:23.180 --> 46:24.500]  That's a good point.
[46:25.180 --> 46:28.240]  Because of the terseness, it can work against readability.
[46:29.440 --> 46:31.220]  However, I did...
[46:31.220 --> 46:40.840]  My last employment, before I decided to retire, I was working on an APL codebase that was a couple million lines long.
[46:41.200 --> 46:44.620]  And has been in development since the 1980s.
[46:45.240 --> 46:46.760]  And is still in operation.
[46:47.440 --> 46:53.520]  However, it is not one I would recommend as a poster child, because it has evolved over the years.
[46:54.860 --> 46:57.860]  So, yeah, it can be a problem maintaining a large codebase.
[46:58.000 --> 47:00.740]  But on the other hand, it's really hard to get to a large codebase.
[47:01.200 --> 47:05.260]  I mean, other systems I worked on, I worked on an APL trading system.
[47:05.760 --> 47:11.300]  And it was maybe 15,000 lines of code.
[47:11.840 --> 47:13.180]  And it was a...
[47:13.180 --> 47:14.260]  It did a...
[47:14.260 --> 47:18.460]  We traded stocks, bonds, and currencies in ten different countries.
[47:18.740 --> 47:21.200]  And it was a Bayesian statistical model.
[47:21.360 --> 47:23.840]  So it was a pretty sophisticated statistical model.
[47:23.980 --> 47:25.680]  And it was 15,000 lines of code.
[47:27.520 --> 47:34.120]  Subsequently, the guy who wrote the model left the company, along with a bunch of other people when we got taken over by Deutsche Bank.
[47:34.560 --> 47:37.760]  And they went to another bank and started running the same system.
[47:38.000 --> 47:39.720]  Only at that point, he rewrote it in J.
[47:40.980 --> 47:46.440]  So, you know, if anyone ever came after him for stealing code, he could say, look, these are clearly very different.
[47:47.820 --> 47:54.300]  And of the few people in the world who know both languages and could say otherwise, well, I'm his friend, so I'm not going to say that.
[47:56.780 --> 47:57.960]  But yeah, it's really...
[47:57.960 --> 47:59.940]  And this way, talk about unmaintainable code.
[47:59.940 --> 48:03.360]  So this guy, he's really into tacit coding.
[48:03.360 --> 48:05.160]  And he runs a large book.
[48:05.320 --> 48:07.380]  He runs like a billion dollars with his J code.
[48:07.840 --> 48:09.640]  And he's really into...
[48:09.640 --> 48:12.700]  There's a point-free form of J where you don't have variables.
[48:12.700 --> 48:13.800]  It's just the symbols.
[48:14.360 --> 48:16.020]  And his whole system is like that.
[48:16.700 --> 48:20.140]  And so he doesn't even bother, like, encrypting it or anything.
[48:21.220 --> 48:23.980]  Because, you know, no one but him can possibly understand.
[48:23.980 --> 48:25.580]  He does have one guy who works with him.
[48:27.020 --> 48:29.160]  But most people wouldn't even realize it's code.
[48:29.620 --> 48:32.320]  If you look at a line of J code, you'd probably think it's line noise.
[48:32.620 --> 48:35.100]  But, you know, it's a pretty large system.
[48:35.320 --> 48:38.940]  And he basically, he and one other guy maintain it.
[48:38.980 --> 48:41.040]  And they run billions of dollars on it.
[48:42.340 --> 48:44.140]  So it's definitely doable.
[48:46.840 --> 48:48.680]  Any other questions or comments?
[48:48.680 --> 48:53.320]  Is the code compilable for execution by someone who didn't write it?
[48:53.560 --> 49:09.140]  For example, if I made something that used encrypted a message, put it into a picture like you've been doing there, could I make a tool that encrypts and decrypts using J that's been compiled for someone else?
[49:09.340 --> 49:10.800]  Okay, J is only interpreted.
[49:10.920 --> 49:11.580]  It is only interpreted.
[49:11.660 --> 49:11.780]  Yeah.
[49:11.940 --> 49:16.540]  I mean, there are all sorts of easy obfuscations you can do.
[49:18.000 --> 49:20.800]  But no, it's all, you know, open.
[49:21.140 --> 49:23.720]  And the interpreter itself is open-source.
[49:25.140 --> 49:26.180]  And, you know.
[49:26.280 --> 49:28.860]  You can see the upper battle to teach that to someone though.
[49:29.060 --> 49:29.400]  Mm-hmm.
[49:29.520 --> 49:29.700]  Yeah.
[49:31.660 --> 49:32.120]  Yes?
[49:32.620 --> 49:40.540]  Is it fully composable that there's any adverb that can be combined with any verb and any adverb or verb combination can be...
[49:40.540 --> 49:40.960]  Yes.
[49:41.500 --> 49:50.440]  In fact, I once had this idea that I'm gonna make it, okay, I'm gonna make a bejeweled but using J symbols instead of jewels.
[49:50.760 --> 49:58.100]  And if you put together a valid, you know, sequence of three or more symbols, it'll do the, you know, it'll collapse and do all the bejeweled, candy crush stuff.
[49:58.500 --> 50:05.580]  And so as a test of concept, I'd say, okay, let's just take a random 10 by 10 matrix of J symbols and then evaluate it.
[50:05.580 --> 50:13.200]  It turns out in that 10 by 10 matrix, there are hundreds of valid three or more sequences already there.
[50:13.520 --> 50:16.120]  Because, like I said, the language is kind of promiscuous in that sense.
[50:16.180 --> 50:18.360]  You can just jam stuff together and it works.
[50:18.620 --> 50:19.640]  It does something.
[50:19.820 --> 50:21.120]  It might not do what you want.
[50:21.760 --> 50:27.680]  But in this 10 by 10, every other one, there was even a 10 symbol sequence that was valid.
[50:27.680 --> 50:30.860]  So the game would never work because it would just sit there playing itself.
[50:32.560 --> 50:38.340]  But that's, yeah, and no other language is really like that, that you can just combine things and they do something.
[50:39.500 --> 50:39.940]  Yes?
[50:49.540 --> 50:51.400]  NumPy got a lot of its ideas from APL.
[50:52.940 --> 50:54.820]  And it's way more composed.
[50:54.980 --> 50:58.160]  And like I say, it's extremely regular.
[50:58.780 --> 51:01.900]  And it's saying verb, adverb, noun.
[51:02.260 --> 51:03.840]  And that will do something.
[51:03.840 --> 51:06.320]  It usually will do something and not give you an error.
[51:06.420 --> 51:07.340]  Which can also be a problem.
[51:08.020 --> 51:13.540]  If you're, especially if you're in the, you know, dick around until a compiled school of programming.
[51:14.040 --> 51:15.080]  That's not going to work here.
[51:18.060 --> 51:18.600]  Any other?
[51:19.160 --> 51:19.500]  Yes?
[51:19.680 --> 51:19.860]  Okay.
[51:19.940 --> 51:20.760]  I think we're out of time.
[51:21.240 --> 51:21.760]  Thank you all.
[51:22.300 --> 51:24.220]  And please get in touch with me if you'd like.