[00:01.160 --> 00:04.900] I guarantee you we will not fill the whole hour, but there will be plenty of time for Q&A at the end. [00:08.400 --> 00:11.200] So here's, I don't know, about three quarters of the networking team. [00:12.160 --> 00:13.860] I'll let everybody introduce themselves in a moment. [00:14.180 --> 00:16.460] But what did people think? [00:16.580 --> 00:17.100] Did it seem to work? [00:23.050 --> 00:28.710] As of right now, I believe it's the first time we've ever not had a major network failure during the con. [00:34.680 --> 00:37.320] And I'll let people introduce themselves and then we'll get started. [00:37.560 --> 00:39.080] So I'm Mike or Dragorn. [00:39.300 --> 00:43.840] I do Wi-Fi stuff and overall wrangling network stuff. [00:45.720 --> 00:46.380] Hi, I'm Bill. [00:46.560 --> 00:48.500] I'm the network manager at Cold Spring Harbor Lab. [00:48.680 --> 00:49.980] Plugged in a few devices here. [00:51.800 --> 00:52.840] I'm Nick Binary. [00:53.140 --> 00:55.080] I have 15 years experience with Internet pumps. [00:56.640 --> 00:57.500] I'm Cedric. [00:57.680 --> 00:59.640] I'm a general UNIX sysadmin kind of guy. [01:01.100 --> 01:01.900] Hi, I'm Fred. [01:02.220 --> 01:04.260] I drove some of the gear here and took some photos. [01:06.740 --> 01:07.610] I'm Porkchop. [01:07.970 --> 01:11.300] Here I did some of the wiring and otherwise I'm a network admin by trade. [01:12.880 --> 01:15.880] And also hiding back there is Pyro who wrangled... [01:15.880 --> 01:16.280] Woo! [01:16.540 --> 01:17.040] Wave more. [01:17.240 --> 01:17.920] Stand up. [01:18.220 --> 01:18.640] Stand up. [01:18.880 --> 01:19.860] Gonna embarrass you. [01:20.160 --> 01:21.480] He's the man who wrangled... [01:21.480 --> 01:21.560] He's got applause. [01:22.260 --> 01:23.840] Yeah, he's Santa Claus. [01:24.080 --> 01:27.920] He wrangled all the dishes, the uplink, the bandwidth, the IP space. [01:28.260 --> 01:30.940] So, you know, the fiber, it wouldn't have worked without him. [01:31.940 --> 01:32.460] So... [01:39.500 --> 01:40.160] All right. [01:40.600 --> 01:41.120] All right. [01:42.700 --> 01:46.340] Basically, this year we have a 50 megabit uplink to knack.net. [01:46.460 --> 01:49.780] This is a picture of the antenna of the dish on the roof. [01:49.780 --> 01:51.580] It points to Times Square. [01:52.880 --> 01:54.260] Any more specifics on that? [01:54.440 --> 01:55.720] Like, whose office that goes to? [01:56.380 --> 01:58.160] Was it rainbow.net or something like that? [01:58.940 --> 02:05.240] Yeah, it's going through rainbow as the provider, which is also the provider that the hotel uses for their commercial Internet service. [02:06.640 --> 02:12.980] And they did a great job getting us a link from a hotel that doesn't have a lot of uplink capability. [02:12.980 --> 02:18.360] So, if anybody remembers, you know, six years ago we were looking at DSL lines and a megabit. [02:18.580 --> 02:21.980] And now we have 50 shot across Manhattan. [02:22.700 --> 02:23.600] That's pretty cool. [02:23.920 --> 02:24.600] It's pretty stable. [02:26.540 --> 02:33.020] The hardware of the network this year consisted of a core 6509, dual suit twos, that's gigabit and copper line cards in it. [02:33.220 --> 02:38.120] The edge consisted of Cisco 3550s and 3560 PoE switches. [02:38.480 --> 02:41.640] And the wireless edge is 18 Aruba AP70 access points. [02:42.820 --> 02:45.760] There's a picture of... let's see if I can zoom this out. [02:47.000 --> 02:48.300] That helps cool the switch down. [02:48.520 --> 02:49.900] It's vital for performance of the network. [02:50.080 --> 02:50.140] Yes, it is. [02:53.780 --> 03:01.040] All packet filtering was done on a PIX 535 and services provided with BSD jails running under FreeBSD. [03:02.240 --> 03:03.960] That's a picture of the Aruba access controller. [03:04.160 --> 03:07.640] Yeah, so everybody's Wi-Fi traffic went through that one wire to the one little box. [03:11.060 --> 03:13.220] This is a simplified picture of the network architecture. [03:13.780 --> 03:17.880] Basically, we have our 50 megabit link coming in to a switch here on the 18th floor. [03:18.160 --> 03:20.540] There's another switch located under this stage. [03:21.140 --> 03:24.160] 13 access points are strapped to the various poles around here. [03:24.680 --> 03:29.180] And then we have a multi-mode fiber run that goes from the 18th floor down to the second through the stairwell. [03:29.900 --> 03:33.500] The hotel really didn't have any infrastructure to do anything else. [03:33.860 --> 03:36.940] There wasn't any risers or anything like that, so we have to run our own. [03:37.840 --> 03:39.280] There are risers in the hotel. [03:39.440 --> 03:42.500] Unfortunately, they are on the other side of the elevators where everybody's walking. [03:42.700 --> 03:43.960] Sorry, accessible risers for us. [03:43.960 --> 03:50.140] So we could have everybody walk on the network cable and break it because there's, you know, 2,000 people walking over it 10 times a day. [03:50.280 --> 03:51.600] Or we could run it down the stairwell. [03:52.100 --> 03:52.540] All right. [03:52.740 --> 03:55.540] So the core of the network, again, consisted of the 6509. [03:55.940 --> 03:59.300] There's the PIX535, the various VLANs are chunked between it. [04:00.260 --> 04:03.360] The edge switches on the second floor, we had a colo cage switch. [04:03.640 --> 04:07.740] We had two mezzanine switches that connected to the various data jacks on the second floor. [04:07.740 --> 04:11.480] There was the fiber Aruba AP70s deployed. [04:11.940 --> 04:14.040] The AMD project had a dedicated switch. [04:14.340 --> 04:15.420] And then there's a controller. [04:16.360 --> 04:18.960] Also up on the 18th floor, people may have noticed or not. [04:19.560 --> 04:25.900] We learned in previous years that if we're going to run cables to every AP up here, we're looking at about 8,000 feet of Cat5. [04:26.540 --> 04:29.960] So what we're actually running is a dual channel 5 gigahertz mesh. [04:30.180 --> 04:39.420] So the APs on the poles in the back act as mesh controllers that pipe to one of the APs stuck to the poles here that then tunnel the other APs back in as if they were plugged into an Ethernet. [04:40.300 --> 04:41.340] Yep, that's pretty cool. [04:41.760 --> 04:46.500] So we saved on a couple hours worth of effort and a couple hundred bucks worth of cabling. [04:47.380 --> 04:48.600] Thank you for that, by the way. [04:49.740 --> 04:50.060] Yeah. [04:51.060 --> 04:53.220] He's speaking as the person who got told to do the cabling. [04:55.600 --> 04:59.700] Right, so here's a basic breakdown of the IP allocation services. [05:00.360 --> 05:07.300] We had quite a few more VLANs than we had in the past hope just because of the various services and different access levels that people required. [05:09.280 --> 05:17.280] Everything of the Wi-Fi except for the public networks was piped through this Pix535, which had multiple interfaces and multiple VLANs. [05:17.300 --> 05:17.780] I just have a question. [05:17.880 --> 05:19.600] Is there any Cisco employees in the audience right now? [05:20.820 --> 05:21.200] No? [05:21.680 --> 05:21.820] Okay. [05:21.960 --> 05:22.480] Just wondering. [05:22.660 --> 05:32.440] So one of the reasons for all these interfaces, I found this really interesting issue where if you go and span the same VLAN to multiple contexts, the firewall just stops processing ARP. [05:32.800 --> 05:37.060] And so I figured that out around, I don't know, 4:30 in the morning on Wednesday. [05:37.480 --> 05:39.120] Not a really good time to figure that out. [05:39.260 --> 05:43.780] So I just got around that by just basically presenting more physical interfaces to the firewall. [05:44.120 --> 05:48.320] So the firewall itself is carved up into three different contexts here. [05:48.820 --> 05:56.440] An admin that's completely isolated from any other user traffic or the data plane of it, just so, you know, higher security. [05:56.620 --> 05:58.720] There's only one IP allowed to manage it. [05:59.240 --> 06:02.600] And there was another context on there just for admin traffic. [06:02.880 --> 06:08.700] And then the rest were the various VLANs, the WPA, and the open, and then a wired NAT network. [06:10.300 --> 06:12.480] Services are deployed through FreeBSD jails. [06:12.640 --> 06:13.500] We had five of them running. [06:13.880 --> 06:20.300] Bind DNS, ISC's DHCP, syslog service, MRTG, and NetFlow collector using NTOP. [06:23.880 --> 06:25.500] And now we have some stats. [06:28.360 --> 06:36.060] On the Wi-Fi side, everything ran through the Aruba controller, and then all the WPA stuff was terminated on the Aruba controller as well using an internal user. [06:36.580 --> 06:50.920] You can do it externally with a love radius server and dedicated user accounts, but I, you know, carried a lot of this gear down, and a lot of it got shipped down in a car, so we didn't want to carry more gear. [06:51.380 --> 06:54.440] And I'm just talking while you're all reading that slide because it's a lot funnier. [06:58.420 --> 06:58.900] All right. [06:59.020 --> 06:59.620] Some network stats. [07:00.980 --> 07:06.200] We showed around 1,200 unique MAC addresses that were handed out by the DHCP server. [07:06.960 --> 07:07.100] Around... [07:07.100 --> 07:07.260] 2,100. [07:07.260 --> 07:08.920] 2,100, excuse me. [07:10.480 --> 07:13.820] Around 20 million flows throughout the weekend through the NetFlow collector. [07:14.700 --> 07:17.500] Two mesh networks, as Dragon explained, for the 18th floor. [07:18.800 --> 07:23.940] Also on the Wi-Fi, we normally saw about 200 people on at any given time. [07:24.320 --> 07:28.880] So about 10% of everybody here was using the Wi-Fi at any given point. [07:28.880 --> 07:32.760] But there were some, you know, obvious significant peaks and valleys throughout the day for that. [07:33.540 --> 07:34.820] You want to talk about the... [07:34.820 --> 07:35.240] Oh, yeah. [07:35.380 --> 07:38.280] So we were running IDS stuff on the Wi-Fi the whole time. [07:38.740 --> 07:42.820] We saw about 50 AP spoofing attacks, which is, you know, all right. [07:43.020 --> 07:43.400] Big deal. [07:43.780 --> 07:44.260] Good work, guys. [07:45.280 --> 07:46.420] About 10 DOS attacks. [07:46.560 --> 07:49.280] I'm glad people finally figured out that it's kind of boring to do that, so... [07:49.280 --> 07:50.340] It's just annoying now. [07:50.400 --> 07:50.860] I mean, come on. [07:50.940 --> 07:51.060] Yeah. [07:52.340 --> 07:55.960] Also, we had so many APs up that it would be pretty hard for you to take them all out at once. [07:55.960 --> 07:57.900] And that's really not a challenge, but go ahead. [07:58.020 --> 07:58.780] You got an hour left. [07:59.140 --> 07:59.240] Yeah. [08:01.100 --> 08:05.460] I think the load on the 6500 got up to 2% max throughout the weekend. [08:05.760 --> 08:06.380] So, yeah. [08:07.300 --> 08:10.760] There were about 20 non-HOPE APs that were plugged into the network somewhere. [08:11.440 --> 08:16.500] Mainly that means our IDS system sent out a probe that got replicated out them and went, well, you're plugged into our network somewhere. [08:17.960 --> 08:18.820] So, all right. [08:19.100 --> 08:19.840] People did that. [08:20.120 --> 08:21.240] I thought it was kind of an interesting number. [08:21.240 --> 08:26.960] And the most interesting number is there's about 190 APs that aren't ours that are overlapping with us right now. [08:28.260 --> 08:33.260] So, hooray for New York City, but we had enough power to drown them out pretty effectively. [08:35.780 --> 08:39.340] This shows the bandwidth used over the 24-hour period. [08:40.100 --> 08:40.700] Not much. [08:40.800 --> 08:46.120] We didn't really go over, I think we hit 48 megabits for the speed test once, and that was about it. [08:46.840 --> 08:58.640] I think we were pretty much hitting the packet per second limit on the link more than we were hitting the bandwidth limit just because, you know, we were seeing, you know, 4,000, 5,000, 8,000 packets a second of just traffic hammering out. [09:00.180 --> 09:06.020] And this graph shows the overall throughput of the network, and that's hosts talking amongst themselves. [09:06.240 --> 09:09.580] So, we hit, what, a max of 248 megabits. [09:10.400 --> 09:11.460] Really interesting stuff. [09:11.460 --> 09:15.500] Here are some more boring slides of traffic distribution by protocol, if anybody cares. [09:16.220 --> 09:21.520] HTTP is the, you know, dominant protocol, followed by a bunch of Windows boxes, speaking NetBIOS. [09:22.680 --> 09:23.120] And... [09:23.120 --> 09:23.840] Oh, okay. [09:23.960 --> 09:24.600] So, that was on the internal. [09:24.720 --> 09:26.380] On the external link, it was an interesting breakdown. [09:26.520 --> 09:29.220] It was HTTP is number one by a factor of about 15. [09:30.060 --> 09:31.460] HTTPS came right after that. [09:32.160 --> 09:34.340] SSH came in third for the outbound traffic. [09:34.460 --> 09:35.300] That was kind of nice to see. [09:35.500 --> 09:37.600] Unfortunately, the fourth outbound traffic was POP3. [09:38.960 --> 09:39.400] Yeah. [09:39.920 --> 09:40.380] Really? [09:40.620 --> 09:41.320] Really, people? [09:41.560 --> 09:41.920] POP3? [09:42.960 --> 09:44.360] Not POP3S, mind you. [09:44.620 --> 09:45.080] POP3. [09:45.340 --> 09:45.760] Yeah. [09:46.240 --> 09:49.020] We also saw Telnet come in around number 10. [09:49.760 --> 09:50.140] Yeah. [09:51.180 --> 09:51.640] Really? [09:51.640 --> 09:55.420] I assume people are, like, tunneling something over that, but I didn't bother to look. [09:55.620 --> 09:57.900] But, yeah, I think they're tunneling stupid over Telnet, pretty much. [10:01.720 --> 10:02.420] All right. [10:02.660 --> 10:03.860] There's also a good amount of IMAP. [10:04.020 --> 10:05.040] Again, not IMAPS. [10:05.280 --> 10:06.080] Just IMAP. [10:06.360 --> 10:07.200] Actually, the... [10:07.200 --> 10:08.840] There was some IMAPS. [10:11.520 --> 10:12.100] Oh, yeah. [10:12.240 --> 10:17.840] No, there was significant POPS and IMAPS, but that's not as funny as the fact that there was also significant unencrypted. [10:19.340 --> 10:19.860] Mm-hmm. [10:20.220 --> 10:20.700] Do you have the video? [10:21.580 --> 10:21.840] Right. [10:23.000 --> 10:23.420] And... [10:23.420 --> 10:23.740] Let's see. [10:24.320 --> 10:26.260] Oh, is it shift control F? [10:27.900 --> 10:32.440] So we made a little video that was 15 minutes of the network. [10:39.030 --> 10:46.310] So this was Packet-O-Matic dumping all the images that were going over unencrypted HTTP for about 15 minutes, middle of the day yesterday. [10:46.610 --> 10:47.510] Are you sure this is right? [10:47.670 --> 10:48.650] I ain't seen no porn yet. [10:49.070 --> 10:49.630] That's what... [10:49.630 --> 10:50.130] There's... [10:51.390 --> 10:52.990] That's exactly what we said. [10:53.110 --> 10:53.790] Where's the porn? [10:54.150 --> 10:56.250] There's a couple of crotches that flash by. [10:56.250 --> 10:57.690] I'm like... [10:58.330 --> 10:59.430] Any that aren't yours? [11:01.690 --> 11:02.570] You hope... [11:02.830 --> 11:03.030] You hope. [11:03.270 --> 11:04.250] I just saw some. [11:07.210 --> 11:11.070] So that was about, you know, a gig and a half of images that went by. [11:14.990 --> 11:15.570] All right. [11:15.750 --> 11:17.230] So we did encounter one issue. [11:17.390 --> 11:19.350] And luckily that was before the con started. [11:19.770 --> 11:21.730] And that made us realize a few things. [11:22.850 --> 11:23.250] Uh... [11:23.250 --> 11:24.590] Pessimism is the new optimism. [11:31.440 --> 11:32.700] Sure it's like one month there. [11:36.320 --> 11:40.900] So the first thing we have to do is run that riser fiber from the 18th floor to the second floor. [11:41.100 --> 11:46.500] And we ended up using a new fiber last year, which was just a duplex jacketed multi-mode. [11:46.600 --> 11:48.100] Just a really, really long patch cable. [11:48.600 --> 11:51.980] And I'm assuming when it got coiled up or had some weight on it, one of the strands broke. [11:52.100 --> 11:53.800] So we spent a while trying to troubleshoot that. [11:54.380 --> 11:55.300] Couldn't get it to work. [11:55.420 --> 11:58.020] So we had to go to our backup fiber, which unfortunately wasn't long enough. [11:58.260 --> 11:59.800] And really wasn't terminated very well. [12:00.860 --> 12:01.300] So... [12:02.400 --> 12:02.840] Um... [12:02.840 --> 12:03.600] We ended up with this. [12:03.760 --> 12:05.200] And that's a banister it's taped to. [12:05.480 --> 12:08.620] And that's pretty much what the entire network is running through all weekend. [12:11.540 --> 12:14.040] Well, we hoped and prayed that nobody bumped into it. [12:14.600 --> 12:15.140] That's right. [12:15.760 --> 12:17.080] Well, it looks really ghetto. [12:17.440 --> 12:18.480] It's a good termination. [12:18.620 --> 12:19.480] It is really ghetto. [12:19.940 --> 12:20.260] Okay. [12:21.800 --> 12:22.740] But it works. [12:22.740 --> 12:22.760] Thanks. [12:23.520 --> 12:24.860] And that's it for the slides. [12:26.420 --> 12:27.400] Oh, 15 minutes. [12:27.540 --> 12:28.000] That's pretty good. [12:28.860 --> 12:29.500] Any questions? [12:31.560 --> 12:32.200] I believe... [12:32.200 --> 12:33.200] Are the microphones on in the middle? [12:34.040 --> 12:34.540] Okay, yeah. [12:34.700 --> 12:36.040] Apparently the microphones are on in the middle. [12:36.180 --> 12:37.660] So if you have questions, queue on up. [12:38.400 --> 12:44.300] And no, unfortunately, this year we don't have a photo of one of the network staff teabagging the router for anybody who was, you know, hoping to see that. [12:44.300 --> 12:44.960] But, uh... [12:44.960 --> 12:45.580] Run downstairs. [12:46.520 --> 12:46.920] Yeah. [12:47.080 --> 12:49.980] Render's offering to run downstairs and get a picture taken for us. [12:50.120 --> 12:51.540] But, uh... [12:51.540 --> 12:53.500] I saw a couple of NANDs in the network traffic. [12:53.640 --> 12:55.840] Did you shape Gnutella, et cetera, into oblivion? [12:56.040 --> 12:58.020] Or did you handle it some other way? [12:58.020 --> 12:58.900] Uh... [12:58.900 --> 13:00.100] We handled it with, uh... [13:01.160 --> 13:01.340] Um... [13:01.340 --> 13:02.240] There's a big pipe. [13:02.880 --> 13:03.520] And, uh... [13:03.520 --> 13:04.240] It's a hacker con. [13:04.660 --> 13:05.220] So... [13:05.220 --> 13:05.320] Whatever. [13:06.160 --> 13:06.720] Ambivalence. [13:07.060 --> 13:07.620] Based... [13:07.620 --> 13:08.420] It's just... [13:08.420 --> 13:11.040] There wasn't any particular filtering of protocols. [13:11.760 --> 13:12.120] Uh... [13:12.120 --> 13:13.080] Actually, I... [13:13.080 --> 13:14.320] I filtered NetBIOS inbound. [13:14.840 --> 13:15.600] Just for port scanning. [13:16.680 --> 13:17.040] Well... [13:17.620 --> 13:17.980] But... [13:17.980 --> 13:18.640] So... [13:18.640 --> 13:20.040] Porkchop actually did bring a packet here. [13:20.220 --> 13:21.960] And then we went, wow, it's four in the morning. [13:23.260 --> 13:23.520] Too bad. [13:23.700 --> 13:24.100] Too much work. [13:24.780 --> 13:26.380] It doesn't seem was necessary anyway. [13:26.500 --> 13:32.140] The only thing we were going to do is, you know, make it so that everybody could get out equally except for the speakers who could get a little bit of priority. [13:32.420 --> 13:32.720] But... [13:32.720 --> 13:34.880] But that was all we were ever planning on doing with it. [13:34.960 --> 13:38.260] Not to shape down any peer peer or anything like that. [13:38.420 --> 13:40.940] But as it turns out, you know, we didn't even really hit the max. [13:41.040 --> 13:41.700] At any point. [13:42.380 --> 13:42.820] So... [13:42.820 --> 13:43.100] Yeah. [13:43.600 --> 13:45.400] We had a plan, but it wasn't needed. [13:46.320 --> 13:46.760] Yeah. [13:46.760 --> 13:50.580] We were lucky enough that we were able to get a big enough pipe that we didn't have to worry too much about throttling. [13:52.060 --> 13:52.500] Um... [13:52.500 --> 13:55.860] Any throttling that went on was most likely caused by, uh... [13:55.860 --> 13:59.660] People tossing the Wi-Fi or just the packets per second getting so high that the, uh... [13:59.660 --> 14:01.540] The uplink stuttered a little bit or something like that. [14:02.900 --> 14:03.340] Uh... [14:03.340 --> 14:06.220] One other problem we did see speaking of brief issues was, uh... [14:06.220 --> 14:09.740] Apparently OSX doesn't like WPA2 all that well. [14:10.380 --> 14:10.820] Because... [14:10.820 --> 14:12.700] A whole bunch of people came and complained and I... [14:12.700 --> 14:14.600] Had to kind of say, I... [14:14.600 --> 14:14.820] I... [14:14.820 --> 14:15.420] I don't have a Mac. [14:16.340 --> 14:16.780] So... [14:16.780 --> 14:17.040] Okay. [14:17.460 --> 14:20.060] Windows 7 gave me more shit than my Mac did. [14:20.060 --> 14:22.420] He says Windows 7 gave him more shit than his Mac. [14:22.500 --> 14:23.680] Well, I don't use Windows 7 either. [14:24.300 --> 14:24.660] So... [14:25.700 --> 14:26.600] My Mac works. [14:26.960 --> 14:27.320] So... [14:27.320 --> 14:27.640] Okay. [14:27.980 --> 14:29.960] I found with mine I just had to... [14:29.960 --> 14:31.900] And a couple I saw that I just had to... [14:31.900 --> 14:33.980] After setting it up, uh... [14:33.980 --> 14:36.040] That I just had to, uh... [14:36.040 --> 14:38.180] Stop the wireless, turn it back on, and then it... [14:38.180 --> 14:41.160] Then it found the certificate and you were able to import it. [14:41.160 --> 14:44.460] Mine took about five minutes to give me the prompt for certificate. [14:44.820 --> 14:45.960] After I saved it up. [14:46.460 --> 14:47.340] I was testing it. [14:47.420 --> 14:48.500] It pops up and says, hey! [14:49.960 --> 14:50.380] Okay. [14:50.420 --> 14:50.600] Yeah. [14:50.700 --> 14:53.260] All I had to do was run TCP Dome to try to diagnose the problem. [14:53.520 --> 14:54.300] And, uh... [14:54.300 --> 14:55.940] It just worked the second I hit enter. [14:58.180 --> 14:58.600] Okay. [14:58.820 --> 15:02.060] One question arose during the first talk of this conference. [15:02.520 --> 15:06.740] Why did you only have legacy IP v4 instead of real Internet? [15:08.500 --> 15:09.240] And, uh... [15:09.240 --> 15:10.020] Sir, where are you from? [15:10.920 --> 15:11.740] From Germany. [15:12.700 --> 15:13.720] And where are we right now? [15:21.550 --> 15:21.990] Um... [15:21.990 --> 15:22.710] Eventually it would be nice. [15:23.090 --> 15:23.930] But, um... [15:23.930 --> 15:27.130] There really just isn't the demand to deploy v6. [15:27.290 --> 15:28.070] Nor, um... [15:28.070 --> 15:29.970] Were we able to get an allocation for that. [15:30.270 --> 15:30.710] Um... [15:30.710 --> 15:33.650] So, if you weigh the effort that it takes to get what we have today... [15:34.210 --> 15:34.570] Um... [15:34.570 --> 15:35.030] Up and running. [15:35.250 --> 15:36.750] And then put v6 on top of that. [15:37.470 --> 15:37.610] Um... [15:37.610 --> 15:40.030] The number of people that would probably utilize it would be pretty low. [15:40.190 --> 15:41.030] Sure, it's fun to play with. [15:41.210 --> 15:42.050] But, um... [15:42.050 --> 15:42.110] But, um... [15:42.110 --> 15:44.450] If not us, who will start using it? [15:44.770 --> 15:45.630] Are you volunteering? [15:46.570 --> 15:47.450] For helping with that? [15:47.530 --> 15:48.970] If you ask me on the next hope. [15:49.150 --> 15:49.650] On the... [15:49.650 --> 15:50.370] Whatever it's called. [15:50.610 --> 15:50.970] Possibly. [15:51.370 --> 15:52.930] But I would really think it would be cool. [15:53.090 --> 15:54.010] Even if it's tunneled. [15:54.110 --> 15:55.930] But just to provide it so people can play around. [15:56.050 --> 15:56.370] I mean... [15:56.370 --> 15:58.990] On German hacker conferences, we're doing it for the last years. [15:59.150 --> 16:00.990] And, of course, not everybody uses it. [16:01.070 --> 16:04.330] But it's just a good option to tell people here, try it out. [16:04.570 --> 16:05.030] It's the future. [16:05.770 --> 16:06.210] Sure. [16:06.590 --> 16:11.650] As a point of reference, not the last Nanog meeting, but the meeting before that in Dearborn, Michigan. [16:12.450 --> 16:17.850] The IPv6 stuff ended up being a total of 2% or so of the outbound traffic. [16:17.970 --> 16:19.890] This is for the North American network operators group. [16:20.070 --> 16:22.930] The people who would probably be using v6 the most. [16:23.070 --> 16:25.390] The people who have it at their corporate offices and stuff. [16:26.250 --> 16:28.350] And, yeah, that'd be great. [16:28.450 --> 16:31.050] But, yeah, you're talking about limiting by time. [16:31.510 --> 16:31.870] Yeah. [16:32.050 --> 16:33.710] I mean, I think v6 is fantastic. [16:33.950 --> 16:35.430] You know, I've played with it a bunch. [16:35.570 --> 16:41.990] But the allocation we were given for this weekend, too, is just so big that other than just people playing with it, there really wasn't much need. [16:42.010 --> 16:46.130] And you can always just set up, you know, a four to six tunnel yourself on your laptop. [16:46.330 --> 16:47.390] I mean, the Mac's really great. [16:48.330 --> 16:49.250] It's pretty much automatic. [16:49.910 --> 16:54.890] So I figured if people wanted to play with v6, they would just set up their own little, you know, microtunnel nodes. [16:57.170 --> 16:58.050] Hey, how are you guys? [16:58.210 --> 16:59.330] First of all, thank you. [16:59.510 --> 17:02.670] It was really nice to come here and have Internet without worry. [17:02.890 --> 17:03.730] I have VPN, personally. [17:06.110 --> 17:06.990] I'm glad it all worked. [17:10.430 --> 17:12.530] But my next question, I may have already been answered. [17:12.590 --> 17:13.250] I just walked in. [17:13.570 --> 17:15.250] I had no problem with DHCP. [17:15.310 --> 17:16.130] I don't know anyone that did. [17:16.130 --> 17:21.610] Were there any attempts for exhaustion or any, you know, new things that you guys saw? [17:21.970 --> 17:26.590] Did you plan for people to do DHCP exhaustion or anything like that? [17:26.770 --> 17:29.310] And follow up, which actually has nothing to do with it. [17:29.750 --> 17:34.330] What was the deal with that happy little JavaScript on badge.hope.net? [17:34.890 --> 17:35.870] What are you looking for? [17:36.970 --> 17:38.690] Yell at the badge people for that. [17:39.070 --> 17:39.290] Yeah. [17:39.510 --> 17:41.070] We are just a layer of two pipe. [17:42.450 --> 17:44.150] Yeah, we're just a common carrier. [17:44.150 --> 17:44.710] No response. [17:44.710 --> 17:44.910] Okay. [17:45.790 --> 17:46.710] I love you all anyway. [17:47.150 --> 17:47.590] I'm sorry. [17:47.650 --> 17:48.750] That came out a little meaner than I intended. [17:50.250 --> 17:50.730] Kind of. [17:52.570 --> 17:55.830] But seriously, I don't know the logistics of what they were setting up. [17:55.990 --> 17:57.970] So I don't know. [17:58.030 --> 18:00.210] I think they're probably going to have a little bit of a talk during closing ceremonies. [18:00.290 --> 18:01.870] There might be some time for some questions for them then. [18:02.690 --> 18:03.910] So they could probably tell you that. [18:04.230 --> 18:09.970] I think it was an attempt to look at your browser history through little CSS hacks to just build profiles of people. [18:10.410 --> 18:10.750] But... [18:11.610 --> 18:11.950] Yeah. [18:12.230 --> 18:12.390] Okay. [18:12.510 --> 18:13.690] Well, we noticed, so... [18:13.690 --> 18:13.930] Okay. [18:14.070 --> 18:14.210] Whatever. [18:14.970 --> 18:16.530] How about the first question there? [18:16.650 --> 18:21.590] To answer your question, I was planning on deploying D2P snooping on some of the switches. [18:21.790 --> 18:24.310] And I just didn't see anybody doing anything naughty, which is great. [18:25.010 --> 18:28.130] Oh, I wondered if you had expected it, because we would have liked to play with it. [18:28.210 --> 18:29.670] But I didn't know what the rules were. [18:29.790 --> 18:29.810] A little bit. [18:29.810 --> 18:36.630] What I ended up doing was, if you noticed, on the Open NAT and the WPA NAT, I assigned slash 22s as well as the public. [18:37.910 --> 18:41.210] And I set really, really low lease times on the DHCP server. [18:41.410 --> 18:46.050] The only DHCP issue I did notice is the MAC DHCP implementation. [18:46.630 --> 18:51.510] Sometimes it would get unhappy if you switch networks somewhat quickly. [18:53.710 --> 18:56.430] It would have issues when the DHCP server would send a NAC back. [18:56.650 --> 18:57.850] It would say, no, I still want this IP. [18:58.030 --> 18:59.010] And it would say, no, you can't have it. [18:59.110 --> 19:00.690] It would just keep going, no, I really want it. [19:00.810 --> 19:04.610] And it would just go through that iteration five or six times and finally get it. [19:04.770 --> 19:05.090] But... [19:05.090 --> 19:05.350] Cool. [19:05.350 --> 19:06.470] We did have a couple of plans. [19:06.650 --> 19:15.790] You know, if we did see something, there's some mechanisms on the Aruba AP controller to control DHCP leases and rogue DHCP servers. [19:16.050 --> 19:19.050] And then we could have just deployed the DHCP snooping on the iOS side. [19:19.910 --> 19:28.430] It's also marginally more difficult to perform DHCP flood attacks with Wi-Fi because you have to have the whole Wi-Fi handshake first before that MAC address is acknowledged by the AP. [19:28.690 --> 19:28.950] Indeed. [19:29.470 --> 19:32.990] I mean, not that that would really stop someone who is particularly determined, but, you know. [19:33.990 --> 19:40.110] Yeah, and given the nature of this environment, I didn't really want to put too many security, you know, controls in place. [19:40.330 --> 19:44.930] This is the first year I put port security on everything so you couldn't flood, you know, any cam tables on the switches. [19:45.370 --> 19:48.270] And I didn't really see any evidence, any attempts on that too. [19:48.570 --> 19:49.290] So, which is good. [19:49.710 --> 19:49.990] All right. [19:50.230 --> 19:50.910] Did you try anything? [19:51.710 --> 19:53.270] Um, not that I'm willing to admit to. [19:55.130 --> 19:56.070] No, I... [19:56.070 --> 19:57.650] For the record, it's a hacker con. [19:57.790 --> 19:59.790] We kind of expect people to screw with it, so... [19:59.790 --> 20:00.170] Oh, good. [20:00.350 --> 20:01.350] I'll know for next time then. [20:01.350 --> 20:02.390] Did anybody else raise your hands? [20:02.390 --> 20:03.470] Did anybody, you know, screw with it? [20:03.710 --> 20:03.810] All right. [20:03.810 --> 20:04.790] I noticed, um... [20:05.310 --> 20:06.350] All so respectful. [20:06.650 --> 20:07.230] Hey, listen, guys. [20:07.370 --> 20:08.290] Thank you once again. [20:08.470 --> 20:09.610] This is my first hope. [20:09.810 --> 20:12.270] And I am just amazed at what I have seen here. [20:12.450 --> 20:14.490] I saw heroes on this stage before. [20:15.030 --> 20:15.830] And, uh... [20:16.230 --> 20:17.130] One more, just for me. [20:17.350 --> 20:17.650] All right? [20:18.110 --> 20:18.790] Thank you, guys. [20:24.200 --> 20:24.920] Thank you. [20:24.960 --> 20:27.540] Uh, just one more vote for V6. [20:27.540 --> 20:27.600] Thanks. [20:27.840 --> 20:30.020] The deal is that if you never have... [20:30.020 --> 20:33.840] If no one ever turns it on anywhere, then, of course, there'll never be any traffic on it. [20:34.000 --> 20:36.840] Because, after all, no one never turns it on, so they don't have it. [20:37.040 --> 20:38.000] So they... [20:38.000 --> 20:45.720] But the thing is, if it's too much of a pain in the ass, I will get the allocation and turn it on and configure the routers if you want. [20:45.800 --> 20:46.680] And we can definitely look into that. [20:46.680 --> 20:47.600] My money wears my outfit. [20:47.840 --> 20:51.220] It's really up to the upstream provider, because should we get an allocation... [20:51.220 --> 20:54.920] You can tunnel to people, and there are people who, for a weekend, will be happy to do it. [20:55.700 --> 20:56.060] Yeah. [20:56.120 --> 21:00.960] Yeah, for the record, this is something that I'm interested in seeing at some point as well. [21:01.340 --> 21:05.080] Um, whether it's, uh, whether it's the next time we do this... [21:05.080 --> 21:06.200] I think he's volunteering. [21:06.880 --> 21:07.600] Yeah, well... [21:08.380 --> 21:09.040] Sounds like it. [21:09.140 --> 21:09.280] Yeah. [21:09.760 --> 21:10.080] But, uh... [21:10.080 --> 21:10.880] All right, Cedric. [21:11.080 --> 21:14.780] But if you want to follow up, you can send emails to, uh... [21:14.780 --> 21:21.840] You can send an email and, uh, you can get chatting, you know, as part of the preparation process for next time and see what happens. [21:22.000 --> 21:22.240] Okay. [21:25.520 --> 21:33.040] Hey, as a network admin myself, I know that sometimes when everything is miraculously working for once and, uh, the day is a bit slow, you have... [21:33.040 --> 21:38.540] One of the fun things you could do is kind of poke around and see what everybody else is doing with the network. [21:38.880 --> 21:51.540] And I remember somewhere between, hope, three and six, uh, one of the guys from the knock gave a little, uh, half hour presentation on interesting, uh, unencrypted messages he had caught while, uh, hanging out. [21:51.760 --> 21:59.300] I was wondering if you guys had found anything interesting, anything noteworthy to talk about because I haven't heard, uh, anything like that in quite some time. [21:59.500 --> 22:04.260] Um, I noticed most people port scanning everybody else and then in turn port scanning everybody else. [22:04.360 --> 22:09.240] There was, you know, probably 30, 40 port scans going on continuously throughout the weekend. [22:09.240 --> 22:18.160] Um, but I ended up spending most of the free time I had, um, fighting with NTOP, getting that working, um, as a Nephilo collector. [22:18.540 --> 22:25.100] So, didn't really look at it too much, but, um, basically the, the picture capture is, is one thing that... [22:25.100 --> 22:29.080] Well, we sort of have a bit of an unofficial policy in that what you don't log, it can't be subpoenaed. [22:30.120 --> 22:30.640] All right. [22:32.680 --> 22:39.620] Yeah, actually no, no, no traffic, um, egressing the building was logged on our side. [22:39.800 --> 22:41.180] And that's just for that same reason. [22:41.760 --> 22:47.720] Uh, I poked around a little bit with like IP traffic just watching the, uh, the, uh, the, the protocol, uh, rates and whatnot. [22:48.220 --> 22:54.800] Um, and I ran packet automatic there and dumped all the images out on the HTTPS stream just to see, uh, see what was going on with that. [22:55.200 --> 22:58.420] Um, yeah, things didn't die this year, but... [22:58.420 --> 22:58.880] Yeah, I know. [22:59.080 --> 23:03.780] We, uh, we, uh, we kind of kept busy looking at things all the time, hoping nothing was going to fall down. [23:03.940 --> 23:06.760] So there wasn't too much digging around like that and... [23:06.760 --> 23:06.940] Okay. [23:10.880 --> 23:14.820] But, uh, yeah, so, uh, I looked at some of the URLs flying by. [23:14.960 --> 23:18.120] There were a whole lot of Twitter feeds and then I went, ah, I'm bored with Twitter. [23:21.420 --> 23:21.780] Uh, [23:25.940 --> 23:28.740] how much VPN or VLC traffic did you notice? [23:30.840 --> 23:35.560] Uh, well, so, HTTPS was the number two protocol and SSH was the number three. [23:36.100 --> 23:38.340] Uh, either one of those could be VPN tunnels. [23:38.920 --> 23:44.880] Um, I think I saw like some of the standard VPN ports show up around like eight or nine. [23:46.720 --> 23:47.120] Um... [23:47.120 --> 23:50.360] I saw a few NAT-T sessions and that was about it. [23:50.360 --> 23:55.680] Again, we weren't really looking, um, at the traffic just for the hell of looking at the traffic. [23:56.000 --> 24:00.760] It was more of, is anybody doing anything that might impact the network and, you know, take down service. [24:02.740 --> 24:08.580] Speaking of VPNs, I did try to use a PPTP VPN on both open networks. [24:08.860 --> 24:13.360] It did connect on the open, open one, but it failed consistently on the NAT-ed one. [24:13.640 --> 24:15.440] Um, PPTP doesn't like to be NAT-ed. [24:15.440 --> 24:21.000] Yeah, the, the reason, the reason for that is that, uh, is it doesn't cross NAT gateways very well. [24:21.160 --> 24:25.520] And that's part of why that, why that, uh, open public network was there. [24:25.940 --> 24:30.120] The, uh, we got a slash 20 allocation, so about 16 class Cs or whatever. [24:30.860 --> 24:32.400] Uh, if I did the math right there. [24:32.620 --> 24:35.500] So, you know, a few thousand IP addresses, right? [24:35.500 --> 24:42.940] And so if you connected to the public one, you got one of those many IP addresses and you just had a straight up public Internet IP and it would work just fine. [24:43.300 --> 24:43.680] Yes, it did. [24:43.680 --> 24:56.820] The thing with, the thing with PPTP is it's a GRE tunnel and it's, uh, when it crosses NAT, um, there's, there's just nothing really in the GRE traffic that says how to line it up. [24:56.820 --> 25:03.480] It doesn't use protocol numbers and it's, um, uh, checksums get rewritten and it basically just breaks the protocol. [25:03.820 --> 25:04.080] Yeah. [25:04.240 --> 25:06.940] So you have to have a protocol helper that makes some educated guesses. [25:07.200 --> 25:09.640] That's what the NAT helpers for PPTP do. [25:09.820 --> 25:11.260] They make educated guesses. [25:11.540 --> 25:20.220] So basically if the protocol helper is installed on the NAT gateway, then it can reliably work for, you know, one or two people, but beyond that, never. [25:20.360 --> 25:20.600] One session. [25:21.120 --> 25:21.320] Yeah. [25:21.320 --> 25:26.260] And most, um, NAT implementations will, can identify one GRE tunnel and it'll only allow you to have one. [25:26.980 --> 25:29.700] Um, but again, that's why we have the different networks available. [25:30.200 --> 25:30.220] Yeah. [25:30.720 --> 25:34.400] Also for the record, PPTP is relatively vulnerable. [25:34.680 --> 25:37.560] So if you can, uh, L2TP better. [25:38.160 --> 25:38.580] Yeah. [25:38.680 --> 25:41.240] Probably good to, you know, migrate at some point. [25:42.080 --> 25:47.240] Uh, yeah, my provider offers a choice of that or SSL and I was just being cheap. [25:48.000 --> 25:49.960] The SSL being a tad more expensive. [25:50.640 --> 25:52.560] What, what is best coupon world? [25:54.200 --> 26:00.320] If you, I checked the public IPs a number of times at IP to location and it kept coming up with that domain. [26:00.520 --> 26:02.500] Um, that's probably the last people to use our net block. [26:03.120 --> 26:03.240] Yeah. [26:03.320 --> 26:06.440] There would have been an old, uh, there would have been an old SWIP for that. [26:06.880 --> 26:12.940] Like, uh, that they would have had that before and that would have gotten registered in the ARIN database and stuff like that. [26:13.580 --> 26:15.500] And so, but they're gone now. [26:16.740 --> 26:17.600] Thank you very much. [26:20.980 --> 26:23.180] Any other questions, comments, flames? [26:23.640 --> 26:23.740] Anything? [26:25.940 --> 26:29.260] We didn't really give them much ammo for the flames, you know, I don't think. [26:29.260 --> 26:29.760] Well, it's something. [26:31.840 --> 26:34.220] I like playing around with some of the networking stuff as well. [26:34.800 --> 26:36.080] Where did you guys get your kit? [26:36.180 --> 26:37.580] Is this just coming out of your basements? [26:37.740 --> 26:39.300] Did you borrow it from work or...? [26:40.680 --> 26:45.400] Alright, so the question, because I think your mic cut in halfway, was, uh, where do we get the kit? [26:45.800 --> 26:48.580] Um, so all the WiFi stuff came out of my test lab. [26:48.740 --> 26:50.020] Uh, I work for Aruba Networks. [26:50.560 --> 26:55.920] Uh, so, um, they let me bring all the gear down here, uh, to help run the cons. [26:56.260 --> 26:56.920] Field test. [26:57.260 --> 26:58.200] Yeah, field test. [26:58.680 --> 27:04.380] I, uh, my day job, I, um, run a large campus LAN and a large research computing environment. [27:04.620 --> 27:05.860] So you get a lot of hand-me-downs. [27:05.860 --> 27:11.620] Um, so sometimes a switch will get discarded with a bad power supply and then another switch will get discarded with a fails post. [27:11.920 --> 27:13.740] Put them together, you have a working switch. [27:14.040 --> 27:19.940] So just accumulate lots and lots of hardware, um, that I'm able to use for something like this. [27:20.040 --> 27:23.400] Oh, and I forgot to mention, um, if anybody's interested, all the Cisco hardware is for sale. [27:23.620 --> 27:26.940] Uh, you can come see me after the talk and, uh, work something out. [27:31.340 --> 27:32.540] I don't want to drive it home. [27:40.010 --> 27:40.410] Yeah. [27:41.490 --> 27:41.890] Maybe. [27:41.890 --> 27:45.510] We'll, uh, we'll, we'll, we'll, we'll negotiate offline. [27:50.060 --> 27:50.520] All right. [27:50.600 --> 27:51.180] Any other questions? [27:53.340 --> 27:53.900] All right. [27:54.160 --> 27:54.480] All right. [27:54.760 --> 28:04.480] Uh, just, uh, thanks again to all the, uh, the network guys who helped get this all going and, uh, for the other guys that I've probably forgotten because even though I've been sleeping this weekend, I'm completely not awake right now. [28:04.820 --> 28:11.480] So I'm sure I'm forgetting people to thank and my apologies to them, but, uh, great work for everybody on the network team because... [28:21.660 --> 28:23.580] And, and thanks everyone for showing up. [28:23.720 --> 28:25.460] It would have been, uh, real weird without you. [28:25.760 --> 28:25.940] Yeah. [28:29.080 --> 28:29.600] All right. [28:29.680 --> 28:29.840] Thanks. [28:30.080 --> 28:33.600] So I think that gives you all about half an hour to go get some food before closing ceremonies. [28:46.470 --> 28:47.850] What am I supposed to do? [28:51.210 --> 28:52.570] I love you. [28:52.850 --> 28:52.950] I love you. [28:53.310 --> 28:53.450] I love you. [28:53.970 --> 28:54.530] I love you.