[00:00.000 --> 00:01.080]  Your projector hasn't started.
[00:04.720 --> 00:25.460]  Sensors, augmented reality, virtual reality, alternate reality, games, GPS, things that allow us to reposition ourselves in time and space in various ways to locate ourselves where we currently are and then reposition ourselves into different places, different times,
[00:26.220 --> 00:30.840]  different, as you'll find out, even different people in a way.
[00:32.280 --> 00:36.440]  So, what I'd like to do is, first off, introduce the members so that you know them ahead of time.
[00:36.980 --> 00:41.200]  Here on the far, what is your left, is Ryan O'Horo.
[00:42.900 --> 00:44.600]  And here we have Noah Zerkin.
[00:45.380 --> 00:47.460]  And would you two like to introduce yourselves as well?
[00:47.600 --> 00:48.800]  I forgot one thing I've got to do.
[00:49.220 --> 00:49.600]  Sure.
[00:50.420 --> 00:51.860]  Hi, I'm Ellie Mehrmand.
[00:52.700 --> 00:54.060]  Hi, Micah Cardenas.
[00:56.200 --> 00:57.500]  Okay, I am set now.
[00:58.120 --> 00:58.580]  Okay.
[00:59.040 --> 01:09.880]  So, my particular section here, we're going to have three sections, essentially, which is that I'm going to talk a little bit about hacking the P5 virtual reality glove.
[01:10.380 --> 01:14.580]  And then we're going to have folks on this side talking a little bit about work that they've done.
[01:14.660 --> 01:17.640]  Ryan will talk about HUD-Kit, a project that he did.
[01:18.340 --> 01:21.220]  Noah will talk about augmented reality.
[01:22.220 --> 01:28.340]  And then these folks here are going to talk about several different projects that they've done.
[01:28.660 --> 01:32.020]  So, there's a lot of information to cover.
[01:32.380 --> 01:36.640]  And also, I'm going to be sending around a piece of paper in a few minutes.
[01:36.640 --> 01:39.840]  When I get done with my section, I'm going to send around a piece of paper.
[01:39.840 --> 01:44.980]  If you're going to be here throughout the next, like, 24 hours...
[01:46.000 --> 01:49.860]  I wasn't able to bring it with me on the plane, but my friends are driving it up.
[01:50.040 --> 01:54.780]  I'm actually going to be giving away a P5 virtual reality glove.
[01:55.960 --> 02:00.160]  So, put your name on the piece of paper, and it'll circulate around.
[02:00.260 --> 02:03.080]  Put your name, and then I'm actually going to put it on the glove and just go like this.
[02:03.400 --> 02:07.140]  And the name that it lands on, you want yourself a P5 virtual reality glove.
[02:07.140 --> 02:10.160]  If you can't stay, that's okay, I'll send it out to you.
[02:11.940 --> 02:15.980]  So, let's start out with a little bit about who I am.
[02:16.200 --> 02:19.380]  My name is Pan, and I am the founder of Sensory Research.
[02:19.720 --> 02:29.040]  And it's an organization of engineers, designers, and artists and musicians who are dedicated to providing useful tools for human augmentation.
[02:29.340 --> 02:41.220]  Human augmentation, of course, is improving what our senses already do, improving the ways that we get information, the ways in which we work with tools.
[02:42.840 --> 02:45.620]  And it's been a passion of mine for many, many years.
[02:46.120 --> 02:59.940]  And I'm a developer of interactive software for mobile devices, as well as for human interface devices of all different kinds, everything from the glove to even doing stuff with the Apple remote and hacking the Apple remote.
[03:02.560 --> 03:04.580]  And I'm also a performance artist.
[03:05.000 --> 03:09.280]  And I'll be talking about that towards the end of my little section here.
[03:09.640 --> 03:20.300]  And I'm also the host of a radio show called Interactive Technologies, which has been going on for almost 15 years now, talking about technology as we interact with it and as it helps us in human augmentation.
[03:22.200 --> 03:31.040]  And sort of what I hope you can get out of this or what you gain out of this is some general concepts about sensors and how they work.
[03:31.520 --> 03:40.700]  Data mapping, thinking about how to take data from sensors and map it to other things that you want to do, other tasks.
[03:40.700 --> 03:52.280]  And also think about motion of the human body and coding at the bit level, how to deal with minute code.
[03:52.600 --> 04:07.140]  And new levels of creativity and utility, new ways of expressing ourselves as musicians, artists, developers, performers, even just interacting in our society.
[04:07.960 --> 04:09.740]  And also community.
[04:10.100 --> 04:13.440]  And that's going to come out in this, I think, through everyone's talk.
[04:13.600 --> 04:35.440]  And I should point out, starting this, that if it wasn't for the community of 2600Net that Rodent runs and also people at CalIT and various other communities of explorers and hackers, I wouldn't have been nearly motivated to do the things that I do.
[04:35.440 --> 04:38.720]  All of these people inspire me and I hope that they inspire you.
[04:39.640 --> 04:41.560]  So, about the P5 glove.
[04:41.800 --> 04:50.440]  The P5 glove was a virtual reality glove and it was released by Essential Reality Incorporated in 2002.
[04:51.440 --> 04:53.920]  This is the glove right here and this is the receiver.
[04:56.020 --> 05:04.360]  And it was designed for ES by a guy named Jonathan Clark of Nitric Limited, a development company.
[05:04.840 --> 05:08.580]  And the initial market for the glove was gaming.
[05:09.080 --> 05:11.880]  And this is... remember, this is early 2002.
[05:12.040 --> 05:13.660]  This is before the Wii, before all that stuff.
[05:14.000 --> 05:17.620]  And gaming gloves were always very expensive and other types of...
[05:17.620 --> 05:25.520]  unless you're thinking of Pong, things were very expensive and not really in the reach of your average gamer.
[05:25.860 --> 05:29.580]  And it came bundled with a couple games, Tiger Hunt and Hitman.
[05:30.360 --> 05:31.380]  And it...
[05:33.840 --> 05:44.080]  And it was initially really, really popular at first because people were excited because there was going to be a cheap virtual reality glove available.
[05:44.940 --> 05:46.460]  That didn't last too long.
[05:47.160 --> 05:52.840]  Due to the lack of games, there was only like three games, two or three games that came bundled, Tiger Hunt, Hitman and one other one.
[05:53.620 --> 05:58.100]  They came bundled with the game, but no new games came out that utilized it.
[05:58.520 --> 06:06.780]  So due to the lack of games, due to the lack of solid developer documentation, their SDK was not so hot and it was originally only written for Windows.
[06:07.240 --> 06:09.700]  And there was a lot of Linux people and Mac people.
[06:09.700 --> 06:16.440]  And at this point, for the Mac folks, it was only developed and the SDK only covered Mac OS 9.
[06:16.620 --> 06:19.540]  And at that point, Mac OS 10 was already in full force.
[06:20.580 --> 06:21.980]  So it didn't do really well.
[06:22.320 --> 06:26.200]  And they also lacked in promotion and they focused just on games.
[06:26.320 --> 06:28.340]  And I think that that was a big mistake of theirs, right?
[06:29.320 --> 06:31.780]  So Essential Reality was sold.
[06:31.780 --> 06:33.340]  It was actually a reverse acquisition.
[06:34.760 --> 06:40.400]  The guy who ran the company ended up merging with this higher up company, but he took over.
[06:40.520 --> 06:41.320]  It was a strange thing.
[06:41.560 --> 06:42.880]  And they changed their focus.
[06:43.080 --> 06:46.820]  And this was in 2004, actually.
[06:47.020 --> 06:49.020]  Sorry, that says 2002, but it's actually 2004.
[06:49.320 --> 06:54.400]  They became alliance distributors, game distributors, here on the East Coast.
[06:54.680 --> 06:56.020]  The glove went into limbo.
[06:56.300 --> 07:00.140]  And the intellectual property of it, no one was really sure what was going to happen with it.
[07:01.260 --> 07:05.100]  Stocks of the glove were sold off to different companies to distribute and sell.
[07:05.600 --> 07:08.020]  And no one was really sure what was going to happen.
[07:08.520 --> 07:17.560]  Many attempts were made by the user community to get information out of the people who had worked on the glove development, people who had worked for Essential Reality.
[07:17.640 --> 07:22.620]  Everyone was trying to tap into the glove and utilize it because it seemed like a really big waste.
[07:24.940 --> 07:33.080]  And I'll get to this in a moment, but it was eventually embraced by a community of people that they never intended and took it much farther.
[07:33.380 --> 07:37.220]  So the technical specs of the glove, let me give you just a sense of how this works here.
[07:37.220 --> 07:48.500]  So there are five finger measurements, five independent finger measurements, bend right here, from zero to 90 degrees.
[07:48.940 --> 07:53.280]  And there's also optical tracking with this here.
[07:53.820 --> 07:55.800]  And so you have it like this.
[07:56.700 --> 07:58.280]  And you have the glove on.
[08:00.220 --> 08:05.620]  And you're moving in front of it like this for your X, Y, and Z.
[08:05.760 --> 08:09.920]  For folks that maybe aren't into animation or VR or anything like that.
[08:10.140 --> 08:12.860]  So X, if you remember any of your geometry or anything.
[08:13.000 --> 08:17.120]  So X, Y, and Z, right?
[08:17.640 --> 08:18.100]  Depth.
[08:19.680 --> 08:32.380]  And the optical tracking also did pitch, yaw, and roll, which of course are this, this, and roll is like this, right?
[08:32.620 --> 08:35.220]  So that, that, and like that.
[08:35.360 --> 08:39.180]  And that was actually extrapolated from the X, Y, and Z.
[08:39.340 --> 08:43.500]  So it wasn't really necessarily read directly.
[08:43.900 --> 08:47.080]  And that had about one degree of accuracy.
[08:47.080 --> 08:51.820]  Oh, and X, Y, and Z, as I note here, was about half an inch of accuracy.
[08:52.200 --> 08:54.720]  And there are three trigger buttons on the glove.
[08:55.120 --> 08:56.920]  A, B, and C.
[08:57.200 --> 08:58.560]  There's also a power button.
[08:58.960 --> 09:02.660]  And the gloves, the buttons on the glove could be mapped to anything.
[09:04.440 --> 09:07.780]  And the refresh rate is about 60 hertz.
[09:08.000 --> 09:14.800]  So it's pretty fast for getting data to the user from the glove.
[09:18.530 --> 09:23.030]  And the data, the X, Y, and Z location coordinates are plus or minus 800.
[09:23.470 --> 09:27.610]  And the finger bend sensors have values of 0 to 63, right?
[09:27.830 --> 09:29.470]  So that's 63.
[09:30.670 --> 09:35.810]  And the buttons, data would be 0 for off, 1 for on.
[09:35.810 --> 09:47.490]  And it was a USB 1.3 at the time that it came out, with a type A mail connector, which is your typical USB mail connector you'd plug into a laptop computer, for example.
[09:47.490 --> 09:59.690]  The data itself, coming out of the glove and into your computer, is BCD, binary coded decibel, which is nibbles, basically.
[09:59.690 --> 10:08.810]  So very minute data packed together to try to get as much information in a small space.
[10:09.350 --> 10:22.050]  And it was accessible via standard USB libraries and bit manipulation that you can do in C, C++, Java, and many other languages that can do bitwise operations.
[10:22.050 --> 10:25.750]  And I'm not going to get too technical, so don't worry if that seems a little bit.
[10:26.010 --> 10:27.850]  So this will be the most technical part right here.
[10:28.310 --> 10:30.110]  And this is the actual format of it.
[10:30.150 --> 10:33.270]  If you look, you can see these are nibbles.
[10:35.570 --> 10:38.570]  And that's like one reading of the glove.
[10:38.790 --> 10:40.650]  That's one data reading of the glove there.
[10:40.650 --> 10:45.850]  And you can see that packed 6-bit bend sensor data, index, middle ring, pinky thumb.
[10:46.570 --> 10:49.170]  Button data, A, B, and C.
[10:49.370 --> 10:53.630]  IR sensor, report index as to what the sensors are.
[10:53.750 --> 10:57.070]  The sensors are all back inside there.
[10:58.510 --> 11:03.770]  And packed 30-bit info for the X, Y, and Z.
[11:07.350 --> 11:08.710]  And massaging the data.
[11:08.710 --> 11:24.970]  This is something that I wanted to talk about as we get into this, because a lot of folks here that work with sensors know that any time you work with sensors, really, you need to massage the data, you need to clean it up a little bit, because they are imperfect things.
[11:25.110 --> 11:26.150]  Sensors are very imperfect.
[11:26.350 --> 11:28.570]  They can actually be very simple, they can be very complex.
[11:29.170 --> 11:32.890]  But regardless, there's always a little bit of cleaning up of the data you've got to do.
[11:32.990 --> 11:34.610]  We call it data smoothing, right?
[11:34.910 --> 11:37.850]  So working with any sensors is going to require some data smoothing.
[11:38.290 --> 11:43.750]  The sensors provide varying data, because not all the sensors may be receiving at a time.
[11:43.970 --> 11:50.950]  So in here, your hand may be off a little bit, and one or two of them may not be picking up any data.
[11:51.230 --> 11:54.850]  So you need to figure out how to get rid of that.
[11:55.050 --> 11:57.710]  And then occasionally you'll get erroneous values that just...
[11:57.710 --> 12:06.110]  You know, you'll have four values from four of the sensors that all say the same thing, but then you'll get a value from another one that's like zero or something that's totally erroneous.
[12:06.330 --> 12:13.570]  So to get the most usable data, erroneous values have to be removed, and remaining values have to be averaged.
[12:13.930 --> 12:21.810]  You take the total of your sensors and you say, okay, of a particular type of sensor, and you say, okay, what's the average of all of these?
[12:21.910 --> 12:24.050]  And that's a good way to clean up the data kind of quickly.
[12:25.250 --> 12:26.850]  Now I want to talk a little bit about the community.
[12:27.010 --> 12:31.030]  So I talked about how the gloves sort of went into limbo, and people didn't really know what was going to happen.
[12:31.230 --> 12:33.010]  But we wanted to get our hands on it.
[12:33.670 --> 12:35.530]  Again, the initial target market was gaming.
[12:35.690 --> 12:43.190]  However, the P5 became really popular with artists and musicians, who eventually built a huge community around it for a while.
[12:44.650 --> 12:57.010]  And the community was able to move forward due to some important contributions that I want to note from developers and performers, some folks who are in Australia, who did some amazing work.
[12:57.130 --> 13:00.410]  And that's Jason McMullen, Ross Bencina, and Tim Kreger.
[13:00.670 --> 13:08.130]  And they released some of the first third-party stuff after the glove had been sort of given up by Essential Reality.
[13:08.390 --> 13:18.150]  They released some third-party software that actually showed how you could decode the data format coming out and showed how you could use it in music and video manipulation.
[13:19.610 --> 13:31.630]  And the really cool thing that they did that I think just pushed this ahead so much was they introduced the use of OpenSoundControl.
[13:31.710 --> 13:36.370]  Now, OpenSoundControl was a protocol developed here in the United States.
[13:38.330 --> 13:39.930]  SceneMat came up with it.
[13:40.030 --> 13:42.210]  It's a protocol that's open.
[13:42.450 --> 13:43.730]  The spec is open.
[13:43.730 --> 13:47.290]  And it's for audio-video controller information over a network.
[13:47.550 --> 13:58.750]  So essentially, you have information from a controller device, and you'd like to get that to your audio or video or VR software or whatever.
[13:58.970 --> 14:01.970]  So there's this open sound control protocol, which shows...
[14:02.370 --> 14:07.410]  It's got a specification that shows you how to delineate your information and send it over a network.
[14:07.610 --> 14:17.890]  And then the receiving application takes that apart, parses it, gets the relevant information, and then uses it to control whatever you want, right?
[14:18.430 --> 14:24.250]  It's an easily implementable specification.
[14:24.850 --> 14:28.350]  And it's now popular in many audio-video applications.
[14:32.330 --> 14:46.210]  Basically, SuperCollider, and Reason, and MaxMSP, and any of the applications that deal with sort of audio-video manipulation, all handle OpenSoundControl protocol.
[14:47.550 --> 14:50.690]  And the OC format is relatively simple.
[14:51.010 --> 14:57.010]  You have an address pattern, which is basically where you want... what device or what...
[14:58.810 --> 15:03.810]  what place you would like to direct the information to once it's received by the receiver over the network.
[15:04.490 --> 15:16.490]  And then you have argument type tags, basically a string of single letters that just say, this is an int that I'm sending an integer.
[15:16.930 --> 15:18.790]  This is a string that I'm sending.
[15:19.470 --> 15:22.210]  Or this is a blob, which just means any type of data.
[15:22.450 --> 15:24.790]  Or a float, which is like 1.5, right?
[15:24.890 --> 15:25.670]  A floating point number.
[15:26.430 --> 15:33.710]  The address pattern, again, simple recursive pattern, similar to file directory structure, slash whatever, slash whatever, slash whatever, slash whatever.
[15:34.510 --> 15:37.350]  Based on argument types are string, influence, blob, as I said.
[15:37.930 --> 15:40.250]  And the arguments will match that format.
[15:40.250 --> 15:45.810]  So whatever... however many type tags that you have, that's how many arguments you're actually providing.
[15:46.070 --> 15:58.590]  And the example that I have here for the P5Glove is, you'll have the address pattern, P5Glove, and then you'll have a comma, and then you'll have the address... I'm sorry, the argument types.
[15:59.110 --> 16:01.570]  And in this case, it's all integers coming from the glove.
[16:01.730 --> 16:11.730]  And then you'll have the values, like 63 for the fingers, 800 for the X, Y, and Z, and then for the buttons 0 or 1.
[16:14.810 --> 16:17.830]  And I embraced this.
[16:17.890 --> 16:23.170]  As soon as I found out that the glove was being released, I bought a whole bunch of them and got really, really excited.
[16:23.570 --> 16:32.330]  And then when Essential Reality went out of business or changed their business, they put the gloves up for sale in quantity.
[16:32.610 --> 16:37.870]  So my business actually bought like a whole boatload of them, and several other companies did.
[16:37.870 --> 16:46.110]  A company in Canada, Cyber World, and a few other distribution companies bought just tons of the gloves.
[16:46.330 --> 16:53.290]  And so we've been providing them to the community, and it allowed us to keep it going.
[16:53.290 --> 16:55.450]  Otherwise, they would be sitting in a warehouse somewhere.
[16:56.670 --> 17:00.890]  So I immediately started using it for music and video stuff.
[17:00.890 --> 17:13.690]  I put on some performances in 2004 and 2005 that were called Sequential Progression, 1.0 and 1.1 respectively.
[17:13.990 --> 17:18.030]  And they utilized the glove for audio, video, and virtual reality control.
[17:18.790 --> 17:36.570]  I do what is called QuickTime Panoramic Virtual Reality Photography, QuickTime VR, which is I take pictures in a circle or in a sphere, several angles around, and then I stitch all of those together, and it creates a virtual reality movie that you can navigate through.
[17:36.710 --> 17:48.510]  And these are often used in like real estate and other companies or businesses or individuals who want to give you an immersive effect of navigating through something, right?
[17:49.070 --> 17:59.190]  So what I did is I wrote some software that allowed the glove to control navigation through these QuickTime virtual reality things, and then also sound, mapping it to simple sound.
[17:59.350 --> 18:01.610]  And this is what I was talking about in terms of mapping.
[18:01.630 --> 18:12.170]  So the idea that as you move the glove, you're going to map the data that's coming from the glove into functionality for controlling sound and video.
[18:12.370 --> 18:13.930]  So I did some obvious things.
[18:15.050 --> 18:25.710]  The first was pitch, and that seemed to me to map quite directly to the y-axis, right?
[18:25.890 --> 18:28.470]  So pitch, up or down?
[18:28.970 --> 18:31.450]  As I move my hand up, pitch goes up.
[18:31.690 --> 18:34.030]  So I move my hand down, the pitch goes down.
[18:34.510 --> 18:38.490]  It seemed to me that x mapped very well to phase, right?
[18:38.490 --> 18:49.410]  Phase is where the wave is, crests and troughs, in your sample of sound information, or any information for that matter, any wave information.
[18:49.750 --> 18:53.730]  So this changed the phase of the sound.
[18:53.990 --> 19:01.890]  And then volume, it seemed to me, closer, louder, further away, quiet, louder, quiet.
[19:01.890 --> 19:11.350]  So I did that really simple mapping, and then did some modifications for the yaw and stuff like that, just to sort of tweak the sounds in the samples.
[19:11.550 --> 19:12.910]  And it worked really well.
[19:13.070 --> 19:32.750]  And it was, again, an example of, you can take information from sensors, and you can think about ways in which you can layer sort of a template for other activities, other things that you want to do, and really experiment and do some amazing things.
[19:32.890 --> 19:35.990]  And really, your mind is the limit on this.
[19:36.590 --> 19:43.710]  So after I did the performances, or actually, while preparing for the performances, I started writing software for the desktop as well.
[19:43.910 --> 19:48.650]  And I came out with what's called Thought Conduit desktop application.
[19:48.650 --> 19:54.270]  And Thought Conduit records these manipulations...
[19:54.270 --> 20:12.810]  Well, it actually records the glove data, right, of your X, Y, and Z, finger bends, all that stuff, and allows you to play it back and then send it using the open sound control protocol to manipulate things or actually view a virtual reality movie or control sound right in the application itself.
[20:13.590 --> 20:20.550]  And you can export that to other formats so that you can send files to people and stuff like that.
[20:22.590 --> 20:25.510]  And I'll talk a little bit more about software that I've done.
[20:26.010 --> 20:33.630]  Let me give you a quick demonstration of the application.
[20:34.350 --> 20:36.910]  And this just shows you the interface and stuff.
[20:37.010 --> 20:43.770]  I don't want to spend too much time on it because you'll actually get a chance to see it in a live performance if you're going to be around this weekend.
[20:44.170 --> 20:48.810]  So this is actually just an example of the interface of the application.
[20:49.030 --> 20:52.950]  It's got a window for viewing your virtual reality or your movie.
[20:53.050 --> 20:56.050]  You can control linear movies as well, scratching forward.
[20:56.050 --> 21:06.670]  If it's an audio, like if you wanted to do a quick time movie with just audio, you can scratch forward, scratch back, and stuff like that, or fast forward, reverse, whatnot, play.
[21:06.850 --> 21:09.530]  So it's like a typical tracked application.
[21:09.530 --> 21:11.090]  You've got tracks that you can create.
[21:11.210 --> 21:12.010]  You've got a viewing window.
[21:12.210 --> 21:25.490]  And then on the side, I created some windows so that you could see what you were manipulating and also connect a camera to your computer and see what the camera was picking up.
[21:25.670 --> 21:48.970]  And the reason I did this is because I'm really into rehabilitation software and working with trying to improve people who have been in accidents or have robotic limbs and other things that basically they have to train themselves again to use this thing.
[21:49.170 --> 22:11.890]  Well, research has shown that if you see yourself moving a limb, you actually train yourself faster and become more accustomed to the change, or if you're being rehabilitated using, you know, your damaged limb and rehabilitating, you improve the rate at which that happens if you see yourself doing it.
[22:11.990 --> 22:13.190]  And that fascinated me.
[22:13.210 --> 22:22.070]  So I said, okay, why not let people see themselves from the side and also see what they're... so they can see what they're navigating and they can see from the side how they're navigating.
[22:22.070 --> 22:24.170]  And I thought that that was really useful.
[22:27.010 --> 22:30.670]  So, let me go back to the presentation here.
[22:30.770 --> 22:34.970]  Oh, let me show you a quick video of how the glove gets put on here.
[22:35.130 --> 22:37.430]  So this is just using the glove.
[22:39.850 --> 22:41.870]  Lightweight computer controller, we discussed that.
[22:42.770 --> 22:47.090]  And finger bend sensors, control buttons, hand harness.
[22:47.950 --> 22:49.610]  That goes around your hand.
[22:49.610 --> 22:50.350]  You slide.
[22:53.940 --> 22:54.900]  Go like that.
[22:55.740 --> 22:59.960]  Put your hand in the finger stirrups like this.
[23:01.920 --> 23:07.400]  And before you put the thumb on, you wrap this little thing around.
[23:07.760 --> 23:09.160]  A little harness there.
[23:09.380 --> 23:11.500]  Then you put your thumb on and it stays snug.
[23:11.620 --> 23:13.460]  And this is called an exoskeleton.
[23:13.460 --> 23:21.860]  So anytime you hear of virtual reality or augmented reality controllers that are exoskeleton, it means that it goes over your hand.
[23:21.860 --> 23:25.220]  It's not like a fake hand that you're controlling.
[23:25.220 --> 23:28.960]  It actually is a shell that goes over what you're doing.
[23:31.600 --> 23:40.520]  And, yeah, then you can basically, as I said, put the harness on and slip your thumb in.
[23:41.220 --> 23:42.440]  And you're ready to go.
[23:42.820 --> 23:44.280]  Situate your body a couple feet away.
[23:44.400 --> 23:45.160]  Two feet away is good.
[23:45.260 --> 23:53.340]  You can go up to four feet away and still get a good result in terms of receiving.
[23:54.940 --> 23:57.440]  So, get back to my presentation here.
[24:03.840 --> 24:14.540]  The bigger picture, really what I want to leave you with in terms of this project, is that sensor programming and open protocols allow for diverse applications.
[24:15.100 --> 24:16.780]  Navigating things that you want to show.
[24:16.920 --> 24:20.720]  Like I said, real estate, medical, things like that.
[24:21.020 --> 24:23.520]  Education, having students navigate.
[24:23.520 --> 24:37.500]  If you can't go to the Parthenon, why not make a QuickTime VR or some of the new, like X3D, which is the new VR protocol for the web that's going to be, hopefully, ratified with HTML5.
[24:38.920 --> 24:40.840]  X3D is a really great standard.
[24:41.120 --> 24:48.940]  And so, with things like that, you can actually very easily create tools that are immersive and that allow you to navigate and whatnot.
[24:49.160 --> 24:53.320]  Again, for rehabilitation, for video, music, animation, and VR.
[24:53.620 --> 24:58.500]  And lastly, the thing that they originally intended it for the glove, gaming.
[24:58.720 --> 25:00.380]  And I'm actually not much of a gamer.
[25:00.560 --> 25:02.780]  I'm more into the AR and VR stuff.
[25:03.000 --> 25:05.220]  But that's also there as well.
[25:06.460 --> 25:22.440]  And what I wanted to do is, because it's so simple, and because it's something that I want to motivate people to do, I've been working for the past few months on a little project, and I'm happy to announce here at HOPE right now, I've released something that's available on my website.
[25:22.440 --> 25:24.200]  It's called Thought Conduit OSC.
[25:24.200 --> 25:33.300]  It's a COCO framework, so it's for Mac developers who want to write software that utilizes the open sound protocol.
[25:33.320 --> 25:48.060]  It allows you to just easily add data to a dictionary, basically, and just say send, and then your data can go out over the Internet to any application that can receive the open sound protocol.
[25:48.360 --> 25:50.660]  Again, it's COCO framework, very easy to use.
[25:50.880 --> 25:53.240]  It's available on the sensory research website now.
[25:53.240 --> 25:54.500]  There's no charge for it.
[25:54.600 --> 25:56.500]  I just want you to do something amazing.
[25:57.280 --> 26:00.240]  And again, there are so many applications that utilize the protocol.
[26:00.860 --> 26:07.000]  You're not limited, and once you understand how to send, you can think about receiving and parsing that protocol.
[26:07.160 --> 26:10.540]  And as you saw, it's very, very easy, and their specification is online.
[26:11.360 --> 26:13.080]  So very, very simple stuff.
[26:13.460 --> 26:18.660]  And lastly, I wanted to let you know that at 2 a.m.
[26:19.040 --> 26:25.480]  tomorrow night, so Sunday morning, 2 a.m., after the chiptune concert, my band is going to be performing.
[26:25.480 --> 26:28.820]  The band that did the sequential progression stuff a couple years ago.
[26:28.880 --> 26:32.820]  We're actually going to be performing in the Video Temple.
[26:34.700 --> 26:40.500]  And it's a full band, and we incorporate the glove for controlling audio, video, and manipulation.
[26:40.740 --> 26:52.060]  And I also generated QuickTime VR panoramas for this event, and also did some 3D animations, and some other really cool things that are going to be projected.
[26:52.220 --> 26:59.040]  And you'll actually see me controlling the sound and the video and incorporating that with all of the other stuff that we do.
[26:59.040 --> 27:01.280]  So I encourage you to come and check us out.
[27:01.360 --> 27:02.880]  I know it's a late thing.
[27:02.880 --> 27:05.460]  It's 2 a.m., and there's a lot of things you could be doing at that time.
[27:05.580 --> 27:07.960]  But I guarantee you, you will not be disappointed.
[27:07.960 --> 27:09.800]  We put on an awesome show.
[27:10.540 --> 27:11.920]  So come check us out.
[27:12.020 --> 27:12.920]  And I'd like to thank you.
[27:13.060 --> 27:16.260]  You can find out more, again, at www.sensoryresearch.net.
[27:16.480 --> 27:20.160]  You can email me at info at sensoryresearch.net.
[27:20.160 --> 27:39.520]  And right now, what I'd like to do is turn things over to Ryan and have him talk a little bit about the projects that he's done, and a particular one that I think really stands out in terms of being creative with interactivity, and in this case, HUD stuff.
[27:39.800 --> 27:40.480]  So, Ryan.
[27:41.400 --> 27:41.860]  What?
[27:42.900 --> 27:43.260]  What?
[27:43.840 --> 27:44.880]  Can't get up quick.
[27:50.040 --> 27:51.280]  Hello, my old dongs.
[27:54.260 --> 27:55.400]  It's a Mac.
[27:55.520 --> 27:56.500]  I don't know what the hell is going on.
[27:57.040 --> 27:57.660]  Just a...
[27:57.660 --> 27:58.120]  Yeah, there you go.
[27:58.420 --> 27:58.800]  Thank you.
[28:00.160 --> 28:01.020]  Oh, cool.
[28:01.260 --> 28:03.000]  I have both of these microphones at home.
[28:03.380 --> 28:04.080]  Good work, guys.
[28:05.040 --> 28:06.260]  My name is Ryan O'Horo.
[28:06.880 --> 28:10.500]  Some of you might know me better as that f*cking douchebag.
[28:11.560 --> 28:12.140]  I just...
[28:12.140 --> 28:12.860]  What?
[28:15.160 --> 28:17.560]  I do a lot of DIY stuff.
[28:18.260 --> 28:19.760]  I work with the maker people.
[28:20.580 --> 28:23.840]  I hang out with the NYC resistor kids and wish I was one of them.
[28:25.160 --> 28:37.180]  But in particular, I was involved with Make Magazine for a little while and had an article published in the magazine on the Make blog and whatnot.
[28:37.440 --> 28:42.680]  And one of the editors at PC Magazine saw it and said, Hey, you live in New York now.
[28:42.860 --> 28:44.240]  Why don't you do a story for us?
[28:44.760 --> 28:45.120]  And...
[28:45.600 --> 28:51.600]  Basically, what the challenge was given to me is that they wanted me to create a mobile office and a car.
[28:53.580 --> 29:03.920]  And as part of that, I created this system called the HUD kit, which never really materialized as a kit itself for sale for retail... targeted for the retail market.
[29:04.220 --> 29:06.020]  But nonetheless, it was fun for me.
[29:06.140 --> 29:10.460]  And hopefully, other people have gotten some utility from the project.
[29:12.580 --> 29:14.800]  I'm also involved in Make NYC.
[29:15.120 --> 29:16.120]  I'm one of the organizers.
[29:16.580 --> 29:19.540]  And I also help out with Tink with Eric Moore.
[29:20.160 --> 29:22.700]  You can find out more about all those projects here at these websites.
[29:24.800 --> 29:35.840]  The idea behind HUD kit was to take information from this mobile office and put it up in front of the driver in the virtual environment that is beyond the windshield.
[29:37.640 --> 29:53.080]  A lot of problems come up from the car PC that you want to be aware of all the information all the time, waiting for emails to come in, looking for information about projects or whatever, current information about the car and things like that.
[29:53.520 --> 30:07.420]  The car PC provided all this information through a standard car PC, email client, diagnostic connection to the car so you can get speed, other interesting facts about what's currently going on with the car.
[30:07.640 --> 30:12.580]  But you'd have to look at the screen to do that and be distracted from your job of driving.
[30:13.460 --> 30:17.180]  So I decided to do something fun and I put a heads-up display in this car.
[30:19.480 --> 30:28.380]  Basically, this project was a VFD, vacuum fluorescent display, put into a little metal box that clipped to your visor.
[30:30.640 --> 30:36.420]  Inside this little device, I used a VFD because it's high intensity and gives good contrast.
[30:37.200 --> 30:43.080]  A front surface mirror, it has low light loss, low distortion.
[30:43.740 --> 30:47.120]  And a plastic magnifier lens in the form of Fresnel.
[30:48.660 --> 31:01.120]  It's created like a little triangle where the image is reflected and projected onto a piece of glass in front of the driver.
[31:01.500 --> 31:08.420]  A little bit of smoked film to put some contrast between the image and the actual world outside of you.
[31:09.020 --> 31:15.360]  Then in your field of vision, you have a series of information scrolling by you as new data is added.
[31:16.720 --> 31:19.080]  Emails, speed, things like that.
[31:20.840 --> 31:23.400]  It looked a little bit like this when it was all done.
[31:24.580 --> 31:25.600]  Kind of hard to see.
[31:25.700 --> 31:28.660]  I took this image off the PC Magazine website.
[31:28.840 --> 31:31.060]  I lost all the original images from the project.
[31:31.160 --> 31:32.520]  I apologize for its crudeness.
[31:32.520 --> 31:37.940]  But you see it reads here, the ultimate mobile office, which was the title of this project.
[31:39.160 --> 31:41.840]  Great because PC Magazine gave me an unlimited budget.
[31:42.340 --> 31:44.440]  Bad because I only had a month to do it.
[31:45.600 --> 31:50.580]  Ended up being put together with a glue gun and looked something like this in its initial stages.
[31:53.800 --> 32:07.660]  I feel this project is important because in order to relate virtual reality, augmented reality, and sensor input from humans, individual development is really critical.
[32:08.040 --> 32:09.920]  The DIY community is really critical.
[32:10.700 --> 32:15.960]  As you can see with the commercial failure of this device, it was picked up by the DIY community and improved.
[32:15.960 --> 32:25.240]  And I feel that really the only efficient entry into the commercial market for these type of technologies is through individual and DIY development.
[32:26.820 --> 32:34.700]  This is a really good example of how you can get involved in what is now the trailing edge of augmented reality technology.
[32:36.000 --> 32:39.820]  The automotive HUD first appeared in cars, I believe, in the 1980s.
[32:40.500 --> 32:46.740]  Some American cars had statistics about your speed and things like that that were projected onto the windscreen.
[32:47.500 --> 32:54.320]  I did not use the windscreen for this project because, unfortunately, you have to put the VFD in the dash below the screen.
[32:54.480 --> 32:59.980]  And that creates a problem because if you put sunlight on the VFD for a long time, it's just going to crap out on you eventually.
[33:02.220 --> 33:06.300]  But you would think so, but these things didn't disappear over time.
[33:06.300 --> 33:08.720]  There's actually a 2001 Pontiac.
[33:09.080 --> 33:12.480]  Pontiac has these heads-up displays in their cars.
[33:12.700 --> 33:19.160]  Up until now, actually, there's a Buick that has one, a 2010 Buick that has a heads-up display.
[33:19.560 --> 33:22.560]  I find these devices really fun and I wish they were in all cars.
[33:22.560 --> 33:25.700]  I don't understand what is preventing them from being there.
[33:25.800 --> 33:29.200]  Maybe people find them annoying or distracting, just stupid.
[33:29.200 --> 33:33.140]  I don't know, but I like to be able to see what's going on in my car without looking down.
[33:34.100 --> 33:37.280]  Radio station, MP3, whatever, what have you.
[33:38.520 --> 33:43.420]  I wish I had one now, even though I don't have a car because I live here in New York City.
[33:43.720 --> 33:55.680]  But this application of augmented reality, bringing data that is in the periphery into your critical world, and it's the world you see in front of you.
[33:57.640 --> 34:06.480]  I really... I wish I had a pair of those glasses that would do that for me and put all sorts of cool information from my cell phone in there and not have it be obtrusive.
[34:08.440 --> 34:09.780]  That's about it for this project.
[34:09.900 --> 34:12.120]  I do have another sensor project that I currently have.
[34:12.280 --> 34:19.480]  I don't know if you've seen it, but the pinball machine in the art space on the mezzanine floor that you can't play with because I'm afraid you guys are going to break it.
[34:19.480 --> 34:20.620]  I'm really sorry.
[34:20.840 --> 34:26.560]  I spent a couple of years working on this project and I'd really like to preserve it for Maker Faire in September.
[34:27.060 --> 34:28.860]  But that contains a shooter.
[34:29.300 --> 34:37.260]  It's a pinball machine and, of course, the shooter has to pull out and the ball speed really depends on how fast and how far you pull the shooter.
[34:37.480 --> 34:41.700]  Well, the only really good way to determine that is to get a measurement of the distance.
[34:42.260 --> 34:53.960]  And really the only practical application that I could find was to use a bend sensor, which when the sensor is bent the resistivity increases.
[34:54.280 --> 34:58.340]  You measure that resistivity on an Arduino and you can tell how far it's been bent.
[34:58.680 --> 35:05.120]  And as Pan mentioned, I've forgotten Pan's real name.
[35:08.860 --> 35:11.080]  I only know him from IRC.
[35:11.080 --> 35:15.600]  Anyway, so you take this data, which is really crude.
[35:15.880 --> 35:21.440]  You get a value of 1 to 10, I'm sorry, 1 to 1,000 as to the voltage measurement.
[35:21.580 --> 35:28.340]  And you have to massage that data down to correct for errors and for basically human fallacy.
[35:31.380 --> 35:36.700]  When you work with these type of sensors, humans move in kind of funny ways.
[35:36.700 --> 35:49.140]  And if you accept data readily, if you trust it explicitly, then obviously the user is going to do something, expect one thing and get another because they're not doing exactly what you expect them to.
[35:49.460 --> 35:52.860]  But the data has to be massaged and I cut those values down to five increments.
[35:53.080 --> 35:57.880]  So my pinball machine has five different speeds at which the ball can be shot with that device.
[35:58.340 --> 35:59.700]  Come take a look at the machine.
[36:00.020 --> 36:03.780]  It's on the mezzanine level in the art space near the corner of the walkway.
[36:04.020 --> 36:08.160]  I'll be doing a talk on it on Sunday at 1 p.m.
[36:08.320 --> 36:08.820]  I believe.
[36:09.200 --> 36:09.580]  Look for it.
[36:10.700 --> 36:11.600]  Thank you very much.
[36:20.860 --> 36:22.220]  So right now I'm going to...
[36:26.160 --> 36:29.760]  Oh, there's going to be some revenge for that.
[36:30.620 --> 36:32.420]  So I'm going to take this piece...
[36:33.660 --> 36:34.540]  You're first.
[36:35.060 --> 36:39.400]  So I'm going to take this piece of paper and a pen and I'm going to start here.
[36:40.260 --> 36:41.220]  And just...
[36:41.760 --> 36:42.900]  It's a big piece of paper.
[36:43.040 --> 36:44.020]  Put your name on it.
[36:44.520 --> 36:48.840]  And I do have this glove, but I don't want to give this one away because it's been used.
[36:49.380 --> 36:57.000]  My band is actually bringing up a boxed version of the glove that has everything all packaged and hasn't been opened and everything like that.
[36:58.480 --> 37:02.760]  So if you're going to be here tomorrow, then I'd be able to hand it off to you.
[37:02.900 --> 37:07.540]  If you're not going to be here, talk to me after the session and we'll arrange to get the glove to you.
[37:07.660 --> 37:09.720]  So again, this is to win a P5 glove.
[37:09.880 --> 37:10.740]  Put your name on here.
[37:11.300 --> 37:15.280]  And at the end of the session, in a little bit, I'll...
[37:15.780 --> 37:16.660]  You ran your name?
[37:16.820 --> 37:17.920]  I'll do your dirty work.
[37:18.520 --> 37:18.820]  Thank you.
[37:19.440 --> 37:19.760]  Okay.
[37:20.140 --> 37:21.680]  And Noah, did you want to come up now?
[37:22.560 --> 37:25.120]  Well, since I'm a last-minute addition, I don't want to...
[37:25.880 --> 37:26.420]  No, no, no.
[37:26.420 --> 37:26.900]  Go right in.
[37:27.020 --> 37:27.580]  No, go right in.
[37:27.660 --> 37:27.680]  Okay.
[37:27.980 --> 37:28.240]  Go right in.
[37:28.620 --> 37:30.040]  Noah Zirkin, ladies and gentlemen.
[37:30.720 --> 37:31.140]  Woo-hoo!
[37:35.690 --> 37:38.180]  So my name, as they said, is Noah Zirkin.
[37:39.900 --> 37:43.180]  And let's see, where to start when it comes to augmented reality?
[37:43.740 --> 37:53.180]  I've been following AR since, I guess about since I was 14, and took a field trip during summer camp to the MIT MediaWeb.
[37:53.180 --> 37:58.040]  I met Steve Mann and Thad Starner, who were both pioneers in wearable computing.
[37:59.480 --> 38:03.580]  And they referred to it as augmented reality.
[38:03.880 --> 38:08.340]  And so I, sorry, started following the field.
[38:09.340 --> 38:11.100]  Sorry, I don't have a prepped talk.
[38:12.140 --> 38:13.600]  Like I said, last-minute addition.
[38:13.600 --> 38:23.940]  Anyhow, AR has become a buzzword in the past two years in marketing circles and online in general.
[38:24.200 --> 38:38.040]  And people have started associating it largely with the printed fiducial markers held up in front of a webcam with a superimposed 3D image or animation on top of it.
[38:38.040 --> 38:42.440]  And it's really sort of a misuse of the term.
[38:44.980 --> 38:54.100]  AR, to me, really denotes the visual overlay, the fusion of the virtual and the real.
[38:54.320 --> 38:59.860]  It encompasses contextual and pervasive computing, location-based computing.
[38:59.860 --> 39:25.000]  And my own AR, VR project actually came out of a frustration with devices like the P5, the Power Glove, the, let's see, the Cyber Glove, which is the research targeted, the glove targeted towards the research market.
[39:25.000 --> 39:44.320]  That includes, I think, up to 20 sensors, two flex sensors per finger, then abduction sensors between the fingers, that's finger spread, wrist sensors, let's see.
[39:44.320 --> 39:48.040]  So, it's incredibly expensive to track it in space.
[39:48.600 --> 40:03.840]  You either needed to use traditional means of motion capture, like a Vicon system, OptiTrack, which is an infrared tracking system, or inertial sensors.
[40:04.480 --> 40:11.840]  And an enthusiasm for inertial sensors is actually what got me started building these sorts of devices.
[40:13.200 --> 40:13.900]  Let's see.
[40:14.200 --> 40:14.340]  Let's see.
[40:14.340 --> 40:19.840]  So, examples of an inertial motion tracking system would be, for instance, the XSense Movin.
[40:22.140 --> 40:45.340]  Well, can't really see hands, but an IMU, which is an inertial measurement unit, which generally includes three-axis accelerometer, three-axis magnetometer, three-axis gyro, will basically give you your six-degree freedom, information, information about a given position,
[40:45.540 --> 40:46.420]  or at least orientation.
[40:47.000 --> 40:51.480]  So, it doesn't do... because, as they were saying, these sensors aren't ideal.
[40:52.580 --> 40:56.600]  It's difficult to do dead reckoning tracking through space.
[40:56.600 --> 41:03.280]  So, even better than massaging the data, there's something called an extended Kalman filter.
[41:03.480 --> 41:12.920]  It's a covariance filter that basically gives you reliable results for orientation.
[41:12.920 --> 41:22.960]  So, that's rotation about the X, Y, and Z axis, either in Euler angles or quaternions or a full rotation matrix.
[41:25.930 --> 41:27.920]  And basically orientation.
[41:27.920 --> 41:41.420]  So, you get a tilt compensated heading using the tilt data from the accelerometer and the heading component from the magnetometer, which is basically a compass.
[41:41.420 --> 41:49.820]  And using the gyro, you can basically get smoother yaw information.
[41:50.050 --> 41:54.020]  Magnetometers don't really update at a very high rate in most systems.
[41:54.920 --> 41:59.280]  I can show you guys a video of the glove system that I built.
[41:59.280 --> 42:11.460]  And really the impetus for this project was that for a glove interface to be viable as part of an augmented reality system, it has to be location independent.
[42:12.340 --> 42:23.080]  And most of the even remotely affordable VR interface devices, glove-based devices, require an external reference point for their tracking in space.
[42:23.080 --> 42:25.180]  For instance, this.
[42:27.040 --> 42:33.280]  I basically came up with a system using an elbow-mounted potentiometer.
[42:34.640 --> 42:37.580]  Initially a two-axis gyro, now a three-axis gyro.
[42:37.820 --> 42:39.480]  It's all arm-mounted.
[42:40.740 --> 42:43.120]  And a three-axis accelerometer.
[42:43.580 --> 42:46.600]  And then eventually added a magnetometer with the tilt compensation.
[42:48.540 --> 42:52.500]  For tracking at least relative to the body, arm position.
[42:52.940 --> 42:57.820]  And then I took a paintball glove, put 15 flex sensors, multiplexed, read by an Arduino.
[42:58.580 --> 43:02.580]  Transmitted wirelessly using the XP, just as a wireless UART.
[43:03.720 --> 43:09.280]  And wrote a dinky little VR environment and processing.
[43:09.550 --> 43:12.000]  You can use the glove to interface with that.
[43:12.880 --> 43:16.120]  You can have the processing environment output, OSC.
[43:18.020 --> 43:26.900]  Since ported a lot of it to open frameworks, which is another open 3D interactive development environment.
[43:26.900 --> 43:30.890]  But yeah, this video is kind of tacky.
[43:31.880 --> 43:35.040]  A friend of mine wanted to commercialize this.
[43:36.020 --> 43:38.900]  So he added it together for me.
[43:40.820 --> 43:43.340]  The name was his idea.
[43:44.900 --> 43:46.940]  And I objected.
[43:47.500 --> 43:50.440]  It's the Zerkin glove, also potentially red.
[43:50.840 --> 43:51.480]  Zerkin glove.
[43:52.020 --> 43:53.520]  It looks like a huge star.
[43:54.120 --> 43:55.700]  That was not my sight.
[44:03.480 --> 44:06.180]  It hasn't come all that far from this.
[44:06.340 --> 44:12.100]  I actually presented this back in about a year and a half ago at Notacon.
[44:12.660 --> 44:20.680]  Jerry Ellsworth asked me to present it at the Fat Man and Circuit Girl live show at Notacon.
[44:20.680 --> 44:23.200]  That was a fair bit of fun.
[44:26.480 --> 44:27.480]  Oh, no audio?
[44:28.960 --> 44:30.080]  Quite all right.
[44:30.300 --> 44:30.620]  You know what?
[44:30.780 --> 44:31.780]  That's just fine.
[44:32.020 --> 44:32.400]  Go ahead.
[44:32.760 --> 44:33.060]  Go ahead.
[44:33.640 --> 44:34.060]  Okay.
[44:35.540 --> 44:37.860]  I take no responsibility for the editing.
[44:38.480 --> 44:39.920]  I really did a fair job of it.
[44:40.120 --> 44:42.260]  And it says it was done in one day.
[44:42.260 --> 44:46.920]  I basically went over to this guy's place and built one for him so he could go around and demo it too.
[44:48.960 --> 44:53.600]  And it was actually over the course of two days, but total build time probably under 10 hours.
[44:58.550 --> 44:59.790]  He's still loading.
[45:00.850 --> 45:02.870]  And still no audio.
[45:03.370 --> 45:03.590]  Audio.
[45:08.560 --> 45:10.980]  Imagine Raging Against the Machine playing.
[45:11.380 --> 45:11.500]  Yep.
[45:17.420 --> 45:20.400]  Have you guys seen World Builder?
[45:20.580 --> 45:24.770]  It's a great little short that sort of...
[45:33.250 --> 45:35.090]  So, all reference-less.
[45:55.630 --> 46:01.470]  And no, it wasn't controlling the music although I've done that basically.
[46:03.230 --> 46:08.690]  Mapping various aspects of a piece of sound.
[46:10.390 --> 46:16.570]  Sound-generating strips to the positions and sizes of various objects.
[46:19.510 --> 46:20.310]  Yeah.
[46:24.650 --> 46:28.510]  Anyway, I could go on about AR for a very long time.
[46:28.510 --> 46:29.070]  I don't know.
[46:29.210 --> 46:29.490]  I don't know.
[46:44.290 --> 46:54.670]  By the way, I would say that probably the definitive AR log out there right now worth checking out is gamesalfresco.com.
[46:55.510 --> 47:03.310]  It's run by Ori Anbar, the same guy who edited this, although he's not actually posting to it much these days.
[47:03.530 --> 47:10.350]  He just organized an AR conference out in Santa Clara, which was keynoted by Bruce Starling and Will Wright.
[47:10.350 --> 47:22.170]  And also check out the work of Stephen Feiner at Columbia, Patty Mays at MIT.
[47:22.430 --> 47:33.550]  In fact, there's kind of a dispute, a friendly dispute that came up at ISMARC, which is the symposium on augmenting mixed reality last year.
[47:33.550 --> 47:34.370]  Don't worry about it.
[47:34.870 --> 47:35.290]  Um...
[47:36.170 --> 47:36.970]  Yep.
[47:41.800 --> 47:50.120]  Anyway, Patty Mays at the MIT Media Lab believes that AR encompasses little augmentations.
[47:50.720 --> 48:03.860]  Things like, well, you may have seen Pranic Mysteries Sixth Sense project, which was a projector worn around the neck with the camera.
[48:05.100 --> 48:06.300]  Excellent project.
[48:06.520 --> 48:07.280]  Excellent project.
[48:08.000 --> 48:28.720]  Um, and, um, Patty also is a proponent of, uh, for instance, a spoon, which has a little vibration motor on the handle and can tell based on accelerometric data if you're eating too fast, for instance, and will vibrate based on, on these cues to indicate this to you.
[48:28.720 --> 48:37.860]  Um, so, so building augmentations into objects, um, rather than into wearable systems and, uh, haptic systems.
[48:38.260 --> 48:42.800]  Also, uh, uh, the Phantom, fantastic device for haptic interaction.
[48:43.460 --> 48:47.900]  Um, there are also a whole bunch of see-through, uh, HUDs on the horizon.
[48:48.300 --> 48:49.040]  Really close.
[48:49.600 --> 48:55.940]  Um, Lumis Optical, Microvision, um, has a great, uh, laser-based display.
[48:55.940 --> 49:14.720]  Um, uh, they only made 10 of them, but they're currently being, uh, funded, uh, by DARPA, um, through the Lockheed Martin UltraViz, uh, contract to, to develop, uh, uh, head-mounted display, initially for the military market.
[49:14.720 --> 49:21.600]  Um, but, uh, that, that should see, uh, the light of day in the consumer market, uh, at least in a few years.
[49:22.160 --> 49:28.220]  Uh, Canon, best, the, the best AR, uh, display that I've used is a Canon HUD.
[49:28.500 --> 49:29.700]  Uh, it's big.
[49:29.700 --> 49:49.300]  But it'll get smaller, stereoscopic, um, and, uh, uh, I saw that down at the, uh, uh, University of Southern Australia, which, um, uh, has a great wearable computing lab and, uh, the, the, uh, Magic Vision Lab, uh, run by Christian Sandor.
[49:49.940 --> 49:53.620]  Um, anyway, uh, I'll, uh, hand it off.
[49:53.620 --> 50:04.620]  Also, I have a blog that I haven't been, uh, updating much recently, but all the early entries, uh, basically express my views pretty accurately, uh, about AR.
[50:05.580 --> 50:10.040]  Um, and that's, uh, augmentation.wordpress.com.
[50:10.600 --> 50:12.940]  Um, might post something to it again.
[50:13.140 --> 50:13.320]  We'll see.
[50:13.720 --> 50:15.040]  Anyway, thanks.
[50:15.600 --> 50:16.120]  Um,
[50:32.620 --> 50:33.380]  if you want, yeah.
[50:33.740 --> 50:35.240]  Let's see, does this reach up here?
[50:37.810 --> 50:38.630]  No, it doesn't.
[50:38.630 --> 50:39.370]  It'll be a tight fit.
[50:39.370 --> 50:40.130]  Okay.
[50:48.480 --> 50:49.240]  For the ladies!
[50:49.640 --> 50:50.060]  Woo!
[50:50.500 --> 50:50.760]  Hi.
[50:51.840 --> 50:52.260]  Woo!
[50:52.260 --> 50:52.460]  Woo!
[50:54.740 --> 50:55.080]  Thanks.
[50:55.660 --> 50:56.500]  It's inviting.
[50:59.300 --> 50:59.920]  Did you get the power?
[51:00.140 --> 51:01.260]  Yeah, just give us a second.
[51:01.460 --> 51:01.580]  Sorry.
[51:01.900 --> 51:02.360]  Grrr.
[51:02.620 --> 51:04.700]  Very smooth transition, that was.
[51:05.400 --> 51:06.320]  Perfectly smooth.
[51:06.560 --> 51:07.980]  All about smooth transitions.
[51:08.440 --> 51:08.660]  Okay.
[51:09.320 --> 51:11.180]  I think that that is working now.
[51:12.280 --> 51:14.620]  So now for something totally different.
[51:18.050 --> 51:22.290]  Um, and talk is coming up.
[51:22.630 --> 51:23.190]  Okay.
[51:23.590 --> 51:24.870]  Micah and Ellie, by the way.
[51:25.150 --> 51:25.370]  Oh yeah.
[51:25.790 --> 51:26.830]  I'm Ellie Mehrmand.
[51:27.250 --> 51:29.070]  This is Micah Cardenas.
[51:29.590 --> 51:29.870]  Hi.
[51:30.390 --> 51:34.310]  Um, so we're from, uh, we're, we're from San Diego.
[51:34.310 --> 51:37.390]  We, uh, teach at UCSD and we work with the Bang Lab.
[51:38.170 --> 51:41.850]  Um, and this f*cking open office is not working.
[51:42.070 --> 51:43.370]  And we don't know how to turn on it.
[51:43.450 --> 51:44.030]  There we go.
[51:44.390 --> 51:45.190]  Thank you very much.
[51:45.350 --> 51:45.470]  Okay.
[51:45.750 --> 51:46.150]  Good.
[51:46.150 --> 51:50.190]  So this is the title of the talk, Crisis Culture and Emergency Aesthetics in the Bang Lab.
[51:50.710 --> 51:55.670]  Um, so we're going to talk about two projects, really, that look at these issues.
[51:56.110 --> 52:00.770]  And, uh, then one, we're going to talk about three projects, one earlier project and a few inspirations.
[52:01.390 --> 52:05.950]  And, um, yeah, let's do that.
[52:07.330 --> 52:22.510]  So, um, the, the kind of motivation for the talk is for us to be thinking about these, these questions of crisis culture and really of like, how do we as artists and hackers and, uh, hacktivists and artivists or whatever the f*ck we call ourselves, uh,
[52:22.790 --> 52:24.390]  how do we respond to crisis?
[52:24.850 --> 52:25.330]  Right?
[52:25.490 --> 52:34.550]  Like, uh, economic crisis, ecological crisis, uh, humanitarian crisis, these, all these crises were faced on a daily basis, faced with on a daily basis.
[52:34.550 --> 52:36.550]  And, uh, how do we respond?
[52:37.190 --> 52:57.370]  And, um, part of what, part of what we want to be thinking about is this idea, this strategy of world building, that part of the, part of the rhetoric of crisis, like financial crisis, is, is just a way for, uh, systems of power to be able to push through changes that they want.
[52:57.670 --> 53:02.210]  Um, so instead we want to say that in a moment of crisis that why not reimagine things?
[53:02.210 --> 53:12.570]  Um, we could, why don't we imagine what the future is going to be and start building that future instead of, uh, just responding to, uh, what other people want.
[53:13.370 --> 53:23.330]  Um, so just starting with some inspirations, um, we're really inspired by, I mean, we're really think of ourselves as artists, I think, performance artists, new media artists.
[53:23.330 --> 53:27.370]  Um, and so we're really inspired by this history of performance art.
[53:27.930 --> 53:35.290]  And, um, part of what we wanted to do is to show some of these, these two older projects to say that, um, life didn't start with Arduino.
[53:35.990 --> 53:45.950]  Um, but that people have been, uh, thinking about things like sensors and, uh, and like creating relational objects and imagining their own technologies for a long time.
[53:45.950 --> 53:48.610]  So there's an artist named Ligia Clark from Brazil.
[53:48.930 --> 53:50.930]  And, uh, this is a project she did in the sixties.
[53:51.010 --> 53:53.070]  She was making relational objects.
[53:53.930 --> 54:01.870]  Um, uh, Marina Bramovic and Uley, uh, this piece, Rest Energy, which is really interesting for us.
[54:02.070 --> 54:04.850]  Um, this project you'll see called Technosexual.
[54:05.330 --> 54:14.610]  Um, well, which we will explain in a moment, but, uh, we're interested in the, they actually mic'd her heartbeat sound.
[54:14.790 --> 54:19.590]  And we didn't know that at the time that we were working on this project that you will see in just a moment.
[54:19.790 --> 54:22.510]  But we just think that's really interesting for us.
[54:22.730 --> 54:24.790]  Just the earlier project using heartbeat.
[54:25.530 --> 54:45.750]  Um, but also thinking about inspirations, uh, this is a good image, I think, because there's, there's so many millions of examples from pop culture of this intersection of sex and, uh, technology and the feminine and, um, all those, those things that are wrapped up with the desire,
[54:45.970 --> 54:47.990]  the future we want, what we want, what not.
[54:50.030 --> 55:01.510]  And this is a shot from, uh, Guillermo Gomez-Pena's group La Pocha Nostra, and they use this thing called Mayan technology, which is kind of this idea that it doesn't really have to work.
[55:01.850 --> 55:04.730]  Um, which might be a lot of technology, right?
[55:04.730 --> 55:10.350]  Um, but, um, so this is also, um, one of our inspirations, but flying through.
[55:11.690 --> 55:13.610]  Um, do you want to talk about that?
[55:14.130 --> 55:24.450]  Uh, this is just, you know, this is in the bathroom at this hotel, and we like to think about how our everyday objects are changing, um, with these sensors.
[55:24.610 --> 55:30.250]  You know, you just wave your hand and you get soap or water or paper towel and how our everyday lives are changing with sensors.
[55:31.210 --> 55:44.350]  And, and part of this was to say that we're really inspired by the tradition of feminist art, and a lot of that tradition was to say that, uh, the, the things that we interact with on a daily basis are a perfectly good subject for art, right?
[55:44.930 --> 55:49.270]  Uh, and so today, what are the things we interact with on a daily basis?
[55:49.390 --> 55:49.930]  They're sensors.
[55:50.150 --> 55:51.710]  They're like magical infrared buttons.
[55:51.950 --> 55:54.450]  They're things you wave your hands in front of and something happens.
[55:54.990 --> 55:57.030]  Um, did you guys use this thing in the bathroom?
[55:57.030 --> 55:58.210]  Or is it in the men's room too?
[55:58.950 --> 55:59.950]  Yeah, it's not in the men's room.
[56:00.190 --> 56:02.030]  Oh, it's only in the ladies room.
[56:02.070 --> 56:04.830]  So only like five people in this room saw that.
[56:06.230 --> 56:06.550]  Cool.
[56:06.890 --> 56:07.150]  Cool.
[56:07.590 --> 56:07.690]  Woo!
[56:07.850 --> 56:08.470]  Have we got more than that?
[56:08.550 --> 56:08.710]  Good.
[56:09.390 --> 56:11.430]  Um, anybody heard of Ghost in the Shell?
[56:13.030 --> 56:13.910]  You watched it twice.
[56:14.090 --> 56:15.090]  Yeah, yeah, of course.
[56:15.270 --> 56:16.870]  Definitely another inspiration.
[56:17.110 --> 56:18.130]  Lots and lots of sci-fi.
[56:18.750 --> 56:21.490]  Um, and in Ghost in the Shell, they have full prosthetic bodies, right?
[56:21.710 --> 56:24.910]  So part of what we want to think about is, uh, how do we get there?
[56:25.070 --> 56:25.930]  How do we make that happen?
[56:26.330 --> 56:33.450]  And, um, and also that's a good time to say that we're thinking about these things critically, right?
[56:33.730 --> 56:44.510]  Like how do we get to full prosthetic bodies, but not assuming that it's going to be good, or not assuming that using Second Life or virtual sex or cyber sex or any of these things are an improvement on our lives.
[56:44.750 --> 56:47.390]  Um, but to just think about what are the implications?
[56:47.390 --> 56:49.510]  What happens when we have full prosthetic bodies?
[56:50.830 --> 56:58.630]  Uh, and, uh, this is a picture of this conversation I had with Stellark in a previous performance that I did.
[56:58.750 --> 57:00.830]  He's another artist who's done a lot of body augmentation.
[57:01.370 --> 57:05.950]  And, um, he's an artist that we're inspired with because we totally disagree with him.
[57:06.090 --> 57:08.030]  He likes to say things like, the body is obsolete.
[57:08.410 --> 57:10.950]  Um, but I think that's, that's pretty ridiculous.
[57:11.090 --> 57:11.970]  He's just being provocative.
[57:12.230 --> 57:15.890]  Um, but in particular in this talk, he said that, uh, avatars have no organs.
[57:15.890 --> 57:20.650]  Uh, and so, that sounds like crap to me.
[57:21.150 --> 57:25.950]  Um, so, this NEST project kind of responds to that by showing that it's not true.
[57:28.210 --> 57:31.230]  So, um, oh, I just put this in too.
[57:31.350 --> 57:36.610]  So one thing, so we're talking about augmented reality, but, um, here on our slide, a lot of our work we're talking about mixed reality.
[57:37.010 --> 57:45.390]  Um, so technosexuals, this erotic mixed reality performance with biometric sensors to create live audio to bridge virtual and physical space, right?
[57:45.850 --> 57:47.810]  Um, what the hell do we mean by mixed reality?
[57:48.010 --> 57:51.790]  I mean, we, I like this term mixed reality because it's a broader term than augmented reality.
[57:51.990 --> 57:54.770]  Um, and like you said, augmented reality is such a buzzword.
[57:54.890 --> 58:00.310]  And people have the specific idea that it's about translucent headsets or it's about iPhone apps or whatever.
[58:00.530 --> 58:05.350]  Um, but I like mixed reality because it's a broader field of just any combinations of realities.
[58:05.550 --> 58:07.790]  And it's similar to our personal experience.
[58:07.810 --> 58:09.950]  I'm interested in doing work about my personal life.
[58:10.330 --> 58:16.590]  And, uh, this idea of mixing realities is something like mixing genders, and sexualities.
[58:17.230 --> 58:21.090]  And, um, I'm a transgender woman, so I'm interested in those things.
[58:22.230 --> 58:23.450]  Um, yeah.
[58:24.570 --> 58:30.590]  And so, um, this is kind of like a shot from Second Life, from our performance Technosexual.
[58:31.150 --> 58:33.550]  Uh, do we have like a live shot?
[58:33.690 --> 58:34.010]  Oh, okay.
[58:34.090 --> 58:35.490]  More sex in Second Life.
[58:36.290 --> 58:42.950]  But wait, but it's, so that's also to say, okay, so this project Technosexual is where, we're doing performance in real life and Second Life at the same time, right?
[58:43.270 --> 58:48.630]  And thinking about how we can use sensors to extend and expand the tradition of performance art.
[58:48.770 --> 58:58.670]  Where if performance art is about making art with the body, then how does that question change when the body is augmented, distributed, virtual, digital, all of these things.
[58:58.930 --> 59:01.610]  Um, and also working with pop culture, right?
[59:01.750 --> 59:02.850]  And the things that people are using.
[59:02.850 --> 59:07.910]  So, part of why we're interested in Second Life is because there's 15 million users in Second Life.
[59:08.090 --> 59:10.770]  So that means there's millions of people, right?
[59:10.870 --> 59:12.090]  And 50 in World of Warcraft.
[59:12.370 --> 59:13.970]  There are hundreds of millions of users of Virtual World.
[59:14.110 --> 59:27.570]  So that means there's hundreds of millions of people interested in, okay, okay, maybe millions or thousands of people interested in having sex as a cat person, or a bear, or a fox, or a bird, or a dragon.
[59:27.930 --> 59:28.950]  Cyborg cat person.
[59:28.990 --> 59:29.950]  Cyborg cat person.
[59:30.190 --> 59:31.570]  Um, and so what does that mean?
[59:32.390 --> 59:45.590]  We're interested in thinking about these emergent forms of embodiment and how, as we can embody these things more, something like trans-species or trans-real becomes less of a fantastical thing and more of people's everyday reality.
[59:47.070 --> 01:00:01.450]  Also, on that note, thinking of what you were saying, Pam, because coming from a musical background as well, I'm really thinking about the body as an instrument, and so that's kind of where this came from and using biometrics.
[01:00:02.170 --> 01:00:17.050]  And so we created this analog version, this little prototype of this stethoscope with the piezo sensor, like a contact mic attached to it, and we actually recorded the sound of our heartbeats and...
[01:00:17.050 --> 01:00:18.850]  Let me just skip through it.
[01:00:19.650 --> 01:00:19.830]  Oh, yeah.
[01:00:20.450 --> 01:00:26.890]  And then we moved on to this, which is the digital heart rate monitor version.
[01:00:27.070 --> 01:00:29.050]  So it uses this... where's the mouse?
[01:00:29.050 --> 01:00:37.010]  It uses this EKG strap right here to detect the electricity in your heartbeat, and then there's this wireless transmitter right there.
[01:00:37.190 --> 01:00:41.370]  And so this is the wireless receiver from SparkFun, a little bridge circuit to the Arduino.
[01:00:42.370 --> 01:00:45.070]  And so then we were using two of them, so there's two of them.
[01:00:46.170 --> 01:00:49.350]  So this wirelessly reads the rate of your heart.
[01:00:49.450 --> 01:00:52.550]  It just gets a number that tells you the rate.
[01:00:53.450 --> 01:00:58.570]  And then we send that data to PeerData, PD, anyone?
[01:00:58.890 --> 01:00:59.390]  I'm sure.
[01:00:59.590 --> 01:01:04.810]  PeerData's the... it's a descendant of MaxMSP, one of the embankments that I work with.
[01:01:04.830 --> 01:01:08.910]  Right, yeah, so it's like the open source version, which we're also really interested in.
[01:01:09.010 --> 01:01:10.110]  Yeah, we'll show you the...
[01:03:52.130 --> 01:03:54.730]  that you just heard.
[01:03:54.850 --> 01:04:03.050]  It sounds so much better when it's actually live, so... let's see if this... yeah.
[01:04:04.810 --> 01:04:05.830]  Give it a number.
[01:04:06.230 --> 01:04:06.490]  Okay.
[01:04:07.670 --> 01:04:08.730]  You... no, the number's right here.
[01:04:09.230 --> 01:04:10.790]  Okay, you don't get to hear this, PD patch.
[01:04:12.050 --> 01:04:13.790]  Because the heartbeat sample's not in the right place.
[01:04:13.910 --> 01:04:15.730]  But I have two... we have another PD patch we'll show you.
[01:04:16.110 --> 01:04:28.190]  But it's a graphical programming language, so instead of doing assignments, like frequency equals Moses parentheses something, you just draw a line.
[01:04:28.350 --> 01:04:28.510]  Right?
[01:04:28.910 --> 01:04:32.810]  As a procedural programmer, it took a while to get the hang of that.
[01:04:32.810 --> 01:04:48.330]  I should just add that... I should just add that when this session, when the talking is done, and we have our audience participation, I actually have some Max patches that the glove will be connected to, so you can actually see Max in action and play around with the glove and... yeah.
[01:04:48.790 --> 01:04:49.010]  Awesome.
[01:04:49.790 --> 01:04:52.690]  Okay, so let's get back to the slides.
[01:04:52.690 --> 01:04:55.830]  How do you send the data into Second Life ?
[01:04:58.530 --> 01:04:59.230]  Were you...
[01:04:59.230 --> 01:05:02.110]  So we had to reiterate the question for the... because the recording...
[01:05:02.110 --> 01:05:02.250]  Oh, sure.
[01:05:02.590 --> 01:05:04.370]  He said how do you get the data into Second Life?
[01:05:04.690 --> 01:05:04.930]  Yeah.
[01:05:05.610 --> 01:05:05.710]  Yes.
[01:05:05.710 --> 01:05:07.210]  Or into the PD.
[01:05:07.770 --> 01:05:08.250]  Yeah.
[01:05:08.670 --> 01:05:09.590]  Into the patch, right?
[01:05:09.870 --> 01:05:09.930]  Yeah.
[01:05:10.070 --> 01:05:10.770]  Into Second Life.
[01:05:10.950 --> 01:05:11.950]  Oh, into Second Life.
[01:05:12.090 --> 01:05:12.530]  Oh.
[01:05:12.810 --> 01:05:15.870]  So in this case, we're only sending the sound data into Second Life.
[01:05:15.970 --> 01:05:21.670]  So basically, we're using this program called SoundFlower that routes sound from one application to another.
[01:05:22.850 --> 01:05:37.470]  But in another application, we'll show you in a few minutes, we actually patched the Second Life client, and we made the patch available, so that it would read data from a local file, so we can send data from the sensor straight into Second Life.
[01:05:37.810 --> 01:05:39.170]  So we'll get to that in a moment.
[01:05:39.930 --> 01:05:46.110]  But we get the data from the Arduino over the COM port, just... there's a COM port object in PD, so you get the heart rate data right here.
[01:05:47.970 --> 01:05:48.330]  Okay.
[01:05:50.790 --> 01:05:51.790]  And you saw that.
[01:05:52.790 --> 01:05:53.150]  Okay.
[01:05:56.150 --> 01:05:58.190]  So, Virus Circus, do you want to talk about that?
[01:05:58.190 --> 01:05:58.910]  Sure.
[01:05:59.210 --> 01:06:06.750]  So, Virus Circus, so we've been collaborating and also real life partners for a few years now.
[01:06:06.930 --> 01:06:10.470]  And so this is maybe our fourth piece that we've done together.
[01:06:10.690 --> 01:06:16.350]  So, Virus Circus, we started really wanting to explore more wearable electronics.
[01:06:17.010 --> 01:06:20.030]  And so, and creating soft sensors.
[01:06:20.870 --> 01:06:24.850]  And so, we really got into this with Virus Circus.
[01:06:25.770 --> 01:06:27.010]  Do we have...
[01:06:27.010 --> 01:06:27.310]  Do we have...
[01:06:27.310 --> 01:06:27.550]  Oh, okay.
[01:06:27.730 --> 01:06:28.370]  Do you want to talk about that?
[01:06:28.710 --> 01:06:29.070]  Sure.
[01:06:29.390 --> 01:06:34.130]  So, this really gets into this question of how do we respond to crises, right?
[01:06:34.270 --> 01:06:35.170]  Like health crisis.
[01:06:35.390 --> 01:06:44.530]  And so, with this project, we really want to be thinking about how the rhetoric of Virus Hysteria is used to control populations.
[01:06:44.930 --> 01:06:52.130]  Or really how the medical system in general controls people and how we can respond to that critically and imagine something else.
[01:06:52.590 --> 01:07:10.910]  So, this performance is really like a speculative kind of sci-fi series of performances that take place in this alternative future where viruses, because of proliferating viruses like H1N1, this is the CDC, this is like the standard CDC cover your cough poster.
[01:07:10.910 --> 01:07:18.450]  But if you notice in the corner over here, there's like a slight modification, which is failure to comply will result in a minimum of 10 years at a federal penitentiary.
[01:07:19.090 --> 01:07:21.990]  So, we're thinking about how that's really not so far off.
[01:07:22.570 --> 01:07:26.890]  And also this, you may be asked to put on a surgical mask to protect others.
[01:07:27.110 --> 01:07:28.930]  It's another big part of the performance.
[01:07:29.570 --> 01:07:31.030]  And we didn't put that text in.
[01:07:31.190 --> 01:07:32.810]  That's just the CDC poster.
[01:07:33.790 --> 01:07:36.250]  I'm sure people have seen this poster all over the place.
[01:07:36.630 --> 01:07:37.190]  Yes.
[01:07:37.370 --> 01:07:39.890]  At work, at school, on the subway.
[01:07:42.330 --> 01:07:47.610]  So, this is Micah's ultrasonic rangefinder bra.
[01:07:48.230 --> 01:07:49.490]  Did I say that right?
[01:07:49.670 --> 01:07:53.330]  So, we wanted to improve on the previous performances in different ways.
[01:07:53.610 --> 01:08:00.870]  One was to make everything wearable and wireless, just to make everything better and more portable.
[01:08:02.530 --> 01:08:07.430]  So, this is just a standard rangefinder sensor from SparkFun.
[01:08:08.310 --> 01:08:11.670]  And it has three different interfaces.
[01:08:12.270 --> 01:08:14.330]  Analog, pulse width modulation, and serial.
[01:08:14.830 --> 01:08:23.490]  And this whole conversation about massaging the data is really relevant to us because the rangefinder, although it can be really reliable, it's generally very noisy.
[01:08:23.790 --> 01:08:25.550]  And so, we tested all three interfaces.
[01:08:26.090 --> 01:08:30.990]  And pretty much on all three, even though it's supposed to be better over serial, it's not.
[01:08:31.150 --> 01:08:37.810]  If you're holding the rangefinder here, it should say the right value, like 777, and then it'll be like 777, 100, 777, 2, 777.
[01:08:38.850 --> 01:08:41.670]  So, we'll show you some of that code for reading this data.
[01:08:42.830 --> 01:08:54.990]  But also, so here we're using like the LilyPad Arduino, which is the sewable, you know, the wearable electronics one that Leah Buckley from MIT created.
[01:08:55.850 --> 01:09:01.410]  And so, it's really fun, you know, the conductive thread and conductive fabric.
[01:09:02.250 --> 01:09:03.810]  We got really into that.
[01:09:05.130 --> 01:09:06.790]  And what else?
[01:09:07.110 --> 01:09:07.810]  We used Zigbee.
[01:09:08.090 --> 01:09:08.930]  Yeah, you already said that.
[01:09:09.070 --> 01:09:09.210]  Sorry.
[01:09:09.210 --> 01:09:18.730]  And so, you can see here, I mean, the LilyPad Arduino, part of what makes it work is that it has these flat pads to make the connection so that the thread can lay across the pads.
[01:09:21.710 --> 01:09:27.950]  And a lot of the challenge is really just laying out the wiring and then finding all the shorts that might come about.
[01:09:28.130 --> 01:09:29.770]  I mean, okay, so we had some shorts.
[01:09:31.970 --> 01:09:39.790]  Because, like, basically, right where the rangefinder is, all these three wires have to come together in a really tight space because it's just a regular circuit.
[01:09:40.030 --> 01:09:46.910]  And so, conductive thread is unshielded and it has to be... basically, you just have to make sure it's separate.
[01:09:47.030 --> 01:09:49.150]  You can use puffy paint to shield it.
[01:09:51.290 --> 01:09:58.910]  So, there's a close-up of the electronics on that project and just using this little bit of fabric to find places to put the wires.
[01:10:00.890 --> 01:10:05.290]  Oh, another problem was that the Zigbee is an incredible battery hog.
[01:10:05.570 --> 01:10:10.450]  It took us a while to realize that we needed to have batteries, separate batteries for the Zigbee and for the Arduino.
[01:10:11.150 --> 01:10:14.890]  I mean, literally, the Zigbee will kill that battery in, like, an hour.
[01:10:15.870 --> 01:10:19.770]  And then, it just... it doesn't stop working, it just starts wigging out.
[01:10:20.230 --> 01:10:22.970]  It just starts, like, the lights start blinking and then...
[01:10:22.970 --> 01:10:27.910]  I mean, now we know that, like, when the lights start freaking out in a certain way, you just have to change the batteries and test the voltage.
[01:10:28.790 --> 01:10:33.510]  And then, we realized there's something called Li power, which is better for that.
[01:10:33.750 --> 01:10:35.710]  So, it would have been nice to know that beforehand.
[01:10:36.010 --> 01:10:39.130]  But, you know, you always figure things out as you go along.
[01:10:39.890 --> 01:10:41.210]  So, let's just...
[01:10:41.210 --> 01:10:48.130]  Oh, so here's a pressure sensor we created with conductive fabric and conductive thread.
[01:10:48.890 --> 01:10:54.090]  And neoprene and a little piece of Velostat, which is what the electronic components come in.
[01:10:54.090 --> 01:10:56.930]  And we found this on Instructables.
[01:10:57.230 --> 01:11:01.090]  And it's super easy and cheap.
[01:11:01.870 --> 01:11:02.930]  Well, yeah.
[01:11:03.150 --> 01:11:07.310]  We paid $60 for a whole roll of fabric and just used that little bit, but...
[01:11:08.290 --> 01:11:09.070]  Theoretically cheap.
[01:11:09.070 --> 01:11:09.110]  There'll be more.
[01:11:09.810 --> 01:11:11.870]  Cheap if you had the conductive fabric sitting around.
[01:11:12.270 --> 01:11:16.230]  Yeah, but it's really a great sensor and really easy to make.
[01:11:16.230 --> 01:11:22.050]  So, if you're interested in pressure sensors, I highly recommend looking this up.
[01:11:22.210 --> 01:11:23.550]  And it's really reliable.
[01:11:23.910 --> 01:11:25.630]  I mean, it has a really wide range, right?
[01:11:25.730 --> 01:11:28.050]  So, the pressure sensor starts responding when you just touch it.
[01:11:28.110 --> 01:11:30.210]  But if you squeeze it, it responds more.
[01:11:30.330 --> 01:11:32.970]  And then, when you squeeze really hard, the values still continue to change.
[01:11:33.910 --> 01:11:37.130]  And so, that pressure sensor was part of my collar.
[01:11:37.130 --> 01:11:39.330]  And so, it was actually a choking sensor.
[01:11:39.790 --> 01:11:45.610]  So, as Micah would choke me, the sound, it would send the data to PD.
[01:11:45.930 --> 01:11:48.550]  And it would create, like, a distortion on the sound.
[01:11:49.630 --> 01:11:52.790]  And here's my corset thing pre-sewing.
[01:11:53.010 --> 01:11:55.070]  Just the, you know, prototype.
[01:11:56.030 --> 01:11:58.930]  Yeah, we like to joke about how sex shops are like our art stores.
[01:11:59.150 --> 01:12:03.650]  Because we always seem, doing erotic art, we always have to stop at the sex shop and buy things.
[01:12:03.650 --> 01:12:05.590]  Like, so this corset, I mean, we bought the corset.
[01:12:05.670 --> 01:12:07.350]  We didn't sew all this latex stuff together.
[01:12:08.790 --> 01:12:09.490]  Okay, cool.
[01:12:10.090 --> 01:12:11.770]  Yeah, we're really interested in that.
[01:12:12.550 --> 01:12:16.150]  And erotic art as a kind of...
[01:12:16.670 --> 01:12:17.810]  Here's some code porn.
[01:12:18.770 --> 01:12:19.570]  Yeah, erotic.
[01:12:22.430 --> 01:12:24.390]  Yeah, speaking of erotic.
[01:12:27.050 --> 01:12:28.390]  Okay, forget what I was saying.
[01:12:28.610 --> 01:12:29.990]  So, speaking of looking at the code.
[01:12:29.990 --> 01:12:32.150]  So, there's like a few...
[01:12:32.150 --> 01:12:35.570]  There's like maybe three pieces of code that make this performance work.
[01:12:35.710 --> 01:12:36.850]  So, I'll just look at them really briefly.
[01:12:37.250 --> 01:12:39.010]  The Arduino code is actually really simple.
[01:12:39.670 --> 01:12:41.310]  So, this is the Arduino code.
[01:12:41.710 --> 01:12:42.890]  It's based on C.
[01:12:43.090 --> 01:12:45.090]  Probably a lot of you have written Arduino code.
[01:12:45.610 --> 01:12:46.350]  Yes, anybody?
[01:12:47.130 --> 01:12:48.710]  I just see blinking red lights.
[01:12:48.850 --> 01:12:49.490]  You're like the board.
[01:12:49.910 --> 01:12:50.650]  No, no.
[01:12:51.010 --> 01:12:51.950]  Just blinking.
[01:12:52.430 --> 01:12:53.550]  Some of them blink.
[01:12:54.930 --> 01:12:59.890]  So, for people who might know, maybe spend like two seconds explaining exactly what Arduino is.
[01:12:59.890 --> 01:13:01.410]  Oh, okay.
[01:13:01.670 --> 01:13:01.810]  Yes.
[01:13:02.310 --> 01:13:04.810]  So, Arduino is like an electronics prototyping platform.
[01:13:06.350 --> 01:13:10.650]  So, basically it's like a little circuit board that you order that's like 30 bucks.
[01:13:11.010 --> 01:13:14.730]  And it comes with a microprocessor and a bunch of inputs and outputs.
[01:13:15.050 --> 01:13:22.750]  So, it just makes it really easy to plug in things like sensors and batteries and make simple electronics projects.
[01:13:23.270 --> 01:13:27.850]  And really the benefit of it is that it has a programmable microprocessor, right?
[01:13:28.070 --> 01:13:34.150]  So, with Arduino, basically the brains of your circuit mostly can come from this C code, right?
[01:13:34.230 --> 01:13:35.750]  So, if you know C code, that's really convenient.
[01:13:36.430 --> 01:13:37.390]  That's like a really big deal.
[01:13:37.410 --> 01:13:45.690]  Because instead of having to figure out the right resistor to get the right sensor values, you can just take the sensor value and then manipulate it, right?
[01:13:46.690 --> 01:13:52.190]  And instead of having to constantly be changing your circuit, you can just do things in the code, recompile, re-upload.
[01:13:54.010 --> 01:13:56.710]  So, the only important parts of this code are really the analog read.
[01:13:56.890 --> 01:14:03.710]  So, this just gives you a value of the voltage that's on a particular pin on the Arduino.
[01:14:05.270 --> 01:14:06.810]  And so, what is this for?
[01:14:06.910 --> 01:14:07.930]  This is for the pressure sensor.
[01:14:08.810 --> 01:14:11.150]  So, we have voltage going through the pressure sensor.
[01:14:11.290 --> 01:14:13.030]  And then so, as you squeeze it, the resistance changes.
[01:14:13.290 --> 01:14:14.410]  And so, the voltage would change here.
[01:14:15.670 --> 01:14:18.090]  And as you can see, it's like a...
[01:14:18.670 --> 01:14:21.930]  As you can see, it's averaging code.
[01:14:22.490 --> 01:14:25.930]  So, this takes like 10 values from the sensor.
[01:14:26.210 --> 01:14:26.610]  And then...
[01:14:28.490 --> 01:14:29.270]  Five minutes?
[01:14:29.530 --> 01:14:29.610]  Yeah.
[01:14:29.610 --> 01:14:29.750]  Okay.
[01:14:30.610 --> 01:14:31.270]  And talking...
[01:14:31.270 --> 01:14:32.070]  Too much talking about the code.
[01:14:32.070 --> 01:14:33.670]  So, this is the pulse width modulation code.
[01:14:33.790 --> 01:14:35.170]  The important part is pulse in.
[01:14:35.390 --> 01:14:35.530]  Yeah.
[01:14:35.810 --> 01:14:37.030]  This is the PD code.
[01:14:37.230 --> 01:14:38.230]  We already showed you some of that.
[01:14:38.610 --> 01:14:39.010]  We'll...
[01:14:39.010 --> 01:14:40.870]  I can show you it working after we see the video.
[01:14:42.370 --> 01:14:42.770]  Uh...
[01:14:42.770 --> 01:14:44.650]  This is the code for Second Life.
[01:14:45.090 --> 01:14:47.090]  And so, all of this code is on our website.
[01:14:47.570 --> 01:14:47.970]  Uh...
[01:14:47.970 --> 01:14:48.830]  My website...
[01:14:48.830 --> 01:14:49.530]  Put at the beginning.
[01:14:49.750 --> 01:14:51.150]  It's transreal.org.
[01:14:51.410 --> 01:14:53.130]  And hers is eleeleele.org.
[01:14:54.790 --> 01:14:55.590]  And, um...
[01:14:55.590 --> 01:14:56.390]  I'm talking too much.
[01:14:56.910 --> 01:14:58.370]  But all this code is on the website.
[01:14:59.050 --> 01:14:59.450]  Okay.
[01:14:59.990 --> 01:15:00.390]  So...
[01:15:00.950 --> 01:15:02.010]  So, let's see some of this video.
[01:15:02.070 --> 01:15:02.530]  Yeah.
[01:15:03.550 --> 01:15:05.070]  Just a quick little snippet.
[01:15:07.310 --> 01:15:08.370]  A quick snippet?
[01:15:08.630 --> 01:15:09.070]  Yes.
[01:15:10.930 --> 01:15:11.810]  Like, uh...
[01:15:11.810 --> 01:15:12.610]  Here?
[01:15:14.590 --> 01:15:15.030]  Okay.
[01:16:03.840 --> 01:16:04.280]  Okay.
[01:16:04.560 --> 01:16:05.200]  So, Low on Time.
[01:16:05.380 --> 01:16:06.700]  You can watch that on our website.
[01:16:06.960 --> 01:16:08.000]  And all the rest of it, too.
[01:16:09.400 --> 01:16:09.840]  Uh...
[01:16:09.840 --> 01:16:10.580]  So...
[01:16:10.580 --> 01:16:14.640]  The Transporter Immigrant Tool is the last project we're gonna talk about in, like, three minutes.
[01:16:15.260 --> 01:16:15.820]  Uh...
[01:16:15.820 --> 01:16:16.340]  So...
[01:16:16.340 --> 01:16:29.020]  The Transporter Immigrant Tool is a project to take cheap, recycled cell phones off of eBay and write some software so that we can access the GPS in them without having service.
[01:16:29.540 --> 01:16:33.680]  And distribute them in Mexico to people who are hoping to cross the border.
[01:16:33.680 --> 01:16:47.280]  And so we've been mapping out safety sites on the U.S.-Mexico border, like water caches, highways, border patrol stations, whatever might get you to safety in case that you were dying of dehydration.
[01:16:47.480 --> 01:16:50.720]  Because the most common cause of death on the U.S.-Mexico border is dehydration.
[01:16:51.720 --> 01:16:53.000]  And we're talking...
[01:16:53.600 --> 01:16:56.820]  Estimates of, like, over 10,000 people dying crossing the border since 1994.
[01:16:56.840 --> 01:17:00.880]  So this is the response to the humanitarian crisis that I was talking about earlier.
[01:17:02.060 --> 01:17:02.540]  Um...
[01:17:02.540 --> 01:17:07.000]  And this is a project with Ricardo Dominguez who developed this idea of hacktivism.
[01:17:07.520 --> 01:17:07.660]  Right.
[01:17:07.800 --> 01:17:08.160]  If you...
[01:17:08.160 --> 01:17:10.900]  By the way, I should mention just on that slide, um...
[01:17:10.900 --> 01:17:11.540]  If you'll let me interrupt.
[01:17:11.840 --> 01:17:12.320]  Um...
[01:17:12.320 --> 01:17:12.680]  So...
[01:17:12.680 --> 01:17:15.880]  The panel that I did, um...
[01:17:15.880 --> 01:17:20.680]  At the last HOPE was about hacktivism and culture jamming and whatnot.
[01:17:20.720 --> 01:17:22.560]  And Ricardo was one of the panel members.
[01:17:22.760 --> 01:17:27.940]  So if you go back into the HOPE archives from the last one, you can find out more about Ricardo.
[01:17:28.160 --> 01:17:28.860]  And he's...
[01:17:28.860 --> 01:17:30.060]  He actually introduced me.
[01:17:30.500 --> 01:17:32.040]  And that's really great.
[01:17:32.400 --> 01:17:33.980]  He's a professor at UCSD.
[01:17:34.220 --> 01:17:35.240]  Ricardo Dominguez.
[01:17:35.780 --> 01:17:36.240]  Um...
[01:17:36.240 --> 01:17:38.440]  And so this is the interface for the transporter immigrant tool.
[01:17:38.640 --> 01:17:39.120]  You can...
[01:17:39.120 --> 01:17:41.400]  Once you have turned it on, um...
[01:17:41.400 --> 01:17:42.720]  It'll direct you to the local...
[01:17:42.720 --> 01:17:44.460]  The closest water cache.
[01:17:45.500 --> 01:17:46.540]  And, um...
[01:17:46.540 --> 01:17:48.480]  Project resulted in a lot of media attention.
[01:17:49.380 --> 01:17:49.900]  Um...
[01:17:49.900 --> 01:17:55.000]  There was an AP story that was syndicated by 600 news agencies worldwide and, uh...
[01:17:55.000 --> 01:17:55.620]  Video...
[01:17:55.620 --> 01:17:56.180]  I mean, uh...
[01:17:56.180 --> 01:17:57.920]  TV stories about it.
[01:17:58.620 --> 01:17:59.460]  And, uh...
[01:17:59.460 --> 01:18:03.440]  As a result, there's a bunch of investigations around us and the tool.
[01:18:04.060 --> 01:18:04.620]  Um...
[01:18:04.620 --> 01:18:05.660]  So the, uh...
[01:18:05.660 --> 01:18:13.080]  University of California received a letter from three right-wing senators in San Diego saying that they wanted to know, uh...
[01:18:13.080 --> 01:18:19.120]  How much funds were being used on the trans-border immigrant tool and if we had committed a felony by, uh...
[01:18:19.120 --> 01:18:19.760]  By, uh...
[01:18:19.760 --> 01:18:20.160]  What's the word?
[01:18:20.640 --> 01:18:22.560]  Enticing migrant people to cross the border.
[01:18:24.140 --> 01:18:24.580]  Um...
[01:18:24.580 --> 01:18:26.080]  And those investigations are still pending.
[01:18:26.620 --> 01:18:30.100]  And one last thing about the phone is that it includes, uh...
[01:18:30.100 --> 01:18:30.640]  Poetry...
[01:18:30.640 --> 01:18:31.760]  Or poetic sustenance.
[01:18:31.880 --> 01:18:33.380]  Because we think that, um...
[01:18:33.380 --> 01:18:33.500]  Um...
[01:18:33.500 --> 01:18:33.880]  It could help.
[01:18:34.300 --> 01:18:35.780]  So poems like this one.
[01:18:36.120 --> 01:18:37.100]  Poems like this one.
[01:18:37.420 --> 01:18:39.200]  We translated these into audio form.
[01:18:39.480 --> 01:18:41.080]  And you can hear them on the phone.
[01:18:41.680 --> 01:18:42.200]  Um...
[01:18:42.200 --> 01:18:42.920]  And...
[01:18:42.920 --> 01:18:44.740]  There's a legal fund for all the investigations.
[01:18:45.060 --> 01:18:47.420]  Because we have, like, eight lawyers at this point.
[01:18:47.780 --> 01:18:48.440]  And, uh...
[01:18:48.440 --> 01:18:50.900]  Police showing up at our offices at UCSD.
[01:18:51.200 --> 01:18:52.520]  And that's a drag.
[01:18:52.800 --> 01:18:53.280]  Um...
[01:18:53.280 --> 01:18:54.580]  All because of our artwork.
[01:18:55.620 --> 01:18:56.460]  And, um...
[01:18:56.460 --> 01:18:57.320]  This is our next project.
[01:18:57.460 --> 01:18:59.100]  It's a blue box that travels in time.
[01:18:59.920 --> 01:19:01.040]  And, um...
[01:19:01.040 --> 01:19:01.500]  Thanks.
[01:19:08.230 --> 01:19:09.280]  So, um...
[01:19:09.280 --> 01:19:12.520]  I'd like to thank you all for coming and, uh...
[01:19:12.520 --> 01:19:13.400]  Sitting through this.
[01:19:14.060 --> 01:19:16.700]  And right now, I'm gonna grab this piece of paper here.
[01:19:17.680 --> 01:19:18.500]  And, uh...
[01:19:18.500 --> 01:19:19.860]  Put on the...
[01:19:20.640 --> 01:19:22.020]  Almighty glove here.
[01:19:22.720 --> 01:19:24.840]  And, uh...
[01:19:26.970 --> 01:19:28.610]  I'll flip you, that's for sure.
[01:19:32.090 --> 01:19:32.530]  Okay.
[01:19:34.590 --> 01:19:35.030]  So...
[01:19:37.230 --> 01:19:39.330]  Again, this is...
[01:19:39.330 --> 01:19:39.910]  Yeah.
[01:19:40.090 --> 01:19:41.290]  Someone's gonna go home with a new glove.
[01:19:41.330 --> 01:19:44.870]  And again, if you're not gonna be here, uh, throughout the weekend, uh...
[01:19:44.870 --> 01:19:47.290]  The glove will probably be arriving, uh...
[01:19:47.290 --> 01:19:48.210]  Tomorrow evening.
[01:19:48.890 --> 01:19:49.250]  Um...
[01:19:49.250 --> 01:19:50.450]  If you're not gonna be here, just...
[01:19:50.990 --> 01:19:51.290]  Uh...
[01:19:51.290 --> 01:19:52.690]  After this, come and talk with me.
[01:19:52.790 --> 01:19:53.150]  And we'll...
[01:19:53.150 --> 01:19:54.350]  We'll work out a way.
[01:19:54.610 --> 01:19:54.910]  So...
[01:19:54.910 --> 01:19:57.570]  Actually, why don't one of you come up?
[01:19:57.710 --> 01:19:59.510]  Does anyone wanna come up and direct my hand here?
[01:19:59.730 --> 01:20:01.430]  And add something, uh...
[01:20:01.430 --> 01:20:01.970]  I'll do it.
[01:20:01.990 --> 01:20:02.170]  I'll direct it.
[01:20:02.170 --> 01:20:02.490]  You do it?
[01:20:03.350 --> 01:20:03.710]  Yeah.
[01:20:04.890 --> 01:20:05.250]  Okay.
[01:20:05.550 --> 01:20:06.910]  Okay, and I have to go like this.
[01:20:07.570 --> 01:20:07.930]  So...
[01:20:07.930 --> 01:20:07.970]  Yeah.
[01:20:08.270 --> 01:20:08.910]  Okay, don't look.
[01:20:09.110 --> 01:20:09.390]  Okay.
[01:20:09.950 --> 01:20:10.470]  Down?
[01:20:10.750 --> 01:20:11.090]  Down.
[01:20:11.430 --> 01:20:11.950]  Closer?
[01:20:12.630 --> 01:20:13.150]  Or...
[01:20:13.150 --> 01:20:14.510]  I don't like the yaw axis.
[01:20:15.050 --> 01:20:15.930]  Down right there?
[01:20:16.130 --> 01:20:16.550]  Or where?
[01:20:17.770 --> 01:20:19.350]  Do I have to push you down?
[01:20:19.550 --> 01:20:19.690]  Okay.
[01:20:19.910 --> 01:20:20.210]  Okay.
[01:20:20.450 --> 01:20:23.870]  We have Max P. Right here.
[01:20:23.950 --> 01:20:24.530]  Max P?
[01:20:24.750 --> 01:20:25.150]  All right.
[01:20:25.870 --> 01:20:29.370]  Max P, you won yourself a P5 virtual reality glove.
[01:20:29.830 --> 01:20:31.610]  And, uh, we'll talk after this.
[01:20:32.190 --> 01:20:36.950]  Um, so now I'm gonna open up my laptop and connect the glove down on the edge here somewhere.
[01:20:36.950 --> 01:20:43.370]  And if folks wanna play around, if you have any questions from the panel members, please come up and talk with us.
[01:20:43.610 --> 01:20:46.930]  Again, you can find out more information about all of this stuff.
[01:20:46.930 --> 01:20:52.790]  If you go to sensoryresearch.net, I'll have a link to all of the links that everyone has provided here.
[01:20:52.970 --> 01:21:02.490]  And also, um, with the, uh, HOPE archives, they're now trying to include information, uh, and links and whatnot with the recordings.
[01:21:03.090 --> 01:21:06.470]  So, uh, you'll be able to find out more information about all of this.
[01:21:06.650 --> 01:21:10.610]  Um, and again, uh, at, uh, 2 a.m.
[01:21:11.190 --> 01:21:15.570]  Saturday night slash Sunday morning, Paradox Explorer, my band, will be playing.
[01:21:15.770 --> 01:21:20.610]  You'll actually be able to see the glove, manipulating things, and also hear some really good music.
[01:21:21.050 --> 01:21:23.390]  Um, and, uh, guaranteed to be a good show.
[01:21:23.390 --> 01:21:25.110]  So, again, thank you very much for coming.
[01:21:25.310 --> 01:21:28.130]  I hope we gave you something to think about when you go home.
[01:21:28.330 --> 01:21:29.050]  So, thank you.
[01:21:41.080 --> 01:21:50.580]  Also, a, uh, quick plug, if, uh, any of you are still in town on Tuesday, the ARNY meetup, uh, will be held.
[01:21:50.580 --> 01:21:53.160]  Uh, it's run by Ori, who I was telling you about.
[01:21:53.300 --> 01:21:54.750]  Steven Feiner usually shows up.
[01:21:54.930 --> 01:21:56.830]  Uh, worth going to if you're around.
[01:21:57.080 --> 01:21:58.480]  And I think it's like a dollar.
[01:21:59.220 --> 01:21:59.620]  So...