Summary:
The purpose of Brain-Machine Interface (BMI) research is to create outputs from or inputs to the brain that are additional to those that occur naturally. There are two possible directions of data flow (data from the brain to outside machinery and data to the brain from outside machinery). In whole, this research suggests that the brain is a much more adaptable organ than it is commonly thought to be.Brain to machinery data flow (Data out)
The most immediate and practical goal of BMI research is to create a mechanical output from neuronal activity. Data out BMIs capitalize on the fact that motor cortex activity occurs approximately 300ms before its associated motor response. Therefore, a mechanical system that is to mimic a natural motor system has this period of time to create motor output while allowing the subject to maintain the experience of natural motor processing. One major goal of BMI research is to create a system that will allow patients who have damage between their motor cortex and muscular system to bypass the damaged route and activate outside mechanisms by using neuronal signals. This would potentially allow an otherwise paralyzed person to control a motorized wheelchair, computer pointer, or robotic arm by thought alone.
Apparatuses
The data-out BMI consists of multi-neuron population recordings in motor cortex, the output of which is electronically decoded. The neural implants typically consist of a four by four millimeter multi-electrode (25-100+ electrode) array that is implanted into the area of motor cortex that is most active during the motor task of interest (such as moving an arm).Data-out BMIs are a recent phenomenon because they involve several technological feats that are of recent origin: predicting (via FRMI) where the motor activity of interest occurs, the ability permanently implant single cell recording devices into the motor cortex, and the ability to make mathematical and mechanical sense of the neural output in the short time period necessary for such control. The first experiments into BMI were particularly limited by electrodes that had a short usable life (in the range of hours after being implanted) before they lost their ability to transmit useful signals. This was due to both buildup of organic materials on the wire ends and because the wires shapes tended to damage surrounding brain areas during animal movement. Newer, blunt-tipped, flexible, Teflon-coated stainless steel microwires produce very little damage and allow permanent implantation. Permanent implantation is necessary for complex testing because the interface has to be tuned specifically for each subject and for each implantation.
After neural outputs are obtained, additional machinery decodes the neural signals using algorithms (that are based upon the output associations that the system acquired during training) that derive the intended motor output of the system. Although decoding technology is currently customized for each application, the platforms have several characteristics in common. The hardware mechanisms allow for parallel, real-time processing based on the parallel data that was used to create the decoding algorithms. These mathematical algorithms derive a mechanical signal from the brain output and are adjusted by higher order algorithms to compensate for variations caused by the animal feedback learning. That is, the algorithms are sensitive to the fact that the subject's realizes that its thoughts are producing external cues, and the algorithms adjust in real time to compensate for this subject adjustment.
Studies that have demonstrated a functioning data-out BMI
Chapin, Moxon, Markowitz, and Nicolelis (1999) taught rats to obtain water via a bar-pressing task. The bar was electronically connected to a motor that controlled a robot arm. The robot arm had a resting position near the rat's head. When the bar was pressed, the arm rotated away from the cage and toward a water dropper. The degree of bar press was proportional to the speed of the arm movement. A small cup at the tip of the arm was able to catch water from the water dropper, and the rats learned how to coordinate the bar press with the arm movement to obtain water. The task was designed to get the rats to think about the action that the bar was having on the arm, not on the immediate task of pressing the bar: if the rats simply held the bar down then the cup would rotate past the water dropper, and if they never released the bar then the arm would never come back to its resting position near the cage so that the rat could drink the water.
Chapin et al. used the data from the activity of populations of neurons in the motor cortex (16 electrodes, 32 neurons) and thalamus (8 electrodes, 16 neurons) to create neuronal population functions that could explain a rat's forearm movement. This allowed for neural information to be processed by these algorithms in real time, and for control of the robot arm to be controlled by the output of these brain-derived signals. The input for the motor was then switched away from the bar and to the output of the neuronal population functions. Four of six of the rats (not coincidentally, those with the highest number of active task-related neurons) were routinely able to use the brain activity-derived signal to accomplish the task. The rats were theoretically able to control the arm with brain activity alone (e.g. ignoring the bar), and this dissociation between the bar movement and robot arm movement did occur, but only after extended practice with brain activity-derived robot arm control.
Wessberg, Stambaugh, Kralik, Beck, Chapin, Kim, Biggs, Srinivasan, and Nicolelis (2000) found that they could reliably predict and replicate the lever-press activity of an owl monkey in real time using only the signal derived from the monkey's brain. The monkey was trained to press a bar to the left in response to a light on the left and to the right to a light on the right. This study extended the findings of Chapin et al (1999) by adding two functional, robotic arms that moved simultaneously with the monkey's arm. In other words, the robotic arms acted like spatially separate third arms that were able to mimic the movement of the current arm. This study has obvious practical implications to the development of neural prosthetics, but it also allowed for the researchers to project the number of neurons that would need to be sampled to create accurate control of the prosthetic arm. They found that the number of neurons that needed to be sampled was roughly proportional to the accuracy of the arm-- such that 100 neurons will produce movements that are approximately 70% accurate and 500-700 neurons would have to be sampled to produce 95% accuracy. This suggests that neuroprotheses may be able to operate reliably by sampling a feasibly small number of neurons (in the hundreds), at least for the simple tasks with which they were tested.
Serruya, Hatsopoulos, Paninski, Fellows, & Donoghue (2002) demonstrated that signals from neural implants in the motor cortex of Rhesus monkeys could quickly and reliably be translated into a complex computer input to be used for a pointer interface. They also used a similar mechanical setup to Chapin et al (1999), but they trained Rhesus monkeys (that had been given motor implants) to play a pinball-like videogame using a handheld joystick. After input control of the videogame was switched away from the joystick and to the neurally-derived signal, the monkeys were not only able to continue to play the game, but eventually learned to play without a controller at all. Furthermore, the monkeys displayed evidence of learning how to use the neurally-derived signal more effectively. In other words, it appeared as if the monkeys not only accepted the new form of output, but adapted to it as well.
Using a far less mechanically complex system, Kennedy, Bakay, Moore, Adams, and Goldwaithe (2000) implanted a working data-out interface into the motor cortex of a completely paralyzed human patient. He implanted only two electrodes, but those electrodes were surrounded with neurotrophic factors that caused cortical tissue (which normally finds the electrodes disruptive or, at best, inert) to grow into the electrodes. After several weeks of neural growth, the patient was trained to "concentrate" to increase his neural signal, and through a relatively simple algorithmic system, he could reliably move a computer cursor horizontally across a computer screen. This study is important as a proof-of-concept for medical applications. It also demonstrates that humans are able to learn how to control their neural states even using new neural tissue-- a proof-of-concept that could make an almost unlimited variety of data-out applications feasible.
Speculative implications of data-out BMIs
The implications of data-out BMIs have been considered for decades (for a short review see Norman, 1993). Science fiction stories of a blurring line between human and machine are slowly being realized by studies such as those of Kennedy et al. Now that data-out BMIs are a limited reality, future applications will most likely be developed to attain higher bandwidth, higher accuracy, more reliability, and increased availability.Perhaps the most interesting phenomenon will be the way that the human brain adapts to the new interface. Humans are well suited to the task of controlling and directing imagination, and this is the essence of producing neural output. With a high bandwidth interface and adequate training, there is a possibility that humans will come to view brain-derived output as an extension of their bodies. External cognition theorists and philosophers (sometimes referred to as "activity theorists") cite the fact that humans already strongly associate the use of certain tools with their bodies (Nardi, 1996; Heidegger, 1962) and some go as far as to say that using a neural-output arm would be similar to learning other cognitive tools (Norman, 1993). Although remotely controlled limbs are likely find their first uses with the disabled, it is easy to speculate that these neural outputs could guide other machinery and be used to enhance already-functioning human output systems.
Machinery to brain data flow (Data in)
The research up to this point has concerned itself with the medically practical side of the BMI. Although creating an interface that starts with neural impulses and ends with mechanical actions is becoming more technologically feasible and the potential benefits to humanity are large, there may be uses for an interface that employs the opposite direction; one that that starts with a mechanical action and ends with neural impulses. Extra-sensory neural input has already been implemented in animals, although only for a very limited application. However, due to complicated ethical and practical problems, human implementation has not been seriously considered.Apparatus
Data-in BMI implants typically consist of few electrodes. This is due to the fact that the electrodes are implanted for the purpose of electrical stimulation of neurons and a single current can have an effect on many neurons.Demonstration of a working data-in BMI
Mechanical to neural BMI research has found at least one potentially useful application, that of remotely stimulated rat. Talwar, Xu, Hawley, Weiss, Moxon, & Chapin (2002) implanted three stimulating probes into the brains of rats. Two of the probes were implanted into the area of somatosensory cortex that receives input from the left and right whiskers. Stimulation of these probes is interpreted by the rat as a whisker touch. The third probe went into the medial forebrain bundle and was used as a way to reward the rat for following instructions. Probes were fed into a harness that contained the radio equipment and stimulating machinery that was necessary for the rat to receive remote input.
Rats were trained on a maze and were taught to turn left or right when stimulated by the appropriate left or right whisker probe. Rats were reinforced for following instructions with MFB stimulation. Rats were trained to interpret MFB stimulation alone (when it was not preceded by a whisker touch signal) as an indication to move forward. After training, the rats were able to generalize their lab training to tasks outside of the laboratory and were guided in real time via computer-controlled transmitters through a variety of difficult obstacles. It is suggested that these rats could be used for search-and-rescue tasks that require rat-like maneuverability that is not possible with modern robotics.
Data-in BMIs have never been implemented in humans. However, electrical brain-stimulating mechanisms have been implanted in humans for the treatment of epilepsy and to Parkinson's disease (Pearson, 2002). It would be inappropriate to label this a true data-in BMI because it does not input meaningful data. However these human applications serve as proof that simple stimulation from chronic implants is possible in the human brain and may be cited as evidence that other forms of human BMIs are feasible.
Speculation on Data-in BMIs
Although the data-in BMI is currently only a way of influencing the decisions of an animal based upon trained responses to broad neural stimulation, the far future of the BMI has captured the imagination of many scientific theorists. There is tremendous speculative interest in the possibility of chronically implanting a series of stimulating electrodes into a sensory area of human cortex. This implant could stimulate an existing system in a new way (such as adding infrared sensitivity to the visual system) or it could create a completely new form of human sensory experience by tying a portion of the brain to machinery that is attuned to a novel form of energy (such as a computer interface). If this novel system is eventually feasible, it will almost undoubtedly fall short of the hopes of most science fiction writers, because it would almost definitely require extensive training and perhaps implantation within a critical period of development (Norman, 1993).Combining Data in and Data out systems for a natural feedback system
One major criticism of data-out neural implant systems that are designed to bypass the natural motor is that they do not allow for feedback of the movement to inform the person where his or her mechanically-controlled device may be and what it is experiencing. A true neuroprosthetic would take the current data-out system of deriving the mechanical intentions of neural signals and transmit this signal to a mechanical device that not only moves, but also contains tactile, visual, and proprioceptive sensors that can feed into the appropriate sensory brain areas. This system has been proposed as a distant goal, but has not yet been implemented.
Conclusion
The technology to create permanent BMIs is not even a decade old, and proof-of-concept tests have already demonstrated that with as few as two electrodes a brain can create a somewhat useful filtered signal, and, with many more electrodes, motion can be replicated with reasonable accuracy. Furthermore, there is evidence that the brain is not only able to produce this output, but is able to adapt to it as well. In the case of neural stimulation via electrodes, the brain responds to external input extraordinarily well, and this input can severely influence the behavior of the organism.References:
Chapin, J.K, Moxon, K.A., Markowitz, R.S. and Nicolelis, M.A.L. (1999). Real-time control of a robot arm using simultaneously recorded neurons in the motor cortex. Nature Neurosciences, 2:664-670.
Heidegger, M. (1962). Being and Time. New York: Harper & Row.
Kennedy P.R., Bakay R.A., Moore M.M., Adams K.,& Goldwaithe J. (2000). Direct control of a computer from the human central nervous system. IEEE Trans Rehabil Eng. 2000 Jun;8(2):198-202.
Nardi, B.A. Context and consciousness: Activity Theory and Human-Computer Interaction. Cambridge, MA: MIT Press.
Norman, D.A. (1993). Things That Make Us Smart: Defending Human Attributes in the Age of the Machine. Reading, MA: Addison-Wesley.
Pearson, H. (2002). Shocks switch brain on. Nature News Service, Available online: www.nature.com/nsu/021104/021104-13.html.
Serruya, M., Hatsopoulos, N.G., Paninski, L., Fellows, M.R., & Donoghue, J. P. (2002).
Immediate, Real-Time Use of a neurally-based control signal for movement. Nature, 416, 141-142.
Talwar, S.K., Xu, S., Hawley, E. S., Weiss, S. A., Moxon, K. A., & Chapin, J. K.. (2002). Rat navigation guided by remote control. Nature, 417(May 2):37-38.
Wessberg, J., Stambaugh, C.R., Kralik, J.D., Beck, P.D., Chapin, J.K., Kim, J., Biggs, S.J., Srinivasan, M.A. & Nicolelis, M.A.L. (2000). Real-Time Prediction of Hand Trajectory by Ensembles of Cortical Neurons in Primates. Nature, 408, 361-365.