{"title": "A Brain-Machine Interface Operating with a Real-Time Spiking Neural Network Control Algorithm", "book": "Advances in Neural Information Processing Systems", "page_first": 2213, "page_last": 2221, "abstract": "Motor prostheses aim to restore function to disabled patients. Despite compelling proof of concept systems, barriers to clinical translation remain. One challenge is to develop a low-power, fully-implantable system that dissipates only minimal power so as not to damage tissue. To this end, we implemented a Kalman-filter based decoder via a spiking neural network (SNN) and tested it in brain-machine interface (BMI) experiments with a rhesus monkey. The Kalman filter was trained to predict the arm\u2019s velocity and mapped on to the SNN using the Neural Engineer- ing Framework (NEF). A 2,000-neuron embedded Matlab SNN implementation runs in real-time and its closed-loop performance is quite comparable to that of the standard Kalman filter. The success of this closed-loop decoder holds promise for hardware SNN implementations of statistical signal processing algorithms on neuromorphic chips, which may offer power savings necessary to overcome a major obstacle to the successful clinical translation of neural motor prostheses.", "full_text": "A Brain-Machine Interface Operating with a\nReal-Time Spiking Neural Network Control\n\nAlgorithm\n\nJulie Dethier\u2217\n\nDepartment of Bioengineering\nStanford University, CA 94305\njdethier@stanford.edu\n\nChris Eliasmith\n\nCentre for Theoretical Neuroscience\n\nUniversity of Waterloo, Canada\nceliasmith@uwaterloo.ca\n\nShauki A. Elassaad\n\nDepartment of Bioengineering\nStanford University, CA 94305\nshauki@stanford.edu\n\nPaul Nuyujukian\n\nDepartment of Bioengineering\n\nSchool of Medicine\n\nStanford University, CA 94305\npaul@npl.stanford.edu\n\nTerry Stewart\n\nCentre for Theoretical Neuroscience\n\nUniversity of Waterloo, Canada\ntcstewar@uwaterloo.ca\n\nKrishna V. Shenoy\n\nDepartment of Electrical Engineering\n\nDepartment of Bioengineering\nDepartment of Neurobiology\nStanford University, CA 94305\nshenoy@stanford.edu\n\nKwabena Boahen\n\nDepartment of Bioengineering\nStanford University, CA 94305\nboahen@stanford.edu\n\nAbstract\n\nMotor prostheses aim to restore function to disabled patients. Despite compelling\nproof of concept systems, barriers to clinical translation remain. One challenge\nis to develop a low-power, fully-implantable system that dissipates only minimal\npower so as not to damage tissue. To this end, we implemented a Kalman-\ufb01lter\nbased decoder via a spiking neural network (SNN) and tested it in brain-machine\ninterface (BMI) experiments with a rhesus monkey. The Kalman \ufb01lter was trained\nto predict the arm\u2019s velocity and mapped on to the SNN using the Neural Engineer-\ning Framework (NEF). A 2,000-neuron embedded Matlab SNN implementation\nruns in real-time and its closed-loop performance is quite comparable to that of the\nstandard Kalman \ufb01lter. The success of this closed-loop decoder holds promise for\nhardware SNN implementations of statistical signal processing algorithms on neu-\nromorphic chips, which may offer power savings necessary to overcome a major\nobstacle to the successful clinical translation of neural motor prostheses.\n\n\u2217Present: Research Fellow F.R.S.-FNRS, Systmod Unit, University of Liege, Belgium.\n\n1\n\n\f1 Cortically-controlled motor prostheses: the challenge\n\nMotor prostheses aim to restore function for severely disabled patients by translating neural signals\nfrom the brain into useful control signals for prosthetic limbs or computer cursors. Several proof\nof concept demonstrations have shown encouraging results, but barriers to clinical translation still\nremain. One example is the development of a fully-implantable system that meets power dissipation\nconstraints, but is still powerful enough to perform complex operations. A recently reported closed-\nloop cortically-controlled motor prosthesis is capable of producing quick, accurate, and robust com-\nputer cursor movements by decoding neural signals (threshold-crossings) from a 96-electrode array\nin rhesus macaque premotor/motor cortex [1]-[4]. This, and previous designs (e.g., [5]), employ\nversions of the Kalman \ufb01lter, ubiquitous in statistical signal processing. Such a \ufb01lter and its variants\nare the state-of-the-art decoder for brain-machine interfaces (BMIs) in humans [5] and monkeys [2].\nWhile these recent advances are encouraging, clinical translation of such BMIs requires fully-\nimplanted systems, which in turn impose severe power dissipation constraints. Even though it is an\nopen, actively-debated question as to how much of the neural prosthetic system must be implanted,\nwe note that there are no reports to date demonstrating a fully implantable 100-channel wireless\ntransmission system, motivating performing decoding within the implanted chip. This computation\nis constrained by a stringent power budget: A 6\u00d7 6mm2 implant must dissipate less than 10mW to\navoid heating the brain by more than 1\u25e6C [6], which is believed to be important for long term cell\nhealth. With this power budget, current approaches can not scale to higher electrode densities or to\nsubstantially more computer-intensive decode/control algorithms.\nThe feasibility of mapping a Kalman-\ufb01lter based decoder algorithm [1]-[4] on to a spiking neural\nnetwork (SNN) has been explored off-line (open-loop). In these off-line tests, the SNN\u2019s perfor-\nmance virtually matched that of the standard implementation [7]. These simulations provide con\ufb01-\ndence that this algorithm\u2014and others similar to it\u2014could be implemented using an ultra-low-power\napproach potentially capable of meeting the severe power constraints set by clinical translation. This\nneuromorphic approach uses very-large-scale integrated systems containing microelectronic analog\ncircuits to morph neural systems into silicon chips [8, 9]. These neuromorphic circuits may yield\ntremendous power savings\u201450nW per silicon neuron [10]\u2014over digital circuits because they use\nphysical operations to perform mathematical computations (analog approach). When implemented\non a chip designed using the neuromorphic approach, a 2,000-neuron SNN network can consume as\nlittle as 100\u00b5W.\nDemonstrating this approach\u2019s feasibility in a closed-loop system running in real-time is a key,\nnon-incremental step in the development of a fully implantable decoding chip, and is necessary\nbefore proceeding with fabricating and implanting the chip. As noise, delay, and over-\ufb01tting play\na more important role in the closed-loop setting, it is not obvious that the SNN\u2019s stellar open-loop\nperformance will hold up. In addition, performance criteria are different in the closed-loop and open-\nloop settings (e.g., time per target vs. root mean squared error). Therefore, a SNN of a different\nsize may be required to meet the desired speci\ufb01cations. Here we present results and assess the\nperformance and viability of the SNN Kalman-\ufb01lter based decoder in real-time, closed-loop tests,\nwith the monkey performing a center-out-and-back target acquisition task. To achieve closed-loop\noperation, we developed an embedded Matlab implementation that ran a 2,000-neuron version of\nthe SNN in real-time on a PC. We achieved almost a 50-fold speed-up by performing part of the\ncomputation in a lower-dimensional space de\ufb01ned by the formal method we used to map the Kalman\n\ufb01lter on to the SNN. This shortcut allowed us to run a larger SNN in real-time than would otherwise\nbe possible.\n\n2 Spiking neural network mapping of control theory algorithms\n\nAs reported in [11], a formal methodology, called the Neural Engineering Framework (NEF), has\nbeen developed to map control-theory algorithms onto a computational fabric consisting of a highly\nheterogeneous population of spiking neurons simply by programming the strengths of their connec-\ntions. These arti\ufb01cial neurons are characterized by a nonlinear multi-dimensional-vector-to-spike-\nrate function\u2014ai(x(t)) for the ith neuron\u2014with parameters (preferred direction, maximum \ufb01ring\nrate, and spiking-threshold) drawn randomly from a wide distribution (standard deviation \u2248 mean).\n\n2\n\n\fRepresentation\n\nx \u2192 ai(x) \u2192 \u02c6x = \u2211i ai(x)\u03c6 x\nai(x) = G(\u03b1i\n)\n\n(cid:10) \u02dc\u03c6 x\ni \u00b7 x(cid:11) + Jbias\n\ni\n\ni\n\nTransformation\ny = Ax \u2192 b j(A\u02c6x)\nA\u02c6x = \u2211i ai(x)A\u03c6 x\ni\n\n\u02d9x = Ax \u2192 x = h\u2217 A(cid:48)x\n\nDynamics\nA(cid:48) = \u03c4A + I\n\nFigure 1: NEF\u2019s three principles. Representation. 1D tuning curves of a population of 50 leaky\nintegrate-and-\ufb01re neurons. The neurons\u2019 tuning curves map control variables (x) to spike rates\n(ai(x)); this nonlinear transformation is inverted by linear weighted decoding. G() is the neurons\u2019\nnonlinear current-to-spike-rate function. Transformation. SNN with populations bk(t) and a j(t)\nrepresenting y(t) and x(t). Feedforward and recurrent weights are determined by B(cid:48) and A(cid:48), as\ndescribed next. Dynamics. The system\u2019s dynamics is captured in a neurally plausible fashion by\nreplacing integration with the synapses\u2019 spike response, h(t), and replacing the matrices with A(cid:48) =\n\u03c4A + I and B(cid:48) = \u03c4B to compensate.\n\ni , where \u03c6 x\n\ni are the decoding weights.\n\nThe neural engineering approach to con\ufb01guring SNNs to perform arbitrary computations is under-\nlined by three principles (Figure 1) [11]-[14]:\nRepresentation is de\ufb01ned by nonlinear encoding of x(t) as a spike rate, ai(x(t))\u2014represented by\nthe neuron tuning curve\u2014combined with optimal weighted linear decoding of ai(x(t)) to recover\nan estimate of x(t), \u02c6x(t) = \u2211i ai(x(t))\u03c6 x\nTransformation is performed by using alternate decoding weights in the decoding operation to\nmap transformations of x(t) directly into transformations of ai(x(t)). For example, y(t) = Ax(t)\nis represented by the spike rates b j(A\u02c6x(t)), where unit j\u2019s input is computed directly from unit i\u2019s\noutput using A\u02c6x(t) = \u2211i ai(x(t))A\u03c6 x\nDynamics brings the \ufb01rst two principles together and adds the time dimension to the circuit. This\nprinciple aims at reuniting the control-theory and neural levels by modifying the matrices to render\nthe system neurally plausible, thereby permitting the synapses\u2019 spike response, h(t), (i.e., impulse\nresponse) to capture the system\u2019s dynamics. For example, for h(t) = \u03c4\u22121e\u2212t/\u03c4, \u02d9x = Ax(t) is realized\nby replacing A with A(cid:48) = \u03c4A + I. This so-called neurally plausible matrix yields an equivalent\ndynamical system: x(t) = h(t)\u2217 A(cid:48)x(t), where convolution replaces integration.\nThe nonlinear encoding process\u2014from a multi-dimensional stimulus, x(t), to a one-dimensional\nsoma current, Ji(x(t)), to a \ufb01ring rate, ai(x(t))\u2014is speci\ufb01ed as:\n\ni , an alternative linear weighting.\n\nai(x(t)) = G(Ji(x(t))).\n\n(1)\n\nHere G is the neurons\u2019 nonlinear current-to-spike-rate function, which is given by\n\n,\n\n\u03c4ref \u2212 \u03c4RC ln (1\u2212 Jth/Ji(x))\n\nG(Ji(x)) =\n\n(2)\nfor the leaky integrate-and-\ufb01re model (LIF). The LIF neuron has two behavioral regimes: sub-\nthreshold and super-threshold. The sub-threshold regime is described by an RC circuit with time\nconstant \u03c4RC. When the sub-threshold soma voltage reaches the threshold, Vth, the neuron emits a\nspike \u03b4 (t \u2212tn). After this spike, the neuron is reset and rests for \u03c4ref seconds (absolute refractory pe-\nriod) before it resumes integrating. Jth = Vth/R is the minimum input current that produces spiking.\nIgnoring the soma\u2019s RC time-constant when specifying the SNN\u2019s dynamics are reasonable because\nthe neurons cross threshold at a rate that is proportional to their input current, which thus sets the\nspike rate instantaneously, without any \ufb01ltering [11].\nThe conversion from a multi-dimensional stimulus, x(t), to a one-dimensional soma current, Ji, is\nperformed by assigning to the neuron a preferred direction, \u02dc\u03c6 x\ni , in the stimulus space and taking the\ndot-product:\n\nJi(x(t)) = \u03b1i\n\n,\n\n(3)\n\n(cid:110)\n\n(cid:111)\u22121\n\n(cid:10) \u02dc\u03c6 x\ni \u00b7 x(t)(cid:11) + Jbias\n\ni\n\n3\n\n\u22121010200400Stimulus xSpike rate (spikes/s)y(t)B'x(t)A'bk(t)aj(t)y(t)A'x(t)B'h(t)\fi\n\nis a bias current that accounts for background\nis either +1 or \u22121 (drawn randomly), for ON and OFF neurons,\n\nwhere \u03b1i is a gain or conversion factor, and Jbias\nactivity. For a 1D space, \u02dc\u03c6 x\ni\nrespectively. The resulting tuning curves are illustrated in Figure 1, left.\nThe linear decoding process is characterized by the synapses\u2019 spike response, h(t) (i.e., post-synaptic\ncurrents), and the decoding weights, \u03c6 x\ni , which are obtained by minimizing the mean square error.\nA single noise term, \u03b7, takes into account all sources of noise, which have the effect of introducing\nuncertainty into the decoding process. Hence, the transmitted \ufb01ring rate can be written as ai(x(t)) +\n\u03b7i, where ai(x(t)) represents the noiseless set of tuning curves and \u03b7i is a random variable picked\nfrom a zero-mean Gaussian distribution with variance \u03c3 2. Consequently, the mean square error can\nbe written as [11]:\n\n(cid:42)(cid:34)\n\n=\n\n1\n2\n\nx,\u03b7,t\n\nE =\n\n1\n2\n\n(cid:68)\n[x(t)\u2212 \u02c6x(t)]2(cid:69)\n(cid:42)(cid:34)\n\ni\n\nx(t)\u2212\u2211\n(cid:35)2(cid:43)\n\n(cid:35)2(cid:43)\n\nx,\u03b7,t\n\n(ai(x(t)) + \u03b7i)\u03c6 x\ni\n\nwhere (cid:104)\u00b7(cid:105)x,\u03b7 denotes integration over the range of x and \u03b7, the expected noise. We assume that the\nnoise is independent and has the same variance for each neuron [11], which yields:\n\n(4)\n\n(5)\n\n(6)\n\nE =\n\n1\n2\n\nx(t)\u2212\u2211\n\ni\n\nai(x(t))\u03c6 x\ni\n\n1\n2\n\n\u03c3 2\u2211\n\n(\u03c6 x\n\ni )2,\n\ni\n\n+\n\nx,t\n\nwhere \u03c3 2 is the noise variance \u03b7i\u03b7 j. This expression is minimized by:\n\n\u03c6 x\ni =\n\n\u0393\u22121\ni j \u03d2 j,\n\nN\n\n\u2211\n\nj\n\nwith \u0393i j =(cid:10)ai(x)a j(x)(cid:11)\n(cid:10)xa j(x)(cid:11)\n\nx + \u03c3 2\u03b4i j, where \u03b4 is the Kronecker delta function matrix, and \u03d2 j =\nx [11]. One consequence of modeling noise in the neural representation is that the matrix\n\u0393 is invertible despite the use of a highly overcomplete representation. In a noiseless representation,\n\u0393 is generally singular because, due to the large number of neurons, there is a high probability of\nhaving two neurons with similar tuning curves leading to two similar rows in \u0393.\n\n3 Kalman-\ufb01lter based cortical decoder\n\nIn the 1960\u2019s, Kalman described a method that uses linear \ufb01ltering to track the state of a dynamical\nsystem throughout time using a model of the dynamics of the system as well as noisy measure-\nments [15]. The model dynamics gives an estimate of the state of the system at the next time step.\nThis estimate is then corrected using the observations (i.e., measurements) at this time step. The\nrelative weights for these two pieces of information are given by the Kalman gain, K [15, 16].\nWhereas the Kalman gain is updated at each iteration, the state and observation matrices (de\ufb01ned\nbelow)\u2014and corresponding noise matrices\u2014are supposed constant.\nIn the case of prosthetic applications, the system\u2019s state vector is the cursor\u2019s kinematics, xt =\n[velx\nt ,1], where the constant 1 allows for a \ufb01xed offset compensation. The measurement vector,\nyt, is the neural spike rate (spike counts in each time step) of 192 channels of neural threshold\ncrossings. The system\u2019s dynamics is modeled by:\n\nt ,vely\n\nxt = Axt\u22121 + wt ,\nyt = Cxt + qt ,\n\n(7)\n(8)\nwhere A is the state matrix, C is the observation matrix, and wt and qt are additive, Gaussian noise\nsources with wt \u223c N (0,W) and qt \u223c N (0,Q). The model parameters (A, C, W and Q) are \ufb01t with\ntraining data by correlating the observed hand kinematics with the simultaneously measured neural\nsignals (Figure 2).\nFor an ef\ufb01cient decoding, we derived the steady-state update equation by replacing the adaptive\nKalman gain by its steady-state formulation: K = (I + WCQ\u22121C)\u22121 W CTQ\u22121. This yields the\nfollowing estimate of the system\u2019s state:\n\nxt = (I\u2212 KC)Axt\u22121 + Kyt = MDT\n\nx xt\u22121 + MDT\n\ny yt ,\n\n(9)\n\n4\n\n\fFigure 2: Neural and kinematic measurements (monkey J, 2011-04-16, 16 continuous trials) used to\n\ufb01t the standard Kalman \ufb01lter model. a. The 192 cortical recordings fed as input to \ufb01t the Kalman\n\ufb01lter\u2019s matrices (color code refers to the number of threshold crossings observed in each 50ms bin).\nb. Hand x- and y-velocity measurements correlated with the neural data to obtain the Kalman \ufb01lter\u2019s\nmatrices. c. Cursor kinematics of 16 continuous trials under direct hand control.\n\nx = (I\u2212 KC)A and MDT\n\nwhere MDT\ny = K are the discrete time (DT) Kalman matrices. The steady-\nstate formulation improves ef\ufb01ciency with little loss in accuracy because the optimal Kalman gain\nrapidly converges (typically less than 100 iterations).\nIndeed, in neural applications under both\nopen-loop and closed-loop conditions, the difference between the full Kalman \ufb01lter and its steady-\nstate implementation falls to within 1% in a few seconds [17]. This simplifying assumption reduces\nthe execution time for decoding a typical neuronal \ufb01ring rate signal approximately seven-fold [17],\na critical speed-up for real-time applications.\n\n4 Kalman \ufb01lter with a spiking neural network\n\nx(t) = h(t)\u2217(cid:0)A(cid:48)x(t) + B(cid:48)y(t)(cid:1) ,\n\nTo implement the Kalman \ufb01lter with a SNN by applying the NEF, we \ufb01rst convert Equation 9 from\nDT to continuous time (CT), and then replace the CT matrices with neurally plausible ones, which\nyields:\n\nx =(cid:0)MDT\n\nx \u2212 I(cid:1) /\u2206t and MCT\n\ny =(cid:0)MDT\n\ny\n\n(cid:1) /\u2206t, the CT\n\n(10)\n\n, with MCT\n\nx + I, B(cid:48) = \u03c4MCT\n\nwhere A(cid:48) = \u03c4MCT\nKalman matrices, and \u2206t = 50ms, the discrete time step; \u03c4 is the synaptic time-constant.\nThe jth neuron\u2019s input current (see Equation 3) is computed from the system\u2019s current state, x(t),\nwhich is computed from estimates of the system\u2019s previous state (\u02c6x(t) = \u2211i ai(t)\u03c6 x\ni ) and current\ninput (\u02c6y(t) = \u2211k bk(t)\u03c6 y\n\nk ) using Equation 10. This yields:\n\ny\n\nJj(x(t)) = \u03b1 j\n= \u03b1 j\n\n= \u03b1 j\n\nj\n\nj\n\n(cid:10) \u02dc\u03c6 x\nj \u00b7 x(t)(cid:11) + Jbias\n(cid:10) \u02dc\u03c6 x\nj \u00b7 h(t)\u2217(cid:0)A(cid:48) \u02c6x(t) + B(cid:48) \u02c6y(t)(cid:1)(cid:11) + Jbias\n(cid:32)\n(cid:42)\nA(cid:48)\u2211\nj \u00b7 h(t)\u2217\n\u02dc\u03c6 x\n(cid:32)\n(cid:68) \u02dc\u03c6 x\n\u2211\nj B(cid:48)\u03c6 y\n\ni + B(cid:48)\u2211\n\nai(t)\u03c6 x\n\n\u03c9 jkbk(t)\n\nk\n\ni\n\ni\n\nk\n\n(cid:33)(cid:43)\n\n+ Jbias\n\nj\n\nbk(t)\u03c6 y\nk\n\n(cid:33)\n\n+ Jbias\n\nj\n\n(11)\n\n(12)\n\nThis last equation can be written in a neural network form:\n(cid:69)\n\u03c9 jiai(t) +\u2211\n\nJj(x(t)) = h(t)\u2217\n\n(cid:69)\n\nk\n\n(cid:68) \u02dc\u03c6 x\n\nj A(cid:48)\u03c6 x\n\ni\n\nwhere \u03c9 ji = \u03b1 j\nspectively.\n\nand \u03c9 jk = \u03b1 j\n\nare the recurrent and feedforward weights, re-\n\n5 Ef\ufb01cient implementation of the SNN\n\nIn this section, we describe the two distinct steps carried out when implementing the SNN: creating\nand running the network. The \ufb01rst step has no computational constraints whereas the second must\nbe very ef\ufb01cient in order to be successfully deployed in the closed-loop experimental setting.\n\n5\n\nNeuron 1501005002000400060008000100001200014000\u221220020Time (ms)Velocity (cm/s) 0510x\u2212velocityy\u2212velocityabTrials: 0034-0049c1cm\fFigure 3: Computing a 1000-neuron pool\u2019s recurrent connections. a. Using connection weights re-\nquires multiplying a 1000\u00d71000 matrix by a 1000\u00d71 vector. b. Operating in the lower-dimensional\nstate space requires multiplying a 1\u00d71000 vector by a 1000\u00d71 vector to get the decoded state, mul-\ntiplying this state by a component of the A(cid:48) matrix to update it, and multiplying the updated state by\na 1000\u00d7 1 vector to re-encode it as \ufb01ring rates, which are then used to update the soma current for\nevery neuron.\n\n, preferred direction \u02dc\u03c6 x\n\nj\n\nj\n\nj , and decoding weight \u03c6 x\n\nNetwork creation: This step generates, for a speci\ufb01ed number of neurons composing the network,\nthe gain \u03b1 j, bias current Jbias\nj for each neuron. The\npreferred directions \u02dc\u03c6 x\nj are drawn randomly from a uniform distribution over the unit sphere. The\nmaximum \ufb01ring rate, max G(Jj(x)), and the normalized x-axis intercept, G(Jj(x)) = 0, are drawn\nrandomly from a uniform distribution on [200, 400] Hz and [-1, 1], respectively. From these two\nspeci\ufb01cations, \u03b1 j and Jbias\nare computed using Equation 2 and Equation 3. The decoding weights\n\u03c6 x\nj are computed by minimizing the mean square error (Equation 6).\nFor ef\ufb01cient implementation, we used two 1D integrators (i.e., two recurrent neuron pools, with\neach pool representing a scalar) rather than a single 3D integrator (i.e., one recurrent neuron pool,\nwith the pool representing a 3D vector by itself) [13]. The constant 1 is fed to the 1D integrators as\nan input, rather than continuously integrated as part of the state vector. We also replaced the bk(t)\nunits\u2019 spike rates (Figure 1, middle) with the 192 neural measurements (spike counts in 50ms bins),\nwhich is equivalent to choosing \u03c6 y\nk from a standard basis (i.e., a unit vector with 1 at the kth position\nand 0 everywhere else) [7].\nNetwork simulation: This step runs the simulation to update the soma current for every neuron,\nbased on input spikes. The soma voltage is then updated following RC circuit dynamics. Gaussian\nnoise is normally added at this step, the rest of the simulation being noiseless. Neurons with soma\nvoltage above threshold generate a spike and enter their refractory period. The neuron \ufb01ring rates\nare decoded using the linear decoding weights to get the updated states values, x and y-velocity.\nThese values are smoothed with a \ufb01lter identical to h(t), but with \u03c4 set to 5ms instead of 20ms to\navoid introducing signi\ufb01cant delay. Then the simulation step starts over again.\nIn order to ensure rapid execution of the simulation step, neuron interactions are not updated di-\nrectly using the connection matrix (Equation 12), but rather indirectly with the decoding matrix \u03c6 x\nj ,\ndynamics matrix A(cid:48), and preferred direction matrix \u02dc\u03c6 x\nj (Equation 11). To see why this is more ef\ufb01-\ncient, suppose we have 1000 neurons in the a population for each of the state vector\u2019s two scalars.\nComputing the recurrent connections using connection weights requires multiplying a 1000\u00d7 1000\nmatrix by a 1000-dimensional vector (Figure 3a). This requires 106 multiplications and about 106\nsums. Decoding each scalar (i.e., \u2211i ai(t)\u03c6 x\ni ), however, requires only 1000 multiplications and 1000\nsums. The decoded state vector is then updated by multiplying it by the (diagonal) A(cid:48) matrix, another\n2 products and 1 sum. The updated state vector is then encoded by multiplying it with the neurons\u2019\npreferred direction vectors, another 1000 multiplications per scalar (Figure 3b). The resulting to-\ntal of about 3000 operations is nearly three orders of magnitude fewer than using the connection\nweights to compute the identical transformation.\nTo measure the speedup, we simulated a 2,000-neuron network on a computer running Matlab 2011a\n(Intel Core i7, 2.7-GHz, Mac OS X Lion). Although the exact run-times depend on the computing\nhardware and software, the run-time reduction factor should remain approximately constant across\nplatforms. For each reported result, we ran the simulation 10 times to obtain a reliable estimate of\nthe execution time. The run-time for neuron interactions using the recurrent connection weights was\n9.9ms and dropped to 2.7\u00b5s in the lower-dimensional space, approximately a 3,500-fold speedup.\nOnly the recurrent interactions bene\ufb01t from the speedup, the execution time for the rest of the oper-\nations remaining constant. The run-time for a 50ms network simulation using the recurrent connec-\n\n6\n\nx=1000x100010001000a10001000x=1000(1000x(b\fTable 1: Model parameters\n\nSatis\ufb01es \ufb01rst two Bias current\nSatis\ufb01es \ufb01rst two Gain factor\n\n(cid:13)(cid:13)(cid:13) \u02dc\u03c6 x\n\nSymbol\nRange\nmax G(Jj(x))\n200-400 Hz\nG(Jj(x)) = 0 \u22121 to 1\nJbias\nj\n\u03b1 j\n\u02dc\u03c6 x\nj\n\u03c3 2\n\u03c4RC\nj\n\u03c4ref\nj\n\u03c4PSC\nj\n\nj\n0.1\n20 ms\n1 ms\n20 ms\n\n(cid:13)(cid:13)(cid:13) = 1\n\nDescription\nMaximum \ufb01ring rate\nNormalized x-axis intercept\n\nPreferred-direction vector\nGaussian noise variance\nRC time constant\nRefractory period\nPSC time constant\n\ntion weights was 0.94s and dropped to 0.0198s in the lower-dimensional space, a 47-fold speedup.\nThese results demonstrate the ef\ufb01ciency the lower-dimensional space offers, which made the closed-\nloop application of SNNs possible.\n\n6 Closed-loop implementation\n\nAn adult male rhesus macaque (monkey J) was trained to perform a center-out-and-back reaching\ntask for juice rewards to one of eight targets, with a 500ms hold time (Figure 4a) [1]. All animal\nprotocols and procedures were approved by the Stanford Institutional Animal Care and Use Com-\nmittee. Hand position was measured using a Polaris optical tracking system at 60Hz (Northern\nDigital Inc.). Neural data were recorded from two 96-electrode silicon arrays (Blackrock Microsys-\ntems) implanted in the dorsal pre-motor and motor cortex. These recordings (-4.5 RMS threshold\ncrossing applied to each electrode\u2019s signal) yielded tuned activity for the direction and speed of arm\nmovements. As detailed in [1], a standard Kalman \ufb01lter model was \ufb01t by correlating the observed\nhand kinematics with the simultaneously measured neural signals, while the monkey moved his arm\nto acquire virtual targets (Figure 2). The resulting model was used in a closed-loop system to control\nan on-screen cursor in real-time (Figure 4a, Decoder block). A steady-state version of this model\nserves as the standard against which the SNN implementation\u2019s performance is compared.\nWe built a SNN using the NEF methodology based on derived Kalman \ufb01lter parameters mentioned\nabove. This SNN was then simulated on an xPC Target (Mathworks) x86 system (Dell T3400, In-\ntel Core 2 Duo E8600, 3.33GHz). It ran in closed-loop, replacing the standard Kalman \ufb01lter as the\ndecoder block in Figure 4a. The parameter values listed in Table 1 were used for the SNN implemen-\ntation. We ensured that the time constants \u03c4RC\nwere smaller than the implementation\u2019s\ntime step (50ms). Noise was not explicitly added. It arose naturally from the \ufb02uctuations produced\nby representing a scalar with \ufb01ltered spike trains, which has been shown to have effects similar to\nGaussian noise [11]. For the purpose of computing the linear decoding weights (i.e., \u0393), we modeled\nthe resulting noise as Gaussian with a variance of 0.1.\nA 2,000-neuron version of the SNN-based decoder was tested in a closed-loop system, the largest\nnetwork our embedded MatLab implementation could run in real-time. There were 1206 trials\ntotal among which 301 (center-outs only) were performed with the SNN and 302 with the standard\n(steady-state) Kalman \ufb01lter. The block structure was randomized and interleaved, so that there is\nno behavioral bias present in the \ufb01ndings. 100 trials under hand control are used as a baseline\ncomparison. Success corresponds to a target acquisition under 1500ms, with 500ms hold time.\nSuccess rates were higher than 99% on all blocks for the SNN implementation and 100% for the\nstandard Kalman \ufb01lter. The average time to acquire the target was slightly slower for the SNN\n(Figure 5b)\u2014711ms vs. 661ms, respectively\u2014we believe this could be improved by using more\nneurons in the SNN.1 The average distance to target (Figure 5a) and the average velocity of the\ncursor (Figure 5c) are very similar.\n\n, and \u03c4PSC\n\n,\u03c4ref\n\ni\n\ni\n\ni\n\n1Off-line, the SNN performed better as we increased the number of neurons [7].\n\n7\n\n\fFigure 4: Experimental setup and results. a. Data are recorded from two 96-channel silicon electrode\narrays implanted in dorsal pre-motor and motor cortex of an adult male monkey performing a center-\nout-and-back reach task for juice rewards to one of eight targets with a 500ms hold time. b. BMI\nposition kinematics of 16 continuous trials for the standard Kalman \ufb01lter implementation. c. BMI\nposition kinematics of 16 continuous trials for the SNN implementation.\n\nFigure 5: SNN (red) performance compared to standard Kalman \ufb01lter (blue) (hand trials are shown\nfor reference (yellow)). The SNN achieves similar results\u2014success rates are higher than 99% on\nall blocks\u2014as the standard Kalman \ufb01lter implementation. a. Plot of distance to target vs. time both\nafter target onset for different control modalities. The thicker traces represent the average time when\nthe cursor \ufb01rst enters the acceptance window until successfully entering for the 500ms hold time.\nb. Histogram of target acquisition time. c. Plot of mean cursor velocity vs. time.\n\n7 Conclusions and future work\n\nThe SNN\u2019s performance was quite comparable to that produced by a standard Kalman \ufb01lter im-\nplementation. The 2,000-neuron network had success rates higher than 99% on all blocks, with\nmean distance to target, target acquisition time, and mean cursor velocity curves very similar to\nthe ones obtained with the standard implementation. Future work will explore whether these re-\nsults extend to additional animals. As the Kalman \ufb01lter and its variants are the state-of-the-art in\ncortically-controlled motor prostheses [1]-[5], these simulations provide con\ufb01dence that similar lev-\nels of performance can be attained with a neuromorphic system, which can potentially overcome the\npower constraints set by clinical applications.\nOur ultimate goal is to develop an ultra-low-power neuromorphic chip for prosthetic applications\non to which control theory algorithms can be mapped using the NEF. As our next step in this direc-\ntion, we will begin exploring this mapping with Neurogrid, a hardware platform with sixteen pro-\ngrammable neuromorphic chips that can simulate up to a million spiking neurons in real-time [9].\nHowever, bandwidth limitations prevent Neurogrid from realizing random connectivity patterns. It\ncan only connect each neuron to thousands of others if neighboring neurons share common inputs\n\u2014 just as they do in the cortex. Such columnar organization may be possible with NEF-generated\nnetworks if preferred directions vectors are assigned topographically rather than randomly. Imple-\nmenting this constraint effectively is a subject of ongoing research.\n\nAcknowledgment\n\nThis work was supported in part by the Belgian American Education Foundation(J. Dethier), Stan-\nford NIH Medical Scientist Training Program (MSTP) and Soros Fellowship (P. Nuyujukian),\nDARPA Revolutionizing Prosthetics program (N66001-06-C-8005, K. V. Shenoy), and two NIH\nDirector\u2019s Pioneer Awards (DP1-OD006409, K. V. Shenoy; DPI-OD000965, K. Boahen).\n\n8\n\nDecoderNeuralSpikesCursorVelocityab1cmTrials: 2056-2071BMI: Kalman decoderc1cmTrials: 1748-1763BMI: SNN decoder04008000510Time after Target Onset (ms)Distance to Target (cm) Mean distance to target a050010001500010203040Target Acquire Time (ms)Percent of Trials (%)Target acquisition time histogram b0200400600800100001020304050Time after Target Onset (ms)Cursor Velocity (cm/s)Mean cursor velocity Standard Kalman filterHandSpiking Neural Networkc\fReferences\n[1] V. Gilja, Towards clinically viable neural prosthetic systems, Ph.D. Thesis, Department of Computer\n\nScience, Stanford University, 2010, pp 19\u201322 and pp 57\u201373.\n\n[2] V. Gilja, P. Nuyujukian, C.A. Chestek, J.P. Cunningham, J.M. Fan, B.M. Yu, S.I. Ryu, and K.V. Shenoy, A\nhigh-performance continuous cortically-controlled prosthesis enabled by feedback control design, 2010\nNeuroscience Meeting Planner, San Diego, CA: Society for Neuroscience, 2010.\n\n[3] P. Nuyujukian, V. Gilja, C.A. Chestek, J.P. Cunningham, J.M. Fan, B.M. Yu, S.I. Ryu, and K.V. Shenoy,\nGeneralization and robustness of a continuous cortically-controlled prosthesis enabled by feedback con-\ntrol design, 2010 Neuroscience Meeting Planner, San Diego, CA: Society for Neuroscience, 2010.\n\n[4] V. Gilja, C.A. Chestek, I. Diester, J.M. Henderson, K. Deisseroth, and K.V. Shenoy, Challenges and op-\nportunities for next-generation intra-cortically based neural prostheses, IEEE Transactions on Biomedical\nEngineering, 2011, in press.\n\n[5] S.P. Kim, J.D. Simeral, L.R. Hochberg, J.P. Donoghue, and M.J. Black, Neural control of computer\ncursor velocity by decoding motor cortical spiking activity in humans with tetraplegia, Journal of Neural\nEngineering, vol. 5, 2008, pp 455\u2013476.\n\n[6] S. Kim, P. Tathireddy, R.A. Normann, and F. Solzbacher, Thermal impact of an active 3-D microelectrode\narray implanted in the brain, IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol.\n15, 2007, pp 493\u2013501.\n\n[7] J. Dethier, V. Gilja, P. Nuyujukian, S.A. Elassaad, K.V. Shenoy, and K. Boahen, Spiking neural network\ndecoder for brain-machine interfaces, IEEE Engineering in Medicine & Biology Society Conference on\nNeural Engineering, Cancun, Mexico, 2011, pp 396\u2013399.\n\n[8] K. Boahen, Neuromorphic microchips, Scienti\ufb01c American, vol. 292(5), 2005, pp 56\u201363.\n[9] R. Silver, K. Boahen, S. Grillner, N. Kopell, and K.L. Olsen, Neurotech for neuroscience: unifying con-\ncepts, organizing principles, and emerging tools, Journal of Neuroscience, vol. 27(44), 2007, pp 11807\u2013\n11819.\n\n[10] J.V. Arthur and K. Boahen, Silicon neuron design: the dynamical systems approach, IEEE Transactions\n\non Circuits and Systems, vol. 58(5), 2011, pp 1034-1043.\n\n[11] C. Eliasmith and C.H. Anderson, Neural engineering: computation, representation, and dynamics in\n\nneurobiological systems, MIT Press, Cambridge, MA; 2003.\n\n[12] C. Eliasmith, A uni\ufb01ed approach to building and controlling spiking attractor networks, Neural Compu-\n\ntation, vol. 17, 2005, pp 1276\u20131314.\n\n[13] R. Singh and C. Eliasmith, Higher-dimensional neurons explain the tuning and dynamics of working\n\nmemory cells, The Journal of Neuroscience, vol. 26(14), 2006, pp 3667\u20133678.\n\n[14] C. Eliasmith, How to build a brain: from function to implementation, Synthese, vol. 159(3), 2007, pp\n\n373\u2013388.\n\n[15] R.E. Kalman, A new approach to linear \ufb01ltering and prediction problems, Transactions of the ASME\u2013\n\nJournal of Basic Engineering, vol. 82(Series D), 1960, pp 35\u201345.\n\n[16] G. Welsh and G. Bishop, An introduction to the Kalman Filter, University of North Carolina at Chapel\n\nHill Chapel Hill NC, vol. 95(TR 95-041), 1995, pp 1\u201316.\n\n[17] W.Q. Malik, W. Truccolo, E.N. Brown, and L.R. Hochberg, Ef\ufb01cient decoding with steady-state Kalman\n\ufb01lter in neural interface systems, IEEE Transactions on Neural Systems and Rehabilitation Engineering,\nvol. 19(1), 2011, pp 25\u201334.\n\n9\n\n\f", "award": [], "sourceid": 1225, "authors": [{"given_name": "Julie", "family_name": "Dethier", "institution": null}, {"given_name": "Paul", "family_name": "Nuyujukian", "institution": null}, {"given_name": "Chris", "family_name": "Eliasmith", "institution": null}, {"given_name": "Terrence", "family_name": "Stewart", "institution": null}, {"given_name": "Shauki", "family_name": "Elasaad", "institution": null}, {"given_name": "Krishna", "family_name": "Shenoy", "institution": null}, {"given_name": "Kwabena", "family_name": "Boahen", "institution": null}]}