{"title": "A Spike Based Learning Neuron in Analog VLSI", "book": "Advances in Neural Information Processing Systems", "page_first": 692, "page_last": 698, "abstract": null, "full_text": "A spike based learning neuron in analog \n\nVLSI \n\nPhilipp Hiifliger \n\nMisha Mahowald \n\nInstitute of Neuroinformatics \n\nInstitute of Neuroinformatics \n\nETHZjUNIZ \n\nGloriastrasse 32 \nCH-8006 Zurich \n\nSwitzerland \n\nETHZjUNIZ \n\nGloriastrasse 32 \nCH-8006 Zurich \n\nSwitzerland \n\ne-mail: haftiger@neuroinf.ethz.ch \n\ne-mail: misha@neuroinf.ethz.ch \n\ntel: ++41 1 257 26 84 \n\ntel: ++41 1 257 26 84 \n\nLloyd Watts \nArithmos, Inc. \n\n2730 San Tomas Expressway, Suite 210 \n\nSanta Clara, CA 95051-0952 \n\nUSA \n\ne-mail: lloyd@arithmos.com \n\ntel: 408 982 4490, x219 \n\nAbstract \n\nMany popular learning rules are formulated in terms of continu(cid:173)\nous, analog inputs and outputs. Biological systems, however, use \naction potentials, which are digital-amplitude events that encode \nanalog information in the inter-event interval. Action-potential \nrepresentations are now being used to advantage in neuromorphic \nVLSI systems as well. We report on a simple learning rule, based \non the Riccati equation described by Kohonen [1], modified for \naction-potential neuronal outputs. We demonstrate this learning \nrule in an analog VLSI chip that uses volatile capacitive storage for \nsynaptic weights. We show that our time-dependent learning rule \nis sufficient to achieve approximate weight normalization and can \ndetect temporal correlations in spike trains. \n\n\fA Spike Based Learning Neuron in Analog VLSI \n\n693 \n\n1 \n\nINTRODUCTION \n\nIt is an ongoing debate how information in the nervous system is encoded and carried \nbetween neurons. In many subsystems of the brain it is now believed that it is done \nby the exact timing of spikes. Furthermore spike signals on VLSI chips allow the \nuse of address-event busses to solve the problem of the large connectivity in neural \nnetworks [3, 4]. For these reasons our artificial neuron and others [2] use spike signals \nto communicate. Additionally the weight updates at the synapses are determined \nby the relative timing of presynaptic and postsynaptic spikes, a mechanism that \nhas recently been discovered to operate in cortical synapses [5, 7, 6]. \n\nWeight normalization is a useful property of learning rules. In order to perform the \nnormalization, some information about the whole weight vector must be available \nat every synapse. We use the neuron's output spikes (The neuron's output is the \nproduct of the weight and the input vector), which retrogradely propagate through \nthe dendrites to the synapses (as has been observed in biological neurons [5]). In \nour model approximate normalization is an implicit property of the learning rule. \n\n2 THE LEARNING RULE \n\npresynaptic spikes \n\n7 \n\n.~ ____ L-~ ____ ~ ______ ~_ \n\ncorrelation signat \n\n3 ynaptic weight \n\npostsynaptic spikes \n\nFigure 1: A snapshot of the simulation variables involved at one synapse. With \nr = 0.838 \n\nThe core of the learning rule is a local 'correlation signal' c at every synapse. It \nrecords the 'history' of presynaptic spikes. It is incremented by 1 with every presy(cid:173)\nnaptic spike and decays in time with time constant r: \n\nc(tm,O) = 0 \n\nc(tm,n) = e-\n\nttn,n- t 171.,n-l \n\nT \n\nc(tm,n-d + 1 \n\nn>O \n\n(1) \n\n(n > 0) is the time of the \ntm,o is the time of the m'th postsynaptic spike and tm,n \nn'th presynaptic spike after the m'th postsynaptic spike. The weight changes when \nthe cell fires an action potential: \n\n\f694 \n\nP. Hiijliger, M. Mahowald and L Watts \n\nw(tm,o) = W(tm-l,O) + ae-\n\ntTn o-tTTl-l \u2022 \n\n. \n\nT \n\n\u2022 c(tm-1,s) - ,8w(tm- 1,o) \n\ns = max{v : tm-l ,v :s; tm,o} \n\n(2) \n\nwhere w is the weight at this synapse. tm-l,s means the last event (presynaptic \nor postsynaptic spike) before the m'th postsynaptic spike. a and ,8 are parameters \ninfluencing learning speed and weight vector normalization (see (5)). \n\nOur learning rule is designed to react to temporal correlations between spikes in \nthe input signals. However, to show the normalizing of the weights we analyze its \nbehavior by making some simplifying assumptions on the input and output signals; \ne.g. the intervals of the presynaptic and the postsynaptic spike train are Poisson \ndistributed and there is no correlation between single spikes. Therefore we can \nrepresent the signals by their instantaneous average frequencies 0 and i. Now the \nsimplified learning rule can be written as: \n\n:t w = al(O)f - ,8wO \n\n(3) \n\n(4) \n\nl(O) represents the average percentage to which the correlation signal is reduced \nbetween weight updates (output spikes). So when the neuron's average firing rate \nfulfills 0 \u00bb ~, one can approximate l(O) ~ 1. (3) is thus reduced to the Riccati \nequation described by Kohonen [1]. This rule would not be Hebbian, but normalizes \nthe weight vector (see (5)). Note that if the correlation signal does not decay, then \nour rule matches exactly the Riccati equation. We will further refer to it as the \nModified Riccati Rule (MRR). Whereas if 0 \u00ab ~ then l(O) ~ Or, which is a \nHebbian learning rule also described in [1]. \nIf we assume that the spiking mechanism preserves 0 = wT f and insert it in (3), it \nfollows for the equilibrium state: \n\nIIwll = Jl(O)~ \n\n(5) \n\nSince l(O) < 1 the weight vector will never be longer than Ii' This property also \n\nholds when the simplifying assumptions are removed. The vector will always be \nsmaller, as it is with no decay of the correlation signals, since the decay only affects \nthe incrementing part of the rule. \n\nMatters get much more complicated with the removal of the assumption of the \npre- and postsynaptic trains being independently Poisson distributed. With an \nintegrate-and-fire neuron for instance, or if there exist correlations between spikes \nof the input trains, it is no longer possible to express what happens in terms of rate \n\n\fA Spike Based Learning Neuron in Analog VLSI \n\n695 \n\ncoding only (with f and 0). (3) is still valid as an approximation but temporal \nrelationships between pre- and postsynaptic spikes become important. Presynaptic \nspikes immediately followed by an action potential will have the strongest increasing \neffect on the synapse's weight. \n\n3 \n\nIMPLEMENTATION IN ANALOG VLSI \n\nWe have implemented a learning rule in a neuron circuit fabricated in a 2.0f..Lm \nCMOS process. This neuron is a preliminary design that conforms only approxi(cid:173)\nmately to the MRR. The neuron uses an integrate-and-fire mechanism to generate \naction potentials (Figure 2). \n\nFigure 2: Integrate-and-fire neuron. The sarna capacitor holds the somatic mem(cid:173)\nbrane voltage. This voltage is compared to a threshold thresh with a differential \npair. When it crosses this threshold it gets pulled up through the mirrored current \nfrom the differential pair. This same current gets also mirrored to the right and \nstarts to pull up a second leaky capacitor (setback) through a small W / L transistor, \nso this voltage rises slowly. This capacitor voltage finally opens a transistor that \npulls sarna back to ground where it restarts integrating the incoming current. The \nparameters tonic+ and tonic- are used to add or subtract a constant current to \nthe soma capacitor. tre! allows the spike-width to be changed. \n\nNot shown, but also part of the neuron, are two non-learning synapses: one ex(cid:173)\ncitatory and one inhibitory. Each of three learning synapses contains a storage \ncapacitor for the synaptic weight and for the correlation signal (Figure 3). \n\nThe correlation signal c is simplified to a binary variable in this implementation. \nWhen an input spike occurs, the correlation signal is set to 1. It is set to 0 whenever \nthe neuron produces an output-spike or after a fixed time-period (T in (7)) if there \nis no other input spike: \n\nc(tm,o) = 0 \nc(tm,n) = 1 \n\nn > 0 \n\n, \n\ntm,n::; tm+1,O \n\n(6) \n\nThis approximation unfortunately tends to eliminate differences between highly \nactive inputs and weaker inputs. Nevertheless the weight changes with every output \nspike: \n\n\f696 \n\nP. Hiijiiger, M. Mahowald and L. Watts \n\nFigure 3: The CMOS learning-synapse incorporates the learning mechanism. The \nweight capacitor holds the weight, the carr capacitor stores the correlation signal \nrepresentation. The magnitude of the weight increment and decrement are com(cid:173)\nputed by a differential pair (upper left w50). These currents are mirrored to the \nsynaptic weight and gated by digital switches encoding the state of the correlation \nsignal and of the somatic action potential. The correlation signal reset is medi(cid:173)\nated by a leakage transistor, decayin, which has a tonic value, but is increased \ndramatically when the output neuron fires. \n\nif C{tm-l s) = 1 and tm 0 -\n, \notherwise \n\n\" \n\ntm-l s < T \n\n(7) \n\nw is the weight on one synapse, c is the correlation signal of that synapse, and a is \na parameter that controls how fast the weight changes. (See in the previous section \nfor a description of tm,n.) The weight, W50, is the equilibrium value of the synaptic \nweight when the occurrence of an input spike is fifty percent correlated with the \noccurrence of an output spike. This implementation differs from the Riccati rule in \nthat either the weight increment or the weight decrement, but not both, are executed \nupon each output spike. Also, the weight increment is a function of the synaptic \nweight. The circuit was implemented this way to try and achieve an equilibrium \nvalue for the synaptic weight equal to the fraction of the time that the input neuron \nfired relative to the times the output neuron fired. This is the correct equilibrium \nvalue for the synaptic weight in the Riccati rule. The evolution of a synaptic weight \nis depicted in Figure 4. \n\nThe synaptic weight vector normalization in this implementation is accurate only \nwhen the assumptions of the design are met. These assumptions are that there is \none or fewer input spikes per synapse for every output spike. This assumption is \neasier to meet when there are many synapses formed with the neuron, so that spikes \nfrom multiple inputs combine to drive the cell to threshold. Since we have only \nthree synapses, this approximation is usually violated. Nevertheless, the weights \ncompete with one another and therefore the length of the weight vector is limited. \nCompetition between synaptic weights occurs because if one weight is stronger, it \ncauses the output neuron to spike and this suppresses the other input that has not \n\n\fA Spike Based Learning Neuron in Analog VLSI \n\n697 \n\nfired. Future revision of the chip will conform more closely to the MRR. \n\n20 \n\n18 \n\n8.S \n\n9 \n\ntlma \n\n9.S \n\n10 \n\nX 10-3 \n\nFigure 4: A snapshot of the learning behavior of a single VLSI synapse: The top \ntrace is the neuron output (IV/division), the upper middle trace is the synap(cid:173)\ntic weight (lower voltage means a stronger synaptic weight) (25mV /division), the \nlower middle trace is a representation of the correlation signal (1 V /division)(it has \ninverted sense too) and the bottom trace is the presynaptic activity (1 V / division) . \nThe weight changes only when an output spike occurs. The timeout of the correla(cid:173)\ntion signal is realized with a decay and a threshold. If the correlation signal is above \nthreshold, the weight is strengthened. If the signal has decayed below threshold at \nthe time of an output spike, the weight is weakened. The magnitude of the change \nof the weight is a function of the absolute magnitude of the weight. This weight \nwas weaker than W50, so the increments are bigger than the decrements. \n\n4 TEMPORAL CORRELATION IN INPUT SPIKE \n\nTRAINS \n\nFigure 5 illustrates the ability of our learning rule to detect temporal correlations in \nspike trains. A simulated neuron strengthens those two synapses that receive 40% \ncoincident spikes, although all four synapses get the same average spike frequencies. \n\n5 DISCUSSION \n\nLearning rules that make use of temporal correlations in their spike inputs/outputs \nprovide biologically relevant mechanisms of synapse modification [5, 7, 6]. Analog \nVLSI implementations allow such models to operate in real time. We plan to develop \nsuch analog VLSI neurons using floating gates for weight storage and an address(cid:173)\nevent bus for interneuronal connections. These could then be used in realtime \napplications in adaptive 'neuromorphic' systems. \n\nAcknowledgments \n\nWe thank the following organizations for their support: SPP Neuroinformatik des \nSchweizerischen Nationalfonds, Centre Swiss d'Electronique et de Microtechnique, \nU.S. Office of Naval Research and the Gatsby Charitable Foundation. \n\n\f698 \n\nP. Hiijiiger, M. Mahowald and L. Watts \n\nsynapse 2 \n\n0.4 \n\n0.36 \n\n0 .3 \n\n015 \n\n0.1 \n\nO\u00b7060l..----:-O:,00--='200\":----,300~~400.,----='\"=\"--\":-----=-'700 \n\ntimers) \n\nFigure 5: In this simulation we use a neuron with four synapses. All of them get \ninput trains of the same average frequency (20Hz). Two of those input trains are \nthe result of independent Poisson processes (synapses 3 and 4), the other two are the \ncombination oftwo Poisson processes (synapses 1 and 2): One that is independent of \nany other (12Hz) and one that is shared by the two with slightly different time delays \n(8Hz): Synapse 1 gets those coincident spikes 0.01 seconds earlier than synapse 2. \nSynapse 2 gets stronger because when it together with synapse 1 triggered an action \npotential, it was the last synapse being active before the postsynaptic spike. The \nparameters were: Q = 0.004, {3 = 0.02, T = 11ms \n\nReferences \n\n[1] Thevo Kohonen. Self-Organization and Associative Memory. Springer, Berlin, \n\n1984. \n\n[2] D.K. Ferry L.A. Akers and R.O. Grondin. Synthetic neural systems in the 1990s. \nAn introduction to neural and electronic networks, Academic Press (Zernetzer, \nDavis, Lau, McKenna), pages 359-387, 1995. \n\n[3] J. Lazzaro, J. Wawrzynek, M. Mahowald, M. Sivilotti, and D. Gillespie. Silicon \nauditory processors as computer peripherals. IEEE Trans. Neural Networks, \n4:523-528, 1993. \n\n[4] A. Mortara and E. A. Vittoz. A communication architecture tailored for analog \nVLSI artificial neural networks: intrinsic performance and limitations. IEEE \nTranslation on Neural Networks, 5:459-466, 1994. \n\n[5] G. J. Stuart and B. Sakmann. Active propagation of somatic action potentials \n\ninto neocortical pyramidal cell dendrites. Nature, 367:600, 1994. \n\n[6] M. V. Tsodyks and H. Markram. Redistribution of synaptic efficacy between \n\nneocortical pyramidal neurons. Nature, 382:807-810, 1996. \n\n[7] R. Yuste and W. Denk. Dendritic spines as basic functional units of neuronal \n\nintegration. Nature, 375:682-684, 1995. \n\n\f", "award": [], "sourceid": 1322, "authors": [{"given_name": "Philipp", "family_name": "H\u00e4fliger", "institution": null}, {"given_name": "Misha", "family_name": "Mahowald", "institution": null}, {"given_name": "Lloyd", "family_name": "Watts", "institution": null}]}