{"title": "Theory of localized synfire chain: characteristic propagation speed of stable spike pattern", "book": "Advances in Neural Information Processing Systems", "page_first": 553, "page_last": 560, "abstract": null, "full_text": " Theory of Localized Synfire Chain:\n Characteristic Propagation Speed of\n Stable Spike Patterns\n\n\n\n Kosuke Hamaguchi Masato Okada\n RIKEN Brain Science Institute Dept. of Complexity Science and\n Wako, Saitama 351-0198, JAPAN Engineering, University of Tokyo,\n hammer@brain.riken.jp Kashiwa, Chiba, 277-8561, JAPAN\n okada@brain.riken.jp\n\n\n Kazuyuki Aihara\n Institute of Industrial Science, University of Tokyo &\n ERATO Aihara Complexity Modeling Project JST\n Meguro, Tokyo 153-8505, JAPAN\n aihara@sat.t.u-tokyo.ac.jp\n\n\n\n\n Abstract\n\n\n Repeated spike patterns have often been taken as evidence for the synfire\n chain, a phenomenon that a stable spike synchrony propagates through\n a feedforward network. Inter-spike intervals which represent a repeated\n spike pattern are influenced by the propagation speed of a spike packet.\n However, the relation between the propagation speed and network struc-\n ture is not well understood. While it is apparent that the propagation\n speed depends on the excitatory synapse strength, it might also be related\n to spike patterns. We analyze a feedforward network with Mexican-Hat-\n type connectivity (FMH) using the Fokker-Planck equation. We show\n that both a uniform and a localized spike packet are stable in the FMH\n in a certain parameter region. We also demonstrate that the propagation\n speed depends on the distinct firing patterns in the same network.\n\n\n\n1 Introduction\n\nNeurons transmit information through spikes, but how the information is encoded remains a\nmatter of debate. The classical view is that the firing rates of neurons encode information,\nwhile a recent view is that spatio-temporal spike patterns encode the information. For\nexample, the synchrony of spikes observed in the cortex is thought to play functional roles\nin cognitive functions [1]. The mechanism of synchrony has been studied theoretically\nfor several neural network models. Especially, the model of spike synchrony propagation\nthrough a feedforward network is called the synfire chain [2].\n\nThe mechanism of generating synchrony in a feedforward network can be described as fol-\nlows. When feedforward connections are homogeneous with excitatory efficacy as a whole,\n\n\f\n inhibitory excitatory\n - +\n \n\n firing synaptic\n rate input\n n\n \n sitio r(' , t) (\n in , t) ....\n po\n\n\n\n\n - W( - ')\n Mexican-Hat-type Connectivity\n\n\nFigure 1: Network architecture. Each layer consists of N neuron arranged in a circle. Each\nneuron projects its axon to a post-synaptic layer with Mexican-Hat-type connectivity.\n\n\n\npost-synaptic neurons accept similar synaptic inputs. If neurons receive similar temporally\nmodulated inputs, the resultant spike timings will be also similar, or roughly synchronized\neven though the membrane potentials fluctuate because of noise [3]. The question in a feed-\nforward network is, whether the timing of spikes within a layer becomes more synchronized\nor not as the spike packet propagates through a sequence of neural layers. Detailed analyses\nof the activity propagation in feedforward networks have shown that homogeneous feedfor-\nward networks with excitatory synapses have a stable spike synchrony propagation mode\n[4, 5, 6]. Neurons, however, are embedded in more structured networks with excitatory and\ninhibitory synapses. Thus, such network structure would generate inhomogeneous inputs\nto neurons and whether spike synchrony is stable is not a trivial issue.\n\nOne simple way to detect the synfire chain phenomena would be to record from several\nneurons and find significant repeated patterns. If a spike packet propagates through a feed-\nforward network, a statistically significant number of spike pairs would be found that have\nfixed relative time lags (or inter-spike intervals (ISIs).) Such correlated activity has been\nexperimentally observed in the anterior forebrain pathway of songbirds [7], in the pre-\nfrontal cortex of primates [8], both in vivo and in vitro [9], and in an artificially constructed\nnetwork in vitro [10]. To generate fixed ISIs by spike packet propagation, the propaga-\ntion speed of a spike packet must be constant over several trials. The speed depends on\nspike patterns as well as the structure of the network. Conventional homogeneous feedfor-\nward networks have only one stable spike pattern, namely a spatially uniform synchronized\nactivity, but structured networks can generally produce spatially inhomogeneous spike pat-\nterns. In those networks, the relation between propagation speed and differences in the\nspike pattern is not well understood.\n\nIt is therefore an important problem to study a biologically realistic, structured network in\nthe context of the synfire chain. Among suggested network structures, Mexican-Hat-type\n(MH) connectivity is one of the most widely accepted as being representative of connectiv-\nity in the cortex [11]. Studies of a feedforward network with the MH connectivity (FMH)\nhave been reported [12]. In a FMH, a localized activity propagates through the network, and\nthis network is preferable to a homogeneous feedforward network because it can transmit\nanalog information regarding position [12], and both position and intensity [13]. However,\nno detailed analytical work on the structured synfire chain has been reported. In this paper,\nwe use the Fokker-Planck equation to analyze the FMH. The method of the Fokker-Planck\nequation enables us to analyze the collective behavior of the membrane potentials in an\n\n\f\nidentical neural population [14, 15]. When it is applied to the synfire chain [5], the detailed\nanalysis of the flow diagram, the effect of the membrane potential distribution on the spike\npacket evolution, and the interaction of spike packets is possible [5].\n\nThis paper thus examines the feedforward neural network model with Mexican-Hat-type\nconnectivity. Our strategy is, first, to describe the evolution of firing states through order\nparameters, which allows us to measure the macroscopic quantity of the network. Second,\nwe relate the input order parameters to the output ones through the Fokker-Planck equation.\nFinally, we analyze the evolution of spike packets with various shapes, and investigate\nstable firing patterns and their propagation speeds.\n\n\n2 Model\n\nWe analyze the dynamics of a structured feedforward network composed of identical sin-\ngle compartment, Leaky Integrate-and-Fire (LIF) neurons. Each neuron is aligned in a ring\nneural layer, and projects its axon to the next neural layer with the Mexican-Hat-type con-\nnectivity (Fig.1). The input to one neuron generally includes both outputs from pre-synaptic\nneurons and a random noisy synaptic current from ongoing activity. If we assume that the\nthousands of synaptic background inputs that connect to one neuron are independent Pois-\nsonian, instantaneous synapse, and have small amplitude of post synaptic potential (PSP),\nwe can approximate the sum of noisy background inputs as a Gaussian white noise fluc-\ntuating around the mean of the total noisy inputs. The membrane potential of a neuron at\nposition at time t, which receives many random Poisson excitatory and inhibitory inputs,\ncan be approximately described through a stochastic differential equation as follows:\n\n dv(, t) v(, t)\n C = I (, t) + ~\n + D (t), (1)\n dt in - R\n d\n I (, t) = W (\n in - )r( ,t), (2)\n - 2\n 0\n r(, t) = dt r(, t - t )(t ), (3)\n -\n W () = W0 + W1 cos(), (4)\n\nwhere C is membrane capacitance, R is membrane resistance, I (, t) is the synaptic\n in\ncurrent to a neuron at position , ~\n is proportional to the mean of the total noisy input, and\n(t) is a Gaussian random variable with (t) = 0 and (t)(t ) = (t -t ). Here, D is\nthe amplitude of Gaussian noise. The input current I (, t) is obtained from the weighted\n in\nsum of output currents r(, t) generated by pre-synaptic neurons. The synaptic current is\nderived from the convolution of its firing rate r(, t) with the PSP time course (t). Here,\n(t) = 2t exp(-t) where is chosen such that a single EPSP generates a 0.0014\nmV elevation from the resting potential. The Mexican-Hat-type connectivity consists of\na uniform term W0 and a spatial modulation term W1 cos(). We set the reset potential\nand threshold potential as V0 and Vth, respectively. We start simulations from stationary\ndistribution of membrane potentials. The input to the initial layer is formulated in terms of\nthe firing rate on the virtual pre-synaptic layer,\n\n r (t\n r(, t) = 0 + r1 cos()\n exp( - t)2), (5)\n 22 - 22\nwhere r0 and r1 are input parameters, and the temporal profile of activity is assumed to\nbe the Gaussian with = 1 and \n t = 10. Throughout this paper, parameter values are\ngiven as follows: C = 100 pF, R = 100 M, Vth = 15 mV, D = 100, ~\n = 0.075 pA,\nV0 = 0 mV, Vth = 15 mV, = 2, and = 0.00017. Space is divided into 50 regions\nfor the Fokker-Planck equation approach, and 10000 LIF neurons per layer are used in\nsimulations.\n\n\f\n A C\n\n 10 0.2 averaged membrane \n Vm 0 potential distribution\n P (v,t)\n -10 = 0 LIF\n \n 0 5 10 15 20 ,t)] 0.1 t=10.5 [ms]\n B time [ms]\n [P(v\n 10\n V PDE\n m 0\n V0 V\n -10 \n P (v,t=10.5) th\n 0\n - 0 -15 -10 0 10 15\n mV\n\n\n\nFigure 2: A: Dynamics of membrane potential distribution at = 0. B: A snapshot of\nthe membrane potential distributions for a range of [- ] at t = 10.5. C: A snapshot\nof the membrane potential distribution averaged with position . Results by a numerical\nsimulation (LIF) and the Fokker-Planck equation (PDE) are shown.\n\n\n\n3 Theory\n\nThe prerequisites for a full description of the network activity are time series of order\nparameters at an arbitrary time t defined as follows:\n\n 1\n r0(t) = d r(, t), r\n 2 1(t) = (rc(t))2 + (rs(t))2, (6)\n 1 1\n rc(t) = d cos()r(, t), r d sin()r(, t), (7)\n 2 s(t) = 2\n\nwhere r0(t) is the population firing rate of the neuron population, and rc(t) and rs(t) are\ncoefficients of the Fourier transformation of the spatial firing pattern which represent the\nspatial eccentricity of activity at time t. rc(t) and rs(t) depends on the position of the\nlocalized activity, but r1(t) does not. These order parameters play an important role in two\nways. First, we can describe the input to the next layer neuron at in a closed form of order\nparameters. Second, their time integrals, which will be introduced later, become indices of\nthe spike packet shape. Input currents are described with the order parameters as follows:\n\n I (, t) = W (t) + W\n in 0r\n 0 1 (r\n c (t) cos() + r\n s (t) sin()) , (8)\n 0\n r (t) = dt (t ) r\n {0,c,s} {0,c,s}(t - t ). (9)\n -\n\nGiven the time sequence of order parameters in the pre-synaptic layer, the order parameters\nin the post-synaptic layer are obtained through the following calculations. The analytical\nmethod we use here is the Fokker-Planck equation which describes the distribution of the\nmembrane potential of a pool of identical neurons as the probability density P(v, t) of\nvoltage v at time t. The suffix denotes that this neuron population is located at position .\nWe assume that there are a large number of neurons at position . Equation (1) is equivalent\nto the Fokker-Planck equation [16] within the limit of a large neuron number N ,\n \n P J\n t (v, t) = v (v, t) + (v - V0)J(Vth, t), (10)\n v I (, t) + ~\n \n J in\n (v, t) = + D P\n - C v (v, t), (11)\n\n\f\n 2\nwhere J D\n (v, t) is a probability flux and D = 1 . Boundary conditions are\n 2 C\n\n\n P(Vth, t) = 0, (12)\n\n r(, t) = J(V +, t) , t) = J\n 0 - J(V -0 (Vth, t). (13)\nEquation (12) is the absorbing boundary condition at the threshold potential, and Eq. (13) is\nthe current source at the reset potential. From Eq. (13), we obtain the firing rate of a post-\nsynaptic neuron r(, t). The Fokker-Planck equations are solved based on the modified\nChang-Cooper algorithm [17].\n\nFigure 2 shows the actual distribution of the initial layer's membrane potentials and their\ndynamics which accepts virtual layer activity with parameter r0 = 500 and r1 = 350.\nFigure 2A shows the evolution of the probability density P=0(v, t). From white to black,\nthe probability becomes higher. Figure 2B is a snapshot of the probability density at time\nt = 10.5 over the region from - to . As a result of a localized input injection, part\nof neuronal membrane potentials is strongly depolarized. The membrane potential distri-\nbution averaged over the neural layer is illustrated in Fig. 2C. It shows the consistency\nbetween the numerical simulations and the Fokker-Planck equations. The probability flow\ndropping out from the threshold potential Vth is a firing rate. Combined with these firing\nrates at each position and definitions of order parameters in Eqs. (6)-(7), the order pa-\nrameters on the post-synaptic neural layer are again calculated. The closed forms of order\nparameters have been obtained.\n\nSpatio-temporal patterns of firing rates and dynamics of order parameters in response to\na localized input (r0 = 600, r1 = 300) and a uniform input (r0 = 900, r1 = 0) are\nshown in Fig. 3. When an input is spatially localized, the spatio-temporal firing pattern\nis localized with a slightly distorted shape (Fig. 3A). On the other hand, when a uniform\ninput is applied, the spatio-temporal firing pattern is uniform as illustrated in Fig. 3B. We\nshow an example of the time course of r0(t) and r1(t) in Fig. 3C and 3D for both the\nnumerical simulation of 10, 000 LIF neurons and the Fokker-Planck equation. Elevation of\ntime course of r1(t) in Fig. 3C indicates the localized firing. In contrast, the uniform input\ngenerates no response in r1(t) parameter as illustrated in Fig. 3D.\n\nTo quantitatively evaluate the spike packet shape and propagation speed, we define indices\nr0, r1, and . r0 and r1 can be directly defined as\n\n r0 = dt r0(t) - spontaneous firing rate, r1 = dt r1(t). (14)\n\nr0 corresponds to the total population activity, and r1 corresponds to spatial eccentricity of\nthe activity. r0 and r1 are a natural extension of an index used in the study of the synfire\nchain [4] in the sense that an index corresponds to the area of a time varying parameters\nof the system, such as the population firing rate (r0(t)) above the spontaneous firing rate,\nor spatial eccentricity (r1(t)). The basic idea of characterizing the spike packet was to\napproximate the firing rate curve through a Gaussian function [4] as in Eq. (5). Here, the\napproximated Gaussian curve is obtained by minimizing the mean squared error with r0(t)\nand the Gaussian. We also use the index obtained from the variance of the Gaussian, and\n\nt as an index for the arrival time of the spike synchrony taken from the peak time of the\nGaussian (Fig. 3C).\n\n\n4 Results\n\nOur observation of the activities of the FMH with various parameter sets reveals two types\nof stable spike packets. Figure 4 shows the activity of the FMH with four characteristic\nparameter sets of W0 and W1. Here we use r0 = 500 and r1 = 350 for the upper figures\n\n\f\n A B\n firing rate firing rate\n \n\n\n\n 0 0\n\n\n - -\n 5 10 15 20 5 10 15 20\n time [ms] time [ms]\n\n C D\n 1000 Gaussian 1000\n FP r0(t) FP\n Approximation r0(t)\n LIF LIF\n 500 500\n - r\n [spk/s] 1(t) [spk/s]\n t r1(t)\n 0 0\n 5 10 15 20 5 10 15 20\n time [ms] time [ms]\n\n\n\nFigure 3: Activity profiles in response to a localized input (A,C) and a uniform input (B,D).\nA,B: Spatio-temporal pattern of the firing rates from neurons at position - to . C,D:\nTime courses of order parameters (r0(t), r1(t)) calculated from numerical simulations of\na population of LIF neurons (squares and crosses) and the Fokker-Planck equation (solid\nlines). The time course of order parameters in response to a single stimulus is approximated\nby using a Gaussian function. In C, Gaussian approximation of r0(t) is also shown. The\nvariance of the Gaussian and the mean value \n t are used as the indices of a spike packet.\n\n\n\nas a localized input and r0 = 900 and r1 = 0 for the lower ones as a uniform input. The\ncommon parameter is = 1 and \n t = 2. When both W0 and W1 are small, no spike packet\ncan propagate (Non-firing). When the uniform activation term W0 is sufficiently strong,\na uniform spike packet is stable (Uniform Activity). Note that even though the localized\ninput elicits localized spike packets with several layers, it finally decays to the uniform\nspike packet. When the Mexican-Hat term W1 is strong enough, only the localized spike\npacket is stable (Localized Activity). When W0 and W1 are balanced within a certain ratio,\nthere exists a novel firing mode where both the uniform and the localized spike packet are\nstable depending on the initial layer input (Multi-stable). The results show that there are\nfour types of phase and two types of spike packet in the FMH. The stability of a spike packet\ndepends on W0 and W1. In addition, the difference of the arrival times of propagating spike\npackets in the 8th layer shown in the Multi-stable phase indicates that the propagation speed\nof spike packets might differ.\n\nIt is apparent that the propagation speed depends on the strength of the excitatory synapse\nefficacy, however, our results in the Multi-stable phase in Fig. 4 suggest that a spike pattern\nalso determines the propagation speed. To investigate this effect, we plotted propagation\ntime \n t, the difference between propagation time \n tpost and \n tpre for various W1 (Fig. 5B).\nThe speed is analyzed after the spike packet indices r0, r1 and have converged. The\nconvergences of spike packets are shown in the flow diagram in Fig. 5A for (W0, W1) =\n(1, 1.5) case. In Fig. 5B, each triangle indicates the speed of the localized activity, and each\ncircle corresponds to that of the uniform activity. Within the plotted region (W1 = 1.4 \n2), both the uniform and localized activities are stable, and no bursting activity is observed.\nThis indicates that as W1 rises the propagation speed of localized activity becomes higher.\nIn contrast, the propagation speed of the uniform activity does not depends on W1 because\n\n\f\n Non-firing Uniform Activity Localized Activity Multi-stable phase \n (W0,W1) = (0.7, 1) (W0,W1) = (1, 0.6) (W0,W1) = (0.7, 2.5) (W0,W1) = (1, 1.4)\n 1\n Input 2\n ed 3\n iz 45\n cal 67\n yer Lo 8\n La t 12\n npu 34\n rm I 56\n ifon 7\n U 80 5 10 15 20 0 5 10 15 20 0 5 10 15 20 0 5 10 15 20\n\n time [ms]\n\n\nFigure 4: Four characteristic FMH with different stable states. The evolutions of firing rate\npropagation are shown. The upper row panels show the response to an identical localized\npulse input, and the lower row panels show the response to a uniform pulse input.\n\n\n\nuniform activity generates rc(t) = rs(t) = 0.\n\n\n5 Summary\n\nWe have found that there are four phases in the W0 - W1 parameter space; Non-firing,\nLocalized activity, Uniform activity, and Multi-stable phase. Multi-stable phase is the most\nintriguing in that an identical network has completely different firing modes in response to\ndifferent initial inputs. In this phase, the effect of spike pattern on the propagation speed\nof the spike packet can be directly studied. By the analysis of the Fokker-Planck equation,\nwe found that the propagation speed depends on the distinct firing patterns in the same\nnetwork. It implies that observation of repeated spike patterns requires an appropriately\ncontrolled input if the network structure produces a multi-stable state. The characteristic\nspeed of the spike packet also suggests that the speed of information processing in the brain\ndepends on the spiking pattern, or the representation of the information.\n\n\nAcknowledgment\n\nThis study is partially supported by the Advanced and Innovational Research Program in\nLife Sciences, a Grant-in-Aid No. 15016023 for Scientific Research on Priority Areas\n(C) Advanced Brain Science Project, a Grand-in-Aid No. 14084212, from the Ministry of\nEducation, Culture, Sports, Science, and Technology, the Japanese Government.\n\n\nReferences\n\n [1] C. M. Gray, P. Konig, A. K. Engel, and W. Singer, \"Oscillatory responses in cat visual cor-\n tex exhibit inter-columnar synchronization which reflects global stimulus properties,\" Nature,\n vol. 338, pp. 334337, 1989.\n\n [2] M. Abeles, Corticonics: neural circuits of the cerebral cortex. Cambridge UP, 1991.\n\n [3] Z. Mainen and T. Sejnowski, \"Reliability of spike timing in neocortical neurons,\" Science,\n vol. 268, pp. 15031506, 1995.\n\n [4] M. Diesmann, M.-O. Gewaltig, and A. Aertsen, \"Stable propagation of synchronous spiking in\n cortical neural networks,\" Nature, vol. 402, pp. 529533, 1999.\n\n\f\n w0 = 1\n A B\n\n 1.2 local\n 500\n s] uniform\n\n \n - t [m\n 300 1\n \n Localized \n r 1\n\n 100\n 0.8\n 0 Uniform\n 300\n 200\n / 100\n 200 600 1000 1.4 1.6 1.8 2\n 0 r0 w1\n\n\n\n\nFigure 5: A: Flow diagram with parameter (W0, W1) = (1, 1.5) where both localized and\nuniform spike packets are stable. Two attractors in high r1 region (Localized) and high r0\nwith r1 = 0 region (Uniform) are shown. A sequence of arrows indicates the evolution of\na spike packet in the (r0, /, r1) space. B: Plot of propagation time \n t that is necessary\ntime for a spike packet to propagate one neural layer. These results indicate that localized\nspike packets propagate slower than the uniform ones.\n\n\n\n [5] H. C^ateau and T. Fukai, \"Fokker-planck approach to the pulse packet propagation in synfire\n chain,\" Neural Networks, vol. 14, pp. 675685, 2001.\n\n [6] W. M. Kistler and W. Gerstner, \"Stable propagation of activity pulses in populations of spiking\n neurons,\" Neural Comp., vol. 14, pp. 987997, 2002.\n\n [7] R. Kimpo, F. Theunissen, and A. Doupe, \"Propagation of correlated activity through multiple\n stages of a neural circuit,\" J. Neurosci., vol. 23, no. 13, pp. 57505761, 2003.\n\n [8] M. Abeles, H. Bergman, E. Margalit, and E. Vaadia, \"Spatiotemporal firing patterns in the\n frontal cortex of behaving monkeys,\" J. Neurophysiol., vol. 70, pp. 16291638, 1993.\n\n [9] Y. Ikegaya, G. Aaron, R. Cossart, D. Aronov, I. Lampl, D. Ferster, and R. Yuste, \"Synfire\n chains and cortical songs: Temporal modules of cortical activity,\" Science, vol. 304, pp. 559\n 564, 2004.\n\n[10] A. Reyes, \"Synchrony-dependent propagation of firing rate in iteratively constructed networks\n in vitro,\" Nature Neuroscience, vol. 6, pp. 593 599, 2003.\n\n[11] D. H. Hubel and T. N. Wiesel, \"Receptive fields, binocular interaction and functional architec-\n ture in the cat's visual cortex,\" J. Physiol., vol. 160, pp. 106154, 1962.\n\n[12] M. C. W. van Rossum, G. G. Turrigiano, and S. B. Nelson, \"Fast propagation of firing rates\n through layered networks of noisy neurons,\" J. Neurosci., vol. 22, pp. 19561966, 2002.\n\n[13] K. Hamaguchi and K. Aihara, \"Quantitative information transfer through layers of spiking neu-\n rons connected by mexican-hat type connectivity,\" Neurocomputing, vol. 58-60, pp. 8590,\n 2004.\n\n[14] D. J. Amit and N. Brunel, \"Dynamics of a recurrent network of spiking neurons before and\n following learning,\" Network: Computation in Neural Systems, vol. 8, no. 4, pp. 373404,\n 1997.\n\n[15] N. Brunel and V. Hakim, \"Fast Global Oscillations in Networks of Integrate-and-Fire Neurons\n with Low Firing Rates,\" Neural Comp., vol. 11, no. 7, pp. 16211671, 1999.\n\n[16] H. Risken, The Fokker-Planck Equation. Springer-Verlag, 1996.\n\n[17] J. S. Chang and G. Cooper, \"A practical difference scheme for fokker-planck equations,\" J.\n Comp. Phys., vol. 6, pp. 116, 1970.\n\n\f\n", "award": [], "sourceid": 2697, "authors": [{"given_name": "Kosuke", "family_name": "Hamaguchi", "institution": null}, {"given_name": "Masato", "family_name": "Okada", "institution": null}, {"given_name": "Kazuyuki", "family_name": "Aihara", "institution": null}]}