{"title": "Burst Synchronization without Frequency Locking in a Completely Solvable Neural Network Model", "book": "Advances in Neural Information Processing Systems", "page_first": 117, "page_last": 124, "abstract": "", "full_text": "Burst Synchronization Without \n\nFrequency-Locking in a Completely Solvable \n\nNetwork Model \n\nHeinz Schuster \n\nChristof Koch \n\nInstitut fur theoretische Physik \n\nComputation and Neural System Program \n\nUniversitat Kiel \n\nOlshausenstraBe 40 \n\n2300 Kiel 1, Germany \n\nCalifornia Institute of Technology \nPasadena, California 91125, USA \n\nAbstract \n\nThe dynamic behavior of a network model consisting of all-to-all excitatory \ncoupled binary neurons with global inhibition is studied analytically and \nnumerically. We prove that for random input signals, the output of the \nnetwork consists of synchronized bursts with apparently random intermis(cid:173)\nsions of noisy activity. Our results suggest that synchronous bursts can be \ngenerated by a simple neuronal architecture which amplifies incoming coin(cid:173)\ncident signals. This synchronization process is accompanied by dampened \noscillations which, by themselves, however, do not play any constructive \nrole in this and can therefore be considered to be an epiphenomenon. \n\n1 \n\nINTRODUCTION \n\n(1989, 1990) as well as Eckhorn et al. \n\nRecently synchronization phenomena in neural networks have attracted considerable \n(1988) provided \nattention. Gray et al. \nelectrophysiological evidence that neurons in the visual cortex of cats discharge in a \nsemi-synchronous, oscillatory manner in the 40 Hz range and that the firing activity \nof neurons up to 10 mm away is phase-locked with a mean phase-shift of less than \n3 msec. It has been proposed that this phase synchronization can solve the binding \nproblem for figure-ground segregation (von der Malsburg and Schneider, 1986) and \nunderly visual attention and awareness (Crick and Koch, 1990). \nA number of theoretical explanations based on coupled (relaxation) oscillator mod-\n117 \n\n\f118 \n\nSchuster and Koch \n\nels have been proposed for burst synchronization (Sompolinsky et al., 1990). The \ncrucial issue of phase synchronization has also recently been addressed by Bush and \nDouglas (1991), who simulated the dynamics of a network consisting of bursty, layer \nV pyramidal cells coupled to a common pool of basket cells inhibiting all pyramidal \ncells. 1 Bush and Douglas found that excitatory interactions between the pyramidal \ncells increases the total neural activity as expected and that global inhibition leads \nto synchronized bursts with random intermissions. These population bursts appear \nto occur in a random manner in their model. The basic mechanism for the observed \nburst synchronization is hidden in the numerous anatomical and biophysical details \nof their model. These, and the related observation that to date no strong oscilla(cid:173)\ntions have been recorded in the neuronal activity in visual cortex of awake monkeys, \nprompted us to investigate how phase synchronization can occur in the absence of \nfrequency locking. \n\n2 A COINCIDENCE NETWORK \n\nWe consider n excitatory coupled binary McCulloch-Pitts (1943) neurons whose \noutput x:+l E [0,1] at time t + 1 is given by: \n[w ~ t \n~t \n;; L: Xi + '-i -\n\nt+l \nXi = U \n\n(1) \n\n(}J \n\nHere win > 1 is the normalized excitatory all-to-all synaptic coupling, e: represents \nthe external binary input and u[zJ is the Heaviside step function, such that u[z] = 1 \nfor z > 0 and 0 elsewhere. Each neuron has the same dynamic threshold () > O. \nNext we introduce the fraction mt of neurons which fire simultaneously at time t: \n\nmt = - ~ x~ \n1 \nn LJ ' \n\ni \n\n(2) \n\nIn general, 0 < mt < 1; only if every neuron is active at time t do we have mt = l. \nBy summing eq. (1) we obtain the following equation of motion for our simple \nnetwork. \n\nmt+l = - L u[wmt + e: - (}J \n\n1 \nn \n\n. , \n\n(3) \n\nThe behavior of this (n+l)-state automata is fully described by the phase-state \ndiagram of Figure 1. If () > 1 and (}/w > 1, the output of the network mt will \nvary with the input until at some time t', me = O. Since the threshold () is always \nlarger than the input, the network will remain in this state for all subsequent times. \nIf () < 1 and () /w < 1, the network will drift until it comes to the state mt' = l. \nSince subsequent wmt is at all times larger than the threshold, the network remains \nlatched at mt = 1. If () > 1, but (}/w < 1, the network can latch in either the \nmt = 0 or the mt = 1 state and will remain there indefinitely. Lastly, if () < 1, but \n() /w > 1, the threshold is by itself not large enough to keep the network latched \n\n1 This model bears similarities to Wilson and Bower's {1992} model describing the origin \n\nof phase-locking in olfactory cortex. \n\n\fBurst Synchronization without Frequency Locking in a Completely Solvable Network Model \n\n119 \n\no \n\nTh:\"esho Id 8 \n\nFigure 1: Phase diagram for the network described by eq. (3). Different regions \ncorrespond to different stationary output states mt in the long time limit. \n\ninto the mt = 1 state. Defining the normalized input activity as \n\nst = .!. \"e \nn~' , \n\n(4) \n\nwith 0 < st < 1, we see that in this part of phase space mt+1 = st, and the output \nactivity faithfully reflects the input activity at the previous time step. \nLet us introduce an adaptive time-dependent threshold, ot. We assume that ot \nremains at its value 0 < 1 as long as the total activity remains less than 1. If, \nhowever, mt = 1, we increase ot to a value larger than w + 1. This has the effect of \nresetting the activity of the entire network to 0 in the next time step, i.e., mt+l = \n(lin) L:i u(w + ei - (w + 1 + f)) = O. The threshold will then automatically reset \nitself to its old value: \n\nmt+l = ;; ~ u[wmt + e: - O(mt)] \n\n1 \n\n, \n\n(5) \n\nwith \n\nfor mt < 1 \nfor mt = 1 \n\nTherefore, we are operating in the topmost left part of Fig. 1 but preventing the \nnetwork from latching to mt = 1 by resetting it. Such a dynamic threshold bears \nsome similarities to the models of Horn and Usher (1990) and others, but is much \nsimpler. Note that O(mt) exactly mimics the effect of a common inhibitory neuron \nwhich is only excited if all neurons fire simultaneously. \nOur network now acts as a coincidence detector, such that all neurons will \"fire\" \nat time t + 2, i.e., ml+2 = 1 if at least k neurons receive at time t a \"I\" as input. \nk is the smallest integer with k > 0 . nlw. The threshold O(mt) is then transiently \nincreased and the network is reset and the game begins anew. In other words, the \n\n\f120 \n\nSchuster and Koch \n\n.,. \nOutput \n\nInput \n\nS \n\n1 \n\n0.8 \n\n0.6 \n\nD. \" \n\n1 \n\n0.8 \n\n0. 6 \n\nD.\" \no. \n\n100 \nTl.,.. \n\nTi.,.. \n\nFigure 2: Time dependence of the fraction mt of output neurons which fire simul(cid:173)\ntaneously compared to the corresponding fraction of input signals st for n = 20 \nand () /w = 0.225. The input variables e! are independently distributed according to \npeen = p8(e! - 1) + (1 - p)8(en with p = 0.1. If more than five input signals with \ne: = 1 coincide, the entire population will fire in synchrony two time steps later, \ni.e. mt+2 = 1. Note the \"random\" appearance of the interburst intervals. \n\nnetwork detects coincidences and signals this by a synchronized burst of neuronal \nactivity followed by a brief respite of activity (Figure 2). \nThe time dependence of mt given by eq. (5) can be written as: \n\nt 1 \n\nm + = \n{ \n\nfor 0 < mt < 1-\nw \nfor ~ =:; m t < 1 \n\nst \n1 \no for mt = 1 \n\n-\n\n(6) \n\nBy introducing functions A(m), B(m), C(m) which take on the value 1 in the \nintervals specified for m = mt in eq. (6), respectively, and zero elsewhere, m Hl can \nbe written as: \n\n(7) \nThis equation can be iterated, yielding an explicit expression for mt as a function \nof the external inputs sHl, ... sO and the initial value mO: \n\nmt+l = st A(mt) + 1 . B(mt) + O. C(mt) \n\nwith the matrix \n\nA(s) 0 1) \nM(s) = B(s) 0 0 \nC(s) 1 0 \n\n( \n\n(8) \n\n\fBurst Synchronization without Frequency Locking in a Completely Solvable Network Model \n\n121 \n\nEq. (8) shows that the dynamics of the network can be solved explicitly, by itera(cid:173)\ntively applying M, t - 1 number of times to the initial network configuration. \n\n3 DISTRIBUTION OF BURSTS AND TIME \n\nCORRELATIONS \n\nThe synchronous activity at time t depends on the specific realization of the input \nsignals at different times (eq. 8). In order to get rid of this ambiguity we resort to \naveraged quantities where averages are understood over the distribution P{ st} of \n\ninputs Sf = ~ L~=l e:. A very useful averaged quantity is the probability pt(m), \n\ndescribing the fraction m of simultaneously firing neurons at time t. pt(m) is related \nto the probability distribution P{ st} via: \n\npt(m) = (6[m - mt{ st-1, ... sO}]) \n\n(9) \nwhere ( ... ) denotes the average with respect to P{st} and mt{st-l, .. . sO} is given \nby eq. (8). If the input signals e: are un correlated in time, mt+l depends according \nto eq. (7) only on mt, and the time evolution of pt(m) can be described by the \nChapman-Kolmogorov equation. We then find: \n\npt(m) = pOO(m) + [pO(m) - pOO(m)]. /(t) \n\n(10) \n\n(11) \n\nwhere \n\npOO(m) = 1 + 217 [P(m) + 176(m - 1) + 176(m)] \n\n~ \n\n1 \n\nis the limiting distribution which evolves from the initial distribution pO( m) for large \ntimes, because the factor /(t) = 17~ cos(Ot) , where 0 = 7r - arctan[J417 -172 /17]' \ndecays exponentially with time and 17 = f01 P(s)B(s)ds = f(},w P(s)ds. Notice that \no < 17 < 1 holds (for more details, see Koch and Schuster, 1992). \nThe limiting equilibrium distribution poo (m) evolves from the initial distribution \npO( m) in an oscillatory fashion, with the building up of two delta-functions at m = 1 \nand m = 0 at the expense of pO(m). This signals the emergence of synchronous \nbursts, i.e., mt = 1, which are always followed at the next time-step by zero activity, \ni.e., mt+l = 0 (see also Fig. 2). The equilibrium value for the mean fraction of \nsynchronized neurons is \n\nwhich is larger than the initial value (s) = f;dsP(s)s, for (s) <~, indicating an \nincrease in synchronized bursting activity. \n\n(12) \n\nIt is interesting to ask what type of time correlations will develop in the output \n\nof our network if it is stimulated with uncorrelated noise, e:. The autocovariance \n\nfunction is \n\ncan be computed directly since mt and pOO(m) are known explicitly. We find \n\nC( r) = 6T ,oCO + (1 - 6T,0)Cl17ITI/2 cos(Or + cp) \n\n(14) \n\n\f122 \n\nSchuster and Koch \n\n((1:) \n\nC( 1:) \n\n1 \nO. 75 \n0. 6 \n0.25 \n\n-0.25 \n-0. 5 \n-0.75 \n-1 \n\n1 \n0.75 \n0.5 \n0. 26 \n\n-0.25 \n-0.5 \n-0.75 \n-1 \n\n0 35 \n\nto \n\n3 \n\nt \n\n5 \n\n6 \n\n7 \n\n8 \n\nlUTe \n\nFigure 3: Time dependence of the auto-covariance function G( r) for two different \nvalues of.,., = fe/w dsP(s). The top figure corresponds to .,., = 0.8 and a period \nT = 3.09, while the bottom correlation function is for.,., = 0.2 with an associated \nT = 3.50. Note the different time-scales. \n\n1 \n\nA \n\nwith br,o the Kroneker symbol (br,o = 1 for r = 0 and 0 else). Figure 3 shows \nthat G(r) consists of two parts. A delta peak at r = 0 which reflects random \nuncorrelated bursting and an oscillatory decaying part which indicates correlations \nin the output. The period of the oscillations, T = 21r/rl, varies monotonically \nbetween 3 < T < 4 as O/w moves from zero to one. Since.,., is given by fe1/w P(s)ds, \nwe see that the strengths of these oscillations increases as the excitatory coupling \nw increases. The emergence of periodic correlations can be understood in the limit \nO/w --+ 0, where the period T becomes three (and.,., = fo P(s)ds = 1), because \naccording to eq. (6), mt = 0 is followed by mt+l = st which leads for O/w --+ 0 \nalways to mt+2 = 1 followed by mt+3 = O. In other words, the temporal dynamics \nof mt has the form OsI10s41Os7 1Os 1olO.... In the opposite case of O/w --+ 1, .,., \nconverges to 0 and the autocovariance function G( r) essentially only contains the \npeak at r = O. Thus, the output of the network ranges from completely uncorrelated \nnoise for O/w ::::: 1 to correlated periodic bursts for ~ --+ O. The power spectrum of \nthe system is a broad Lorentzian centered at the oscillation frequency, superimposed \nonto a constant background corresponding to uncorrelated neural activity. \n\n1 \n\nA \n\nIt is important to discuss in this context the effect of the size n of the network. If the \n\ninput variables e: are distributed independently in time and space with probabilities \nPi(eD, then the distribution pes) has a width which decreases as l/fo as n --+ 00. \nTherefore, in a large system.,., = fe1/w pes )ds is either 0 if O/w > (s) or 1 if O/w < (s), \nwhere (s) is the mean value of s, which coincides for n --+ 00 with the maximum of \n\n\fBurst Synchronization without Frequency Locking in a Completely Solvable Network Model \n\n123 \n\nreported by Sompolinsky et al. (1989). \n\n/>(8). If.,., = 0 the correlation function is a constant according to eq. (14), while the \nsystem will exhibit undamped oscillations with period 3 for .,., = 1. Therefore, the \nirregularity ofthe burst intervals, as shown, for instance, in Fig. 2, is for independent \ne: a finite size effect. Such synchronized dephasing due to finite size has been \nHowever, for biologically realistic correlated inputs e: , the width of />(8) can remain \nfinite for n ~ 1. For example, if the inputs er, ... , e~ can be grouped into q \ncorrelated sets eI ... eL e~ ... e~, ... ,e: ... e:, with finite q, then the width of />(8) \nscales like 1/ vq. Our model, which now effectively corresponds to a situation with \na finite number q of inputs, leads in this case to irregular bursts which mirror and \namplify the correlations present in the input signals, with an oscillatory component \nsuperimposed due to the dynamical threshold. \n\n4 CONCLUSIONS AND DISCUSSION \n\nWe here suggest a mechanism for burst synchronization which is based on the fact \nthat excitatory coupled neurons fire in synchrony whenever a sufficient number of in(cid:173)\nput signals coincide. In our model, common inhibition shuts down the activity after \neach burst, making the whole process repeatable, without entraining any signals. It \nis rather satisfactory to us that our simple model shows qualitative similar dynamic \nbehavior of the much more detailed biophysical simulations of Bush and Douglas \n(1991). In both models, all-to-all excitatory coupling leads-together with common \ninhibition-to burst synchronization without frequency locking. In our analysis we \nupdated all neurons in parallel. The same model has been investigated numerically \nfor serial (asynchronous) updating, leading to qualitatively similar results. \nThe output of our network develops oscillatory correlations whose range and am(cid:173)\nplitude increases as the excitatory coupling is strengthened. However, these oscil(cid:173)\nlations do not depend on the presence of any neuronal oscillators, as in our earlier \nmodels (e.g., Schuster and Wagner, 1990; Niebur et al.,1991). The period of the \noscillations reflects essentially the delay between the inhibitory response and the \nexcitatory stimulus and only varies little with the amplitude of the excitatory cou(cid:173)\npling and the threshold. The crucial role of inhibitory interneurons in controlling \nthe 40 Hz neuronal oscillations has been emphasized by Wilson and Bower (1992) \nin their simulations of olfactory and visual cortex. Our model shows complete syn(cid:173)\nchronization, in the sense that all neurons fire at the same time. This suggests \nthat the occurrence of tightly synchronized firing activity across neurons is more \nimportant for feature linking and binding than the locking of oscillatory frequencies. \nSince the specific statistics of the input noise is, via coincidence detection, mirrored \nin the burst statistics, we speculate that our network-acting as an amplifier for \nthe input noise-can play an important role in any mechanism for feature linking \nthat exploits common noise correlations of different input signals. \n\nAcknowledgements \n\nWe thank R. Douglas for stimulating discussions and for inspiring us to think about \nthis problem. Our collaboration was supported by the Stiftung Volkswagenwerk. \nThe research of C.K. is supported by the National Science Foundation, the James \n\n\f124 \n\nSchuster and Koch \n\nMcDonnell Foundation, and the Air Force Office of Scientific Research. \n\nReferences \n\nBush, P.C. and Douglas, R.J. \"Synchronization of bursting action potential dis(cid:173)\ncharge in a model network of neocortical neurons.\" Neural Computation 3: 19-30, \n1991. \n\nCrick, F. and Koch, C. ''Towards a neurobiological theory of consciousness.\" Sem(cid:173)\ninars Neurosci. 2: 263-275, 1990. \n\nEckhorn, R., Bauer, R., Jordan, W., Brosch, M., Kruse, W., Munk, M. and Re(cid:173)\nitboeck, H.J. \"Coherent oscillations: a mechanism of feature linking in the visual \ncortex?\" Bioi. Cybern. 60: 121-130, 1988. \nGray, C.M., Engel, A.K., Konig, P. and Singer, W. \"Stimulus-dependent neuronal \noscillations in cat visual cortex: Receptive field properties and feature dependence.\" \nEur. J. Neurosci. 2: 607-619, 1990. \nGray, C.M., Konig, P., Engel, A.K. and Singer, W. \"Oscillatory response in cat \nvisual cortex exhibits inter-columnar synchronization which reflects global stimulus \nattributes.\" Nature 338: 334-337, 1989. \n\nHorn, D. and Usher, M. \"Excitatory-inhibitory networks with dynamical thresh(cid:173)\nolds.\" Int. J. of Neural Systems 1: 249-257, 1990. \n\nKoch, C. and Schuster, H. \"A simple network showing burst synchronization with(cid:173)\nout frequency locking.\" Neural Computation, in press. \n\nMcCulloch, W.S. and Pitts, W.A. \"A logical calculus of the ideas immanent in \nneural nets.\" Bull. Math. Biophys. 5: 115-137, 1943. \n\nNiebur, E., Schuster, H.G., Kammen, D.M. and Koch, C. \"Oscillator-phase coupling \nfor different two-dimensional network connectivities.\" Phys. Rev. A., in press. \nSchuster, H.G. and Wagner, P. \"A model for neuronal oscillations in the visual cor(cid:173)\ntex: I Mean-field theory and the derivation of the phase equations.\" Bioi. Cybern. \n64: 77-82, 1990. \n\nSompolinsky, H., Golomb, D. and Kleinfeld, D. \"Global processing of visual stimuli \nin a neural network of coupled oscillators.\" Proc. Natl. Acad. Sci. USA 87: \n7200-7204, 1989. \n\nvon der Malsburg, C. and Schneider, W. \"A neural cocktail-party processor.\" Bioi. \nCybern. 54: 29-40, 1986. \n\nWilson, M.A. and Bower, J .M . \"Cortical oscillations and temporal interactions in \na computer simulation of piriform cortex.\" J. Neurophysiol., in press. \n\n\f", "award": [], "sourceid": 581, "authors": [{"given_name": "Heinz", "family_name": "Schuster", "institution": null}, {"given_name": "Christof", "family_name": "Koch", "institution": null}]}