{"title": "Non-Boltzmann Dynamics in Networks of Spiking Neurons", "book": "Advances in Neural Information Processing Systems", "page_first": 109, "page_last": 116, "abstract": null, "full_text": "Non-Boltzmann Dynamics in Networks of Spiking Neurons \n\n109 \n\nNon-Boltzmann  Dynamics  in Networks  of \n\nSpiking  Neurons \n\nMichael  C.  Crair and William  Bialek \n\nDepartment of Physics,  and \n\nDepartment of Molecular and  Cell Biology \n\nUniversity of California at Berkeley \n\nBerkeley,  CA 94720 \n\nABSTRACT \n\nWe study networks  of spiking neurons  in which spikes  are fired  as \na  Poisson process.  The state of a  cell is  determined  by the  instan(cid:173)\ntaneous  firing  rate,  and  in the  limit of high firing  rates our model \nreduces  to  that  studied  by  Hopfield.  We  find  that  the  inclusion \nof spiking  results  in several new  features,  such  as  a  noise-induced \nasymmetry between \"on\" and \"off\" states of the cells and probabil(cid:173)\nity currents which destroy the usual description of network dynam(cid:173)\nics  in  terms  of energy  surfaces.  Taking account  of spikes  also  al(cid:173)\nlows us to calibrate network parameters such as  \"synaptic weights\" \nagainst  experiments  on  real synapses.  Realistic  forms  of the  post \nsynaptic  response  alters  the  network dynamics,  which  suggests  a \nnovel dynamical learning mechanism. \n\n1 \n\nINTRODUCTION \n\nIn  1943 McCulloch and Pitts introduced the  concept  of two-state  (binary) neurons \nas elementary building blocks for neural computation.  They showed that essentially \nany finite calculation can be done using these simple devices.  Two-state neurons are \nof questionable biological relevance, yet much of the subsequent work on modeling of \nneural networks has  been based on McCulloch-Pitts type neurons  because  the  two(cid:173)\nstate  simplification makes analytic  theories  more  tractable.  Hopfield  (1982,  1984) \n\n\f110 \n\nCrair and Bialek \n\nshowed that an asynchronous model of symmetrically connected  two-state neurons \nwas equivalent to Monte-Carlo dynamics on an 'energy' surface at zero temperature. \nThe  idea that  the  computational abilities  of a  neural  network can  be  understood \nfrom the structure of an effective energy surface has been the central theme in much \nrecent work. \nIn an effort  to understand  the  effects  of noise,  Amit,  Gutfreund  and  Sompolinsky \n(Amit  et aI.,  1985a;  1985b) assumed  that  Hopfield's  'energy' could  be  elevated  to \nan energy in the statistical mechanics sense,  and solved the Hopfield model at finite \ntemperature.  The  problem is  that  the  noise  introduced  in equilibrium statistical \nmechanics is  of a very special form, and it is not clear that the stochastic properties \nof real neurons are captured by postulating a  Boltzmann distribution on the energy \nsurface. \nHere we try to do a slightly more realistic calculation, describing interactions among \nneurons  through  action  potentials which are fired  according  to probabilistic rules. \nWe  view  such  calculations  as  intermediate  between  the  purely  phenomenological \ntreatment  of neural  noise  by  Amit  et  aI.  and  a  fully  microscopic  description  of \nneural dynamics in  terms of ion channels and their associated  noise.  We  find  that \neven our limited attempt at biological realism results in some interesting deviations \nfrom previous  ideas  on  network dynamics. \n\n2  THE MODEL \nWe consider a model where neurons have a continuous firing rate, but the generation \nof action potentials is  a  Poisson process.  This mean~ that the  \"state\"  of each cell  i \nis described  by the  instantaneous  rate Ti(t),  and  the  probability that  this cell  will \nfire  in a  time  interval  [t, t + dt]  is  given  by Ti(t)dt.  Evidence  for  the  near-Poisson \ncharacter of neuronal firing can be found in the mammalian auditory nerve (Siebert, \n1965;  1968),  and  retinal  ganglion cells  (Teich  et  al.,  1978, Teich and  Saleh,  1981). \nTo stay as  close  as  possible  to  existing  models,  we  assume  that  the  rate  T( t)  of a \nneuron is a sigmoid function, g(x) =  1/(1 +e- Z ), of the  total input x  to the neuron. \nThe  input  is  assumed  to  be  a  weighted  sum of the  spikes  received  from  all  other \nneurons,  so that \n\nr,(t) = rmY  [~~ J,;!(t - til - e,] . \n\n(1) \n\nJii  is  the  matrix  of connection  strengths  between  neurons,  Tm  is  the  maximum \nspike rate  of the neuron, and 0i is  the neuronal threshold.  J(t)  is a  time  weighting \nfunction,  corresponding  schematically to the time course  of post-synaptic currents \ninjected  by a  pre-synaptic spike; a  good first  order approximation for  this function \nis  J(t)  -- e- t / r ,  but we  also  consider  functions  with  more  than  one  time constant. \n(Aidley,  1980,  Fetz and Gustafsson,  1983). \n\nWe  can  think of the  spike  train from  the  itA  neuron,  Ep .5(t  - tn,  as  an approx(cid:173)\n\nimation to the  true  firing  rate  Ti(t);  of course  this  approximation improves  as  the \n\n\fNon-Boltzmann Dynamics in Networks of Spiking Neurons \n\n111 \n\nspikes come  closer  together at high firing  rates.  If we  write \n\nL <5(t  - tn = ri(t) + 7]i(t) \n\nIJ \n\n(2) \n\nwe  have defined  the  noise  TJi  in  the  spike  train.  The  equations  of motion for  the \nrates then become \n\n(3) \n\nwhere  Ni(t)  =  L:j  Jij7]j(t)  and  f  0  rj(t)  is  the  convolution of f(t)  with  the  spike \nrate  rj(t).  The  statistics  of the  fluctuations  in  the  spike  rate  7]j(t)  are  (7]j(t\u00bb  = \n0, \n\n(7]i(t)7]j(t'\u00bb = <5ij(t - t')rj(t). \n\n3  DYNAMICS \nto  obtain  a  first  order  equation  for  the  normalized  spike  rate  Yi(t)  = ri{t)/rm. \nIf the  post-synaptic  response  f(t)  is  exactly  exponential,  we  can  invert  Eq.  (3) \n\nMore precise descriptions of the post-synaptic response  will yield  higher order  time \nderivatives with coefficients that depend  on the relative time constants in  f(t).  vVe \nwill  comment  later on the  relevance  of these  higher  order  terms,  but consider  first \nthe lowest order description.  By inverting Eq.  (3)  we obtain a stochastic differential \nequation analogous to the Langevin equation describing  Brownian motion: \n\ndg-1(Yd  __ dE  N.() \ndYi  +  \u2022 t  , \n\ndt \n\n-\n\nwhere  the deterministic forces  are  given  by \n\n(4a) \n\n(4b) \n\nNote  that  Eq.  (4)  is  nearly  equivalent  to  the  \"charging equation\"  Hopfield  (1984) \nassumed in his discussion of continuous neurons, except  we  have explicitly included \nthe  noise  from  the spikes.  This system is  precisely  equivalent  to  the  Hopfield  two(cid:173)\nstate  model  in  the  limit  of large  spike  rate  (rm T  =:}  00, Jii  =  constant),  and  no \nIn  a  thermodynamic  system  near  equilibrium,  the  noise  \"force\"  Ni (t)  is \nnoise. \nrelated  to  the  friction  coefficient  via  the  fluctuation  dissipation  theorem.  In  this \nsystem however,  there  is no analogous relationship. \n\nA standard transformation, analogous to deriving Einstein's diffusion equation from \nthe Langevin equation (Stratonovich, 1963,  1967), yields a  probabilistic description \nfor  the evolution of the neural system, a form of Fokker-Planck equation for  the time \nevolution of P( {y;}),  the probability that the network is in a  state described  by the \nnormalized rates {y;};  we  write the Fokker-Planck equation below for a simple case. \n\n\f112 \n\nCrair and Bialek \n\nA  useful interpretation to consider is  that the system, starting in a  non-equilibrium \nstate, diffuses  or evolves in phase space,  to a  final  stationary state. \nWe  can  make  our  description  of  the  post-synaptic  response  f(t)  more  accurate \nby  including  two  (or  more)  exponential  time  constants,  corresponding  roughly  to \nthe  rise  and  fall  time  of the  post  synaptic  potential.  This  inclusion  necessitates \nthe  addition  of a  second  order  term  in  the  Langevin  equation  (Eq.  4).  This  is \nanalogous to including an inertial term in a  diffusive description, so that the system \nis  no longer  purely dissipative.  This additional complication has some  interesting \nconsequences.  Adjusting  the  relative  length  of the  rise  to  fall  time  of  the  post \nsynaptic potential effects  the rate of relaxation  to local  equilibrium of the system. \nIn  order  to  perform most  efficaciously  as  an  associative memory,  a  neural system \nwill  \"choose\"  critical damping time  constants,  so  that relaxation is fastest.  Thus, \nby adjusting the time course of the post synaptic potential, the system can  \"learn\" \nof a  local  stationary  state,  without  adjusting  the  synaptic  strengths.  This  novel \nlearning mechanism could be a form of fine  tuning of already established memories, \nor could  be  a  unique form of dynamical short-term memory. \n\n4  QUALITATIVE RESULTS \nIn  order  to  understand  the  dynamics  of our  Fokker-Planck equation,  we  begin  by \nconsidering the case of two neurons interacting with each other.  There are two lim(cid:173)\niting behaviors.  If the neurons  are  weakly coupled  (J  < Je , Je  = 4/rm  T),  then  the \nonly stable state of the system is with both neurons firing at a mean firing rate, ! rm. \nIf the  neurons are  strongly  (and  positively) coupled  (J  > Je ),  then  isolated  basins \nof attraction,  or stationary  states  are  formed,  one  stationary  state  corr..:sponding \nto  both neurons  being active,  the other state  has both neurons  relatively (but not \nabsolutely)  quiescent.  In  the  strong  coupling  limit,  one  can  reduce  the  problem \nto motion along  the  a  collective coordinate connecting  the  two stable states.  The \nresulting one dimensional Fokker-Planck equation is \n\nat P(y, t) = ay  U'(y)P(y, t) + ay T(y)P(y, t) \na \n\na  [ \n\na \n\n1 \n\n, \n\nwhere  U(y)  is an effective  potential energy, \n\nU  Y  = y(l - y) \n'(  ) \n\n[9- 1(y) \n\nT \n\n1 \n2 \n\n- -rmJ  y - -) + -J rmy(3 - 5y)], \n\n( \n\n1 \n2 \n\n1  2 \n4 \n\n(5) \n\n(6) \n\nand  T(y)  is  a  spatially  varying  effective  temperature,  T(y)  = ~J2rmy3(1 _  y)2. \nOne  can solve  to find  the size  of the  stable  regions,  and  the stationary probability \ndistribution, \n\n[(I U'(y)  1 \n\u2022 \nP  (y)  - T(y) exp  - J,  T(y) dy \n\nB \n\n. \n\n-\n\n(7) \n\nWe have done numerical simulations which confirm the qualitative predictions of the \none dimensional Fokker-Planck equation.  This analysis shows that the non-uniform \n\n\fNon-Boltzmann Dynamics in Networks of Spiking Neurons \n\n113 \n\nand asymmetric  temperature  distribution alters  the  relative stability of the stable \nstates, in the favor of the 'off' state.  This effect does have some biological pertinence, \nas  it  is  well  known  that  on  average  neurons  are  more  likely  to  be  quiescent  then \nactive.  In our model  the  asymmetry  is  a  direct  consequence  of the  Poisson nature \nof the neuronal firing. \n\nProbability Current \n\n\u2022 \n\n...  -\n\n'\"  -\nI -r \n\ni \n\no \n\nII \n\n2 \n\n\u2022 \n\nI \n\n14 \n\n\u2022  \u2022 \n\n! \n\u2022 \n\nrX ... \n\n\u2022 \n\n10 \n\n\u2022 \n\n12 \n\nFigure  1:  Probability  current  in  the  stationary  state  for  two  neurons  that  are \nstrongly  interacting.  Computed  as  a  ratio  of the  number  of excess  excursions  in \none  dire<:tion  to  the  total  number  of excursions,  in  percent.  In  thermodynamic \nequillibrium,  detailed  balance  would  force  the  current  to  be  zero.  Shown  as  a \nfunction of the number of spikes in an e-folding time of the post-synaptic response. \n\nThere are further  surprises  to be found  in the simple two neuron model.  Since  the \ninteraction between the neurons  is  not  time  reversal invariant, detailed  balance  is \nnot  maintained  in  the  system.  Thus,  even  the  stationary  probability distribution \nhas non-zero probability current, so that the system tends to cycle  probabilistically \nthrough state space.  The presence  of the  current further  alters  the relative proba(cid:173)\nbility of the  two stable states,  as confirmed  by numerical simulations, and renders \nthe application of equilibrium statistical mechanics inappropriate. \n\nSimulations also confirm (Fig.  1)  that the  probability current falls off with increas(cid:173)\ning maximum spike  rate (rmT),  because  the effective  noise  is  suppressed  when the \nspike  rate  is  high.  However,  at biologically reasonable  spike  rates  (rm  - 150s- 1), \nthe  probability current is  significant.  These  currents destroy any sense  of a  global \n\n\f114 \n\nCrair and Bialek \n\nenergy function or  thermodynamic temperature. \nOne advantage of treating spikes explicitly is that we can relate the abstract synaptic \nstrength  J  to observable parameters.  In  Fig.  2 we  compare  J  with the experimen(cid:173)\ntally accessible spike number to spike number transfer across the synapse, for  a  two \nneuron system.  Note that critical coupling (see  above) corresponds to a rather large \nvalue of,...- 4/5 th  of a  spike  emitted  per spike received. \n\nSpikes Generated per Spike Input \n\n. -----------------------------\n\n.' \n\no \n\n. 0 \n\n, \n\n~ ~ \ni I. \n\ne \ne \n\n0.0 \n\nD.5 \n\n1.0 \n\n1.5 \n\n2.0 \n\n2.5 \n\nFigure  2:  Single  neuron  spike  response  to  the  receipt  of a  spike  from a  coupled \nneuron.  Since response is probabilistic, fractional spikes are relevant.  Computed as \na  function of J  /Jcritical,  where  Jcritical  is  the minimum synaptic strength necessary \nfor  isolated basins of attraction. \n\nMany of the  simple ideas we  have introduced for  the two neuron system carryover \nto  the  multi-neuron case.  If the  matrix of connection strengths  obeys  the  \"Hebb\" \nrule  (often  used  to model associative  memory), \n\n(8) \n\nthen a stability analysis yields the same critical value for  the connection strength J \n(note  that we have scaled by N,  and the sum on 11  runs from 1 to p, the  number of \nmemories  to be  stored).  Calculation of the  spike-out/spike-in ratio for  the  multi(cid:173)\nneuron system at critical coupling shows that it scales like (a/N)t, where  p = aN. \n\n\fNon-Boltzmann Dynamics in Networks of Spiking Neurons \n\n115 \n\nSince  most neural systems naturally have a  small spike-out/spike-in ratio,  this (to(cid:173)\ngether  with Fig.  2)  suggests  that small networks  will have to be strongly driven in \norder  to achieve isolated  basins of attraction for  \"memories;\"  this  is  in agreement \nwith  the  one  available experiment  (Kleinfeld  et  aI.,  1990).  In contrast,  large  net(cid:173)\nworks achieve criticality with more  natural spike  to spike ratios.  For instance,  if a \nnetwork of 104  - 105  connected neurons is  to have multiple stable \"memory\" states \nas  in  the  original  Hopfield  model,  we  predict  that  a  neuron  needs  to  receive  100-\n500  contiguous action  potentials  to stimulate the  emission  of its own  spike.  This \nprediction agrees with experiments done on the hippocampus (McNaughton et  al., \n1981),  where  about 400  convergent inputs are needed  to discharge a  granule cell. \n\n5  CONCLUSIONS \nTo conclude,  we  will just summarize our major points: \n\n\u2022  Spike  noise  generated  by  the  Poisson firing  of neurons  breaks  the  symmetry \n\nbetween on/off states,  in favor of the  \"off\"  state. \n\n\u2022  State dependent  spike noise  also destroys any sense  of a  global energy  func(cid:173)\n\ntion,  let  alone  a  thermodynamic  'temperature'.  This  makes  us  suspicious  of \nattempts to apply standard techniques  of statistical mechanics. \n\n\u2022  By  explicitly  modeling  the  interaction  of neurons  via spikes,  we  have direct \n\naccess  to experiments which can guide,  and be  guided  by our  theory.  Specif(cid:173)\nically, our theory  predicts  that for  a  given connection  strength  between  neu(cid:173)\nrons,  larger net Norks  of neurons will function as  memories at naturally small \nspike-input to spike-output ratios. \n\n\u2022  More realistic forms  of post synaptic response  to the receipt  of action poten(cid:173)\ntials alters  the network dynamics.  By adjusting the  relative rise  and fall  time \nof the post-synaptic potential, the network speeds  the relaxation ,to  the  local \nstable state.  This  implies  that more  efficacious  memories,  or  \"learning\", can \nresult  without  altering the strength of the synaptic weights. \n\nFinally, we  comment on  the  dynamics of networks in the  N  -+ 00  limit.  \\Ve  might \nimagine that some of the complexities we find in the two-neuron case would go away, \nin particular the probability currents.  We have been able to prove  that this does not \nhappen in any rigorous sense for  realistic forms of spike noise,  although in  practice \nthe  currents  may  become  small.  The  function  of the  network  as  a  memory  (for \nexample) would then depend on a clean separation of time scales between relaxation \ninto a  single  basin of attraction and noise-driven transitions to neighboring basins. \nArranging for  this separation  of time  scales  requires  some  constraints on  synaptic \nconnectivity and firing rates which might be testable in experiments on real circuits. \n\n\f116 \n\nCrair and Bialek \n\nReferences \n\nD. J. Aidley (1980),  Physiology of Excitable Cells,  2nd Edition, Cambridge Univer(cid:173)\nsity Press,  Cambridge. \nD. J. Amit, H. Gutfreund and H.  Sompolinsky (1985a), Phys.  Rev.  A, 2,  1007-1018. \nD.  J.  Amit,  H.  Gutfreund  and  H.  Sompolinsky  (1985b),  Phys.  Rev.  Lett.,  55, \n1530-1533. \nE.  E.  Fetz  and B.  Gustafsson  (1983),  J.  Physiol.,  341, 387. \nJ.  J.  Hopfield  (1982),  Proc.  Nat.  Acad.  Sci.  USA,  79,2554-2558. \n\nJ. J.  Hopfield  (1984),  Proc.  Nat.  Acad.  Sci.  USA,  81,3088-3092. \n\nD.  Kleinfeld,  F.  Raccuia-Behling,  and  H.  J.  Chiel  (1990),  Biophysical  Journal,  in \npress. \n\nW.  S.  McCulloch and  W.  Pitts (1943),  Bull.  of Math.  Biophys., 5,  115-133. \n\nB.  L.  McNaughton,  C.  A.  Barnes  and  P.  Anderson  (1981),  J.  Neurophysiol.  46, \n952-966. \n\nW. M.  Siebert  (1965),  Kybernetik,  2,  206. \n\nW.  M.  Siebert  (1968)  in  Recognizing  Patterns,  p104,  P.A.  Kohlers  and  ~L Eden, \nEds.,  MIT  Press,  Cambridge. \nR.  1. Stratonovich (1963,1967),  Topics  in  the  Theory oj Random Noise,  Vol.  I and \nII,  Gordon  &  Breach,  New  York. \n\nM.  C.  Teich, L.  Martin and  B.1.  Cantor (1978), J.  Opt.  Soc.  Am., 68,  386. \nM.  C.  Teich and  B.E.A. Saleh  (1981),  J.  Opt.  Soc.  Am.,71, 771. \n\n\f", "award": [], "sourceid": 276, "authors": [{"given_name": "Michael", "family_name": "Crair", "institution": null}, {"given_name": "William", "family_name": "Bialek", "institution": null}]}