{"title": "Network Model of State-Dependent Sequencing", "book": "Advances in Neural Information Processing Systems", "page_first": 283, "page_last": 290, "abstract": null, "full_text": "NETWORK MODEL OF STATE-DEPENDENT \n\nSEQUENCING \n\nJeffrey P. Sutton: Adam N. Mamelakt and J. Allan Hobson \n\nLaboratory of Neurophysiology and Department of Psychiatry \n\nHarvard Medical School \n\n74 Fenwood Road, Boston, MA 02115 \n\nAbstract \n\nA network model with temporal sequencing and state-dependent modula(cid:173)\ntory features is described. The model is motivated by neurocognitive data \ncharacterizing different states of waking and sleeping. Computer studies \ndemonstrate how unique states of sequencing can exist within the same \nnetwork under different aminergic and cholinergic modulatory influences. \nRelationships between state-dependent modulation, memory, sequencing \nand learning are discussed. \n\n1 \n\nINTRODUCTION \n\nModels of biological information processing often assume only one mode or state \nof operation. In general, this state depends upon a high degree of fidelity or mod(cid:173)\nulation among the neural elements. In contrast, real neural networks often have \na. repertoire of processing states that is greatly affected by the relative balances of \nvarious neuromodulators (Selverston, 1988; Harris-Warrick and Marder, 1991). One \narea where changes in neuromodulation and network behavior are tightly and dra(cid:173)\nmatically coupled is in the sleep-wake cycle (Hobson and Steriade, 1986; Mamelak \nand Hobson, 1989). This cycle consists of three main states: wake, non-rapid eye \n\n\u2022 Also in the Center for Biological Information Processing, Whitaker College, E25-201, \n\nMassachusetts Institute of Technology, Cambridge, MA 02139 \n\nt Currently in the Department of Neurosurgery, University of California, San Francisco, \n\nCA 94143 \n\n283 \n\n\f284 \n\nSutton, Mamelak, and Hobson \n\nmovement (NREM) sleep and rapid eye movement (REM) sleep. Each state is char(cid:173)\nacterized by a unique balance of monoaminergic and cholinergic neuromodulation \n(Hobson and Steriade, 1986; figure 1). In humans, each state also has character(cid:173)\nistic cognitive sequencing properties (Foulkes, 1985; Hobson, 1988; figure 1). An \nintegration and better understanding of the complex relationships between neuro(cid:173)\nmodulation and information sequencing are desirable from both a computational \nand a neurophysiological perspective. In this paper, we present an initial approach \nto this difficult neurocognitive problem using a network model. \n\nSTATE \n\nMODULATION \ntonic \nphasic \n\namlDerglc cholinergic \n\n(tf) \n\n(6) \n\nWAKE \n\nhigh \n\nlow \n\nNREM \n\nSLEEP \n\nintcr-\n\nUlC(liatc \n\nlow \n\nREM \n\nSLEEP \n\nlow \n\nhigh \n\nSEQUENCING \n\nprogrt!6llive \n\nAl ~ A2 --~ A3 \nJ, +- illput \nBl -7 B2 \n\nperseverative \n\nAl \n\nl' \\, \n\nA3~ A2 \n\nbizarre \n\nAl -7 A2 \n\nJ, +-rGO \nA2/Bl \nPGO -+ J, \n\nD2 ~ B3 \n\nFigure 1: Overview of the three state model which attempts to integrate aspects of \nneuromodulation and cognitive sequencing. The aminergic and cholinergic systems \nare important neuromodulators that filter and amplify, as opposed to initiating or \ncarrying, distributed information embedded as memories (eg. A1, A2, A3) in neural \nnetworks. In the wake state, a relative aminergic dominance exists and the asso(cid:173)\nciated network sequencing is logical and progressive. For example, the sequence \nA1 -+ A2 transitions to B1 -+ B2 when an appropriate input (eg. B1) is present \nat a certain time. The NREM state is characterized by an intermediate aminergic(cid:173)\nto-cholinergic ratio correlated with ruminative and perseverative sequences. Unex(cid:173)\npected or \"bizarre\" sequences are found in the REM state, wherein phasic choliner(cid:173)\ngic inputs dominate and are prominent in the ponto-geniculo-occipital (PGO) brain \nareas. Bizarreness is manifest by incongruous or mixed memories, such as A2/ B1, \nand sequence discontinuities, such as A2 -+ A2/ B1 -+ B2, which may be associated \nwith PGO bursting in the absence of other external input. \n\n\fNetwork Model of State-Dependent Sequencing \n\n285 \n\n2 AMINERGIC AND CHOLINERGIC \n\nNEUROMODULATION \n\nAs outlined in figure 1, there are unique correlations among the aminergic and \ncholinergic systems and the forms of information sequencing that exist in the states \nof waking and NREM and REM sleep. The following brief discussion, which un(cid:173)\ndoubtably oversimplifies the complicated and widespread actions of these systems, \nhighlights some basic and relevant principles. Interested readers are referred to the \nreview by Hobson and Steriade (1986) and the article by Hobson et al. \nin this \nvolume for a more detailed presentation. \n\nThe biogenic amines, including norepinephrine, serotonin and dopamine, have \nbeen implicated as tonic regulators of the signal-to-noise ratio in neural networks \n(eg. Mamelak and Hobson, 1989). Increasing (decreasing) the amount of aminergic \nmodulation improves (worsens) network fidelity (figure 2a). A standard means of \nmodeling this property is by a stochastic or gain factor, analogous to the well-known \nBoltzmann factor f3 = l/kT, which is present in the network updating rule. \nComplex neuromodulatory effects of acetylcholine depend upon the location and \ntypes of receptors and channels present in different neurons. One main effect is \nfacilitatory excitation (figure 2b). Mamelak and Hobson (1989) have suggested \nhow the phasic release of acetylcholine, involving the bursting of PGO cells in the \nbrainstem, coupled with tonic aminergic demodulation, could induce bifurcations \nin information sequencing at the network level. The model described in the next \nsection sets out to test this notion. \n\na. \n\nh. \n\n1.0 r--------7\":::::::O~==-_, \n\nbe 0.8 \n.9 \n~ \n\"0 0.6 \n~ \n~ 0.4 \n'\" e \n\nc.. 0.2 \n\n\u00b73 \n\n-2 \n\n-1 \n\no \nb-8 \n\n1 \n\n2 \n\n3 \n\nMembrane Potential Relative to Threshold \n\nInitial Activity \n\n------------(] \n\u00b7\u00b7\u00b7\u00b7A\u00b7\u00b7\u00b7A\u00b7 8-6 \n\nEPSP ~ -6 \n\nResultant Activity \n\n------(] A \n\nno efFow:t \n\naction potential subthreshold \n\ninduced \n\nadivity pp.rRists \n\nFigure 2: (a) Plot of neural firing probability as a function of the membrane proten(cid:173)\ntial, h, relative to threshold, 9, for values of aminergic modulation f3 of 0.5, 1.0, 1.5 \nand 3.0. (b) Schematic diagram of cholinergic facilitation, where EPSPs of magni(cid:173)\ntude 6 only induce a change in firing activity if h is initially in the range (9 - 6, 9). \nModified from Mamelak and Hobson (1989). \n\n\f286 \n\nSutton, Mamelak, and Hobson \n\n3 ASSOCIATIVE SEQUENCING NETWORK \n\nThere are several ways to approach the problem of modeling modulatory effects on \ntemporal sequencing. We have chosen to commence with an associative network \nthat is an extension of the work on models resembling elementary motor pattern \ngenerators (Kleinfeld, 1986; Sompolinsky and Kanter, 1986; Gutfreund and Mezard, \n1988). We consider it to be significant that recent data on brainstem control systems \nshow an overlap between sleep-wake regulators and locomotor pattern generators \n(Garcia-Rill et al., 1990). \nThe network consists of N neural elements with binary values S, = \u00b11, i = 1, .'\" N, \ncorresponding to whether they are firing or not firing. The elements are linked \ntogether by two kinds of a priori learned synaptic connections. One kind, \n\np \n\nJH) = ~ I: ere;, \n\ni:/; j, \n\n(1) \n\n#,=1 \n\nencodes a set of p uncorrelated patterns {er}[~l! J.L = 1, ... ,p, where each er takes \nthe value \u00b1l with equal probabilities. These patterns correspond to memories that \nare stable until a transition to another memory is made. Transitions in a sequence \nof memories J.L = 1 -+ 2 -+ ... -+ q < p are induced by a second type of connection \n\n9- 1 \n\nJ~~) = ~ \"c~+lc~. \n\nN L...J Ii., \n\n'3 \n\n1i.3 \n\n(2) \n\n#,=1 \n\nHere, ~ is a relative weight of the connection types. The average time spent in a \nmemory pattern before transitioning to the next one in a sequence is T. At time t, \nthe membrane potential is given by \n\nN \n\nh,(t) = ~ [IN) Sj(t) + J,~') Sj(t - T) 1 + 6,(t) + 1;(t). \n\n(3) \n\nThe two terms contained in the brackets reflect intrinsic network interactions, while \nphasic PGO effects are represented by the 6,(t). External inputs, other than PGO \ninputs, to ~(t) are denoted by Ii(t). Dynamic evolution of the network follows the \nupdating rule \n\nwith probability \n\n{ 1 + .'F'/I[h\u00ab.)-\u2022\u2022 (.)) } -1 \n\n(4) \n\nIn this equation, the amount of aminergic-like modulation is parameterized by {3. \nWhile updating could be done serially, a parallel dynamic process is chosen here for \nconvenience. In the absence of external and PGO-like inputs, and with {3 > 1.0, \nthe dynamics have the effect of generating trajectories on an adiabatically varying \nhyper surface that molds in time to produce a path from one basin of attraction \nto another. For {3 < 1.0, the network begins to lose this property. Lowering {3 \nmostly affects neural elements close to threshold, since the decision to change firing \nactivity centers around the threshold value. However, as {3 decreases, fluctuations \nin the membrane potentials increase and a larger fraction of the neural elements \nremain, on average, near threshold. \n\n\fNetwork Model of State-Dependent Sequencing \n\n287 \n\n4 SIMULATION RESULTS \nA network consisting of N = 50 neural elements was examined wherein p = 6 \nmemory patterns (A1, A2, A3, B1, B2 and B3) were chosen at random \n(pi N = 0.12). These memories were arranged into two loops, A and B, accord(cid:173)\ning to equation (2) such that the cyclic sequences A1 --+ A2 --+ A3 --+ A1 --+ \u2022\u2022\u2022 and \nB1 --+ B2 --+ B3 --+ B1 --+ \u2022\u2022\u2022 were stored in loops A and B, respectively. For sim(cid:173)\nplicity, c5i(t) = c5(t) and 9.(t) = 0, 'Vi. The transition parameters were set to A = 2.5 \nand T = 8 for all the simulations to ensure reliable pattern generation under fully \nmodulated conditions (large /3, c5 = OJ Somplinsky and Kanter, 1986). Variations in \n/3, c5(t) and I.(t) delineated the individual states that were examined. \nIn the model wake state, where there was a high degree ofaminergic-like modulation \n(eg. /3 = 2.0), the network generated loops of sequential memories. Once cued into \none of the two loops, the network would remain in that loop until an external input \ncaused a transition into the other loop (figure 3). \n\n..: \n\nu \n\nso \n\n;, \n\n\\ \n\u2022 \n\n.ao \n\nILS, \n\n:CI.II~'\" \n\n(lOll, \n\nZ$' \n' \n\n... \n\u2022 \"~ ____ -~ \n\n. - , ~ \n. \n. ' \n\nl, \n, \n\"'.II~\"\" \nt. \n, \n\", 1. \n, \n'~~------------\n\n__ ; __________________ _ \n\n\"I 2 J ' \n, \n, \n\n,;, \n\n.:Jo \n\n1..00 \n\n.21 \n\n,5(1 \n\n\u2022 \n\n, \n\nI \n\n_ \n\nI \n\n_ \n\n\u2022 \n\nII \n\n1 I l0 - - - - - - - ! . ' \n\n~ :~~ , \n. \n, \n, \n\n. \n\n.. 1.11 \nos \nAI \n\n_ \n\u2022 \n\n' \n\n\" \n\n~ \n\n~ \n\n, \n\nI \n\n, \n\n, \n, \n\n. \n~\\ \" \" \n\n., \n\n~ \n\nII \n\n, \n\nI \n\nI \n\n, \n\n~f.llr \n\no.s \nU \n\n\u2022 \n\n\u2022 \n:Is \n\n\u2022 \nItJ 7r~'\" \n\n'. \n\n~', \n, \n\n'I \n\n, \n\n, \n\n; :~f------------~ \n\n~ ~ ~ \n\n~ \n\nA~ \n\n, \n\nn \n\nI/me \n\nJL-\n\nBI \n\nFigure 3: Plot of overlap as a function of time for each of the six memories A1, \nA2, A3, B1, B2, B3 in the simulated wake state. The overlap is a measure of the \nnormalized Hamming distance between the instantaneous pattern of the network \nand a given memory. f3 = 2.0, c5 = 0, A = 2.5, T = 8. The network is cued in pattern \nA1 and then sequences through loop A. At t = 75, pattern B1 is inputted to the \nnetwork and loop B ensues. The dotted lines highlight the transitions between \ndifferent memory patterns. \n\n\f288 \n\nSutton, Mamelak, and Hobson \n\nSlmuillflHl NREII Sf..\" S,.,. \n\n;: \n\no \n\nu \n\n~ \n\nn \n\nf~ \n\nfa \n\nf~ \n\n/lme \n\nFigure 4: Graph of overlap VB. time for each of the six memories in the simulated \nNREM sleep state. {3 = 1.1, 6 = 0, A = 2.5, T = 8. Initially, the network is cued \nin pattern Al and remains in loop A. Considerable fluctuations in the overlaps are \npresent and external inputs are absent. \n\nAs {3 was decreased (eg. (3 = 1.1), partially characterizing conditions of a model \nNREM state, sequencing within a loop was observed to persist (figure 4). However, \ndecreased stability relative to the wake state was observed and small perturbations \ncould cause disruptions within a loop and occasional bifurcations between loops. \nNevertheless, in the absence of an effective mechanism to induce inter-loop transi(cid:173)\ntions, the sequences were basically repetitive in this state. \nFor small f3 (eg. 0.8 < f3 < 1.0) and various PGO-like activities within the simulated \nREM state, a diverse and rich set of dynamic behaviors was observed, only some of \nwhich are reported here. The network was remarkably sensitive to the timing of the \nPGO type bursts. With f3 = 1.0, inputs of 6 = 2.5 units in clusters of 20 time steps \noccurring with a frequency of approximately one cluster per 50 time steps could \ninduce the following: (a) no or little effect on identifiable intra-loop sequencing; \n(b) bifurcations between loops; (c) a change from orderly intra-loop sequencing \nto apparent disorder;l(d) a change from apparent disorder to orderly progression \nwithin a single loop (\"defibrillation\" effect); (e) a change from a disorderly pattern \nto another disorderly pattern. An example of transition types (c) and (d), with the \noverall effect of inducing a bifurcation between the loops, is shown in figure 5. \n\n10n detailed inspection, the apparent disorder actually revealed several sequences in \n\nloops A and/or B running out of phase with relative delays generally less than T. \n\n\fNetwork Model of State-Dependent Sequencing \n\n289 \n\nIn general, lower intensity (eg. 2.0 to 2.5 units), longer duration (eg. >20 time steps) \nPGO-like bursting was more effective in inducing bifurcations than higher intensity \n(eg. 4.0 units), shorter duration (eg. 2 time steps) bursts. PGO induced bifurcations \nwere possible in all states and were associated with significant populations of neural \nelements that were below, but within 6 units of threshold. \n\nSlmu/afMI REJI SllHIp SIIIIII \n\n:c ::~0: ~ A--~ \n\nu,!:-...... ,-~zs~. -~~:-'-. -+.7S,....\\--:'\u00b1;DII:-----:'2$=--~,~ \n\n, \n~'.II', \n\n'. \n' \n\n\" \n\" \n\n' \n\" \n\noslt\\-..\u00a5~ \nu \n\n~ \n\n, \n\n\\ \n\n, \n\n\\ \n\n' \n' \n\n, \n, \n\n~ ,.0 \nU \n\n~~ \n\nH \n\n~ \n\nn \n\n~ ~ _ \n\n&O~~ , \n\n, \n\n. \n, \n\n' \n, \n\n; ~ \n\n, \n\nu \n\nn \n\n~ \n\n-PGO \n\n~ ~ ~ \n\n11\",. \n\n-PGO \n\nFigure 5: REM sleep state plot of overlap VB. time for each of the six memories. \nf3 = 1.0, 6 = 2.5, A = 2.5, T = 8. The network sequences progressively in loop \nA until a cluster of simulated PGO bursts (asterisks) occurs lasting 40 < t < 60. \nA complex output involving alternating sequences from loop A and loop B results \n(note dotted lines). A second PGO burst cluster during the interval 90 < t < 110 \nyields an output consisting of a single loop B sequence. Over the time span of the \nsimulation, a bifurcation from loop A to loop B has been induced. \n\n5 STATE-DEPENDENT LEARNING \n\nThe connections set up by equations (1) and (2) are determined a priori using \na standard Hebbian learning algorithm and are not altered during the network \nsimulations. Since neuromodulators, including the monoamines norepinephrine and \nserotonin, have been implicated as essential factors in synaptic plasticity (Kandel \net al., 1987), it seems reasonable that state changes in modulation may also affect \nchanges in plasticity. This property, when superimposed on the various sequencing \nfeatures of a network, may yield possibly novel memory and sequence formations, \nassociations and perhaps other unexamined global processes. \n\n\f290 \n\nSutton, Mamelak, and Hobson \n\n6 CONCLUSIONS \n\nThe main finding of this paper is that unique states of information sequencing \ncan exist within the same network under different modulatory conditions. This \nresult holds even though the model makes significant simplifying assumptions about \nthe neurophysiological and cognitive processes motivating its construction. Several \nobservations from the model also suggest mechanisms whereby interactions between \nthe aminergic and cholinergic systems can give rise to sequencing properties, such as \ndiscontinuities, in different states, especially REM sleep. Finally, the model provides \na means of investigating some of the complex and interesting relationships between \nmodulation, memory, sequencing and learning within and between different states. \n\nAcknowledgeInents \n\nSupported by NIH grant MH 13,923, the HMS/MMHC Research & Education Fund, \nthe Livingston, Dupont-Warren and McDonnell-Pew Foundations, DARPA under \nONR contract N00014-85-K-0124, the Sloan Foundation and Whitaker College. \n\nReferences \n\nFoulkes D (1985) Dreaming: A Cognitive-Psychological Analysis. Hillsdale: Erl(cid:173)\nbaum. \n\nGarcia-Rill E, Atsuta Y, Iwahara T, Skinner RD (1990) Development of brainstem \nmodulation of locomotion. Somatosensory Motor Research 7 238-239. \nGutfreund H, Mezard M (1988) Processing of temporal sequences in neural net(cid:173)\nworks. PhYI Rev Lett 61 235-238. \nHarris-Warrick RM, Marder E (1991) Modulation of neural networks for behavior. \nAnnu Rev Neurolci 14 39-57. \n\nHobson JA (1988) The Dreaming Brain. New York: Basic. \n\nHobson JA, Steriade M (1986) Neuronal basis of behavioral state control. \nIn: \nMountcastle VB (ed) Handbook of Physiology - The Nervous System, Vol IV. \nBethesda: Am Physiol Soc, 701-823. \n\nKandel ER, Klein M, Hochner B, Shuster M, Siegelbaum S, Hawkins R, et al. (1987) \nSynaptic modulation and learning: New insights into synaptic transmission from \nthe study of behavior. In: Edelman GM, Gall WE, Cowan WM (eds) Synaptic \nFunction. New York: Wiley, 471-518. \n\nKleinfeld D (1986) Sequential state generation by model neural networks. Proc Naa \nAcad Sci USA 83 9469-9473. \n\nMamelak AN, Hobson JA (1989) Dream bizarrenes as the cognitive correlate of \naltered neuronal behavior in REM sleep. J Cog Neurolci 1(3) 201-22. \n\nSelverston AI (1988) A consideration of invertebrate central pattern generators as \ncomputational data bases. Neural Networks 1 109-117. \nSompolinsky H, Kanter I (1986) Temporal association in asymmetric neural net(cid:173)\nworks. Phys Rev Lett 57 2861-2864. \n\n\f", "award": [], "sourceid": 524, "authors": [{"given_name": "Jeffrey", "family_name": "Sutton", "institution": null}, {"given_name": "Adam", "family_name": "Mamelak", "institution": null}, {"given_name": "J.", "family_name": "Hobson", "institution": null}]}