{"title": "Hippocampally-Dependent Consolidation in a Hierarchical Model of Neocortex", "book": "Advances in Neural Information Processing Systems", "page_first": 24, "page_last": 30, "abstract": null, "full_text": "Hippocampally-Dependent Consolidation in a \n\nHierarchical Model of Neocortex \n\nSzabolcs Ka1i1,2 \n\nPeter Dayan1 \n\n1 Gatsby Computational Neuroscience Unit \n\nUniversity College London \n\n17 Queen Square, London, England, WCIN 3AR. \n\n2Department of Brain and Cognitive Sciences \n\nMassachusetts Institute of Technology \n\nCambridge, MA 02139, U.S.A. \n\nszabolcs@gatsby.ucl.ac.uk \n\nAbstract \n\nIn  memory consolidation, declarative memories  which  initially  require \nthe  hippocampus  for  their recall,  ultimately  become independent of it. \nConsolidation has  been the focus of numerous experimental and qualita(cid:173)\ntive modeling studies, but only little quantitative exploration. We present \na consolidation model  in  which  hierarchical  connections in  the  cortex, \nthat  initially  instantiate  purely  semantic  information  acquired  through \nprobabilistic  unsupervised learning,  come  to  instantiate episodic  infor(cid:173)\nmation  as  well.  The hippocampus is  responsible  for  helping complete \npartial input patterns before consolidation is  complete, while also train(cid:173)\ning the cortex to perform appropriate completion by itself. \n\n1  Introduction \n\nThe hippocampal formation and adjacent cortical areas have long been believed to  be in(cid:173)\nvolved in the acquisition and retrieval of long-term memory for events and other declarative \ninformation.  Clinical studies in humans and animal experiments indicate that damage to \nthese regions results in  amnesia,  whereby the ability to acquire new declarative memories \nis  impaired and some of the memories acquired before the damage are lost [I].  The obser(cid:173)\nvation that recent memories are more likely to be lost than old memories in these cases has \ngenerally been interpreted as evidence that the role of these medial temporal lobe structures \nin the storage and/orretrieval of declarative memories is only temporary.  In particular, sev(cid:173)\neral investigators have  advocated the  general  idea that,  in  the  course  of a relatively  long \ntime period (from several days in rats up to decades in humans), memories are reorganized \n(or consolidated) so  that  memories  whose  successful recall initially  depends  on  the hip(cid:173)\npocampus gradually become independent of this  structure (see Refs.  2-4).  However, other \npossible interpretations of the data have also been proposed [5]. \nThere  have  been  several  analyses  of the  computational  issues  underlying  consolidation. \nThere is  a general consensus that memory recall involves the reinstatement of cortical ac(cid:173)\ntivation patterns  which characterize  the  original episodes,  based  only  on  partial  or noisy \n\n\finput.  Thus the computational goal for the memory systems is cortical pattern completion; \nthis  should be  possible  after just a single presentation of the  particular pattern when  the \nhippocampus is  intact,  and  should be possible independent of the presence or absence of \nthe hippocampus once consolidation is complete. The hippocampus plays a double role:  a) \nsupporting one-shot learning and subsequent completion of patterns in the cortical areas it \nis  directly connected to, and b) directing consolidation by reinstating these stored patterns \nin those same cortical regions and allowing the efficacies of cortical synapses to change. \nDespite  the  popularity  of the  ideas  outlined  above,  there  have  been  surprisingly  few  at(cid:173)\ntempts  to  construct  quantitative  models  of memory  consolidation.  Alvarez  and  Squire \n(1994)  is  the  only  model  we  could  find  that  has  actually  been  implemented  and  tested \nquantitatively.  Although it embodies the general principles above, the authors themselves \nacknowledge that the  model  has  some rather serious  limitations,  largely  due  to  its  spar(cid:173)\ntan  simplicity  (eg  it  only  considers  2 perfectly  orthogonal patterns  over 2 cortical  areas \nof 8 units  each)  which also  makes it hard to  test comprehensively.  Perhaps most impor(cid:173)\ntantly,  though  (and  this  feature  is  shared  with  qualitative models  such  as  Murre  (1997\u00bb, \nthe model requires  some way  of establishing and/or strengthening functional connections \nbetween neurons in disparate areas of neocortex (representing different aspects of the same \nepisode) which would not normally be expected to enjoy substantial reciprocal anatomical \nconnections. \nIn this paper, we consider consolidation using a model whose complexity brings to the fore \nconsideration of computational  issues  that  are  invisible to  simpler proposals.  In  particu(cid:173)\nlar,  it  treats  cortex  as  a hierarchical  structure,  with  hierarchical  codes  for  input patterns \nacquired through a process of unsupervised learning.  This allows us to  study the relation(cid:173)\nship between coding for generic patterns, which forms a sort of semantic memory, and the \ncoding for  the  specific patterns  through consolidation.  It also  allows  us  to  consider con(cid:173)\nsolidation  as  happening in  hierarchical  connections  (in  which  the  cortex  abounds)  as  an \nalternative to  consolidation  only between disparate areas  at the  same  level  of the hierar(cid:173)\nchy.  The next section  of the  paper describes  the  model  in  detail  and  section  3 shows  its \nperformance. \n\n2  The Model \n\nFigure  la shows  the  architecture of the  model,  which  involves three cortical  areas  (A,  B, \nand C)  that represent different aspects  of the  world.  We  can understand consolidation as \nfollows :  across  the  whole  spectrum  of possible inputs,  there  is  structure  in  the  activity \nwithin  each  area;  but  there  are  no  strong correlations  between  the  activities  in  different \nareas  (these  are  the  generic  patterns  referred  to  above).  Thus,  for  instance,  nothing  in \nparticular can be concluded about the pattern of activity in area C given just the activities \nin areas A and B. However, for the specific patterns that form particular episodes, there are \ncorrelations in  these activities.  As  a result of this,  it becomes possible to  be much  more \ndefinite about the pattern in C given activities in A and B that reinstate part of the episode. \nBefore consolidation,  information about these  correlations  is  stored  in  the  hippocampus \nand related structures; after consolidation, the information is  stored directly in the weights \nthat construct cortical representations. \nThe model does  not assume that there are any direct connections between the cortical ar(cid:173)\neas.  Instead, as  a closer match to the available anatomical data, we assume a hierarchy of \ncortical regions  (in  the  present model  having just two  layers) below the hippocampus.  It \nis  hard  to  establish  an exact correspondence between model components  and  anatomical \nregions, so we tentatively call the model region on the top of the cortical hierarchy entorhi(cid:173)\nnal/parahippocampal/perirhinal area (E/P), and lump together all parts of the hippocampal \nformation into an entity we call hippocampus (HC). EIP is connected bidirectionally to all \nthe cortical areas. \n\n\f", "award": [], "sourceid": 1842, "authors": [{"given_name": "Szabolcs", "family_name": "K\u00e1li", "institution": null}, {"given_name": "Peter", "family_name": "Dayan", "institution": null}]}