{"title": "A Model of Feedback to the Lateral Geniculate Nucleus", "book": "Advances in Neural Information Processing Systems", "page_first": 409, "page_last": 416, "abstract": null, "full_text": "A  Model of Feedback to  the Lateral \n\nGeniculate Nucleus \n\nComputation and  Neural  Systems Program \n\nCalifornia Institute of Technology \n\nCarlos D.  Brody \n\nPasadena, CA 91125 \n\nAbstract \n\nSimplified  models  of the  lateral geniculate  nucles  (LGN)  and stri(cid:173)\nate cortex  illustrate the  possibility  that feedback  to the  LG N may \nbe  used  for  robust,  low-level  pattern  analysis.  The  information \nfed  back  to the  LG N is  rebroadcast  to cortex  using the LG N 's full \nfan-out,  so  the  cortex-LGN-cortex pathway  mediates extensive \ncortico-cortical communication while  keeping the number of neces(cid:173)\nsary  connections small. \n\n1 \n\nINTRODUCTION \n\nThe lateral geniculate nucleus (LGN)  in  the  thalamus is often  considered  as just a \nrelay station on  the way from  the retina to visual cortex, since receptive field  prop(cid:173)\nerties of neurons in  the  LGN  are very similar  to retinal  ganglion  cell  receptive field \nproperties.  However,  there  is  a  massive  projection  from  cortex  back  to  the  LGN: \nit  is  estimated  that 3-4  times  more  synapses in  the  LG N are  due  to corticogenicu(cid:173)\nlate  connections than those due  to retinogeniculate  connections  [12].  This suggests \nsome  important  processing  role  for  the  LGN,  but  the  nature  of the  computation \nperformed has  remained far  from  clear. \n\nI  will  first  briefly  summarize  some  anatomical  facts  and  physiological  results  con(cid:173)\ncerning the corticogeniculate  loop,  and then present a  simplified  model in  which  its \nfunction  is  to (usefully)  mediate communication between cortical cells. \n\n409 \n\n\f410 \n\nBrody \n\n1.1  SOME ANATOMY AND PHYSIOLOGY \n\nThe  LG N  contains  both  principal  cells,  which  project  to  cortex,  and  inhibitory \ninterneurons.  The  projection  to cortex sends  collaterals  into a  sheet  of inhibitory \ncells  called  the  perigeniculate  nucleus (PGN).  PGN  cells,  in  turn,  project  back  to \nthe  LGN.  The  geniculocortical  projection  then  proceeds  into cortex,  terminating \nprincipally in  layers 4 and 6 in  the cat [11,  12].  Areas  17,  18,  and to a lesser extent, \n19  are  all  innervated.  Layer  6  cells  in  area  17  of  the  cat  have  particularly  long, \nnon-end-stopped  receptive  fields  [2].  It  is  from  layer  6  that  the  corticogeniculate \nprojection  back  originates.1  It,  too,  passes  through  the  PGN,  sending  collaterals \ninto it, and then cont.acts both principal cells and interneurons in  the LGN,  mostly \nin  the  more  distal  parts  of  their  dendrites  [10,  13].  Both  the  forward  and  the \nbackward projection  are  retinotopically ordered. \nThus there is  the possibility of both excitatory and inhibitory effects in  the cortico(cid:173)\ngeniculate  projection,  which  is  principally  what shall  be used  in  the model. \n\nThe  first  attempts  to study  the  physiology  of the  corticogeniculate  projection  in(cid:173)\nvolved inactivating cortex in some way  (often cooling cortex) while observing genic(cid:173)\nulate  responses  to simple  visual  stimuli.  The results  were  somewhat  inconclusive: \nsome investigators reported that the projection was excitatory, some  that it was in(cid:173)\nhibitory, and still others t.hat it had no observable effect at all.  [1,  5,9] Later studies \nhave emphasized  the need for  using stimuli which optimally excite the cortical cells \nwhich  project  to  the  LGN;  inactivating cortex  should  then  make  a  significant  dif(cid:173)\nference  in  the  inputs  to  geniculate  cells.  This  has  helped  to reveal  some  effects: \nfor  example,  LGN  cells  with  corticogeniculate  feedback  are  end-stopped  (that  is, \nrespond  much  less to long  bars than  to short  bars),  while  the end-stopping is  quite \ndearly reduced  when  the  cortical  input  is  removed  [8]. \n\nOne  study  [13]  has  used  cross-correlat.ion  analysis  between  cortical  and geniculate \ncells to suggest that there is spatial structure in the corticogeniculate projection:  an \nexcitatory corticogeniculate interaction was found  if cells had receptive field  centers \nthat were close to each other, while an inhibitory interaction was found  if the centers \nwere farther apart.  However, the precise spatial structure of the projection  remains \nunknown. \n\n2  A  FEEDBACK MODEL \n\nI  will  now  describe  a  simplified  model  of the  LGN  and  the  corticogeniculate loop. \nThe  very  simple  connection  scheme  shown  in  fig  1 originated  in  a  suggestion  by \nChristof Koch  [3]  that the long receptive fields  in layer 6 might  be used to facilitate \ncontour  completion  at  t.he  LGN  level.  In  the  model,  then,  striate  cortex  simple \ncells  feed  back  positively  to  the  LGN,  enhancing  the  conditions  which  gave  rise \nto  their firing.  This reinforces,  or  completes,  the oriented bar or  edge  patterns to \nwhich they are tuned.  Assuming that the visual features of interest are for  the most \npart oriented, while  much of the  noise  in  images  may  be  isotropic  and  unoriented, \nenhancing the oriented features  improves the signal-to-noise ratio. \n\n1 In  all  areas innervated  by the  LGN. \n\n\fA Model of Feedback to the Lateral  Geniculate Nucleus \n\n411 \n\nOO[i] \n///~ \n~~ \nLGN ~ 00 \n\nCELLS \n\nD--~~ \n\nRETINA \n\n[f] \n~~ \n\nVI CELLS \n\nFigure  1:  Basic  model  connectivity:  A  schematic diagram  showing  the  connections \nbetween  different.  pools of units in  the single spatial frequency  channel  model.  LGN  cells \nfirst  filter  the image  linearly  through  a center-surround  filter  (V 2 G),  the  result  of which \nis  then  passed  through  a  sigmoid  nonlinearity  (tanh).  (In  the simulations presented  here \nG  was  a  Gaussian  with  standard  deviation  1.4  pixels.)  VI  cells  then  provide  oriented \nfiltering,  which is  also  passed  through  a  nonlinearity (logistic;  but see  details in  text) and \nfed  back  positively  to  the  LGN  to reinforce detected  oriented edges.  VI  cells excite  LGN \ncells  which  have  excitat.ory  connections  to  them,  and  inhibit  those  t.hat  have  inhibitory \nconnections  to  them.  Inhibition  is  implicitly  assumed  to  be  mediated  by  interneurons. \n(Note that there are no  intracortical or intrageniculate connections:  communication takes \nplace entirely  through  the feedback  loop.)  See  text  for  further  details. \n\nFor simplicity, only striate cortex simple  \"edge-detecting\"  cells were modeled.  Two \nmodels  are  presented. \nIn  the  first  one,  all  cortical  cells  have  the  same  spatial \nfrequency characteristics.  In the second one, two channels, a  high frequency channel \nand  a  low  frequency  channel, interact simultaneously. \n\n2.1  SINGLE  SPATIAL FREQUENCY CHANNEL MODEL \n\nA srhematic  diagram  of the  model  is  shown  in  figure  1.  The retina is  used  simply \nas  an  input  layer.  To  each  input  position  (pixel)  in  the  retina  there  corresponds \none  LGN  unit.  Linear  weights  from  the  retina  to  the  LGN  implement  a  '\\l2C \nfilter,  where G(x,y) is  a  two-dimensional Gaussian.  The LGN units then project to \neight.  different  pools  of \"orientation-t.uned\"  cells  in  VI.  Each  of these pools  has  as \nmany units as t.here  are pixels  in  the input  \"retina\".  The  weights in  the projection \nforward  to VI  represent eight rotations of the template shown in figure  2a, covering \n360 degrees.  This simulates basic  orientation tuning in  VI.  Cortical cells then feed \n\n\f412 \n\nBrody \n\nback positively to the geniculus,  using rotations of the template shown in  fig  2(b). \nThe precise dynamics of the  model are as follows:  Ri  are real-valued retinal inputs, \nLi are geniculate unit outputs, and V;  are cortical cell outputs.  Gji represent weights \nfrom  retina  - LGN,  Fji  forward  weights  from  LGN  - VI,  and  Bji  backward \nweights  from  VI  - LGN.  o:,/3,,,,(,TCl  and  TC2  are  all  constants.  For  geniculate \nunits: \n\ndl-\n_J =-\"111.+ \ndt \n\nI  J \n\nL \n\nG .. Q\u00b7 + \n\nJI~~ \n\nL \n\nB ' LVIe \n\nJ~ \n\nLj  = tanh(/j ) \n\ni \n\nIe \n\nWhile for  cortical cell  units: \n\ndVj  = -o:v' + ~ Y\u00b7L\u00b7 - /3(~ IY\u00b7IL \u00b7)2 \ndt \n\nJI \n\nJI \n\nJ \n\nI \n\nL.....J \ni \n\nL.....J \ni \n\nI \n\nV;={  g(Vj  - Tcd \n\no \n\nif vi  > TC2 \notherwise \n\nHere gO  is  the logistic  function. \n\n\"receptIYe field le.atII\" \n\n\u2022\u2022\u2022\u2022\u2022\u2022\u2022 \n\u2022\u2022\u2022\u2022\u2022\u2022\u2022 \n\u2022\u2022\u2022\u2022\u2022\u2022\u2022 \n\n0000000 \n0000000 \n0000000 \n\n\u2022 \u2022\u2022\u2022\u2022\u2022\u2022 \n\n\u2022  \u2022  \u2022  \u2022  \u2022  \u2022  \u2022 \n\u2022  \u2022  \u2022  \u2022  \u2022  \u2022  \u2022 \n0000000 \no \n0 \no  0 \n\n0  000  \n\n0  000  \n\n0 \n\n0 \n\n(b) \n\nFigure 2:  Weights between the LGN and VI.  Figure 2(a):  Forward weights, from the \nLGN to VI.  Each circle represents the weight from  a cell in the LGN; dark circles represent \npositive weights,  light circles  negative weights  (assumed  mediated  by  interneurons).  The \nradius  of each  circle  represent.s  the strength  of the corresponding  weight.  These  weights \ncreate  \"edge-detecting\"  neurons in  VI.  Figure 2(b):  Backwards weights,  from  VI  back  to \nthe LGN.  Only cells close  to  the contrast edge receive strong feedback. \n\nIn the scheme described above many cortical cells have overlapping receptive fields, \nboth in  the forward  projection from  the geniculus and in  the backwards projection \nfrom  cortex.  A cell  which  is  reinforcing  an  edge  within  its  receptive field  will  also \npartially reinforce the edge  for  retinotopically nearby cortical cells.  For nearby cells \nwith  similar  orientation  tuning,  the  reinforcement  will  enhance  their  own  firing; \nthey  will  then  enhance  the  firing  of ot.her,  similar,  cells  farther  along;  and  so  on. \nThat  is,  the  overlapping feedback  fields  allow  the edge  detection  process  to follow \ncontours (note that the process is  tempered  at the geniculate level by actual  input \nfrom  the  retina).  This  is  illustrated  in figure  3. \n\n\fA Model of Feedback to  the Lateral  Geniculate Nucleus \n\n413 \n\nFigure 3:  Following contours:  This figure  shows the effect on  the LGN of the feedback \nenhancement.  The image on the left is the retinal input:  a very weak, noisy horizontal edge. \nThe center image  is  the  LGN  after  two  iterations of the  simulation.  Note  that  initially \nonly  certain sectors of the edge are detected  (and  hence enhanced).  The  rightmost image \nis  the  LGN  after  8  iterations:  the  enhanced  region  has  spread  to  cover  the  entire  edge \nthrough  the effect  of  horizontally  oriented,  overlapping  receptive  fields.  This is  the  final \nstable point of the dynamics. \n\n2.2  MULTIPLE  SPATIAL  FREQUENCY CHANNELS MODEL \n\nIn the model described above the LGN  is integrating and summarizing the informa(cid:173)\ntion  provided by  each of the orientation-tuned pools of cortical cells.2  It does so in \na  way  which  would easily ext.end  to cover other types of cortical cells  (bar  or  grat(cid:173)\ning  \"detectors\" , or varying spatial frequency  channels).  To experiment simply with \nthis  possibility,  an  extra  set  of eight  pools  of orientation-tuned  \"edge-detecting\" \ncortical cells  was added.  The  new  set's weights were similar to the original  weights \ndescribed  above, except t.hey  had a  \"receptive field  length\"  (see figure  2) of 3 pixels: \nthe original set had a  \"receptive field  length\"  of 9  pixels. \n\nThus one  set  was  tuned  for  detecting short  edges,  while  the  other  was  tuned  for \ndetecting long edges.  The effect  of using both of these sets is  illustrated  in  figure  4. \nBoth  sets  interact  nonlinearly  to produce  edge  detection  rather  more  robust  than \neither set used alone:  the image produced  using both simultaneously is  not a  linear \naddit.ion  of those  produced  using  each  set  separately.  Note  how  little  noise  is  ac(cid:173)\ncepted  as  an  edge.  The same model,  running  with  the  same  parameters  but  more \npixels,  was also tested  on  a  real  image.  This is  shown  in  figure  5. \n\n3  DISCUSSION  ON CONNECTIVITY \n\nA major function fulfilled  by the LG N in  this model is that of providing a communi(cid:173)\ncat.ions pathway between cortical cells,  both between cells of similar orientation but \ndifferent location or spatial frequency  tuning, and between cells of different orienta-\n\n2 A function  not  unlike  that suggested  by  Mumford  [7],  except  that  here  the  \"experts\" \n\nare extremely  low-level  orient.ation-tuned channels. \n\n\f414 \n\nBrody \n\n~::.  ' ''1 \n\n:r\" \n\nFigure 4:  Combined spatial frequency  channels:  The leftmost  image is  the  retinal \ninput,  a  weak  noisy  edge.  (The other  three  images  are  \"summary  outputs\",  obtained  as \nfollows:  the model  produces activations in  many pools of cortical cell units;  the activations \nfrom  all  VI  units corresponding to a  particular retinotopic position  are added  together to \nform  a  real-valued  number  corresponding  to  that  position;  and  this is  then  displayed  as \na  grey-scale  pixel.  Since  only  \"edge-detecting\"  units  were  used,  this  provides  a  rough \nestimate of the certainty of there being an  edge at that point.)  Second from  left we see the \nsummary output of  the  model  after  20  iterations  (by  which  time it  has stabilized),  using \nonly  the  low  spatial  frequency  channel.  Only  a  single  segment  of  the  edge  is  detected. \nThird  from  left  is  the  output  after  20  iterations  using  only  the  high  frequency  channel. \nOnly isolat.ed, short, segment.s of the edge are detected.  The rightmost image is the output \nusing  both  channels  simultaneously.  Now  the  segments  detected  by  the  high  frequency \nchannel  can  combine  with  the  original  image  to  provide  edges  long  enough  for  the  low \nfrequency  channel  t.o  detect and  complete into a single,  long continuous edge. \n\ntion tuning:  for example,  these last compete to reinforce their particular orientation \npreference  on  the  geniculus.  The model  qualitatively  shows  that  such  a  pathway, \nwhile  mediated  by  a  low-level  representation  like  that  of the  LGN,  can  neverthe(cid:173)\nless  be used  effectively, producing contour-following and  robust edge-detection.  We \nmust  now  ask  whether such  a  function  could  not  be  performed  without  feedback. \nClearly, it could be done without feedback to the LGN, purely through intracortical \nconnections, since  any feedback net.work can in principle be  \"unfolded in  time\"  into \na  feedforward  network  which  performs  the  same  computation- provided  we  have \nenough  units and  connections available. \n\nIn other words, any sugg{'st.ed functional role for corticogeniculate feedback must not \nonly include an account of the proposed computation performed, but also an account \nof why it  is  preferable  to perform that computation  through a  feedback  process,  in \nterms  of some  efficiency  measure  (like  the  number  of cells  or  synapses  necessary, \nfor  example).  There can  be  no other  rationale,  apart  from  fortuitous  coincidence, \nfor  constructing  an  elaborate  feedback  mechanism to perform  a  computation  that \ncould just  as  well  be done  without  it. \n\nWith this view in mind, it.  is worth re-stating that in  this model any two cortical cells \nwhose receptive fields overlap are connected (disynaptically) through the LG N.  How \nmany connections would we require in order to achieve similar communication if we \nonly  used  direct  connections  between  cortical  orientation-tuned  cells  instead?  In \nmonkey, each cell's receptive field  overlaps with approximately 106  others [4]- thus, \n\n\fA  Model of Feedback to the Lateral  Geniculate Nucleus \n\n415 \n\n~\u00b71~ . . ' \n. ~  \\ . , \n,  , \n.... ~  :o~, \n;...=:: \n... \n\n.. ::~:.:-\n\n, \u00b7;:~Hk.;~.~ \n\ni \n\n\" \n\n}: \n\n/ .:.-r. \n\n5*1 \n\n.-' ... ' ,'., ......... '. , '., ,. \n,6 ,  'CrT5S \n\nt ~\\ \n\n: :  \n\n' romw gg- SF' \n\nff',.P.-... ~ .. II., .... \u00b7;i.;\\'\"W.,.,-W, ... , ........... __ ...... ,..... \ne '. \nil \nfi\nw~~\\  r\u00b7~:,,~~,\u00b7 \n:l~~s't]~k.il~:~~~~~~: \n\n~ ::' \"\"\"'\"  . '::~?~:t;\\' :I:~i~J:~~:\u00b7::;.:;~:;:;:;~~;~:~:~I:~:; :;:';:.~: _:', \n\n~ 1 :' ' ........ ~., .. \" .... ; .. ~ \n\n\u2022\u2022 \n\n. , \n\n\u2022 \n\n\u2022 \u2022 ...;y;; .. ~ _ \n\n. .. ....... -.\u2022\u2022\u2022 .r \u2022\u2022\u2022\u2022\u2022\u2022 \n\nFigure 5:  A  real image:  The top image is the retinal input.  Stippling is due to printing \nonly.  The  center  image  is  that  obtained  through  detecting  the  zero-crossings  of v 2c. \nTo  reduce  spurious  edges.  a  minimum  slope  threshold  was  placed  on  the  point  of  the \nzero-crossing  below  which  edges  were  not  accepted.  The image  shown  here  was  the  best \nthat could  be obtained  through  varying both  the  width of the  Gaussian  G  and  the slope \nthreshold  value.  The  last  image  shows  the  summary  output  from  the  model,  using  two \nsimultaneous spatial frequency cha.nnels.  Note how noise is reduced compared to the center \nimage, straight lines  are smoother,  and  resolution  is  not impaired,  but is  better in  places \n(group of people at lower left.  or  \"smoke stacks\"  atop launcher). \n\n\f416 \n\nBrody \n\nany  cortical  cell  would  need  to synapse  onto at least  106  cells.  If the  information \ncan be sent via the LGN, geniculate cell fan-out can reduce the number of necessary \nsynapses  by  a  significant  factor.  It is  estimated  that  geniculate  cells  (in  the  cat) \nsynapse onto at least 200  cortical cells (probably more)  [6],  reducing  the number of \nnecessary  connections considerably. \n\n4  BIOLOGY AND  CONCLUSIONS \n\nIn  section  1.1  I  noted  one  important study  [8J  which established that corticogenic(cid:173)\nulate  input  reduces firing  of geniculate cells  for  long  bars;  this  is  in  direct  contra(cid:173)\ndiction  to the prediction  which  would  be made  by this model,  where the feedback \nenhances firing for  long features (here, edges).  Thus, the model  does not agree with \nknown  physiology. \n\nHowever,  the  model's  value  lies  simply  in  clearly  illustrating  the  possibility \nthat  feedback  in  a  hierarchical  processing  scheme  like  the  corticogeniculate  loop \ncan  be  utilized  for  robust,  low-level  pattern  analysis,  through  the  use  of  the \ncortex-+LGN-+cortex communications  pathway.  The possibility  that  a  great  deal \nof different  types  of  information  could  be  flowing  through  this  pathway  for  this \npurpose should  not  be  left unconsidered. \n\nAcknowledgements \n\nThe  author  is  supported  by  fellowships  from  the  Parsons  Foundation  and  from \nCONACYT (Mexico).  Thanks are  due  to Michael  Lyons for  careful reading of the \nmanuscript. \n\nReferences \n\n[IJ  Baker,  F.  H.  and  Malpeli,  J.  G.  1977  Exp.  Brain  Res.  29 pp.  433-444 \n[2J  Gilbert,  C.D.  1977,  J.  Physiol.,  268, pp.  391-421 \n[3J  Koch,  C.  1992,  personal communication. \n[4J  Hubel,  D.H.  and  Wiesel,  T. N.  1977,  Proc.  R.  Soc.  Lond.  (B) 198 pp.  1-59 \n[5]  Kalil,  R.  E.  and  Chase,  R.  1970,  J.  Neurophysiol.  33 pp.  459-474 \n[6]  Martin, K.A.C.  1988,  Q.  J.  Exp.  Phy.  73 pp.  637-702 \n[7]  Mumford,  D.  1991  Bioi.  Cybern.  65 pp.  135-145 \n[8]  Murphy,  P.C. and  Sillito,  A.M.  1987,  Nature 329 pp. 727-729 \n[9]  Richard.  D.  et. al.  1975,  Exp.  Brain  Res.  22 pp.  235-242 \n[10]  Robson,  J. A.  1983.  J.  Compo  Neurol.  216 pp.  89-103 \n[11]  Sherman,  S.  M.  1985.  Prog.  in  Psychobiol.  and  Phys.  Psych.  11  pp.  233-314 \n[12J  Sherman, S.M.  and  Koch,  C.  1986,  Exp.  Brain  Res.  63 pp.  1-20 \n[13J  Tsumoto, T. et. al.  1978,  Exp.  Brain  Res.  32 pp.  345-364 \n\n\f", "award": [], "sourceid": 670, "authors": [{"given_name": "Carlos", "family_name": "Brody", "institution": null}]}