{"title": "Stability and Observability", "book": "Advances in Neural Information Processing Systems", "page_first": 1171, "page_last": 1172, "abstract": null, "full_text": "Stability and  Observability \n\nMax Garzon \n\nFernanda Botelho \n\ngarzonmGhermea.maci.memat.edu botelhofGhermea.maci.memat.edu \nInstitute for  Intelligent Systems  Department of Mathematical Sciences \n\nMemphis State University \nMemphis,  TN 38152  U.S.A. \n\nThe theme was the effect of perturbations of the defining parameters of a neural net(cid:173)\nwork due to:  1)  mea\"urement\" (particularly with analog networks); 2)  di\"cretization \ndue to a)  digital implementation of analog nets; b)  bounded-precision implementa(cid:173)\ntion  of digital networks;  or c)  inaccurate evaluation of the  transfer function(s};  3) \nnoise in or incomplete input and/or output of the net or individual cells  (particu(cid:173)\nlarly with analog networks). \n\nThe workshop presentations address these  problems in various ways.  Some develop \nmodels  to  understand  the  influence  of errors/perturbation  in the  output,  learning \nand  general behavior of the net  (probabilistic in Piche  and  TresPi optimisation in \nRojas;  dynamical systems  in  Botelho  k  Garson).  Others  attempt  to  identify  de(cid:173)\nsirable  properties that are  to be  preserved  by neural network solutions  (equilibria \nunder faster convergence in Peterfreund  &  Baram; decision  regions  in Cohen).  Of \nparticular interest is  to develop  networks that compute robustly, in the sense  that \nsmall perturbations of their parameters do not  affect  their dynamical and observ(cid:173)\nable  behavior  (stability  in  biological  networks  in  Chauvet  &  Chauvet;  oscillation \nstability in learning in Rojas; hysterectic finite-state machine simulation in  Casey). \nIn particular, understand how biological networks cope with uncertainty and errors \n(Chauvet &  Chauvet)  through the type of stability that they exhibit. \n\nQUESTIONS AND ANSWERS \n\nSome  questions served  to focus  the presentations  and discussion.  Some were  (par(cid:173)\ntially)  answered,  and others were barely touched: \n<>  What are  the  mod \"ignificant error\" in defining  parameter\" with  re\"pect to  output \nbehavior?  By evidence  presented, i/o and weights seem to be  the most sensitive. \n<>  Is  there  an  essential  difference  between perturbations in weights  (long-term  mem(cid:173)\nory) and inputs (short-memory)?  They seem to playa symmetric role in feedforward \nand, to some extent, recurrent  nets.  But evidence is not conclusive. \n<>  How can  the  effects of perturbation\" be  kept under control or eliminated altogether'! \nIf one  is  only  interested  in  dynamical qualitative features,  small  enough  errors  of \nany  kind  (as  incurred  in digital implementations for  example)  are  not relevant for \nmost  nets  (What you see on the screen is what should  be  happening). \n<>  Are they architecture  (in)dependentf  On the other hand, they spread rapidly un(cid:173)\nder iteration and exact quantification varies with the architecture. \n<>  Are  stability  and  implementation  based  on  dynamical  features  the  only  ways  to \n\n1171 \n\n\f1172 \n\nGarzon and Botelho \n\ncope  with  error!/perturbatiofU f  The difficulty  to quantify  (perhaps  due  to lack of \nresearch)  seems to indicate so.  Stability worth a  closer look for  its own sake. \n<>  Doe,  requiring  robud  computation  really  redrict  the  capabilitie,  of neural  net(cid:173)\nwork, f  Apparently  not,  since  in  all  likelihood  there  exist  universal  neural  nets \nwhich tolerate small errors  (see  talk by Botelho &  Garlon).  Wide open. \n\nTALKS AND SHORT ABSTRACTS \n\n\u2022  TraJ~tory Control  of  Convergent  Networks,  Natan  Peterfreund  and  Y. \nBaram.  We  present  a  class  of  feedback  control  functions  which  accelerate  con(cid:173)\nvergence rates of autonomous nonlinear dynamical systems such as neural network \nmodels, without affecting the basic convergence properties (e.g.  equilibrium points). \nnatanOtx.technion.ac.il \n\u2022  Sensitivity  of Neural  Network  to  Errors,  Steven  Piche.  Using  stochastic \nmodels,  analytic expressions for  the effects of such errors are  derived for  arbitrary \nfeedforward neural networks.  Both, the degree of nonlinearity and the relationship \nbetween  input  correlation  and  the  weight  vectors,  are  found  to  be  important  in \ndetermining the effects of errors.  picheOlllcc. COm \n\u2022  Stability  of Learning  in  Neural  Networks,  Raul  Roja!.  Finding  optimal \ncombinations of learning  and  momentum rates  for  the  standard  backpropagation \ninvolves difficult tradeoffs across fractal boundaries.  We show that statistic prepro(cid:173)\ncessing can bring error functions  under control.  rOjaaOinf. fu-berlin.de \n\u2022  Stability of Purklnje Cells in Cerebellar Cortex, Gilbert Ohauvet and Pierre \nOhauvet.  The cerebellar cortex (involved in learning and retrieving) is a hierarchical \nfunctional  unit built  around  a  Purkinje cell,  which  has  its own functional proper(cid:173)\nties.  We have shown experimentally that Purkinje dynamical systems have a unique \nsolution,  which is  asymptotically stable.  It  seems  possible to give  a  general expla(cid:173)\nnation of stability in biological systems.  chauvetOibt. uni v-angers. fr. \n\u2022  Recall  and  Learning  with  Deficient  Data,  Volker  Tresp,  Subutai  Ahmad, \nRalph Neuneier.  Mean values and maximum likelihood estimators are  not the best \nways to cope with noisy data.  See their LA:5 poster summary in these proceedings \nfor  an extended abstract.  treapOzfe. aiemena. de \n\u2022  Computation  Dynamics  in  Discrete-Time  Recurrent  Networks,  Mike \nOasey.  We  consider  training  recurrent  higher-order  neural  networks  to  recognize \nregular languages,  using  the  cycles  in  their diagrams for  hysterectic simulation of \nfinite  state machines.  The latter suggests a  general logical approach to solving the \n'neural code' problem for living organisms, necessary for understanding information \nprocessing in the nervous system.  mcaseyOsdcc. ucsd. edu \n\u2022  Synthesis  of Decision  Regions  in  Dynamical  Systems,  Mike  Oohen.  As \na  first  step  toward  a  representation  theory  of decision  functions  via  neural  nets, \nhe  presented  a  method  which  enables  the  construction  of a  system  of differential \nequations exhibiting a  given finite  set of decision regions and equilibria with a very \nlarge  class of indices consistent with the  Morse  inequalities.  mikeOpark. bu. edu \n\u2022  Observability of Discrete and Analog Networks, F.  Botelho and M.  Garzon. \nWe show that most networks (with finitely many analog or infinitely many boolean \nneurons)  are  observable  (i.e.,  all  their corrupted pseudo-orbits actually reflect  true \norbits).  See  their DS:2  poster summary in these proceedings. \n\n\f", "award": [], "sourceid": 734, "authors": [{"given_name": "Max", "family_name": "Garzon", "institution": null}, {"given_name": "Fernanda", "family_name": "Botelho", "institution": null}]}