{"title": "A Silicon Axon", "book": "Advances in Neural Information Processing Systems", "page_first": 739, "page_last": 746, "abstract": null, "full_text": "A  Silicon Axon \n\nBradley A.  Minch, Paul Hasler,  Chris Diorio, Carver Mead \n\nPhysics of Computation Laboratory \nCalifornia Institute of Technology \n\nPasadena,  CA  91125 \n\nbminch, paul, chris,  carver@pcmp.caltech.edu \n\nAbstract \n\nWe  present  a  silicon  model of an  axon  which  shows  promise  as  a \nbuilding  block for  pulse-based  neural  computations involving cor(cid:173)\nrelations of pulses  across  both space  and time.  The  circuit shares \na  number  of features  with  its  biological  counterpart  including  an \nexcitation  threshold,  a  brief refractory  period  after  pulse  comple(cid:173)\ntion, pulse amplitude restoration, and pulse width restoration.  We \nprovide  a simple explanation of circuit operation and present  data \nfrom  a  chip fabricated  in  a standard  2Jlm  CMOS  process  through \nthe MOS  Implementation Service  (MOSIS).  We  emphasize the ne(cid:173)\ncessity of the restoration of the width of the pulse in time for stable \npropagation in  axons. \n\n1 \n\nINTRODUCTION \n\nIt is  well  known  that axons are  neural  processes  specialized  for  transmitting infor(cid:173)\nmation over  relatively  long  distances  in  the  nervous  system.  Impulsive  electrical \ndisturbances  known  as  action  potentials  are  normally initiated  near  the  cell  body \nof a  neuron  when  the  voltage across  the cell  membrane crosses  a  threshold.  These \npulses  are  then  propagated  with a  fairly  stereotypical shape  at a  more or less  con(cid:173)\nstant velocity down  the length of the  axon.  Consequently, axons  excel  at precisely \npreserving the relative timing of threshold  crossing events but do  not preserve  any \nof the initial signal shape.  Information , then , is presumably encoded in the relative \ntiming of action potentials. \n\n\f740 \n\nBradley A. Minch,  Paul Hasler,  Chris Diorio,  Carver Mead \n\nThe  biophysical  mechanisms  underlying  the  initiation  and  propagation  of action \npotentials  in axons have  been  well  studied  since  the  seminal work  of Hodgkin  and \nHuxley  on  the  giant  axon  of Loligo.  (Hodgkin &  Huxley,  1952)  Briefly,  when  the \nvoltage across a small patch of the cell membrane increases to a certain level, a pop(cid:173)\nulation  of ion  channels  permeable  to  sodium opens,  allowing  an  influx  of sodium \nions, which, in turn, causes the membrane voltage to increase further and a pulse to \nbe initiated.  This population of channels rapidly inactivates, preventing the passage \nof additional ions.  Another  population of channels  permeable to potassium opens \nafter  a  brief delay  causing  an  efflux  of potassium ions,  restoring  the  membrane to \na  more negative potential and terminating the pulse.  This cycle  of ion migration is \ncoupled  to  neighboring  sections  of the  axon,  causing  the  action  potential  to prop(cid:173)\nagate.  The sodium channels  remain inactivated for  a  brief interval of time  during \nwhich  the  affected  patch  of membrane will  not  be  able  to support  another  action \npotential.  This period  of time is  known  as  the  refractory  period.  The  axon  circuit \nwhich  we  present  in  this  paper  does  not  attempt to  model  the  detailed  dynamics \nof the  various populations of ion  channels,  although such  detailed  neuromimes are \nboth  possible  (Lewis,  1968;  Mahowald  &  Douglas,  1991)  and  useful  for  learning \nabout  natural  neural  systems.  Nonetheless,  it  shares  a  number  of important fea(cid:173)\ntures with its biological counterpart including having a threshold for excitation and \na  refractory period. \n\nIt is  well  accepted  that the  amplitude of the  action  potential must  be  restored  as \nit  propagates.  It is  not  as  universally  understood  is  that  the  width  of the  action \npotential must be restored  in time if it is to propagate over any appreciable distance. \nOtherwise,  the  pulse  would  smear out in  time  resulting  in  a  loss  of precise  timing \ninformation, or it would shrink down to nothing and cease to propagate altogether. \nIn  biological  axons,  restoration  of  the  pulse  width  is  accomplished  through  the \ndynamics of sodium channel  inactivation and potassium channel  activation.  In our \nsilicon  model,  the  pulse  width  is  restored  through  feedback  from  the  successive \nstage.  This feedback  provides an inactivation which is similar to that of the sodium \nchannels in biological axons and is  also the underlying cause of refractoriness in our \ncircuit . \n\nIn  the  following  section  we  provide  a  simple  description  of how  the  circuit  be(cid:173)\nhaves.  Following this, data from a chip fabricated in a standard 2p.m CMOS process \nthrough  MOSIS  are  presented  and discussed. \n\n2  THE SILICON  AXON  CmCUIT \n\nAn axon circuit which is to be used as a building block in large-scale computational \nsystems  should  be  made  as  simple  and  low-power  as  possible,  since  it  would  be \nreplicated many times in any such system.  Each stage of the axon circuit described \nbelow  consists of five  transistors and two small capacitors,  making the axon circuit \nvery  compact.  The  axon  circuit  uses  the  delay  through  a  stage  to time the  signal \nwhich is fed  back to restore the pulse width, thus avoiding the need for an additional \ndelay  circuit  for  each  section.  Additonally,  the  circuit  operates  with  low  power; \nduring  typical  operation  (a  pulse  of width  2ms  travelling  at  103 stages/s),  pulse \npropagation costs about 4pJ / stage of energy.  Under these circumstances, the circuit \nconsumes about 2nW/stage of static power. \n\n\fA  Silicon  Axon \n\n741 \n\nFigure  1:  Three sections  of the axon circuit. \n\nThree stages of the axon circuit  are depicted  in Figure 1.  A single stage consists  of \ntwo  capacitors  and  what  would  be  considered  a  pseudo-nMOS  NAND  gate  and  a \npseudo-nMOS  inverter  in digital  logic  design.  These  simple circuits  are  character(cid:173)\nized  by  a  threshold  voltage  for  switching  and  a  slew  rate  for  recharging.  Consider \nthe inverter  circuit.  If the input is  held  low  for  a sufficiently long time, the pull-up \ntransistor  will  have  charged  the  output  voltage  almost completely  to  the  positive \nrail.  If the  input  voltage  is  ramped up  toward the  positive  rail,  the  current  in  the \npull-down transistor will increase rapidly.  At some input voltage level, the current in \nthe pull-down transistor will  equal the saturation current  of the pull-up transistor; \nthis voltage is  known  as  the threshold.  The output  voltage will  begin to discharge \nat  a  rapidly increasing  rate  as  the  input  voltage is  increased  further .  After  a  very \nshort time, the output will have  discharged  almost all the way  to the negative rail. \nNow,  if the  input  were  decreased  rapidly, the  output  voltage  would  ramp  linearly \nin time (slew)  up  toward the  positive rail at a  rate set  by the saturation current  in \nthe  pull-up  transistor  and  the  capacitor  on  the  output  node.  The  NAND  gate  is \nsimilar except  both  inputs must be  (roughly speaking)  above  the threshold in order \nfor  the output to go  low.  If either input goes  low , the output will charge toward the \npositive rail.  Note  also that if one input of the NAND  gate is  held  high,  the  circuit \nbehaves exactly as  an inverter. \n\nThe axon circuit is formed by  cascading multiple copies of this simple five  transistor \ncircuit  in series.  Let  the  voltage on  the first  capacitor  of the  nth  stage  be  denoted \nby  Un  and  the  voltage on the second  capacitor  by  vn .  Note  that there  is  feedback \nfrom  Un +1  to the  lower input of the  NAND  gate of the  nth  stage.  Under  quiescent \nconditions,  the input  to the first  stage  is  low  (at the  negative  rail) , the  U  nodes  of \nall stages are  high  (at the positive rail),  and all  of the  v  nodes  are held  low  (at the \nnegative  rail).  The feedback  signal  to  the  final  stage  in  the  line  would  be  tied  to \nthe positive rail.  The level  of the bias voltages  71  and  72  determine whether  or  not \na  narrow  pulse  fed  into  the  input  of the  first  stage  will  propagate  and,  if so,  the \nwidth and  velocity  with which  it  does. \n\nIn  order  to  obtain  a  semi-quantitative understanding  of how  the  axon  circuit  be(cid:173)\nhaves,  we  will  first  consider  the  dynamics  of a  cascade  of simple  inverters  (three \nsections of which  are  depicted  in  Figure  2)  and then consider  the  addition of feed(cid:173)\nback.  Under  most  circumstances, discharges  will occur on a  much faster  time scale \n\n\f742 \n\nBradley A.  Minch,  Paul Hasler,  Chris Diorio,  Carver Mead \n\nthan  the  recharges,  so  we  make  the  simplifying assumption  that  when  the  input \nof an inverter reaches  the threshold  voltage,  the output discharges  instantaneously. \nAdditionally,  we  assume  that saturated transistors  behave  as  ideal current  sources \n(i.e.,  we  neglect  the  Early effect)  so  that the recharges  are  linear ramps in time. \n\nTTT \n\nTTT \n\nTTT \n\nTTT \n\nTTT \n\nTTT \n\nFigure  2:  A  cascade of pseudo-nMOS  inverters. \n\nLet  hand h  be the saturation currents in the pull-up transistors with bias voltages \n1\"1  and  1\"2,  respectively.  Let  6 1  and  62  be  the  threshold  voltages of the  first  and \nsecond  inverters in a single stage, respectively.  Also,  let u =  IdC and v = I2/C be \nthe  slew  rates for  the  U  and v nodes,  respectively.  Let a 1  = 6dv and a 2  = 6 2/u \nbe the time required for  Un  to charge from the negative rail up to E>1  and for  Vn  to \ncharge from the negative rail up to 6 2 ,  respectively.  Finally, let  v~ denote  the peak \nvalue attained by the v  signal of the  nth  stage. \n\nV\u00b7 \n\nv\u00b7 71 \n\nvn \n\nIE \n\n~I \n\n~IE \n\n671  a 2 \n(a)  II  > h \n\nIE \n\n~I \n\n~IE \n\n671  a 2 \n(b)  h  < 12 \n\nFigure 3:  Geometry of the idealized Un  and Vn  signals under the bias conditions (a) \nh  > 12  and  (b)  II  < 12, \n\nConsider  what  would  happen  if Vn -l exceeded  6 1  for  a  time 671 ,  In  this  case,  Un \nwould  be  held  low  for  671  and  then  released.  Meanwhile,  Vn  would  ramp  up  to \nv6n .  Then,  Un  would begin  to charge toward the positive rail  while  Vn  continues to \ncharge.  This continues for a time a2 at which point Un  will have reached 6 2  causing \n\n\fA  Silicon  Axon \n\n743 \n\nVn  to  be  discharged  to  the  negative  rail.  Now,  Un+l  is  held  low  while  Vn  exceeds \n8 1 this  interval of time is  precisely  On+!.  Figures 3a and 3b depict  the geometry of \nthe Un  and Vn  signals in this scenario under the bias conditions h  > 12  and h  < 12, \nrespectively.  Simple  Euclidean  geometry  implies  that  the  evolution  of On  will  be \ngoverned  by  the first-order  difference  equation \n\nwhich  is  trivially solved  by \n\nThus, the quantity a2-al determines what happens to the width of the pulse as it \npropagates.  In the event  that a 2 < aI, the pulse will shrink down to nothing from \nits  initial  width.  If a 2 > aI,  the  pulse  width  will  grow  without  bound  from  its \ninitial width.  The pulse width is preserved only if a 2  = a l .  This last case,  however, \nis  unrealistic.  There  will  always  be  component  mismatches (with  both systematic \nand  random parts),  which  will  cause  the  width  of the  pulse  to grow  and shrink  as \nit propagates down  the line,  perhaps cancelling on  average.  Any systematic offsets \nwill cause the pulse to shrink to nothing or to grow  without bound as it propagates. \nIn  any  event,  information about  the  detailed  timing of the  initial  pulse  will  have \nbeen  completely lost. \nNow,  consider  the action of the feedback  in  the axon  circuit  (Figure  1).  If Un  were \nto  be  held  low  for  a  time  longer  than  a l  (i.e.,  the  time  it  takes  Vn  to  charge  up \nto 8d, Un+1  would  come  back and release  Un,  regardless of the state of the  input. \nThus,  the  feedback  enforces  the  condition  On  ::;  a l .  If 11  > h  (i.e.,  a 2 < ad, a \npulse  whose  initial width  is  larger  than a 1  will  be  clipped  to a 1  and  then shrink \ndown  to nothing and disappear.  In the event  that II < h  (i.e.,  a 2 > ad, a  pulse \nwhose  initial  duration  is  too  small  will  grow  up  until  its  width  is  limited  by  the \nfeedback.  The axon  circuit  normally operates  under the latter bias condition.  The \ndynamics of the simple inverter chain cause  a  pulse which  is  to narrow  to grow and \nthe  feedback  loop  serves  to  limit the  pulse  width;  thus,  the  width  of the  pulse  is \nrestored  in  time.  The feedback  is  also the source of the  refractoriness  in the  axon; \nthat is,  until U n +l  charges  up  to (roughly)  8 1 ,  Vn -l can have  no effect  on  Un. \n\n3  EXPERIMENTAL DATA \n\nIn  this  section,  data from  a  twenty-five  stage  axon  will  be  shown.  The  chip  was \nfabricated  in  a  standard 2f.lm  p-well  (Orbit)  CMOS  process  through MOSIS. \n\nUniform Axon \n\nA full space-time picture of pulse  (taken at the  v  nodes of the circuit)  propagation \ndown  a  uniform axon is  depicted  on  the left  in Figure 4.  The graph on the right  in \nFigure 4 shows  the same data from  a different  perspective.  The lower sloped  curve \nrepresents  the  time of the  initial  rapid  discharge  of the  U  node  at  each  successive \nstage-this time marks the leading edge of the pulse taken at the v node of that stage. \n\n\f744 \n\nBradley A.  Minch.  Paul Hasler. Chris Diorio.  Carver Mead \n\nThe upper sloped curve  marks the time of the final  rapid  discharge of the  v  node of \neach stage-this time is  the end of the  pulse  taken at the v  node of that stage.  The \npropagation velocity of the pulse is given  (in units of stages/s)  by the reciprocal  of \nthe slope  of the lower  inclined  curve.  The third curve  is  the difference  of the other \ntwo  and represents  the  pulse  width  as  a  function  of position  along  the  axon.  The \ngraph  on  the  left  of Figure  5  shows  propagation  velocity  as  a  function  of the  72 \nbias  voltage- so  long as  the  pulse propagates,  the  velocity  is  nearly  independent  of \n71.  Two orders  of magnitude of velocity  are shown in the  plot;  these  are especially \nwell  matched to the time scales of motion in auditory and visual sensory  data.  The \ncircuit  is  tunable over  a  much  wider  range  of velocities  (from  about  one stage  per \nsecond  to well  in excess  of 104 stages/ s).  The graph on the right of Figure 5 shows \npulse  width  as  a  function  of 71  for  various  values  of 72-the  pulse  width  is  mainly \ndetermined  by  71  with 72  setting  a lower limit. \n\nTapered Axon \n\nIn biological axons,  the propagation velocity of an action potential is  related to the \ndiameter of the  axon- the bigger the diameter,  the greater the  velocity.  If the  axon \nwere  tapered,  the velocity of the action potential would change as it propagated.  If \nthe bias transistors in the axon circuit are operated in their subthreshold region,  the \neffect of an exponentially tapered axon can be simulated by applying a small voltage \ndifference to the ends of each ofthe 71  and 72  bias lines.  (Lyon &  Mead, 1989) These \nnarrow wires are made with a relatively resistive layer (polysilicon); hence, putting a \nvoltage difference  across  the ends will linearly interpolate the bias voltages for  each \nstage along the line.  In subthreshold,  the bias currents  are exponentially related to \nthe bias voltages.  Since the pulse width and velocity are related to the bias currents, \nwe  expect  that a  pulse  will either speed  up  and get  narrower  or slow  down and get \nwider  (depending on the sign of the applied voltage)  exponentially as  a  function  of \nposition along the line.  The graph on the left of Figure 6 depicts the boundaries of \na  pulse  as  it  propagates along  of the  axon  circuit for  a  positive  (*'s)  and  negative \n(x 's)  voltage  difference  applied  to  the  7  lines.  The  graph  on  the  right  of Figure \n6  shows  the  corresponding  pulse  width  for  each  applied  voltage  difference.  Note \nthat in each case,  the width  changes  by  more than an order of magnitude, but the \npulse maintains its integrity.  That is,  the pulse does not disappear nor  does  it split \ninto multiple pulses-this behavior would not be possible if the pulse width were  not \nrestored  in time. \n\n4  CONCLUSIONS \n\nIn  this  paper  we  have  presented  a  low-power,  compact  axon  circuit,  explained  its \noperation, and presented  data from a working chip fabricated through MOSIS.  The \ncircuit shares  a  number of features  with its biological counterpart  including an ex(cid:173)\ncitation threshold , a brief refractory period after pulse completion, pulse amplitude \nrestoration ,  and  pulse  width  restoration.  It is  tunable  over  orders  of magnitude \nin  pulse  propagation  velocity-including  those  well  matched  to  the  time  scales  of \nauditory and  visual signals- and shows  promise for  use  in  synthetic  neural systems \nwhich perform computations by correlating events which occur over  both space and \ntime such as those presented  in  (Horiuchi  et  ai,  1991)  and (Lazzaro & Mead,  1989). \n\n\fA  Silicon  Axon \n\nAcknowledgements \n\n745 \n\nThis material is  based upon work supported in part under a National Science  Foun(cid:173)\ndation  Graduate  Research  Fellowship, the  Office  of Naval  Research,  DARPA,  and \nthe  Beckman Foundation. \n\nReferences \n\nA.  1. Hodgkin and  A.  K.  Huxley, (1952).  A Quantitative Description of Membrane \nCurrent  and  its  Application  to  Conduction  and  Excitation  in  Nerve.  Journal  of \nPhysiology,  117:6,  500-544. \nT.  Horiuchi,  J.  Lazzaro,  A.  Moore,  and  C.  Koch,  (1991) .  A  Delay-Line  Based \nMotion  Detection  Chip.  Advances  in  Neural  Information  Processing  Systems  3. \nSan  Mateo,  CA:  Morgan  Kaufmann Publishers,  Inc.  406-412 . \n\nJ.  Lazzaro and C.  Mead,  (1989).  A Silicon Model  of Auditory  Localization.  Neural \nComputation,  1:1 , 47-57. \n\nR . Lyon  and  C.  Mead,  (1989).  Electronic  Cochlea.  Analog  VLSI  and  Neural  Sys(cid:173)\ntems.  Reading,  MA:  Addison-Wesley Publishing  Company, Inc.  279-302. \n\nE.  R.  Lewis,  (1968).  Using  Electronic  Circuits  to  Model  Simple  Neuroelectric  In(cid:173)\nteractions.  Proceedings  of the  IEEE, 56:6,  931-949. \n\nM.  Mahowald  and  R.  Douglas,  (1991).  A Silicon  Neuron.  Nature , 354:19, 515-518. \n\no .o35 r - - - - - . , - - - - - . , - - - - - - - - ,  \n\n2 \n\n1.5 \n\n~ \n., \n.&  0 .5 \n'0 > \n\n0 \n\n-0.5 \n30 \n\n0.03 \n\n0.025 \n\n0 .02 \n\n~ \nf=  0.015 \n\n0.06 \n\nStage \n\no  0 \n\nTilTIe (s) \n\n\u00b00~--~1~0---~2~0---~30 \n\nStage \n\nFigure 4:  Pulse propagation along a uniform axon.  (Left)  Perspective view.  (Right) \nOverhead  view .  *:  pulse  boundaries,  x:  pulse  width.  T1  = 0.720V,  T2  = 0.780V . \nVelocity = 1, 100stages/s , Width = 3.8ms \n\n\f746 \n\nBradley A.  Minch,  Paul Hasler,  Chris Diorio,  Carver Mead \n\n10\"  r----~--~--~--____, \n\nII< \n\n\"\" \n\"\" \n\"\" \n\"\" \n\"\" \n\"\" \n\nII< \n'ttl\" \n\nII< \n\nII< \n\n\"\" \n\"\" \n... \n\"\" \n\nII< \n\n\"\" \nII< \n~ \n\"\" \n\"\" \n\no o \n\n00  Tau2 =  0 .660 V \n\n~ \n110 \n11<... \n\nTau2 =  0.700 V \n\n~ Xx  Tau2 = 0 .740 \n\n~ + \n\n+d> \n\nTau2 =  0.7  0  V \n\n~o  T a u2 = \n\n.820 V \n\n~ \n\n, \n\nTa\\lp =  0 .860 V \n\n10.2 \n\n3: \n~ \n~ \nl!l \n\u00a3 \n\n10-3 \n\n10~~. 6-------0~.~7-------0~. 8-------0~. 9 \n\n10~~.5-----0~.6-----0~.~7-----0~.8~---0~.9 \n\nTau2 (V) \n\nTaul  (V) \n\nFigure  5:  Uniform  axon.  (Left)  Pulse  velocity  as  a  function  of r2.  (Right)  Pulse \nwidth as  a function  of rl  for  various values of r2. \n\n0 . 14r-------~-------~----~ \n\n1 0\"  r------~--------~------___, \n\n0 .12 \n\n0 .1 \n\nil:!  0 .08 \nE::: \n\nII< \n\n)0( \n\n)0( \n\n)0( \n\n)O()O( \n\n)0( \n\n)0( \n\n)0( \n\n)0( \n\n0.06  x \n\n)0( \n\nx \n\nx \n\nx \n\n0.04# .... \n\n)0( \n\n. . . .   )0()0( \n\nII< \n\n)0( \n\n)0( \n\n)O()O( \n\n)0( \n\nx \n\nx \n><\"XxX \nX \n\nX \n\nX \n\n0 . 020~------~1~0~------2~0~------370 \n\nStage \n\n1 O\u00b73~------,'---------_'_------__=' \n30 \n\n10 \n\n20 \n\no \n\nStage \n\nFigure  6:  (Left)  Pulse  propagation along  a  tapered  axon.  (Right)  Pulse  width  as \na  function  of position  along  a  tapered  axon.  *:  rieft  = 0.770V,  r;ight  = 0.600V , \nr~eft = 0.B20V, r;ight = 0.650.  x :  rieft = 0.600V, r;ight = 0.770V, r~eft = 0.650V, \nr;ight = 0.B20V. \n\n\f", "award": [], "sourceid": 903, "authors": [{"given_name": "Bradley", "family_name": "Minch", "institution": null}, {"given_name": "Paul", "family_name": "Hasler", "institution": null}, {"given_name": "Chris", "family_name": "Diorio", "institution": null}, {"given_name": "Carver", "family_name": "Mead", "institution": null}]}