{"title": "VLSI Implementation of Motion Centroid Localization for Autonomous Navigation", "book": "Advances in Neural Information Processing Systems", "page_first": 685, "page_last": 691, "abstract": null, "full_text": "VLSI Implementation of Motion Centroid \nLocalization for Autonomous Navigation \n\nRalph Etienne-Cummings \n\nDept. of ECE, \n\nViktor Gruev \nDept.  of ECE, \n\nMohammed Abdel Ghani \n\nDept.  ofEE, \n\nJohns Hopkins University, \n\nJohns Hopkins University, \n\nS.  Illinois University, \n\nBaltimore, MD \n\nBaltimore, MD \n\nCarbondale, IL \n\nAbstract \n\nA  circuit  for  fast,  compact and  low-power focal-plane  motion  centroid \nlocalization  is  presented.  This  chip,  which  uses  mixed  signal  CMOS \ncomponents  to  implement  photodetection,  edge  detection,  ON-set \ndetection  and  centroid  localization,  models  the  retina  and  superior \ncolliculus.  The  centroid  localization  circuit  uses  time-windowed \nasynchronously  triggered  row  and  column  address  events  and  two \nlinear  resistive  grids  to  provide  the  analog  coordinates  of the  motion \ncentroid.  This  VLSI  chip  is  used  to  realize  fast  lightweight \nautonavigating  vehicles.  The  obstacle  avoiding  line-following \nalgorithm is  discussed. \n\n1  INTRODUCTION \nMany  neuromorphic chips  which  mimic  the  analog  and  parallel  characteristics  of visual, \nauditory  and  cortical  neural  circuits  have  been  designed  [Mead,  1989;  Koch,  1995] . \nRecently  researchers  have  started  to  combine  digital  circuits  with  neuromorphic  aVLSI \nsystems  [Boahen,  1996].  The  persistent  doctrine,  however,  has  been  that  computation \nshould  be  performed  in  analog,  and  only  communication  should  use  digital  circuits.  We \nhave  argued  that  hybrid  computational  systems  are  better  equipped  to  handle  the  high \nspeed  processing  required  for  real-world  problem  solving,  while  maintaining \ncompatibility  with  the  ubiquitous  digital  computer  [Etienne,  1998].  As  a  further \nillustration  of  this  point  of  view,  this  paper  presents  a  departure  form  traditional \napproaches for focal  plane centroid localization by  offering a  mixed signal  solution that is \nsimultaneously  high-speed,  low  power  and  compact.  In  addition,  the  chip  is  interfaced \nwith an 8-bit microcomputer to  implement fast  autonomous navigation. \n\nImplementation  of centroid  localization  has  been  either completely  analog  or  completely \ndigital.  The analog implementations, realized  in  the early  1990s, used  focal  plane current \nmode  circuits  to  find  a  global  continuos  time  centroid  of  the  pixels'  intensities \n[DeWeerth,  1992].  Due  to  their  sub-threshold  operation,  these  circuits  are  low  power, \nbut slow.  On the  other hand,  the  digital  solutions do  not compute the  centroid at the  focal \n\n\f686 \n\nR.  Etienne-Cummings,  V.  Grnev and M  A. Ghani \n\nplane.  They  use  standard  CCO cameras,  AID  converters  and  OSP/CPU  to  compute  the \nintensity  centroid  [Mansfield,  1996].  These software  approaches  offer  multiple  centroid \nlocalization with complex mathematical processing.  However,  they  suffer from the usual \nhigh  power  consumption  and  non-scalability  of  traditional  digital  visual  processing \nsystems.  Our  approach  is  novel  in  many  aspects.  We  benefit  from  the  low  power, \ncompactness  and  parallel  organization  of  focal  plane  analog  circuits  and  the  speed, \nrobustness  and  standard  architecture  of asynchronous  digital  circuits.  Furthermore,  it \nuses  event  triggered  analog  address  read-out,  which  is  ideal  for  the  visual  centroid \nlocalization  problem.  Moreover,  our chip  responds  to  moving  targets  only  by  using  the \nON-set of each pixel  in  the  centroid computation.  Lastly,  our chip models  the  retina and \ntwo  dimensional  saccade  motor  error  maps  of  superior  colliculus  on  a  single  chip \n[Sparks,  1990] .  Subsequently,  this chip is  interfaced  with  a IlC for autonomous obstacle \navoidance  during  line-following  navigation.  The  line-following  task  is  similar to  target \ntracking  using  the  saccadic  system,  except  that  the  \"eye\"  is  fixed  and  the  \"head\"  (the \nvehicle) moves to  maintain fixation  on the  target.  Control signals provided to  the  vehicle \nbased on  decisions made by  the IlC are  used for  steering and  accelerating/braking.  Here \nthe  computational  flexibility  and  programmability  of the  IlC  allows rapid  prototyping  of \ncomplex and robust algorithms. \n\n2  CENTROID LOCALIZATION \nThe  mathematical  computation  of  the  centroid  of  an  object  on  the  focal  plane  uses \nintensity  weighted  average  of the  position  of the  pixels  forming  the  object  [OeWeerth, \n1992].  Equation (1) shows this formulation.  The implementation of this representation \n\nN \n\nx=  j =1 \n\nLI,x, \nL,I, \n\nN \n\n,=1 \n\nN \n\nLI,y; \n\nand  y=  ;=1 \n\nN \n\n(1) \n\nL I, \n\n1=1 \n\ncan  be quite  involved  since  a product  between the  intensity  and  position  is  implied.  To \neliminate this  requirement,  the  intensity  of the pixels can be  normalized to  a single value \nwithin  the  object.  This  gives  equation  (2)  since  the  intensity  can  be  factored  out of the \nsummations.  Normalization of the intensity using a simple threshold is  not advised since \n\nIx, \n\nN \n\nIy, \n\nN \n\nx=~ and  y=~ \n\n(2) \n\nIntensity Image \n\nFigure 1:  Centroid computation \n\narchitecture. \n\nX I \n\nX l+l \n\nX 1+2 \n\nX I+3 \nEdges from pixels \n\nx1+4 \n\nFigure 2:  Centroid computation method. \n\nthe  value  of the  threshold  is  dependent  on  the  brightness  of the  image  and  number  of \npixels  forming  the  object  may  be  altered  by  the  thresholding  process.  To  circumvent \nthese problems,  we  take the view  that the centroid of the object is  defined in relation to its \nboundaries.  This implies that edge detection (second order spatial derivative of intensity) \ncan  be  used  to  highlight  the  boundaries,  and  edge  labeling  (the  zero-crossing  of the \nedges) can be  used  to  normalize  the magnitude  of the  edges.  Subsequently,  the  centroid \n\n\fVLSI Implementation of Motion  Centroid Localization for Autonomous Navigation \n\n687 \n\nof the  zero-crossings  is  computed.  Equation  (2)  is  then  realized  by  projecting  the  zero(cid:173)\ncrossing  image onto the  x- and  y-axis  and  performing two  linear centroid determinations. \nFigure (1) shows this process. \n\nThe  determination  of the  centroid  is  computed  using  a  resistance  grid  to  associate  the \nposition  of a  column  (row)  with  a  voltage.  In  figure  2,  the  positions  are  given  by  the \nvoltages  Vi .  By  activating  the  column  (row)  switch  when  a  pixel  of the  edge  image \nappears in  that column (row),  the position voltage is  connected to  the output line through \nthe switch impedance, Rs.  As  more switches are  activated,  the voltage on the output line \napproximates  equation  (2).  Clearly,  since  no  buffers  are  used  to  isolate  the  position \nvoltages,  as  more  switches  are  activated,  the  position  voltages  will  also  change.  This \ndoes not pose a problem since the switch resistors are design to  be larger than the position \nresistors  (the switch currents are small compared to  the  grid  current).  Equation (3)  gives \nthe  error  between  the  ideal  centroid  and  the  switch  loaded  centroid  in  the  worst  case \nwhen Rs  = on.  In  the  equation,  N  is  the  number of nodes,  M  is  the  number  of switches \nset and  Xl  and  xM  are  the  locations  of the  first  and  last set switches,  respectively.  This \n\nerror  is  improved  as  Rs  gets  larger,  and  vanishes  as  N  (M~N) approaches  infinity .  The \n\nterms  Xi  represent an  ascending ordered list  of the  activated switches;  x I  may correspond \nto  column  five,  for  example.  This  circuit is  compact  since  it  uses  only  a  simple  linear \nresistive grid and MOS  switches.  It is  low  power because the total grid resistance, N x R, \ncan  be  large.  It can  be  fast  when  the  parasitic capacitors  are  kept  small.  It provides  an \nanalog position value, but it is triggered by fast digital signals that activate the switches. \n\nerror =  mOll \n\nV  -V  ~[ \nM(N + 1)  .=1 \n\nmin  \u00a3...J  X  __  --'-1-'--_-'--_ \n\u2022  N + 1 + XI  - Xm \n\nxCN+l)  1 \n\n(3) \n\n3  MODELING THE RETINA AND SUPERIOR COLLICULUS \n3.1  System Overview \n\nThe centroid  computation approach  presented in  section 2  is  used  to  isolate  the  location \nof moving  targets  on  a  20  focal  plane  array.  Consequently,  a  chip  which  realizes  a \nneuromorphic  visual  target  acquisition  system  based  on  the  saccadic  generation \nmechanism of primates can be implemented. The biological saccade generation process is \nmediated  by  the  superior  colliculus,  which  contains  a  map  of the  visual  field  [Sparks, \n1990}. In  laboratory  experiments,  cellular recordings  suggest  that  the  superior colliculus \nprovides  the  spatial  location  of targets  to  be  foveated.  Clearly,  a  great  deal  of  neural \ncircuitry exists  between the superior colliculus and the  eye muscle.  Horiuchi has  built an \nanalog  system  which  replicates  most of the  neural  circuits  (including  the  motor system) \nwhich  are  believed  to  form  the  saccadic  system  [Horiuchi,  1996].  Staying  true  to  the \nanatomy forced  his  implementation to  be  a complex multi-chip system with many control \nparameters.  On  the  other hand,  our  approach  focuses  on realizing  a compact  single  chip \nsolution by  only mimicking the behavior of the saccadic system,  but not its structure. \n\n3.2  Hardware Implementation \n\nOur  approach  uses  a  combination  of  analog  and  digital  circuits  to  implement  the \nfunctions  of the  retina  and  superior colliculus  at  the  focal  plane. We  use  simple  digital \ncontrol  ideas,  such  as  pulse-width  modulation  and stepper motors,  to  position  the \"eye\". \nThe retina portion of this chip uses  photodiodes,  logarithmic compression,  edge detection \nand zero-crossing circuits.  These circuits mimic the first three  layers of cells in the retina \n\n\f688 \n\nR.  Etienne-Cummings,  V.  Grnev and M. A.  Ghani \n\nwith  mixed  sub-threshold  and  strong  inversion  circuits.  The  edge  detection  circuit  is \nrealized  with  an  approximation  of  the  Laplacian  operator  implemented  using  the \ndifference  between a smooth (with a resistive grid)  and  unsmoothed  version of the image \n[Mead,  1989].  The  high  gain  of  the  difference  circuit  creates  a  binary  image  of \napproximate  zero-crossings.  After this  point,  the  computation is  performed  using  mixed \nanalog/digital  circuits.  The  zero-crossings  are fed  to  ON-set detectors  (positive temporal \nderivatives)  which  signal  the  location of moving or flashing  targets. These circuits model \nthe  behavior of some  of the  amacrine  and  ganglion  cells  of the  primate retina [Barlow, \n1982].  These  first  layers  of  processing  constitute  all  the  \"direct\"  mimicry  of  the \nbiological models. Figure 3 shows the schematic of these early processing layers. \n\nThe  ON-set detectors  provide  inputs to  the model  of the  superior colliculus circuits.  The \nON-set detectors  allow  us  to  segment moving targets against textured  backgrounds.  This \nis  an  improvement  on  earlier  centroid  and  saccade  chips  that  used  pixel  intensity.  The \nessence of the superior colliculus map is  to  locate  the  target that is  to  be foveated.  In  our \ncase,  the  target chosen  to  be  foveated  will  be  moving.  Here  motion  is  define  simply  as \nthe  change in contrast over time.  Motion,  in  this  sense,  can  be  seen  as  being  the earliest \nmeasurable attribute  of the  target  which  can  trigger a saccade  without requiring any  high \nlevel  decision making. Subsequently, the coordinates of the motion must be  extracted and \nprovided to  the motor drivers. \n\nX  M otion Cenuold \n\n~! ~A  ! \n\nEdge Detc:ctlon \nON-set Detecu on \n\nFigure 3:  Schematic of \nthe model of the retina. \n\nFigure 4:  Schematic of the model of the superior \n\ncollicu Ius. \n\nThe  circuits  for  locating  the  target  are  implemented  entirely  with  mixed  signal,  non(cid:173)\nneuromorphic  circuits.  The  theoretical  foundation  for  our  approach  is  presented  in \nsection 2.  The ON-set detector is  triggered when  an  edge of the  target appears at a pixel. \nAt this  time,  the  pixel  broadcasts  its  location  to  the edge  of the  array  by  activating  row \nand column lines.  This row  (column)  signal  sets a latch at the right (top)  of the array. The \nlatches  asynchronously  activate  switches  and  the  centroid  of the  activated  positions  is \nprovided.  The latches remain set until  they  are cleared by  an  external control  signal.  This \ncontrol  signal  provides a time-window over which  the centroid output is  integrated.  This \nhas the effect of reducing  noise by  combining the  outputs of pixels which  are activated at \ndifferent instances even if they  are triggered  by  the  same motion  (an  artifact of small  fill \nfactor  focal  plane  image  processing).  Furthermore,  the  latches  can  be  masked  from  the \npixels'  output with  a second control signal.  This signal  is used  to de-activate the  centroid \n\n\fVLSI Implementation of Motion  Centroid Localization for Autonomous Navigation \n\n689 \n\ncircuit during  a  saccade  (saccadic  suppression).  A  centroid  valid  signal  is  also generated \nby the chip. Figure 4 shows a portion of the schematic of the superior colliculus model. \n\n3.3  Results \n\nIn  contrast to  previous  work,  this  chip provides  the  2-D  coordinates of the  centroid of a \nmoving target.  Figure 5 shows the  oscilloscope trace of the coordinates as  a target moves \nback  and  forth,  in  and  out  of the  chip's  field  of view.  The  y-coordinate  does  change \nwhile  the  x-coordinate  increases  and  decreases  as  the  target moves  to  the  left  and  right, \nrespectively.  The chip  has  been  used  to  track targets  in  2-D  by  making  micro-saccades. \nIn  this  case,  the chip chases  the  target  as  it  attempts  to  escape  from  the  center.  The eye \nmovement is  performed by  converting  the  analog  coordinates  into  PWM signals,  which \nare  used  to  drive  stepper  motors.  The  system  performance  is  limited  by  the  contrast \nsensitivity  of the  edge  detection  circuit,  and  the  frequency  response  of the  edge  (high \nfrequency  cut-off)  and  ON-set  (low  frequency  cut-off)  detectors.  With  the  appropriate \noptics,  it  can  track  walking  or  running  persons  under  indoor  or  outdoor  lighting \nconditions at close or far distances.  Table I gives a summary of the chip characteristics. \n\nVarYing x\u00b7 \nl'Oordmate \n\nTechnology \nChip Size \nArray Size \nPixel Size \nFill  Factor \nIntensity \nMin. Contrast \nResponse Time \nPower (chip) \n\n1.2um ORBIT \n4mm 1 \n12 x  10 \nllOxllOutn \n11% \n0.lu-1OOmW/cm 2 \n10% \n2-106 Hz(@1  mW/cml) \n5 mW (@l  m W/cm~ Vdd = 6V) \n\nFigure 5:  Oscilloscope trace of 20 \n\ncentroid for  a moving target. \n\nTable I:  Chip characteristics. \n\n4  APPLICATION: OBSTACLE A VOIDANCE DURING LINE(cid:173)\n\nFOLLOWING AUTONA VIGATION \n\n4.1  System Overview \n\nThe frenzy  of activity  towards  developing  neuromorphic  systems  over the  pass  10  years \nhas  been mainly driven by  the  promise that one  day  engineers will develop machines that \ncan  interact with the environment in  a similar way  as  biological organisms.  The prospect \nof having  a  robot  that can  help  humans  in  their daily  tasks  has  been  a  dream  of science \nfiction  for  many  decades.  As  can  be  expected,  the  key  to  success  is  premised  on  the \ndevelopment  of compact systems,  with  large  computational  capabilities,  at  low  cost (in \nterms  of hardware and power). Neuromorphic VLSI systems have closed the gap between \ndreams  and  reality,  but we  are  still  very  far  from  the  android robot.  For all  these  robots, \nautonomous  behavior,  in  the  form  of auto-navigation  in  natural  environments,  must  be \none  of their primary  skills.  For miniaturization,  neuromorphic vision systems performing \nmost  of the  pre-processing,  can  be  coupled  with  small  fast  computers  to  realize  these \ncompact yet powerful sensor/processor modules. \n\n4.2  Navigation Algorithm \n\nThe simplest form  of data driven  auto-navigation  is  the  line-following task.  In  this  task, \nthe  robot must maintain a certain relationship with some visual  cues that guide its  motion. \nIn  the case  of the  line-follower,  the  visual  system provides data regarding  the  state  of the \n\n\f690 \n\nR.  Etienne-Cummings,  V.  Gruev and M  A.  Ghani \n\nline relative to  the vehicle,  which results  in controlling steering and/or speed.  If obstacle \navoidance  is  also  required,  auto-navigation  is  considerably  more  difficult.  Our  system \nhandles line-following and obstacle avoidance by  using two  neuromorphic  visual  sensors \nthat  provide  information to  a micro-controller OlC)  to  steer,  accelerate or decelerate  the \nvehicle.  The  sensors,  which  uses  the  centroid  location  system  outlined  above,  provides \ninformation  on  the  position  of the  line  and  obstacles  to  the  p,C,  which  provides  PWM \nsignals  to  the  servos  for  controlling  the  vehicle.  The  algorithm  implemented  in  the  p,C \nplaces  the  two  sensors  in  competition  with  each  other to  force  the  line  into  a blind  zone \nbetween  the  sensors.  Simultaneously,  if an  object enters  the  visual  field  from  outside,  it \nis  treated  as  an  obstacle  and  the  p,C  turns  the  car  away  from  the  object.  Obstacle \navoidance  is  given  higher  priority  than  line-following  to  avoid  collisions.  The  p,C  also \nkeeps  track of the direction of avoidance such that the vehicle can  be  re-oriented towards \nthe  line  after the  obstacle is  pushed out of the  field  of view.  Lastly,  for  line  following, \nthe  position, orientation and  velocity of drift,  determined from  the temporal derivative of \nthe centroid,  are  used  to  track the line. The control strategy  is to  keep the line in the  blind \nzone,  while  slowing  down  at  corners,  speeding  up  on  straight  aways  and  avoiding \nobstacles.  The  angle  which  the  line  or  obstacle  form  with  the  x-axis  also  affects  the \nspeed.  The  value  of  the  x-centroid  relative  to  the  y-centroid  provides  rudimentary \nestimate of the orientation of the line or obstacle to  the vehicle.  For example, angles less \n\nL Zone \n~ne \ni\n\n: \n\n/  AV~na;ce \n! \n! \n\n. \nAV~8id~nce \n\n\" \n\n..... \n, \n\nFollow \n\n0I0s1ade \n\n'. )\", / ~\\?': ~ .... / \n;i:~~~~\u00b7\u00b7\u00b7 ... \\j  ~ \\;.. .. \u00b7\u00b7\u00b7\u00b7\u00b7\u00b7~i=~s..s \n\nFigure 6:  Block diagram of the \nautonomous line-follower system. \n\nFigure 7:  A picture of the vehicle. \n\n(greater) than  +/- 45  degrees tend to  have  small  (large)  x-coordinates and  large  (small) y(cid:173)\ncoordinates  and  require  deceleration  (acceleration).  Figure  6  shows  the  organization  of \nthe  sensors  on  the  vehicle  and  control  spatial  zones.  Figure  7  shows  the  vehicle  and \nsamples of the line and obstacles. \n\n4.3  Hardware Implementation \n\nThe  coordinates  from  the  centroid  localization  circuits  are  presented  to  the  p,C  for \nanalysis.  The  p,C  used  is  the  Microchip  PIC16C74.  This  chip  is  chosen  because  of its \nfive NO inputs and  three PWM outputs.  The analog coordinates are presented directly to \nthe NO inputs.  Two of the PWM  outputs are  connected to  the steering and speed control \nservos.  The PIC16C74 runs  at 20 MHz  and has  35  instructions, 4K by  8-b ROM  and  80 \nby  20-b RAM.  The program which runs on the  PIC determines the control action to  take, \nbased  on  the  signal  provided  by  the  neuromorphic  visual  sensors.  The  vehicle  used  is  a \nfour-wheel  drive  radio  controlled  model  car  (the  radio  receiver  is  disconnected)  with \nDigital Proportional Steering (DPS). \n\n\fVLSI Implementation of Motion  Centroid Localization for Autonomous Navigation \n\n691 \n\n4.4  Results \n\nThe vehicle was  tested on a  track composed of black tape  on  a  gray  linoleum floor  with \nblack and white obstacles.  The track formed  a closed loop with two sharp turns  and some \nsmooth S-curves.  The  neuromorphic  vision chip was  equipped  with  a  12.5  mm variable \niris  lens,  which  limited  its  field  of view  to  about  100.  Despite the  narrow  field  of view , \nthe  car was  able  to  navigate  the  track at  an  average  speed of 1 mls  without making  any \nerrors.  On less curvy parts of the track,  it accelerated to  about 2  mls and slowed down at \nthe corners.  When  the  speed  of the  vehicle is  scaled up,  the  errors made  are mainly  due \nto over steering. \n\n5  CONCLUSION \nA 2D  model  of the  saccade generating components of the superior colliculus is  presented. \nThis  model  only  mimics  the  functionality  the  saccadic  system  using  mixed  signal  focal \nplane  circuits  that  realize  motion  centroid  localization.  The  single  chip  combines  a \nsilicon  retina  with  the  superior  colliculus  model  using  compact,  low  power  and  fast \ncircuits.  Finally, the centroid chip is interfaced with an  8-bit IlC and vehicle for fast line(cid:173)\nfollowing  auto navigation with  obstacle avoidance.  Here all  of the required computation is \nperformed at the visual sensor, and a standard IlC is  the high-level decision maker. \n\nReferences \n\nBarlow  H.,  The  Senses:  Physiology  of  the  Retina,  Cambridge  University  Press, \n\nCambridge, England,  1982. \n\nBoahen K.,  \"Retinomorphic Vision  Systems II: Communication Channel  Design,\"  ISCAS \n\n96, Atlanta, GA,  1996. \n\nDeWeerth,  S.  P.,  \"Analog  VLSI  Circuits  for  Stimulus  Localization  and  Centroid \n\nComputation,\" Int'l 1. Computer Vision,  Vol.  8,  No.2, pp.  191-202,  1992. \n\nEtienne-Cummings  R.,  J  Van  der  Spiegel  and  P.  Mueller,  \"Neuromorphic  and  Digital \nHybrid  Systems,\"  Neuromorphic  Systems:  Engineering  Silicon from  Neurobiology, L. \nSmith and A.  Hamilton (Eds.), World Scientific, 1998. \n\nHoriuchi  T.,  T.  Morris,  C .  Koch  and  S.  P .  DeWeerth,  \"Analog  VLSI  Circuits  for \nAttention-Based  Visual  Tracking,\"  Advances  in  Neural  Information  Processing \nSystems, Vol. 9, Denver, CO,  1996. \n\nKoch  C.  and  H.  Li  (Eds.),  Vision  Chips:  Implementing  Vision  Algorithms  with  Analog \n\nVLSI Circuits, IEEE Computer Press,  1995. \n\nMansfield, P.,  \"Machine Vision Tackles Star Tracking,\"  Laser Focus  World,  Vol.  30,  No. \n\n26, pp. S21 -S24,  1996. \n\nMead C.  and  M.  Ismail (Eds.), Analog  VLSI  Implementation of Neural Networks,  Kluwer \n\nAcademic Press, Newell, MA,  1989. \n\nSparks  D.,  C.  Lee and W. Rohrer,  \"Population Coding  of the  Direction,  Amplitude  and \nVelocity  of Saccadic  Eye  Movements  by  Neurons  in  the  Superior  Colliculus,\" Proc. \nCold Spring Harbor Symp.  Quantitative Biology, Vol. LV,  1990. \n\n\f", "award": [], "sourceid": 1499, "authors": [{"given_name": "Ralph", "family_name": "Etienne-Cummings", "institution": null}, {"given_name": "Viktor", "family_name": "Gruev", "institution": null}, {"given_name": "Mohammed", "family_name": "Ghani", "institution": null}]}