{"title": "Implementation of Neural Hardware with the Neural VLSI of URAN in Applications with Reduced Representations", "book": "Advances in Neural Information Processing Systems", "page_first": 811, "page_last": 815, "abstract": null, "full_text": "Implementation  of  Neural  Hardware  with \nthe  Neural  VLSI  of URAN  in  Applications \nwith  Reduced  Representations \n\nll-Song  Han \n\nKorea  Telecom  Research  Laboratories \n17,  Woomyun-dong,  Suhcho-ku \n\nSeoul  137-140,  KOREA \n\nHwang-Soo  Lee \n\nDept.  of  Info  and  Comm \n\nKAIST \n\nSeoul,  130-012,  Korea \n\nKi-Chul Kim \n\nDept.  of  Info  and Comm \n\nKAIST \n\nSeoul,  130-012,  Korea \n\nAbstract \n\nimplement  Korean \n\nThis  paper  describes \na  way  of  neural  hardware \nimplementation  with  the  analog-digital  mixed  mode  neural \nchip.  The \nfull  custom  neural  VLSI  of  Universally \nReconstructible  Artificial  Neural  network (URAN)  is  used \nsystem.  A \nto \nmulti-layer  perceptron  with \nis \ntrained \nsuccessfully  under  the  limited  accuracy  in  computations. \nThe  network  with  a  large  frame  input  layer  is  tested  to \nrecognize  spoken  korean  words  at  a  forward  retrieval. \nMultichip  hardware  module  is  suggested  with  eight  chips \nor  more  for  the  extended  performance  and  capacity. \n\nlinear  neurons \n\nspeech \n\nrecognition \n\n\f812 \n\nll-Song  Han,  Hwang-Soo  Lee,  Ki-Chul  Kim \n\n1  INTRODUCTION \n\nIn  general,  the  neural  network  hardware  or  VLSI  has  been  preferred  in \nrespects  of  its  relatively  fast  speed,  huge  network  size  and  effective  cost \ncomparing  to  software  simulation.  Universally  Reconstructible  Artificial \nthe  new  analog-digital  mixed  VLSI  neural \nNeural-network(URAN), \nnetwork,  can  be  used  for  the  implementation  of  the  real  world  neural \nnetwork  applications  with  digital  interface.  The  basic  electronic  synapse \ncircuit  is  based  on  the  electrically  controlled  MOSFET  resistance  and  is \noperated  with  discrete  pulses. \n\nThe  URAN's  adaptability  is  tested  for  the  multi-layer  perceptron  with \nthe  reduced  precision  of  connections  and  states.  The  linear  neuron \nfunction  is  also  designed  for  the  real  world  applications.  The  multi-layer \nnetwork  with  back  propagation  learning  is  designed  for  the  speaker \nindependent  digit/word  recognition.  The  other  case  of  application  is  for \nthe  servo  control,  where  the  neural  input  and  output  are  extended  to \n360 \nthe  servo  control \nsimulation,  the  flexibility  of  URAN  is  proved  to  extend  the  accuracy  of \ninput  and  output  from  external. \n\nthe  suitable  angle  control.  With \n\nlevels \n\nfor \n\n2.  Analog-Digital  Mixed  Chip  - URAN \n\nIn  the  past,  there  have  been  improvements  in  analog  or  analog-digital \nmixed  VLSI  chips.  Analog  neural  chips  or  analog-digital  mixed  neural \nchips  are  still  suffered  from  the  lack  of  accuracy,  speed  or  flexibility. \nWith  the  proposed  analog-digital  mixed  neural  network  circuit  of  URAN, \nlinear \nthe  accuracy \nsynapse  weight  emulation.  The  speed  in \nMOSFET  resistance  for  the \nimproved  by  using \nthe  simple  switch \nneural  computation \nis  also \ncontrolled  by  the  neural  input  as  described  in  previous  works. \n\nthe  voltage-controlled \n\nimproved  by  using \n\nis \n\nThe  general  flexibility  is  attained  by  the  independent  characteristic  of \neach  synapse  cell  and  the  modular  structure  of  URAN  chip.  As  in  Table \n1  of  URAN  chip  feature,  the  chip  is  operated  under  the  flexible  control, \nthat  is,  the  various  mode  of  synaptic  connection  per  neuron  or  the \nextendable  weight  accuracy  can  be  implemented.  It  is  not  limited  for  the \nasynchronous/direct  interchip  expansion  in  size  or  speed.  In  fact,  16  fully \nconnected  module  of  URAN  is  selected  from  external  and  independently \n- it  is  possible  to  select  either  one  by  one  or  all  at  once. \n\nTable  1.  URAN  Chip  Features \n\nTotal  Synapses \nComputation  Speed \nWeight  Accuracy \nModule  No. \nModule  Size \n\n135,424  connections \n200  Giga  Connections  Per  S \n8  Bit \n16 \n92  X  92 \n\n\fImplementation of Neural Hardware with the Neural VLSI of URAN \n\n813 \n\nAs  all  circuits  over  the  chip  except  digital  decoder  unit  are  operated  in \nanalog  transistor  level,  the  computation  speed  is  relatively  high  and  even \ncan  be  improved  substantially.  The  cell  size  including  interconnection \narea  in  conventional  short-channel  technology  is  reduced  less  than  900  1.1 \nm2.  From  its  expected  and  measured  linear  characteristic,  URAN  has  the \naccuracy  more  than  256  linear  levels. \n\ninherent \nin \nThe  accuracy  extendability  and  flexible  modularity  are \nelectrical  wired- OR  characteristics  as  each  synapse  is  an \nindependent \nbipolar  current  source  with  switch.  No  additional  clocking  or  any  limited \nsynchronous  operation  is  required  in  this  case,  while  it  is  indispensible \nin  most  of  conventional  digital  neural  hardware  or  analog-digital  neural \nchip.  Therefore,  any  size  of  neural  network  can  be  integrated  in  VLSI \nor  module  hardware  merely  by  placing  the  cell  in  2  dimensional  array \nwithout  any  timing  limitation  or  loading  effect. \n\n3.  Neural  Hardware  with  URAN  - Module  Expansion \n\nURAN  is  the  full  custom  VLSI  of  analog-digital  mixed  operation.  The \nprototype  of  URAN  chip  is  fabricated  in  1.0tI  digital  CMOS  technology. \nThe  chip  contains  135,424  synapses  with  8  bit  weight  accuracy  on  a  13 \nX  13  mm2  die  size  using  single  poly  double  metal  technology.  As \nsummarized  in  Table  1  of  chip  features, \nthe  chip  allows  the  variety  of \nIn  the  prototype  chip,  16  fully  connected  module  of  92  X \nconfiguration. \nselecting \n92  can  be  selected \nindependent  module  either  one  by  one,  several \nIS \npossible. \n\nfrom  external  and \n\nor  all  at  a  time \n\nindependently \n\n-\n\nWith  URAN's  synapse  circuit  of  linear  Voltage-controlled  bipolar  current \nsource,  the  synaptic  multiplication  with  weight  value  is  done  with  the \nswitching  transistor,  in  a  similar  way  of  analog-sampled  data  type.  The \naccuracy  enhancement  and  flexible  modularity  of  URAN  are  inherent  in \nits  electrical  wired-OR  interface  from  each  independent  bipolar  current \nneural  network  hardware  module  can  be  realized  in \nsource.  And  the \nany  size  with  the  multi  URAN  chips. \n\n4.  Considerations  on  the  Reduced  Precision \n\nURAN  chip  is  applied  for \nthe  case  of  Korean  speaker  independent \nspeech  recognition.  By  changing  numbers  of  hidden  units  and  input \naccuracy,  the  result  of  simulations  has  not  shown  any  problems  in \nIt  means  that  the  overall  performance  is  not \nrecognition  accuracy. \nseverely  affected  from  the  accuracy  of  weight,  input,  and  output  with \nURAN.  Also,  it  was  possible  to  train  with  2  or  1  decimal  accuracy  for \ninput  and  output,  which  is  equivalent  to  8  bit  or  4  bit  precisions.  With \n20  hidden  units  for  the  Korean  spoken  10  digit  recognition,  2  decimal \ninput  accuracy  yields  99.2%  and  l ' decimal  input  accuracy  yields  98.6%, \n\n\f814 \n\nII-Song  Han,  Hwang-Soo  Lee,  Ki-Chul Kim \n\nwhile  binary  I-bit  input  results  96.6%. \n\nThe  following \nresult  is  summarized  in  Table  2. \n\nis  the  condition  for  the  experimentation.  The  general \n\nConditions  for  Training  and  Test \n\u2022  2,000  samples  from  10  women  and  10  men \n\n(  10  times  X  10  digits  X  20  persons) \n\n[]  Training  with  500  spoken  samples  of  10  digits  in  Korean  from  10 \npersons \n\n(5  times  X  10  digits  X  [  5  women  and  5  men  ]  )  from  2,000 \n\nsamples \n[]  Recognition  Test  with  1,000  spoken  samples  from \n\nthe  other  10  persons  of  women  and  men. \n\nPreprocessing  of samples \n[]  sampled  at  10KHz  with  12bit  accuracy \n[]  preemphasis  with  0.95 \n[]  Hamming  window  of  20ms \n[]  17  channel  critical-band  filter  bank \n[]  noise  added  for  the  SNR  of  3OdB,  2OdB,  10dB,  OdB \n\nTable  2.  Low  Accuracy  Connection  with  Linear Neuron \n\nSNR  Ratio \n\nclean \n30  dB \n20  dB \n10  dB \no dB \n\nInput  /  Output  Accuracy \n\n2  decimal \n\n1  decimal \n\n1  bit \n\n97.5% \n96.2% \n90.1% \n59.8% \n30.8% \n\n97.2% \n96.6% \n91.3% \n59.9% \n29.5% \n\n90.7% \n90.5% \n86.6% \n68.0% \n38.5% \n\nindustrial  purpose  is \nIn  case  of  servo  control,  the  digital  VCR  for \nSix  inputs  are  used  to  minimize  the \nmodelled  for  the  application. \nnumber  of  hidden  units  and  20  hidden  units  are  configured  for  one \noutput.  For  the  adaptation  to  URAN,  the  linear  neuron  function  is  used \nduring  the  simulation.  The  weight  accuracy  during  the  learning  phase \nusing  conventional  computer  is  4  byte  and  that  in  the  recall  phase  using \nURAN  chip  is  1  byte.  With  this  limitation,  the  overall  performance  is \nnot  severely  degraded,  that  is,  the  reduction  of  error  is  attained  up  to \n70%  improvement  comparing  to  the  conventional  method.  The  nonideal \nfactor  of  30%  results  from  the  limitation  in  learning  data  as  well  as  the \nlimited  hardware.  Current  results  are  suitable  for  the  digital  VCR  or \ncompact  cam coder  in  noisy  environment \n\n\fImplementation of Neural Hardware with the Neural VLSI of URAN \n\n815 \n\n5.  Conclusion \n\nthe  application  to  the \nIn  this  paper,  it  is  proved  to  be  suitable  for \nmulti-layer  perceptron  with  the  use  of  URAN  chip,  which  is  fabricated \nin  conventional  digital  CMOS \n1.0#  single  poly  double \nmetal.  The  reduced  weight  accuracy  of  1  byte  is  proved  to  be  enough \nto  obtain  high  perfonnance  using  the  linear  neuron  and  URAN. \n\ntechnology  -\n\nWith  8  test  chips  of  135,424  connections,  it is  now  under  development  of \nthe  practical  module  of  neural  hardware  with  million  connections  and \ntera  connections  per  second  - comparable  to  the  power  of  biological \nneuro-system  of  some  insects.  The  size  of  the  hardware  is  smaller  than \nA4  size  and  is  designed  for  more  general  recognition  system.  The \nflexible  modularity  of  URAN  makes  it  possible  to  realize  a  1,000,000 \nconnections  neural  chip  in  0.5#  CMOS  technology  and  a  general  purpose \nneural  hardware  of  hundreds  of  tera  connections  or  more. \n\nReferences \n\nII  Song  Han  and  Ki-Hwan  Ahn, \nImplementation  of  Analog-Digital  Mixed  Operation \n100,000  Connections\"  MicroNeuro'93,  pp.  159-162,  1993 \n\n\"Neural  Network  VLSI  Chip \nthan \n\nfor  more \n\nM.  Brownlow,  L.  Tara s senko ,  A.  F.  Murray,  A.  Hamilton,  I  SHan,  H. \nM.  Reekie,  \"Pulse  Firing  Neural  Chips  Implementing  Hundreds  of \nNeurons,\"  NIPS2,  pp.  785-792,  1990 \n\n\f\f", "award": [], "sourceid": 886, "authors": [{"given_name": "Il", "family_name": "Han", "institution": null}, {"given_name": "Ki-Chul", "family_name": "Kim", "institution": null}, {"given_name": "Hwang-Soo", "family_name": "Lee", "institution": null}]}