{"title": "Grammar Transfer in a Second Order Recurrent Neural Network", "book": "Advances in Neural Information Processing Systems", "page_first": 67, "page_last": 73, "abstract": null, "full_text": "Grammar  Transfer  in  a  Second  Order \n\nRecurrent  Neural  Network \n\nMichiro  N egishi \n\nDepartment of Psychology \n\nRutgers  University \n\nStephen Jose Hanson \nPsychology Department \n\nRutgers  University \n\n101  Warren St.  Smith Hall  #301 \n\n101  Warren St.  Smith Hall  #301 \n\nNewark,  NJ 07102 \n\njose@psychology.rutgers.edu \n\nNewark,  NJ 07102 \n\nnegishi@psychology.rutgers.edu \n\nAbstract \n\nIt  has  been  known  that  people,  after  being  exposed  to  sentences \ngenerated  by  an  artificial  grammar,  acquire  implicit  grammatical \nknowledge and are able to transfer the knowledge to inputs that are \ngenerated  by  a  modified  grammar.  We  show  that  a  second  order \nrecurrent neural network is able to transfer grammatical knowledge \nfrom one language (generated by a Finite State Machine) to another \nlanguage which differ  both in vocabularies and syntax.  Representa(cid:173)\ntion of the grammatical knowledge in the network is analyzed using \nlinear  discriminant analysis. \n\n1 \n\nIntroduction \n\nIn the field  of artificial grammar learning,  people are known  to be able to transfer \ngrammatical knowledge  to a  new  language which  consists  of a  new  vocabulary  [6]. \nFurthermore,  this  effect  persists  even  when  the  new  strings  violate  the  syntactic \nrule slightly as long  as  they are similar to the old  strings  [1].  It has  been  shown in \nthe  past  studies  that recurrent neural  networks  also  have  the  ability  to  generalize \npreviously  acquired  knowledge  to  novel  inputs.  For  instance,  Dienes  et  al. \n([2]) \nshowed  that  a  neural  network  can  generalize  abstract  knowledge  acquired  in  one \ndomain  to  a  new  domain.  They  trained  the  network  to  predict  the  next  input \nsymbol in grammatical sequences in the first  domain,  and showed that the network \nwas  able  to  learn  to  predict  grammatical  sequences  in  the  second  domain  more \neffectively  than  it  would  have  learned  them  without  the  prior  learning.  During \nthe training in  the second  domain, they had to freeze  the  weights  of a  part of the \nnetwork to prevent catastrophic forgetting.  They used this simulation paradigm to \nemulate and analyze domain transfer,  effect  of similarity between training and test \nsequences,  and the effect of n-gram information in human data.  Hanson  et  al.  ([5]) \nalso  showed  that  a  prior  learning  of a  grammar facilitates  the  learning  of a  new \ngrammar in the cases where either the syntax or the vocabulary was kept constant. \n\nIn this study we investigate grammar transfer by a neural network, where both syn(cid:173)\ntax and vocabularies are different from the source grammar to the target grammar. \nUnlike Dienes  et al.'s network, all weights in the network are allowed to change dur-\n\n\fing the learning of the target grammar, which  allows  us  to investigate interference \nas  well  as transfer from  the source grammar to the target grammar. \n\n2  Simulation Design \n\n2.1  The  Grammar Transfer  Task \n\nIn  the  following  simulations,  a  neural  network  is  trained  with  sentences  that  are \ngenerated by  a  Finite  State Machine  (FSM)  and  is  tested  whether  the  learning of \nsentences generated by another FSM is  facilitated.  Four pairs of FSMs used for  the \ngrammar transfer task are shown in Fig.  2.  In each FSM diagram, symbols (e.g.  A, \nB, C,  ... )  denote words, numbers represent states, a state number with an incoming \narrow  with  no  state  numbers  at  the  arrow  foot  (e.g.  state  1  in  the  left  FSM  in \nFig.  2A)  signifies  the  initial  state,  and  numbers  in  circles  (e.g.  state 3 in  the  left \nFSM  in  Fig.  2A)  signify  the  accepting  states.  In  each  pair  of diagrams,  transfer \nwas  tested  in  both  directions:  from  the  left  FSM  to  the  right  FSM,  and  to  the \nopposite direction.  Words in a  sentence are generated by an FSM and presented to \nthe network one word at a  time.  At each time,  the next word is  selected randomly \nfrom  next  possible  words  (or  end  of sentence  where  possible)  at the  current  FSM \nstate with the  equal  probability,  and the  FSM  state is  updated  to  the  next  state. \nThe sentence length is  limited to 20  words, excluding START. \n\nThe task for  the network is  to predict the  correct termination of sentences.  If the \nnetwork  is  to  predict  that  the  sentence  ends  with  the  current  input,  the  activity \nof  the  output  node  of  the  network  has  to  be  above  a  threshold  value,  otherwise \nthe  output  has  to  be  below  another  threshold  value.  Note  that  if  a  FSM  is  at \nan  accepting  state  but  can  further  transit  to  another  state,  the  sentence  mayor \nmay not end.  Therefore, the prediction may succeed or fail.  However,  the network \nwill  eventually learn to yield  higher  values  when  the FSM is  at an accepting state \nthan  when  it  is  not.  After  the  network  learns  each  training  sentence,  it  is  tested \nwith randomly generated 1000 sentences and the training session is  completed only \nwhen  the network makes  correct end  point judgments  for  all  sentences.  Then  the \nnetwork is trained with sentences generated by another FSM. The extent of transfer \nis  measured  by  the  reduction  of  the  number  of  sentences  required  to  train  the \nnetwork on an FSM after a prior learning of another FSM, compared to the number \nof sentences required to train the network on the current FSM from  scratch. \n\n2.2  The  Network  Architecture  and the Learning Algorithm \n\nThe network is a second order recurrent neural network, with an added hidden layer \nthat receives first  order connections from  the input layer (Fig.  1).  The network has \nan  input  layer  with  seven  nodes  (A,  B,  C,  ...  F,  and  START),  an  output  layer \nwith  one  node,  an  input  hidden  layer  with  four  nodes,  a  state  hidden  layer  with \nfour  nodes,  and  a  feedback  layer  with  four  nodes.  Recurrent  neural  networks  are \noften used for  modeling syntactic processing  [3].  Second order networks  are suited \nfor  processing  languages  generated  by  FSMs  [4] .  Learning  is  carried  out  by  the \nweight  update rule for  recurrent networks  developed  by Williams  and  Zipser  ([7]), \nextended to second order connections  ([4])  where  necessary.  The learning rate and \nthe momentum are 0.2 and 0.8, respectively.  High and low thresholds are initialized \nto 0.20 and 0.17 respectively and are adapted after the network have processed the \ntest  sentences  as  follows.  The  high  threshold  is  modified  to  the  minimum  value \nyielded  for  all  end  points  in  the  test  sentences  minus  a  margin  (0.01).  The  low \nthreshold  is  modified  to  the  high  threshold  minus  another  margin  (0.02).  These \nthresholds are used  in  the next training and test. \n\n\fOutput Layer \n\nState Hidden Laye r \n\n~--'1c:----=, Feedback \n'-+-+_-+---\"  Laye r \n\nInput Laye r \n\nFigure  1:  A  second  order  recurrent  network  used  in  simulations.  The  network \nconsists of an input layer that receives words, an output layer that predicts sentence \nends,  two  hidden  layers  (an  input  hidden  layer  and  a  state  hidden  layer) ,  and  a \nfeedback layer that receives  a  copy of the state hidden layer activities. \n\n3  The Simulation Results \n\n3.1  The  Transfer  Effects \n\nNumbers of required trainings and changes in number of trainings averaged over 20 \nnetworks with different initial weights are shown in Fig.  2.  Numbers in parentheses \nare standard errors of number  of trainings.  Changes  are shown  with either  a  \"+\" \nsign  (increase)  or  a  \"-\"  sign  (reduction).  For  instance,  Fig.  2A  shows  that  it \nrequired 14559 sentence presentations for the network to learn the left FSM after the \nnetwork was trained on the right FSM. On the other hand, it required 20995 sentence \npresentation  for  the  network  to  learn  the  left  FSM  from  the  scratch.  Therefore \nthere  was  30.7%  reduction  in  the  transfer  direction  from  right  to  left.  Note  that \nthe  network  was  trained  only  once  on  sentences  from  the  source  grammar  to  the \ncriteria and then  only once  on  the sentences from  the target grammar.  Thus after \nthe  completion  of  the  target  grammar  learning,  the  knowledge  about  the  source \ngrammar is  disrupted to some extent.  To  show that the network eventually learns \nboth grammars , number of required training was examined for  more than one cycle. \nAfter ten cycles,  number of required trainings was  reduced to 0.13%  (not shown). \n\n3.2  Representation of Grammatical Knowledge \n\nTo  analyze  the  representation  of  grammatical  knowledge  in  the  network,  Linear \nDiscriminant  Analysis  (LDA)  was  applied  to  hidden  layer  activities.  LDA  is  a \ntechnique which finds  sets of coefficients that defines  a  linear combination of input \nvariables  that  can  be  used  to  discriminate  among  sets  of  input  data that  belong \nto  different  categories.  Linear  combinations  of hidden  layer  node  activities  using \nthese coefficients  provide low-dimensional views  of hidden layer activities that best \nseparate specified  categories  (e.g.  grammatical functions).  In  this  respect,  LDA  is \nsimilar to Principal Component Analysis  (PCA) except that PCA finds  dimensions \nalong  which  the  data  have  large  variances,  whereas  LDA  finds  dimensions  which \ndifferentiate the specified  categories. \n\n\fA \n\nB \n\nc \n\nD \n\n9731 \n(961) \n+3.81 \u00a5o \n\n9374 \n(674) \n\n25314 \n(3959) \n\" \n\n-40.9% \u2022 \n\n42826 \n(4575) \n\n20995 \n(1119) \n\n\u2022 -30.7% \n\n/ \n\\ \n14559 \n(1332) \n\n40943 \n\n(2905) , \n< 28448 \n\n-30.5% \n\n(7924) \n\n20995 \n(1l20) \n\nD \n\n67521 \n\n-~.:~:) \n/:121~ \n. ~ ~ \n\nF  F \n\n\" 1 \n\ni. \n\n~E \n~ \n\n+6l2% \n~<-------\n68858 \n35097 \n(10255) \n(3423) \n\n20995 \n44114 \n(1119) \n(3604) \n-------,-+ 1~2.ma \n\n, \n\n-25.8% \n( \n15575 \n(1149) \n\n\u2022 \n\n39372 \n(2271) \n\nD \n\nE \n\nGJ--/\n-----) 1 \n\n2 \n\n] \nD \n\nF \n~ \nF  GGE \nCV \n\nFigure  2:  Initial  savings observed in various grammar transfer tasks.  Numbers  are \nrequired number of training averaged over 20 networks with different initial weights. \nNumbers in parentheses are standard errors.  Numbers shown with \"%\"  are change \nin number of training due to transfer.  A negative change means reduction (positive \ntransfer)  and a  positive change means increase  (negative transfer,  or interference). \n\n\fe \n8  e \n\n0 \n\n0 \n\n0 \n\n~ \nm \n\ne \nee \nrJ' \n\nID \n\n00 \n\n0 \n\n0 \n\n0 \n\n0 \n\n0 0 \n\n0 \n\n<i> \n\nI!J \n\n<i> \n\n<i> \n0 \n\n0 \n\n0 \n00 \n0 \n\n0 \n\n0 \n\n<i> \n\n0 8  0 \n0 \n0 \n\no~oo \n\nfJO \n\n<i> \n\n<i>  <i> \n\n0 \n\n000 \n\n00 \n0\u00b000 \n0 \n8 \n0 \n\n0 \n\n0 \n\n0 \n\n0 \n\ngo 0 \n\n-2 \n\n-I \n\n0 \n\nLinear OiSO\"iminanl  1 \n\nN \n\n~  I \n\"E \n\"\u00a7 \n15 \n2 0 \n5 \n\n0 \n0 \n00 \n\n- I \n\n-2 \n\n-3 \n\n-3 \n\nFigure 3:  State space organization for  a grammar transfer task (a case of Fig_  2B)_ \nState  space  activities  corresponding  to  FSM  states  1,  2,  and  3  are  plotted  with \nsquares,  diamonds,  and  circles,  respectively_ State space  activities  that  belong  to \nthe target FSM have dots in the plots, whereas those that belong to the source FSM \ndo not have fill  in  patterns_ \n\no \n\no \n\nesponding \n\nState 3 reg ion  corresponding \n'GO inputs ~.ACAC\u00b7\u00b7o 0 \n\no \n\no \n\n0 \n\n0 \n\u00b000 \n\nStat  3 reg ion  corresponding \n\n~;::;;;=~~~~~p..,.:l:.~JI~:.O~in~Sbts .. B8B ... \n\n- I \n\n~ \n\no \n\no \n\n0 \n\n~ ~~a~~~~J ,~~~~Tc~~,ing \n\n(state  3) \n\n-3 ~----~------~----~------~----~------~ \n-3 \n\n-I \n\n-2 \n\n0 \n\nLinear Oisaiminanl  1 \n\nFigure 4:  Trajectories corresponding to loops  in  Fig_  2B  in  the state hidden  layer \nstate space_ The  broken  line  corresponds  to a  hypothetical  shared  discrimination \ncue a  hypothetical boundary described in  4_ It is  the between white  diamonds and \nwhite circles  (i. e.  states 2 and 3 in the source grammar), as  well  as it can be one of \nthe  discrimination  boundaries  between  diamonds  with dots  and squares  with dots \n(i . e.  states 2 and  1 in the target grammar).  The triangular shape shows  the three \nFSM  state trajectory corresponding  to inputs  BCCBCC ....  Ellipses  show  to state \nspace activities involved in one state loops  (at state 1 and at state 3)  and two state \nloops  (at state 2 and 3). \n\n\f4  Discussion \n\nIn  the  first  grammar  transfer  task  (Fig.  2A) ,  only  the  initial  and  the  accepting \nstates  in  the  FSMs  were  different,  so  the  frequency  distribution  of subsequences \nof  words  were  very  similar  except  for  short  sentences.  In  this  case,  31 %  saving \nwas  observed in one transfer direction  although there was  little change in  required \ntraining in the other direction.  In the second grammar transfer task, directions of all \narcs in the FSMs were reversed.  Therefore the mirror images of sentences accepted \nin  one  grammar  were  accepted  in  the  other  grammar.  Although  the  grammars \nwere  very  different,  there  were  significant  amount  of  overlaps  in  the  permissible \nshort subsequences.  In  this  case,  there were  31%  and  41 % savings  in  training.  In \nthe third and fourth  grammar transfer tasks,  the source  and the target grammars \nshared less subsequences.  In  the third case (Fig.  2C)  for  instance, the subsequences \nwere  very different  because the source grammar had two one-state loops  (at  states \n1 and 3)  with the same word A,  whereas two one-state loops in the target grammar \nconsisted of different  words  (D  and E).  In  this  case, there was  little change in  the \nnumber of learnings  required  in one  transfer  direction  but there was  67%  increase \nin  the  other  direction.  In  the  fourth  case  (Fig  2.  D),  there  was  26%  reduction in \none  direction  but  there  was  12%  increase  in  the  other  direction  in  the  number  of \nlearnings  required.  From  these  observations  we  hypothesize  that,  as  in  the  case \nof  syntax  transfer  ([5]) ,  if  the  acquired  grammar  allows  frequent  subsequence  of \nwords  that  appears  in  the  target  grammar  (after  the  equivalent  symbol  sets  are \nsubstituted)  the transfer is  easier and thus there are more savings. \n\nWhat is  the source  of savings in grammar transfer?  It  is  tempting to  say that,  as \nin the vocabulary transfer task ([5]),  the source of savings is the organization of the \nstate hidden layer activity which directly reflects  the FSM states.  Fig.  3 shows the \nstate space organization after the grammar transfer shown in Fig.  2B.  Fig.  4 shows \nthe  change  in  the  state hidden  layer  activities  drawn over  the  state  space  organi(cid:173)\nzation.  The triangular lines  are the trajectories as the network receives  BCCBCC, \nwhich creates the 3-state loops  (231)(231)  in the FSM.  Regions of trajectories cor(cid:173)\nresponding to the 2-state loop  (23)  and two  I-state loops  (1)  and (3)  are also shown \nin  Fig.  4,  although the  trajectory lines  are  not  shown  to  avoid  a  cluttered figure. \nIt can be seen that  state space activities  that belong to  different  FSM  state loops \ntend to be  distinct even  when they belong to the same FSM  state,  although there \nseem to be  some tendencies  that they are allocated in  vicinities.  Unlike  in  the vo(cid:173)\ncabulary transfer, regions belonging to different  FSM loops tend to be interspersed \nby  regions  that belong  to  the  other  grammar,  causing state space  structure to  be \nmore fragmented.  Furthermore, we  found  that there was  no  significant  correlation \nbetween  the  correct  rate  of the  linear  discrimination  with  respect  to  FSM  states \n(which  reflects  the  extent  to  which  the  state  space  organization reflects  the  FSM \nstates)  and savings  (not shown). \n\nOne  could  reasonably  argue that the saving  is  not  due  to transfer  of grammatical \nknowledge but is  due to some more low-level processing specific to neural networks. \nFor instance, the network may have to move  weight  values  to an appropriate range \nat the first stage of the source grammar learning, which might become unnecessary \nfor  the leaning of the target grammar.  We  conducted a  simulation to examine the \neffect  of altering the initial random weights  using the source and target grammars. \nThe  space  limitation  does  not  permit  us  to  present  the  details,  but  we  did  not \nobserve the effect  of initializing the bias  and the weights to appropriate ranges. \n\nIf neither  the  state  space  organization  nor  the  lower-level  statistics  was  not  the \nsource  of savings,  what  was  transferred?  As  already mentioned,  state space orga(cid:173)\nnization observed in grammar transfer task is  more fragmented  than that observed \n\n\fin  vocabulary  transfer  task  (Fig.  3).  These  fragmented  regions  have  to  be  dis(cid:173)\ncriminated  as  far  as  each  region  (which  represents  a  combination  of the  current \nnetwork  state  and  the  current  vocabulary)  has  to  yield  a  different  network  state. \nState hidden nodes provide clues for the discrimination by placing boundaries in the \nnetwork  state  space.  Boundary lines  collectively  define  regions  in  the state  space \nwhich  correspond to  sets of state-vocabulary combinations that  should  be treated \nequivalently  in  terms  of the  given  task.  These  boundaries  can  be  shared:  for  in(cid:173)\nstance,  a  hypothetical  boundary shown  by  a  broken line  in  the  Fig.  4  can  be  the \ndiscrimination  boundary  between  white  diamonds  and  white  circles  (i. e.  states  2 \nand 3 in the source grammar), as well  as it can be one of the discrimination bound(cid:173)\naries  between  diamonds  with  dots  and  squares  with  dots  (i. e.  states  2  and  1  in \nthe  target  grammar).  We  speculate  that  shared  boundaries  may be  the  source  of \nsavings.  That is, boundaries created for  the source grammar learning can be used, \npossibly with some modifications, as one of the boundaries for the target grammar. \nIn  other words,  the source of savings  may not be as high level  as  FSM  state space \nbut some lower level features  at the syntactic processing level. \n\n5  Conclusion \n\nWe  investigated the  ability  of a  recurrent  neural  network to transfer  grammatical \nknowledge of a previously acquired language to another.  We found that the network \nwas  able to transfer  the grammatical knowledge to a  new  grammar with a  slightly \ndifferent  syntax defined  over a  new  vocabulary  (grammar transfer).  The extent  of \ntransfer  seemed  to  depend  on  the  subsequences  of  symbols  generated  by  the  two \ngrammars, after the equivalence sets are translated, although the  results presented \nin this paper are admittedly very restricted in the type of syntax covered and the size \nof syntactic rules and vocabularies.  We  hypothesize that the ability of the network \nto  transfer  grammatical knowledge  comes  from  sharing  discrimination  boundaries \nof input and vocabulary combinations.  In  sum, we  hope to have demonstrated that \nneural  networks  do  not  simply  learn  associations  among  input  symbols  but  they \nacquire structural knowledge from  inputs. \n\nReferences \n\n[1]  Brooks,  L. R. ,  and Vokey,  J .  R.  (1991)  Abstract  analogies  and abstracted grammars: \nComments on Reber (1989)  and Mathews  et  al.  (1090).  Journal  of Experimental  Psychol(cid:173)\nogy:  Gen eral, 120,  316-323. \n\n[2]  Dienes,  Z. ,  Altmann,  and G. ,  Gao,  S-J.  (1999)  Mapping  across  domains without feed(cid:173)\nback:  A  neural  network  model  of transfer  of  implicit  knowledge,  Cognitive  Science  23, \n53-82. \n\n[3]  Elman,  J.  L.  (1991)  Distributed representation, simple recurrent neural networks, and \ngrammatical structure.  Machine  Learning,  7, 195-225. \n\n[4]  Giles,  C.  L. ,  Miller,  C.  B. ,  Chen,  D. ,  Chen,  H.  H. , Sun,  G.  Z. ,  and Lee, Y.  C.  (1992) \nLearning and Extracting Finite State Automata with Second-Order Recurrent Neural Net(cid:173)\nworks,  it  Neural  Computation, 4 , 393-495. \n\n[5]  Hanson,  S.  J.,  Negishi,  M.,  (2001)  The  emergence  of explicit  knowledge  (symbols  & \nrules)  in  (associationist)  neural networks,  Submitted. \n\n[6]  Reber,  A.  (1969)  Transfer  of syntactic  structure  in  synthetic  languages.  Journal  of \nExperimental  Psychology,  81 ,  115-119. \n\n[7]  Williams, R . J.  and Zipser,  D.  (1989)  A learning algorithm for  continually running fully \nrecurrent neural networks,  Neural  Computation, 1  (2) ,  270. \n\n\f", "award": [], "sourceid": 1971, "authors": [{"given_name": "Michiro", "family_name": "Negishi", "institution": null}, {"given_name": "Stephen", "family_name": "Hanson", "institution": null}]}