{"title": "Learning Human-like Knowledge by Singular Value Decomposition: A Progress Report", "book": "Advances in Neural Information Processing Systems", "page_first": 45, "page_last": 51, "abstract": null, "full_text": "Learning Human-like Knowledge by Singular \n\nValue Decomposition: A Progress Report \n\nThomas  K.  Landauer \nDarrell  Laham \nDepartment of Psychology  &  Institute of Cognitive Science \nUniversity of Colorado at Boulder  Boulder, CO  80309-0345 \n\n{landauer,  dlaham}@psych.colorado.edu \n\nPeter  Foltz \n\nDepartment of Psychology \n\nNew Mexico State University  Las Cruces, NM  88003-8001 \n\npfoltz@crl.nmsu.edu \n\nAbstract \n\nSingular value decomposition  (SVD)  can  be  viewed  as  a  method  for \nunsupervised training of a network that associates two  classes  of events \nreciprocally by linear connections  through  a single  hidden  layer.  SVD \nwas used to learn and represent relations among  very large numbers  of \nwords  (20k-60k)  and  very  large numbers  of natural  text  passages  (lk-\n70k)  in  which  they  occurred.  The  result  was  100-350  dimensional \n\"semantic spaces\" in which any trained or newly aibl word or passage \ncould be represented as  a vector,  and  similarities  were measured by  the \ncosine  of  the  contained  angle  between  vectors.  Good  accmacy  in \nsimulating  human judgments and behaviors has  been  demonstrated  by \nperformance  on  multiple-choice  vocabulary  and  domain  knowledge \ntests, emulation of expert essay evaluations,  and  in  several other ways. \nExamples are also given of how the kind of knowledge extracted by  this \nmethod can be applied. \n\n1  INTRODUCTION \nTraditionally,  imbuing  machines  with  human-like  knowledge  has  relied  primarily  on \nexplicit coding of symbolic facts into computer data structures and  algorithms.  A  serious \nlimitation of this approach is people's  inability  to  access  and express  the  vast reaches of \nunconscious  knowledge  on  which  they  rely,  knowledge  based  on  masses  of  implicit \ninference and irreversibly melded data. A more important deficiency of this  state of affairs \nis  that  by  coding  the knowledge  ourselves,  (as  we also do when  we  assign  subjectively \nhypothesized rather than objectively identified features to input or output nodes in a neural \nnet)  we  beg  important  questions  of  how  humans  acquire  and  represent  the  cOOed \nknowledge in the fIrSt place. \n\n\f46 \n\nT.  K.  Landauer; D.  Laham and P.  Foltz \n\nThus. from both engineering and scientific perspectives. there are reasons  to  try  to  design \nlearning machines  that  can  ocquire human-like quantities  of human-like knowledge  from \nthe  same  sources as  humans.  The  success of such  techniques  would  not  prove  that  the \nsame mechanisms are  used  by  humans.  but because we presently do not  know  how  the \nproblem can  be solved in  principle,  successful  simulation  may  offer theoretical insights \nas well as practical applications. In the work reported here we have found  a way  to  induce \nsignificant amounts of knowledge about the meanings of passages and of their constituent \nvocabularies  of words  by  training  on  large  bodies of natural  text.  In  general  terms,  the \nmethod  simultaneously  extracts  the  similarity  between  words  (the  likelihood  of  being \nused in passages that convey similar ideas) and  the  similarity  of passages  (the  likelihood \nof  containing  words  of  similar  meaning).  The  conjoint  estimation  of  similarity  is \naccomplished  by  a fundamentally  simple  representational  technique  that exploits  mutual \nconstraints  implicit  in  the  occurrences of very many  words  in  very  many  contexts.  We \nview  the  resultant  system  both  as  a  means  for  automatically  learning  much  of  the \nsemantic content of words and passages.  and as  a potential  computational  model for  the \nprocess that underlies the corresponding human ability. \nWhile the method starts with data about the natural contextual co-occurrences of words.  it \nuses  them  in  a  different  manner  than  has  previously  been  applied.  A  long -standing \nobjection  to  co-occurrence  statistics  as  a  source  of  linguistic  knowledge  (Chomsky's \n1957) is  that  many  grammatically  acceptable  expressions.  for  example  sentences  with \npotentially  unlimited  embedding  structures.  cannot  be  produced  by  a  finite  Markov \nprocess  whose  elements  are  transition  probabilities  from  word  to  word.  If word-word \nprobabilities are insufficient to  generate  language.  then.  it  is  argued, acquiring estimates \nof  such  probabilities  cannot  be  a  way  that  language  can  be  learned.  However,  our \napproach to statistical knowledge learning differs from those considered in the past  in  two \nways.  First.  the  basic  associational  data  from  which  knowledge  is  induced  are  not \ntransition  frequencies  between  successive  individual  words  or  phrases.  but  rather  the \nfrequencies with  which  particular words  appear as components of relatively  large natural \npassages, utterances of the kind that humans use  to  convey complete ideas. The result of \nthis  move  is  that  the  statistical  regularities  reflected  are  relations  among  unitary \nexpressions of meaning.  rather than  syntactic constraints on  word  order  that  may  serve \nadditional purposes such  as output  and input  processing  efficiencies. error protection.  or \nesthetic style.  Second,  the  mutual  constraints inherent in  a multitude  of  such  local  ~ \noccurrence  relations  are jointly  satisfied by  being  forced  into  a  global  representation  of \nlower dimensionality. This constraint satisfaction. a form of induction.  was  accomplished \nby  singular  value  decomposition.  a  linear  factorization  technique  that  produces  a \nrepresentational  structure  equivalent  to  a  three  layer  neural  network  model  with  linear \nactivation functions. \n\n2  THE TEXT  ANALYSIS  MODEL  AND  METHOD \nThe text analysis process that we have explored is called Latent Semantic Analysis  (LSA) \n(Deerwester et al .\u2022  1990; Landauer and Dumais. 1997). It comprises four steps: \n(1) A large body of text is represented as a matrix [ij],  in  which rows  stand for  individual \nword types. columns  for  meaning-bearing  passages such  as  sentences  or paragraphs. mel \ncells contain the frequency with which a word occurs in a passage. \n(2) Cell entries (freqi) are transformed to: \n\nlog(freqi, + I) \n\nL((  fref/,] \n{freq. ]'l \n-1-,  ~fr:~v  *10  ~fre~v ) \n\na measure of the first order association of a word and its context. \n\n\fLearning Human Knowledge by Singular Value  Decomposition: A Progress Report \n\n47 \n\n(3) The matrix is then subjected to singular value decomposition (Berry, 1992): \n\n[ij]  =  [ik]  [kk]  Uk]' \n\nin  which  [ik]  and  Uk]  have orthonormal columns,  [kk]  is  a diagonal matrix  of  singular \nvalues, and k <= max (ij). \n(4) Finally, all but the d largest singular values are  set to  zero.  Pre-multiplication of the \nright-hand matrices  produces  a  least-squares  best  approximation  to  the  original  matrix \ngiven  the  number of dimensions,  d,  (hidden  units  in  a  corresponding  neural  net  model \nrepresentation)  that  are  retained.  The  SVD  with  dimension  reduction  constitutes  a \nconstraint-satisfaction induction process in that it predicts the original observations on the \nbasis  of linear  relations  among  the  abstracted  representations  of  the  data  that  it  has \nretained.  By  hypothesis,  the  analysis  induces  human-like  relationships  among  passages \nand  words  because  humans  also  make  inferences  about  semantic  relationships  from \nabstracted representations based on limited data, and do so by an analogous process. \nIn  the  result,  each  word and passage is  represented as a  vector of length  d.  Performance \ndepends  strongly  on  the  choice  of  number  of  dimensions.  The  optimal  number  is \ntypically  around  300.  The similarity  of any  two  words,  any  two  text  passages,  or  any \nword and any  text passage, are computed by  measures on  their  vectors.  We  have  most \noften used the cosine (of the contained angle  between the  vectors  in  semantic d-space)  -\nwhich  we  interpret  as  the  degree  of qualitative  similarity  of meaning.  The  length  of \nvectors is also useful and interpretable. \n\n3  TESTS  OF  LSA'S  PERFORMANCE \nLSA's  ability  to  simulate human  knowledge and meaning relations has  been  tested in  a \nvariety of ways. Here we describe two relatively direct sources of evidence and briefly  list \nseveral others. \n\n3.1  VOCABULARY  &  DOMAIN  KNOWLEDGE  TESTS \nIn all cases, LSA was ftrst trained on  a  large text corpus  intended to  be representative of \nthe text from  which humans gain  most  of the  semantic knowledge  to  be  simulated.  In a \npreviously reported test (Landauer and Dumais,  1997), LSA was trained on  approximately \nfive million words of text sampled from  a  high-school  level encyclopedia, then  tested on \nmultiple choice items from the Educational Testing Service Test of English  as  a Foreign \nLanguage (TOEFL). These test questions present a target word or short phrase and ask  the \nstudent to  choose  the  one of four  alternative  words  or  phrases  that  is  most  similar  in \nmeaning.  LSA's  answer  was  determined  by  computing  the  cosine  between  the  derived \nvector for the target word or phrase  and each of the  alternatives and choosing  the  largest. \nLSA  was correct on 64%  of the  80  items,  identical  to  the  average of a  large sample of \nstudents  from  non-English  speaking countries  who had  applied  for  admission  to  U.  S. \ncolleges. When in error, LSA made choices positively correlated (product-moment r = .44) \nwith those preferred by students. We have recently replicated this result with training  on  a \nsimilar sized sample from the Associated Press newswire \nIn  a  new  set of tests,  LSA  was trained on  a popular  introductory  psychology  textbook \n(Myers,  1995) and  tested with  the  same  four-alternative  multiple  choice  tests  used  for \nstudents in two large classes.  In  these experiments, LSA's  score  was  about 6O%-lower \nthan  the  class  averages but  above  passing  level,  and far  above  guessing  probability.  Its \nerrors again resembled those of students; it got right about half as many of questions rated \ndifftcult by the test constructors as ones rated easy (Landauer, Foltz and Laham, 1997). \n\n3.2  ESSA Y  TESTS \n\n\f48 \n\nT.  K.  Landauer, D.  Laham and P.  Foltz \n\nWord-wad meaning  similarities  are a  good  test  of knowledge-indeed,  vocabulary  tests \nare  the  best  single  measure of human  intelligence.  However,  they  are  not  sufficient  to \nassess the correspondence of LSA and  human knowledge because people usually  express \nknowledge via larger verbal strings, such as sentences, paragraphs  and articles.  Thus,  just \nas  multiple  choice  tests  of  student  knowledge  are  often  supplemented  by  essay  tests \nwhose content is  then  judged by  humans,  we wished  to  evaluate  the  adequacy of LSA' s \nrepresentation of knowledge in complete passages of text.  We could not  have LSA  write \nessays because it has no means for producing sentences. However, we  were  able  to  assess \nthe  accum::y  with  which  LSA  could  extract  and  represent  the  knowledge  expressed  in \nessays written by students by simulating judgments about their content that were made by \nhuman readers (Landauer, Laham, Rehder, & Schreiner, in press). \nIn these tests, students were asked to  write  short essays  to  cover an assigned topic  or to \nanswer  a  posed  question.  In  various  experiments,  the  topics  included  anatomy  axI \nfunction  of  the  heart,  phenomena  from  introductory  psychology,  the  history  of  the \nPanama Canal, and tolerance of diversity in America.  In  each case, LSA  was first  trained \neither on a large sample of instructional text from the same domain or,  in  the  latter case, \non  combined text  from  the  very large number of essays  themselves,  to  produce a  high(cid:173)\ndimensional  (100-350  dimensions  in  the  various  tests)  semantic  space.  We  then \nrepresented  each  essay  simply  as  the  vector  average  of  the  vectors  for  the  words  it \ncontained. Two properties of these average vectors  were then  used to  measure  the  quality \nand  quantity  of knowledge  conveyed  by  an  essay:  (1)  the  similarity  (measured  as  the \ncosine of the angle between vectors) of the student essay and one or more standard essays, \nand (2) the total amount of domain specific content, measured as the vector length. \nIn each case, two human experts independently rated the overall quality of each essay on  a \nfive or ten point scale. The judges were either university course instructors or professional \nexam readers  from  Educational Testing  Service. The LSA  measures  were calibrated  with \nrespect to the judges' rating scale in  several  different ways,  but  because they  gave nearly \nthe same results only one will  be described  here.  In  this  method,  each  student essay  was \ncompared to a large (90-200) set of essays previously scored by experts, and  the  ten  most \nsimilar  (by  cosine)  identified.  The  target  essay  was  then  assigned  a  \"quality\"  score \ncomponent  consisting  of the  cosine-weighted  average  of  the  ten.  A  second,  \"relevant \nquantity\", score component was the vector length of the student essay. Finally, regression \non expert scores was used to weight the quality and quantity scores (However, the  weights \nin all cases were so close to equal that merely  adding  them  would  have  given  comparable \nresults).  Calibration  was  performed  on  data  independent  of  that  used  to  evaluate  the \nrelation between LSA and expert ratings. \nThe correlation between  the LSA  score  for  an  essay and  that  assigned by  the  average of \nthe human readers was .80, .64, .XX and .84 for  the  four  sets of exams.  The comparable \ncorrelation between one reader and  the other was .83,  .65,  .XX  and  .82,  respectively.  In \nthe  heart topic  case, each  student had also  taken  a carefully  constructed  \"objective\"  test \nover the  same material  (a  short  answer  test  with  near  perfect  scoring  agreement).  The \ncorrelation  between  the  LSA  essay  score  and  the  objective  test  was  .81,  the  average \ncorrelation for the two expert readers .74. \nA striking aspect of these results is  that  the  LSA  representations were based on analyses \nof the  essays  that  took  no  account of word  order,  each  essay  was  treated  as  a  \"bag  of \nwords\". In extracting  meaning  from  a  text,  human  readers  presumably rely on  syntax  as \nwell as the mere combination of words it contains, yet they were no better at agreeing  on \nan essay's quality or in assigning a score that predicted a performance on a  separate  test  of \nknowledge.  Apparently,  either  the  relevant  information  conveyed  by  word  order  in \nsentences is redundant with the information  that  can  be inferred  from  the  combination  of \nwords  in  the  essay,  or  the  processes  used  by  LSA  and  humans  extract  different  but \ncompensatingly useful information. \n\n\fLearning HumanKnowledgc by Singular Value Decomposition: A Progress Report \n\n49 \n\n3.3  OTHER  EVIDENCE \nLSA  has  been  compared  with  human  knowledge  in  several  additional  ways,  some \nconfrrming the correspondence, others indicating limitations. Here are some examples, all \nbased on encyclopedia corpus training. \n(1)  Overall LSA  similarity  between  antonyms  (mean  cos = .18)  was  equivalent  to  that \nbetween  synonyms  (mean  cos  = .17)  in  triplets  sampled  from  an  antonym/synonym \ndictionary  (W  &  R  Chambers,  1989),  both  of  which  significantly  exceeded  that  for \nunrelated  pairs  (mean  cos  =  .01;  ps  <  .0001).  However  for  antonym  (but  not  for \nsynonym)  pairs  a  dominant  dimension  of  difference  could  easily  be  extracted  by \ncomputing a one dimensional unfolding using the LSA cosines from a set of words  listed \nin Rogel's (1992) thesaurus as related respectively to the two members of the pair. \n(2) Anglin (1970) asked children and adults to sort words varying in concept relations  ad \nparts of speech. LSA wont-word similarity correlates .50 with children and .32 with adults \nfor the number of times they sorted two words together.  Conceptual structure is  reflected, \nbut grammatical classification, strong in the adult data, is not. \n(3) When people are asked to decide that a letter string is a wont, they  do so  faster if they \nhave just read a sentence that does not contain the word but implies a related concept (e.g. \nTill, Mross & Kintsch, 1988). LSA mirrors this result with high similarities between the \nsame sentences and words. (Landauer & Dumais, 1997). \n(4) People frequently  make a  logical error,  called the  conjunction  error  by  Tversky  ad \nKahneman (1974), in which they estimate that the probability that an object is  a  member \nof a class is greater than  that  it  is  a member of a superset class  when  the  description of \nthe  object  is  \"similar\"  to  the  description  of  the  subset.  For  example,  when  told  that \n\"Linda is  a  young  woman  who  is  single,  outspoken ... deeply  concerned  with  issues  of \ndiscrimination and social justice,\" over 80%  of even  statistically  sophisticated  subjects \nrate  it  more  likely  that Linda is  a feminist  bank  teller than  that  Linda  is  a  bank  teller \n(Tversky  &  Kahneman,  1980).  LSA  similarities  between  descriptions  of  people  ad \noccupations of this kind taken from Shafir, Smith and Osherson (1990) were computed as \nthe  cosine  between  the  vector  averages  of  words  in  the  paired  person-occupation \ndescriptions. Conjunction error statements were more  similar  to  the  subset than  superset \nstatement in  12 out of 14 cases (p<.01),  showing  that LSA's  representation of sentential \nmeaning reflected similarity relations of the  sort  that have been  hypothesized to  underlie \nthe conjunction fallacy in human judgment. \n(5)  A  semantic  subspace  was  constructed  for  words  from  natural  kind  and  artifact \ncategories whose differential preservation is characteristic of agnosias due to  local damage \nfrom  herpes  simplex  encephalitis  (Warrington  &  Shallice,  1984).  Principal  components \nanalysis  of the  similarities  among  these  words  as  represented  by  LSA  revealed  that \ncategories that tend to be lost contain words  that  are  more  highly  inter-related  than  those \nin preserved categories (Laham, in press). \nOf course, LSA does not capture all of the human  knowledge conveyed by  text.  Some  of \nthe shortfall is probably due merely to the use of training corpora that are still imperfectly \nrepresentative of the  language experience of a  typical  person,  and  to  lack  of knowledge \nfrom non-textual  sources. For example, in  all  these studies,  less  total  text was  used for \nLSA  training  than  even  a  single  educated  adult  would  have  read  However,  a  more \nfundamental restriction is that the analysis does not reflect order relations between words, \nand  therefore  cannot  extract  infonnation  that  depends  on  syntax.  Because  the  analysis \ndiscovers  and represents only  unsigned continuous  similarities,  it can be  used  to  induce \nonly certain classes of structural relations, not including ones that express Boolean, causal \nor other non-commutative logical relations.  As  we  have  seen,  this  lack does  not  prevent \naccurate simulation  of human cognition  in  many  cases,  possibly  because  humans  also \n\n\f50 \n\nT.  K  Landauer, D.  Laham and P.  Foltz \n\nfrequently  rely  on  similarity  rather  than  syntax-based.  discrete  logic  (fversky  axl \nKahneman,  1983);  however,  it  does  limit  the  utility  of the  results  for  populating  the \nsymbolic data structures commonly used to represent knowledge in  traditional  AI.  On the \nother hand, as examples in  the  next  section show,  continuous-valued similarity  relations \ncan be fruitfully applied if appropriate computational use is made of them. \n\n4  SAMPLE  APPLICATIONS \nLSA has  been  used successfully  in  a variety of experimental applications,  including  the \nessay scoring techniques described earlier. Here are some additional examples: \n(1)  The technique  has  been  used  to  improve  automatic information retrieval by  20-30% \nover otherwise identical methods by allowing users' queries  to  match documents  with  the \ndesired conceptual meaning but expressed in different words (Dumais, 1991, 1994). \n(2)  By  training  on  corpora  of  translated  text  in  which  the  words  of  corresponding \nparagraphs in the two languages are combined in the \"bags of words\",  LSA  has  been able \nto provide at least as good retrieval when queries and documents are in a different language \nas when in the same language (Landauer and Littman,  1990). \n(3) LSA-based measures of the  similarity  of student  essays  on  a  topic  to  instructional \ntexts can predict how much an individual student will  learn  from  a particular text (Wolfe \net al., in press; Rehder et al., in press). To  do this,  the  full  set of student essays and the \ntexts in  question  are aligned along  a single  dimension  that  best accommodates  the  LSA \nsimilarities among them. Estimates from one such experiment showed that using  LSA  to \nchoose  the  optimal  one  of  four  texts  for  each  student  (a  text  that  is  slightly  more \nsophisticated  than  the  student)  rather  than  assigning  all  students  the  overall  best  text \n(which LSA also picked correctly) increased the average amount learned by over 40%. \n(4) LSA-based measures of conceptual similarity  between  successive sentences accurately \npredict  differences  in  judged  coherence  and  measured  comprehensibility  of text  (Foltz, \nKintsch and Landauer, in press). \n\n5  SUMMARY \nSVD-based learning of the  structure underlying  the  use  of words  in  meaningful contexts \nhas  been  found capable of deriving  and representing  the  similarity  of meaning  of words \nand text passages in a manner that accurately  simulates  corresponding  similarity  relations \nas  reflected  in  several  sorts  of  human  judgments  and  behavior.  The  validity  of  the \nresulting representation of meaning similarities has been established in a variety  of ways, \nand  the  utility  of its  knowledge  representation  illustrated  by  several  educational  axl \ncognitive psychological research applications. It is obviously too  early  to  assess  whether \nthe  particular computational  model  is  a true analog  of the  process  used  by  the  human \nbrain to  accomplish  the same things.  However, the  basic process,  the  representation  of \nmyriad  local  associative relations between components  and larger contexts of experience \nin a joint space of lower dimensionality,  offers,  for  the  first  time,  a candidate for  such  a \nmechanism that has been  shown  sufficient to  approximate human  knowledge  acquisition \nfrom  natural sources at natural scale. \n\nAcknowledgments \n\nWe thank members of the LSA research group at the University  of Colorado for  valuable \ncollaboration and advice:  Walter Kintsch,  Bob Rehder,  Mike  Wolfe,  &  M.  E.  Shreiner. \nWe especially acknowledge two  participants from  the  Spring  1997  LSA  seminar at  CU \nwhose unpublished work is described: Alan  Sanfey  (3.3.4)  and Michael Emerson  (3.3.1). \nThanks also to Susan T.  Dumais of Bellcore. \n\n\fLearning Human Knowledge by Singular Value Decomposition: A Progress Repol1 \n\n51 \n\nReferences \n\nAnglin, J. M. (1970). The growth of word meaning. Cambridge, MA:  MIT. \nBerry, M. W. (1992). Large scale singular value computations. International Journal of \n\nSupercomputer Applications, 6,  13-49. \n\nDeerwester, S.,  Dumais, S. T., Furnas, G. W., Landauer, T.  K., & Harshman, R.  (1990). \n\nIndexing By Latent Semantic Analysis. Journal of the American Society For \nInformation  Science. 41, 391-407. \n\nDumais, S. T. (1991). Improving the retrieval of infonnation from  external sources. \n\nBehavior Research Methods, Instruments and Computers. 23,229-236. \n\nDumais, S. T. (1994). Latent semantic indexing (LSI) and TREC-2. In D. Harman (Ed.), \n\nNational Institute of Standards and Technology Text Retrieval Conference. NIST \nspecial publication. \n\nFoltz, P. W., Kintsch, W., &  Landauer, T.  K. (in press). Analysis of text coherence \n\nusing Latent Semantic Analysis. Discourse Processes. \n\nLaham, D. (in press). Latent Semantic Analysis approaches to categorization. Proceedings \n\nof the Cognitive Science Society.  1997. \n\nLandauer, T. K., &  Dumais, S. T.  (1997).  A solution to Plato's problem:  The Latent \n\nSemantic Analysis theory of the acquisition, induction, and representation of \nknowledge. Psychological Review, 104,211-240. \n\nLandauer, T.  K., Foltz, P. W., &  Laham, D.  (1997). Latent Semantic Analysis passes \n\nthe test: knowledge representation and multiple-choice testing. Manuscript in \npreparation. \n\nLandauer, T. K., Laham, D., Rehder, B. &  Schreiner, M .E. (in press). How well can \n\npassage meaning be derived without using word order: A comparison of Latent \nSemantic Analysis and humans. Proceedings of the Cognitive Science Society,  1997. \n\nLandauer, T. K., &  Littman, M. L. (1990). Fully automatic cross-language document \n\nretrieval using latent semantic indexing. In Proceedings of the Sixth Annual \nConference of the UW Centre for the New Oxford English Dictionary and Text \nResearch (pp. 31-38). Waterloo, Ontario: UW Centre for the New OED. \n\nMyers, D. G. (1995). Psychology. Fourth Edition. NY, NY: Worth. \nRehder, B., Schreiner, M. E., Wolfe, B. W., Laham, D., Landauer, T. K., &  Kintsch, W. \n\n(in press). Using Latent Semantic Analysis to assess knowledge: Some technical \nconsiderations. Discourse Processes. \n\nShafIf, E., Smith, E. E., & Osherson, D.  N.  (1990). Typicality and reasoning judgments. \n\nMemory  &  Cognition, 3, 229-239. \n\nTill, R.  E., Mross, E.  F.,  &  Kintsch.  W.  (1988).  Time  course  of priming  for  associate \n\nand inference words in discourse context. Memory and Cognition, 16, 283-299. \n\nTversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases. \n\nScience, 18S,  1124-1131. \n\nTversky, A., &  Kahneman, D. (1980). Judgments of and by representativeness. In D. \n\nKahneman, P. Slovic, & A. Tversky (Eds.), Judgment under uncertainty: Heuristics \nand biases. New York: Cambridge University Press. \n\nTversky, A., &  Kahneman, D. (1983). Extensional versus intuitive reasoning: The \n\nconjunction fallacy  in probability judgment. Psychological Review, 90,293-315. \n\nWarrington, E.  K., &  Shallice, T. (1984). Category-specific semantic impairments. \n\nBrain, 107, 829-853. \n\nWolfe,  M. B., Schreiner, M. E., Rehder, B., Laham, D., Foltz, P.  W., Kintsch, W., & \nLandauer, T. K.  (in press). Learning from text:  Matching readers and text by Latent \nSemantic Analysis. Discourse Processes. \n\n\f", "award": [], "sourceid": 1468, "authors": [{"given_name": "Thomas", "family_name": "Landauer", "institution": null}, {"given_name": "Darrell", "family_name": "Laham", "institution": null}, {"given_name": "Peter", "family_name": "Foltz", "institution": null}]}