{"title": "Inductive reasoning about chimeric creatures", "book": "Advances in Neural Information Processing Systems", "page_first": 316, "page_last": 324, "abstract": "Given one feature of a novel animal, humans readily make inferences about other features of the animal. For example, winged creatures often fly, and creatures that eat fish often live in the water. We explore the knowledge that supports these inferences and compare two approaches. The first approach proposes that humans rely on abstract representations of dependency relationships between features, and is formalized here as a graphical model.  The second approach proposes that humans rely on specific knowledge of previously encountered animals, and is formalized here as a family of exemplar models. We evaluate these models using a task where participants reason about chimeras, or animals with pairs of features that have not previously been observed to co-occur. The results support the hypothesis that humans rely on explicit representations of relationships between features.", "full_text": "Inductive reasoning about chimeric creatures\n\nCharles Kemp\n\nDepartment of Psychology\nCarnegie Mellon University\n\nckemp@cmu.edu\n\nAbstract\n\nGiven one feature of a novel animal, humans readily make inferences about other\nfeatures of the animal. For example, winged creatures often \ufb02y, and creatures that\neat \ufb01sh often live in the water. We explore the knowledge that supports these infer-\nences and compare two approaches. The \ufb01rst approach proposes that humans rely\non abstract representations of dependency relationships between features, and is\nformalized here as a graphical model. The second approach proposes that humans\nrely on speci\ufb01c knowledge of previously encountered animals, and is formalized\nhere as a family of exemplar models. We evaluate these models using a task where\nparticipants reason about chimeras, or animals with pairs of features that have not\npreviously been observed to co-occur. The results support the hypothesis that hu-\nmans rely on explicit representations of relationships between features.\n\nSuppose that an eighteenth-century naturalist learns about a new kind of animal that has fur and a\nduck\u2019s bill. Even though the naturalist has never encountered an animal with this pair of features,\nhe should be able to make predictions about other features of the animal\u2014for example, the animal\ncould well live in water but probably does not have feathers. Although the platypus exists in reality,\nfrom a eighteenth-century perspective it quali\ufb01es as a chimera, or an animal that combines two or\nmore features that have not previously been observed to co-occur. Here we describe a probabilistic\naccount of inductive reasoning and use it to account for human inferences about chimeras.\n\nThe inductive problems we consider are special cases of the more general problem in Figure 1a\nwhere a reasoner is given a partially observed matrix of animals by features then asked to infer the\nvalues of the missing entries. This general problem has been previously studied and is addressed\nby computational models of property induction, categorization, and generalization [1\u20137]. A chal-\nlenge faced by all of these models is to capture the background knowledge that guides inductive\ninferences. Some accounts rely on similarity relationships between animals [6, 8], others rely on\ncausal relationships between features [9, 10], and others incorporate relationships between animals\nand relationships between features [11]. We will evaluate graphical models that capture both kinds\nof relationships (Figure 1a), but will focus in particular on relationships between features.\n\nPsychologists have previously suggested that humans rely on explicit mental representations of re-\nlationships between features [12\u201316]. Often these representations are described as theories\u2014for\nexample, theories that specify a causal relationship between having wings and \ufb02ying, or living in\nthe sea and eating \ufb01sh. Relationships between features may take several forms: for example, one\nfeature may cause, enable, prevent, be inconsistent with, or be a special case of another feature. For\nsimplicity, we will treat all of these relationships as instances of dependency relationships between\nfeatures, and will capture them using an undirected graphical model.\n\nPrevious studies have used graphical models to account for human inferences about features but\ntypically these studies consider toy problems involving a handful of novel features such as \u201chas\ngene X14\u201d or \u201chas enzyme Y132\u201d [9, 11]. Participants might be told, for example, that gene X14\nleads to the production of enzyme Y132, then asked to use this information when reasoning about\nnovel animals. Here we explore whether a graphical model approach can account for inferences\n\n1\n\n\f(a)\n\nslow heavy\n\nflies wings\n\n(b)\n\nhippo\n\nrhino\n\nsparrow\n\nrobin\n\nnew\n\no\n\n1\n\n1\n\n0\n\n0\n\n?\n\n1\n\n1\n\n0\n\n0\n\n?\n\n0\n\n0\n\n1\n\n1\n\n1\n\n0\n\n0\n\n1\n\n1\n\n?\n\nFigure 1: Inductive reasoning about animals and features. (a) Inferences about the features of a\nnew animal onew that \ufb02ies may draw on similarity relationships between animals (the new animal is\nsimilar to sparrows and robins but not hippos and rhinos), and on dependency relationships between\nfeatures (\ufb02ying and having wings are linked). (b) A graph product produced by combining the two\ngraph structures in (a).\n\nabout familiar features. Working with familiar features raises a methodological challenge since\nparticipants have a substantial amount of knowledge about these features and can reason about them\nin multiple ways. Suppose, for example, that you learn that a novel animal can \ufb02y (Figure 1a). To\nconclude that the animal probably has wings, you might consult a mental representation similar to\nthe graph at the top of Figure 1a that speci\ufb01es a dependency relationship between \ufb02ying and having\nwings. On the other hand, you might reach the same conclusion by thinking about \ufb02ying creatures\nthat you have previously encountered (e.g. sparrows and robins) and noticing that these creatures\nhave wings. Since the same conclusion can be reached in two different ways, judgments about\narguments of this kind provide little evidence about the mental representations involved.\n\nThe challenge of working with familiar features directly motivates our focus on chimeras. Inferences\nabout chimeras draw on rich background knowledge but require the reasoner to go beyond past\nexperience in a fundamental way. For example, if you learn that an animal \ufb02ies and has no legs, you\ncannot make predictions about the animal by thinking of \ufb02ying, no-legged creatures that you have\npreviously encountered. You may, however, still be able to infer that the novel animal has wings\nif you understand the relationship between \ufb02ying and having wings. We propose that graphical\nmodels over features can help to explain how humans make inferences of this kind, and evaluate our\napproach by comparing it to a family of exemplar models. The next section introduces these models,\nand we then describe two experiments designed to distinguish between the models.\n\n1 Reasoning about objects and features\n\nOur models make use of a binary matrix D where the rows {o1, . . . , o129} correspond to objects,\nand the columns {f 1, . . . , f 56} correspond to features. A subset of the objects is shown in Figure 2a,\nand the full set of features is shown in Figure 2b and its caption. Matrix D was extracted from the\nLeuven natural concept database [17], which includes 129 animals and 757 features in total. We\nchose a subset of these features that includes a mix of perceptual and behavioral features, and that\nincludes many pairs of features that depend on each other. For example, animals that \u201clive in water\u201d\ntypically \u201ccan swim,\u201d and animals that have \u201cno legs\u201d cannot \u201cjump far.\u201d\n\nMatrix D can be used to formulate problems where a reasoner observes one or two features of a\nnew object (i.e. animal o130) and must make inferences about the remaining features of the animal.\nThe next two sections describe graphical models that can be used to address this problem. The\n\ufb01rst graphical model O captures relationships between objects, and the second model F captures\nrelationships between features. We then discuss how these models can be combined, and introduce\na family of exemplar-style models that will be compared with our graphical models.\n\nA graphical model over objects\n\nMany accounts of inductive reasoning focus on similarity relationships between objects [6, 8]. Here\nwe describe a tree-structured graphical model O that captures these relationships. The tree was\nconstructed from matrix D using average linkage clustering and the Jaccard similarity measure, and\npart of the resulting structure is shown in Figure 2a. The subtree in Figure 2a includes clusters\n\n2\n\n\f(a)\n\nr\no\n\nt\n\na\ng\n\ni\nl\nl\n\na\n\nn\na\nm\na\nc\n\ni\n\ne\n\nd\nr\na\nz\n\ni\nl\n \nr\no\n\nr\nu\na\ns\no\nn\nd\n\nm\nr\no\nw\nd\nn\n\ni\n\ni\nl\n\nt\ni\n\nl\ni\n\nd\no\nc\no\nr\nc\n\nb\n\nn\no\nm\n\n(b)\n\na\no\nb\n\na\nr\nb\no\nc\n\nn\no\nh\n\nt\ny\np\n\ne\nk\na\nn\ns\n\nr\ne\np\nv\n\ni\n\na\nn\na\nu\ng\n\no\nk\nc\ne\ng\n\nd\nr\na\nz\n\ni\nl\n\ni\n\ng\no\nr\nf\n\nd\na\no\n\ne\ns\no\n\ni\n\nt\n\ne\n\nl\nt\nr\nu\n\nt\n\nt\nr\no\n\nt\n\ni\n\ng\nn\ni\nr\nr\ne\nh\n\ne\nn\nd\nr\na\ns\n\ny\nv\no\nh\nc\nn\na\n\nd\no\nc\n\nl\n\ne\no\ns\n\nt\n\nu\no\nr\nt\n\np\nr\na\nc\n\ne\nk\np\n\ni\n\nn\no\nm\na\ns\n\nl\n\nl\n\ne\ne\n\ny\na\nr\n\nh\ns\ni\nf\nt\n\na\n\nl\nf\n\ni\n\ne\nc\na\np\n\nl\n\na\nh\nn\na\nr\ni\np\n\nl\n\nn\no\ne\ne\nm\na\nh\nc\n\nr\ne\nd\nn\na\nm\na\na\ns\n\nl\n\nk\nc\na\nb\ne\nk\nc\ni\nt\ns\n\nl\n\ni\n\nd\nu\nq\ns\n\ni\n\nn\nh\np\no\nd\n\nl\n\nh\ns\ni\nf\n\nh\ns\ni\nf\n\nl\n\nd\no\ng\n\nd\nr\no\nw\ns\n\nl\n\n \n\ne\na\nh\nw\nm\nr\ne\np\ns\n\na\nc\nr\no\n\nl\n\ne\na\nh\nw\n\nk\nr\na\nh\ns\n\nt\n\na\nb\n\nx\no\n\nf\n\nf\nl\n\no\nw\n\nt\ni\n\nb\nb\na\nr\n\nn\no\ns\nb\n\ni\n\nl\n\ne\ns\nu\no\nm\n\ne\nr\nr\ni\nu\nq\ns\n\nr\ne\n\nt\ns\nm\na\nh\n\nr\ne\nv\na\ne\nb\n\ng\no\nh\ne\ng\nd\ne\nh\n\nt\n\nn\na\nh\np\ne\ne\n\nl\n\ns\no\nr\ne\nc\no\nn\nh\nr\n\ni\n\ns\nu\nm\na\n\nt\n\no\np\no\np\np\nh\n\ni\n\nn\no\n\ni\nl\n\nr\ne\ng\n\ni\nt\n\nr\na\ne\nb\n\nr\ne\ne\nd\n\na\nm\na\n\nl\nl\n\ne\n\nf\nf\n\na\nr\ni\ng\n\na\nr\nb\ne\nz\n\ny\ne\nk\nn\no\nm\n\no\no\nr\na\ng\nn\na\nk\n\ny\nr\na\nd\ne\nm\no\nr\nd\n\n \nr\na\no\np\n\nl\n\nt\n\na\nc\n\ng\no\nd\n\nw\no\nc\n\ne\ns\nr\no\nh\n\ny\ne\nk\nn\no\nd\n\ng\np\n\ni\n\np\ne\ne\nh\ns\n\ncan swim\n\nhas gills\n\nlives in\nwater\n\nlives in\nthe sea\n\neats nuts\neats grain\neats grass\neats berries\n\ncrawls\n\nhas two\n\nlegs\n\nhas no\n\nlegs\n\ncan jump\n\nfar\n\nhas\n\nfeathers\n\nhas scales\n\nhas mane\n\nslow\n\neats\nfish\n\nlives\nin the\ndesert\n\nlives\nin the\nwoods\n\nlives\n\nin trees\n\nhas six\n\nlegs\n\ncan fly\n\nhas four\n\nlegs\n\nhas fur\n\ncan be\nridden\n\nhas sharp\n\nteeth\n\nheavy\n\nnocturnal\n\ncan see\nin dark\n\nlives\n\nunderground\n\ncan climb\n\nwell\n\nhas wings\n\nstrong\n\npredator\n\nFigure 2: Graph structures used to de\ufb01ne graphical models O and F.\n(a) A tree that captures\nsimilarity relationships between animals. The full tree includes 129 animals, and only part of the\ntree is shown here. The grey points along the branches indicate locations where a novel animal o130\ncould be attached to the tree. (b) A network capturing pairwise dependency relationships between\nfeatures. The edges capture both positive and negative dependencies. All edges in the network are\nshown, and the network also includes 20 isolated nodes for the following features: is black, is blue,\nis green, is grey, is pink, is red, is white, is yellow, is a pet, has a beak, stings, stinks, has a long neck,\nhas feelers, sucks blood, lays eggs, makes a web, has a hump, has a trunk, and is cold-blooded.\n\ncorresponding to amphibians and reptiles, aquatic creatures, and land mammals, and the subtree\nomitted for space includes clusters for insects and birds.\n\nWe assume that the features in matrix D (i.e. the columns) are generated independently over O:\n\nP (D|O, \u03c0, \u03bb) =Yi\n\nP (f i|O, \u03c0i, \u03bbi).\n\nThe distribution P (f i|O, \u03c0i, \u03bbi) is based on the intuition that nearby nodes in O tend to have the\nsame value of f i. Previous researchers [8, 18] have used a directed graphical model where the\ndistribution at the root node is based on the baserate \u03c0i, and any other node v with parent u has the\nfollowing conditional probability distribution:\n\nP (v = 1|u) =(\u03c0i + (1 \u2212 \u03c0i)e\u2212\u03bbil,\n\n\u03c0i \u2212 \u03c0ie\u2212\u03bbil,\n\nif u = 1\nif u = 0\n\n(1)\n\nwhere l is the length of the branch joining node u to node v. The variability parameter \u03bbi captures the\nextent to which feature f i is expected to vary over the tree. Note, for example, that any node v must\ntake the same value as its parent u when \u03bb = 0. To avoid free parameters, the feature baserates \u03c0i\nand variability parameters \u03bbi are set to their maximum likelihood values given the observed values\nof the features {f i} in the data matrix D. The conditional distributions in Equation 1 induce a joint\ndistribution over all of the nodes in graph O, and the distribution P (f i|O, \u03c0i, \u03bbi) is computed by\nmarginalizing out the values of the internal nodes. Although we described O as a directed graphical\nmodel, the model can be converted into an equivalent undirected model with a potential for each\nedge in the tree and a potential for the root node. Here we use the undirected version of the model,\nwhich is a natural counterpart to the undirected model F described in the next section.\nThe full version of structure O in Figure 2a includes 129 familiar animals, and our task requires\ninferences about a novel animal o130 that must be slotted into the structure. Let D\u2032 be an expanded\nversion of D that includes a row for o130, and let O\u2032 be an expanded version of O that includes a\nnode for o130. The edges in Figure 2a are marked with evenly spaced gray points, and we use a\n\n3\n\n\funiform prior P (O\u2032) over all trees that can be created by attaching o130 to one of these points. Some\nof these trees have identical topologies, since some edges in Figure 2a have multiple gray points.\nPredictions about o130 can be computed using:\n\nP (D\u2032|D) =XO\u2032\n\nP (D\u2032|O\u2032, D)P (O\u2032|D) \u221dXO\u2032\n\nP (D\u2032|O\u2032, D)P (D|O\u2032)P (O\u2032).\n\n(2)\n\nEquation 2 captures the basic intuition that the distribution of features for o130 is expected to be\nconsistent with the distribution observed for previous animals. For example, if o130 is known to\n\ufb02y then the trees with high posterior probability P (O\u2032|D) will be those where o130 is near other\n\ufb02ying creatures (Figure 1a), and since these creatures have wings Equation 2 predicts that o130\nprobably also has wings. As this example suggests, model O captures dependency relationships\nbetween features implicitly, and therefore stands in contrast to models like F that rely on explicit\nrepresentations of relationships between features.\n\nA graphical model over features\n\nModel F is an undirected graphical model de\ufb01ned over features. The graph shown in Figure 2b was\ncreated by identifying pairs where one feature depends directly on another. The author and a research\nassistant both independently identi\ufb01ed candidate sets of pairwise dependencies, and Figure 2b was\ncreated by merging these sets and reaching agreement about how to handle any discrepancies.\n\nAs previous researchers have suggested [13, 15], feature dependencies can capture several kinds of\nrelationships. For example, wings enable \ufb02ying, living in the sea leads to eating \ufb01sh, and having\nno legs rules out jumping far. We work with an undirected graph because some pairs of features\ndepend on each other but there is no clear direction of causal in\ufb02uence. For example, there is clearly\na dependency relationship between being nocturnal and seeing in the dark, but no obvious sense in\nwhich one of these features causes the other.\n\nWe assume that the rows of the object-feature matrix D are generated independently from an undi-\nrected graphical model F de\ufb01ned over the feature structure in Figure 2b:\n\nP (D|F) =Yi\n\nP (oi|F).\n\nModel F includes potential functions for each node and for each edge in the graph. These potentials\nwere learned from matrix D using the UGM toolbox for undirected graphical models [19]. The\nlearned potentials capture both positive and negative relationships: for example, animals that live in\nthe sea tend to eat \ufb01sh, and tend not to eat berries. Some pairs of feature values never occur together\nin matrix D (there are no creatures that \ufb02y but do not have wings). We therefore chose to compute\nmaximum a posteriori values of the potential functions rather than maximum likelihood values, and\nused a diffuse Gaussian prior with a variance of 100 on the entries in each potential.\n\nAfter learning the potentials for model F, we can make predictions about a new object o130 using\nthe distribution P (o130|F). For example, if o130 is known to \ufb02y (Figure 1a), model F predicts\nthat o130 probably has wings because the learned potentials capture a positive dependency between\n\ufb02ying and having wings.\n\nCombining object and feature relationships\n\nThere are two simple ways to combine models O and F in order to develop an approach that incorpo-\nrates both relationships between features and relationships between objects. The output combination\nmodel computes the predictions of both models in isolation, then combines these predictions using\na weighted sum. The resulting model is similar to a mixture-of-experts model, and to avoid free\nparameters we use a mixing weight of 0.5. The structure combination model combines the graph\nstructures used by the two models and relies on a set of potentials de\ufb01ned over the resulting graph\nproduct. An example of a graph product is shown in Figure 1b, and the potential functions for this\ngraph are inherited from the component models in the natural way. Kemp et al. [11] use a similar\napproach to combine a functional causal model with an object model O, but note that our structure\ncombination model uses an undirected model F rather than a functional causal model over features.\n\nBoth combination models capture the intuition that inductive inferences rely on relationships be-\ntween features and relationships between objects. The output combination model has the virtue of\n\n4\n\n\fsimplicity, and the structure combination model is appealing because it relies on a single integrated\nrepresentation that captures both relationships between features and relationships between objects.\nTo preview our results, our data suggest that the combination models perform better overall than\neither O or F in isolation, and that both combination models perform about equally well.\n\nExemplar models\n\nWe will compare the family of graphical models already described with a family of exemplar models.\nThe key difference between these model families is that the exemplar models do not rely on explicit\nrepresentations of relationships between objects and relationships between features. Comparing the\nmodel families can therefore help to establish whether human inferences rely on representations of\nthis sort.\n\nConsider \ufb01rst a problem where a reasoner must predict whether object o130 has feature k after ob-\nserving that it has feature i. An exemplar model addresses the problem by retrieving all previously-\nobserved objects with feature i and computing the proportion that have feature k:\n\nP (ok = 1|oi = 1) =\n\n|f k & f i|\n\n|f i|\n\n(3)\n\nwhere |f k| is the number of objects in matrix D that have feature k, and |f k & f i| is the number that\nhave both feature k and feature i. Note that we have streamlined our notation by using ok instead of\no130\nk\nSuppose now that the reasoner observes that object o130 has features i and j. The natural general-\nization of Equation 3 is:\n\nto refer to the kth feature value for object o130.\n\nP (ok = 1|oi = 1, oj = 1) =\n\n|f k & f i & f j|\n\n|f i & f j|\n\n(4)\n\nBecause we focus on chimeras, |f i & f j| = 0 and Equation 4 is not well de\ufb01ned. We therefore\nevaluate an exemplar model that computes predictions for the two observed features separately then\ncomputes the weighted sum of these predictions:\n\nP (ok = 1|oi = 1, oj = 1) = wi |f k & f i|\n\n|f i|\n\n+ wj |f k & f j|\n\n|f j|\n\n.\n\n(5)\n\nwhere the weights wi and wj must sum to one. We consider four ways in which the weights could\nbe set. The \ufb01rst strategy sets wi = wj = 0.5. The second strategy sets wi \u221d |f i|, and is consistent\nwith an approach where the reasoner retrieves all exemplars in D that are most similar to the novel\nanimal and reports the proportion of these exemplars that have feature k. The third strategy sets\n|f i| , and captures the idea that features should be weighted by their distinctiveness [20]. The\nwi \u221d 1\n\ufb01nal strategy sets weights according to the coherence of each feature [21]. A feature is coherent if\nobjects with that feature tend to resemble each other overall, and we de\ufb01ne the coherence of feature\ni as the expected Jaccard similarity between two randomly chosen objects from matrix D that both\nhave feature i. Note that the \ufb01nal three strategies are all consistent with previous proposals from the\npsychological literature, and each one might be expected to perform well.\n\nBecause exemplar models and prototype models are often compared, it is natural to consider a pro-\ntotype model [22] as an additional baseline. A standard prototype model would partition the 129\nanimals into categories and would use summary statistics for these categories to make predictions\nabout the novel animal o130. We will not evaluate this model because it corresponds to a coarser ver-\nsion of model O, which organizes the animals into a hierarchy of categories. The key characteristic\nshared by both models is that they explicitly capture relationships between objects but not features.\n\n2 Experiment 1: Chimeras\n\nOur \ufb01rst experiment explores how people make inferences about chimeras, or novel animals with\nfeatures that have not previously been observed to co-occur. Inferences about chimeras raise chal-\nlenges for exemplar models, and therefore help to establish whether humans rely on explicit rep-\nresentations of relationships between features. Each argument can be represented as f i, f j \u2192 f k\n\n5\n\n\fexemplar\n(wi = 0.5)\nr = 0.42\n\nexemplar\n(wi = |f i|)\nr = 0.44\n\nfeature\n\nF\n\nobject\n\nO\n\n7\n\n5\n\n3\n\n1\n\nr = 0.69\n\n7\n\n5\n\n3\n\n1\n\nr = 0.31\n\n7\n\n5\n\n3\n\n1\n\n7\n\n5\n\n3\n\n1\n\noutput\n\ncombination\n\nr = 0.59\n\n7\n\n5\n\n3\n\n1\n\nstructure\n\ncombination\n\nr = 0.60\n\n0\n\n0.5\n\n1\n\n0\n\n0.5\n\n1\n\n0\n\n0.5\n\n1\n\n0\n\n0.5\n\n1\n\n0\n\n0.5\n\n1\n\n0\n\n0.5\n\n1\n\nr = 0.06\n\n7\n\n5\n\n3\n\n1\n\nr = 0.17\n\n7\n\n5\n\n3\n\n1\n\nr = 0.71\n\n7\n\n5\n\n3\n\n1\n\nr = \u22120.02\n\nr = 0.57\n\n7\n\n5\n\n3\n\n1\n\nr = 0.49\n\n7\n\n5\n\n3\n\n1\n\n0\n\n0.5\n\n1\n\n0\n\n0.5\n\n1\n\n0\n\n0.5\n\n1\n\n0\n\n0.5\n\n1\n\n0\n\n0.5\n\n1\n\n0\n\n0.5\n\n1\n\nr = 0.51\n\n7\n\n5\n\n3\n\n1\n\nr = 0.64\n\n7\n\n5\n\n3\n\n1\n\nr = 0.83\n\n7\n\n5\n\n3\n\n1\n\nr = 0.45\n\n7\n\n5\n\n3\n\n1\n\nr = 0.76\n\n7\n\n5\n\n3\n\n1\n\nr = 0.79\n\n0\n\n0.5\n\n1\n\n0\n\n0.5\n\n1\n\n0\n\n0.5\n\n1\n\n0\n\n0.5\n\n1\n\n0\n\n0.5\n\n1\n\n0\n\n0.5\n\n1\n\nr = 0.26\n\n7\n\n5\n\n3\n\n1\n\nr = 0.25\n\n7\n\n5\n\n3\n\n1\n\nr = 0.19\n\n7\n\n5\n\n3\n\n1\n\nr = 0.25\n\n7\n\n5\n\n3\n\n1\n\nr = 0.24\n\n7\n\n5\n\n3\n\n1\n\nr = 0.33\n\n0\n\n0.5\n\n1\n\n0\n\n0.5\n\n1\n\n0\n\n0.5\n\n1\n\n0\n\n0.5\n\n1\n\n0\n\n0.5\n\n1\n\n0\n\n0.5\n\n1\n\n7\n\n5\n\n3\n\n1\n\n7\n\n5\n\n3\n\n1\n\n7\n\n5\n\n3\n\n1\n\n7\n\n5\n\n3\n\n1\n\nl\nl\n\na\n\nt\nc\n\ni\nl\nf\n\nn\no\nc\n\ne\ng\nd\ne\n\nr\ne\nh\n\nt\n\no\n\nFigure 3: Argument ratings for Experiment 1 plotted against the predictions of six models. The\ny-axis in each panel shows human ratings on a seven point scale, and the x-axis shows probabilities\naccording to one of the models. Correlation coef\ufb01cients are shown for each plot.\nwhere f i and f k are the premises (e.g. \u201chas no legs\u201d and \u201ccan \ufb02y\u201d) and f k is the conclusion (e.g.\n\u201chas wings\u201d). We are especially interested in con\ufb02ict cases where the premises f i and f j lead to\nopposite conclusions when taken individually: for example, most animals with no legs do not have\nwings, but most animals that \ufb02y do have wings. Our models that incorporate feature structure F can\nresolve this con\ufb02ict since F includes a dependency between \u201cwings\u201d and \u201ccan \ufb02y\u201d but not between\n\u201cwings\u201d and \u201chas no legs.\u201d Our models that do not include F cannot resolve the con\ufb02ict and predict\nthat humans will be uncertain about whether the novel animal has wings.\nMaterials. The object-feature matrix D includes 447 feature pairs {f i, f j} such that none of the\n129 animals has both f i and f j. We selected 40 pairs (see the supporting material) and created\n400 arguments in total by choosing 10 conclusion features for each pair. The arguments can be\nassigned to three categories. Con\ufb02ict cases are arguments f i, f j \u2192 f k such that the single-premise\narguments f i \u2192 f k and f j \u2192 f k lead to incompatible predictions. For our purposes, two single-\npremise arguments with the same conclusion are deemed incompatible if one leads to a probability\ngreater than 0.9 according to Equation 3, and the other leads to a probability less than 0.1. Edge cases\nare arguments f i, f j \u2192 f k such that the feature network in Figure 2b includes an edge between f k\nand either f i or f j. Note that some arguments are both con\ufb02ict cases and edge cases. All arguments\nthat do not fall into either one of these categories will be referred to as other cases.\n\nThe 400 arguments for the experiment include 154 con\ufb02ict cases, 153 edge cases, and 120 other\ncases. 34 arguments are both con\ufb02ict cases and edge cases. We chose these arguments based on\nthree criteria. First, we avoided premise pairs that did not co-occur in matrix D but that co-occur in\nfamiliar animals that do not belong to D. For example, \u201cis pink\u201d and \u201chas wings\u201d do not co-occur in\nD but \u201c\ufb02amingo\u201d is a familiar animal that has both features. Second, we avoided premise pairs that\nspeci\ufb01ed two different numbers of legs\u2014for example, {\u201chas four legs,\u201d \u201chas six legs\u201d}. Finally, we\naimed to include roughly equal numbers of con\ufb02ict cases, edge cases, and other cases.\nMethod. 16 undergraduates participated for course credit. The experiment was carried out using a\ncustom-built computer interface, and one argument was presented on screen at a time. Participants\n\n6\n\n\frated the probability of the conclusion on seven point scale where the endpoints were labeled \u201cvery\nunlikely\u201d and \u201cvery likely.\u201d The ten arguments for each pair of premises were presented in a block,\nbut the order of these blocks and the order of the arguments within these blocks were randomized\nacross participants.\nResults. Figure 3 shows average human judgments plotted against the predictions of six models.\nThe plots in the \ufb01rst row include all 400 arguments in the experiment, and the remaining rows show\nresults for con\ufb02ict cases, edge cases, and other cases. The previous section described four exemplar\nmodels, and the two shown in Figure 3 are the best performers overall. Even though the graphical\nmodels include more numerical parameters than the exemplar models, recall that these parameters\nare learned from matrix D rather than \ufb01t to the experimental data. Matrix D also serves as the basis\nfor the exemplar models, which means that all of the models can be compared on equal terms.\n\nThe \ufb01rst row of Figure 3 suggests that the three models which include feature structure F perform\nbetter than the alternatives. The output combination model is the worst of the three models that in-\ncorporate F, and the correlation achieved by this model is signi\ufb01cantly greater than the correlation\nachieved by the best exemplar model (p < 0.001, using the Fisher transformation to convert correla-\ntion coef\ufb01cients to z scores). Our data therefore suggest that explicit representations of relationships\nbetween features are needed to account for inductive inferences about chimeras. The model that\nincludes the feature structure F alone performs better than the two models that combine F with the\nobject structure O, which may not be surprising since Experiment 1 focuses speci\ufb01cally on novel\nanimals that do not slot naturally into structure O.\nRows two through four suggest that the con\ufb02ict arguments in particular raise challenges for the\nmodels which do not include feature structure F. Since these con\ufb02ict cases are arguments f i, f j \u2192\nf k where f i \u2192 f k has strength greater than 0.9 and f j \u2192 f k has strength less than 0.1, the\n\ufb01rst exemplar model averages these strengths and assigns an overall strength of around 0.5 to each\nargument. The second exemplar model is better able to differentiate between the con\ufb02ict arguments,\nbut still performs substantially worse than the three models that include structure F. The exemplar\nmodels perform better on the edge arguments, but are outperformed by the models that include F.\nFinally, all models achieve roughly the same level of performance on the other arguments.\n\nAlthough the feature model F performs best overall, the predictions of this model still leave room for\nimprovement. The two most obvious outliers in the third plot in the top row represent the arguments\n{is blue, lives in desert \u2192 lives in woods} and {is pink, lives in desert \u2192 lives in woods}. Our\nparticipants sensibly infer that any animal which lives in the desert cannot simultaneously live in\nthe woods. In contrast, the Leuven database indicates that eight of the twelve animals that live in\nthe desert also live in the woods, and the edge in Figure 2b between \u201clives in the desert\u201d and \u201clives\nin the woods\u201d therefore represents a positive dependency relationship according to model F. This\ndiscrepancy between model and participants re\ufb02ects the fact that participants made inferences about\nindividual animals but the Leuven database is based on features of animal categories. Note, for\nexample, that any individual animal is unlikely to live in the desert and the woods, but that some\nanimal categories (including snakes, salamanders, and lizards) are found in both environments.\n\n3 Experiment 2: Single-premise arguments\n\nOur results so far suggest that inferences about chimeras rely on explicit representations of relation-\nships between features but provide no evidence that relationships between objects are important. It\nwould be a mistake, however, to conclude that relationships between objects play no role in induc-\ntive reasoning. Previous studies have used object structures like the example in Figure 2a to account\nfor inferences about novel features [11]\u2014for example, given that alligators have enzyme Y132 in\ntheir blood, it seems likely that crocodiles also have this enzyme. Inferences about novel objects can\nalso draw on relationships between objects rather than relationships between features. For example,\ngiven that a novel animal has a beak you will probably predict that it has feathers, not because there\nis any direct dependency between these two features, but because the beaked animals that you know\ntend to have feathers. Our second experiment explores inferences of this kind.\nMaterials and Method. 32 undergraduates participated for course credit. The task was identical\nto Experiment 1 with the following exceptions. Each two-premise argument f i, f j \u2192 f k from\nExperiment 1 was converted into two one-premise arguments f i \u2192 f k and f j \u2192 f k, and these\n\n7\n\n\fexemplar\n\nr = 0.78\n\n0\n\n0.5\n\n1\n\nr = 0.87\n\n0\n\n0.5\n\n1\n\nr = 0.79\n\nl\nl\n\na\n\ne\ng\nd\ne\n\nr\ne\nh\n\nt\n\no\n\n7\n\n5\n\n3\n\n1\n\n7\n\n5\n\n3\n\n1\n\n7\n\n5\n\n3\n\n1\n\noutput\n\ncombination\n\nr = 0.75\n\n0\n\n0.5\n\n1\n\nr = 0.86\n\n0\n\n0.5\n\n1\n\nr = 0.66\n\nfeature\n\nF\n\nobject\n\nO\n\nr = 0.54\n\n0\n\n0.5\n\n1\n\nr = 0.87\n\n0\n\n0.5\n\n1\n\nr = 0.21\n\n7\n\n5\n\n3\n\n1\n\n7\n\n5\n\n3\n\n1\n\n7\n\n5\n\n3\n\n1\n\nr = 0.75\n\n0\n\n0.5\n\n1\n\nr = 0.84\n\n0\n\n0.5\n\n1\n\nr = 0.74\n\n7\n\n5\n\n3\n\n1\n\n7\n\n5\n\n3\n\n1\n\n7\n\n5\n\n3\n\n1\n\n7\n\n5\n\n3\n\n1\n\n7\n\n5\n\n3\n\n1\n\n7\n\n5\n\n3\n\n1\n\n7\n\n5\n\n3\n\n1\n\n7\n\n5\n\n3\n\n1\n\n7\n\n5\n\n3\n\n1\n\nstructure\n\ncombination\n\nr = 0.77\n\n0\n\n0.5\n\n1\n\nr = 0.85\n\n0\n\n0.5\n\n1\n\nr = 0.73\n\n1\n\n0\n\n1\n\n0\n\n0.5\n\n1\n\n0\n\n0.5\nFigure 4: Argument ratings and model predictions for Experiment 2.\n\n0.5\n\n0.5\n\n0\n\n0\n\n0.5\n\n1\n\n1\n\none-premise arguments were randomly assigned to two sets. 16 participants rated the 400 arguments\nin the \ufb01rst set, and the other 16 rated the 400 arguments in the second set.\nResults. Figure 4 shows average human ratings for the 800 arguments plotted against the predictions\nof \ufb01ve models. Unlike Figure 3, Figure 4 includes a single exemplar model since there is no need\nto consider different feature weightings in this case. Unlike Experiment 1, the feature model F\nperforms worse than the other alternatives (p < 0.001 in all cases). Not surprisingly, this model\nperforms relatively well for edge cases f j \u2192 f k where f j and f k are linked in Figure 2b, but the\n\ufb01nal row shows that the model performs poorly across the remaining set of arguments.\n\nTaken together, Experiments 1 and 2 suggest that relationships between objects and relationships\nbetween features are both needed to account for human inferences. Experiment 1 rules out an\nexemplar approach but models that combine graph structures over objects and features perform\nrelatively well in both experiments. We considered two methods for combining these structures and\nboth performed equally well. Combining the knowledge captured by these structures appears to be\nimportant, and future studies can explore in detail how humans achieve this combination.\n\n4 Conclusion\n\nThis paper proposed that graphical models are useful for capturing knowledge about animals and\ntheir features and showed that a graphical model over features can account for human inferences\nabout chimeras. A family of exemplar models and a graphical model de\ufb01ned over objects were\nunable to account for our data, which suggests that humans rely on mental representations that\nexplicitly capture dependency relationships between features. Psychologists have previously used\ngraphical models to capture relationships between features, but our work is the \ufb01rst to focus on\nchimeras and to explore models de\ufb01ned over a large set of familiar features.\n\nAlthough a simple undirected model accounted relatively well for our data, this model is only a\nstarting point. The model incorporates dependency relationships between features, but people know\nabout many speci\ufb01c kinds of dependencies, including cases where one feature causes, enables, pre-\nvents, or is inconsistent with another. An undirected graph with only one class of edges cannot\ncapture this knowledge in full, and richer representations will ultimately be needed in order to pro-\nvide a more complete account of human reasoning.\n\nAcknowledgments\npart by the Pittsburgh Life Sciences Greenhouse Opportunity Fund and by NSF grant CDI-0835797.\n\nI thank Madeleine Clute for assisting with this research. This work was supported in\n\n8\n\n\fReferences\n[1] R. N. Shepard. Towards a universal law of generalization for psychological science. Science, 237:1317\u2013\n\n1323, 1987.\n\n[2] J. R. Anderson. The adaptive nature of human categorization. Psychological Review, 98(3):409\u2013429,\n\n1991.\n\n[3] E. Heit. A Bayesian analysis of some forms of inductive reasoning. In M. Oaksford and N. Chater, editors,\n\nRational models of cognition, pages 248\u2013274. Oxford University Press, Oxford, 1998.\n\n[4] J. B. Tenenbaum and T. L. Grif\ufb01ths. Generalization, similarity, and Bayesian inference. Behavioral and\n\nBrain Sciences, 24:629\u2013641, 2001.\n\n[5] C. Kemp and J. B. Tenenbaum. Structured statistical models of inductive reasoning. Psychological\n\nReview, 116(1):20\u201358, 2009.\n\n[6] D. N. Osherson, E. E. Smith, O. Wilkie, A. Lopez, and E. Sha\ufb01r. Category-based induction. Psychological\n\nReview, 97(2):185\u2013200, 1990.\n\n[7] D. J. Navarro. Learning the context of a category. In J. Lafferty, C. K. I. Williams, J. Shawe-Taylor, R.S.\nZemel, and A. Culotta, editors, Advances in Neural Information Processing Systems 23, pages 1795\u20131803.\n2010.\n\n[8] C. Kemp, T. L. Grif\ufb01ths, S. Stromsten, and J. B. Tenenbaum. Semi-supervised learning with trees. In\nAdvances in Neural Information Processing Systems 16, pages 257\u2013264. MIT Press, Cambridge, MA,\n2004.\n\n[9] B. Rehder. A causal-model theory of conceptual representation and categorization. Journal of Experi-\n\nmental Psychology: Learning, Memory, and Cognition, 29:1141\u20131159, 2003.\n\n[10] B. Rehder and R. Burnett. Feature inference and the causal structure of categories. Cognitive Psychology,\n\n50:264\u2013314, 2005.\n\n[11] C. Kemp, P. Shafto, and J. B. Tenenbaum. An integrated account of generalization across objects and\n\nfeatures. Cognitive Psychology, in press.\n\n[12] S. E. Barrett, H. Abdi, G. L. Murphy, and J. McCarthy Gallagher. Theory-based correlations and their\n\nrole in children\u2019s concepts. Child Development, 64:1595\u20131616, 1993.\n\n[13] S. A. Sloman, B. C. Love, and W. Ahn. Feature centrality and conceptual coherence. Cognitive Science,\n\n22(2):189\u2013228, 1998.\n\n[14] D. Yarlett and M. Ramscar. A quantitative model of counterfactual reasoning.\n\nIn T. G. Dietterich,\nS. Becker, and Z. Ghahramani, editors, Advances in Neural Information Processing Systems 14, pages\n123\u2013130. MIT Press, Cambridge, MA, 2002.\n\n[15] W. Ahn, J. K. Marsh, C. C. Luhmann, and K. Lee. Effect of theory-based feature correlations on typicality\n\njudgments. Memory and Cognition, 30(1):107\u2013118, 2002.\n\n[16] D. C. Meehan C. McNorgan, R. A. Kotack and K. McRae. Feature-feature causal relations and statistical\n\nco-occurrences in object concepts. Memory and Cognition, 35(3):418\u2013431, 2007.\n\n[17] S. De Deyne, S. Verheyen, E. Ameel, W. Vanpaemel, M. J. Dry, W. Voorspoels, and G. Storms. Exem-\nplar by feature applicability matrices and other Dutch normative data for semantic concepts. Behavior\nResearch Methods, 40(4):1030\u20131048, 2008.\n\n[18] J. P. Huelsenbeck and F. Ronquist. MRBAYES: Bayesian inference of phylogenetic trees. Bioinformatics,\n\n17(8):754\u2013755, 2001.\n\n[19] M. Schmidt. UGM: A Matlab toolbox for probabilistic undirected graphical models. 2007. Available at\n\nhttp://people.cs.ubc.ca/\u223cschmidtm/Software/UGM.html.\n\n[20] L. J. Nelson and D. T. Miller. The distinctiveness effect in social categorization: you are what makes you\n\nunusual. Psychological Science, 6:246\u2013249, 1995.\n\n[21] A. L. Patalano, S. Chin-Parker, and B. H. Ross. The importance of being coherent: category coherence,\n\ncross-classi\ufb01cation and reasoning. Journal of memory and language, 54:407\u2013424, 2006.\n\n[22] S. K. Reed. Pattern recognition and categorization. Cognitive Psychology, 3:393\u2013407, 1972.\n\n9\n\n\f", "award": [], "sourceid": 227, "authors": [{"given_name": "Charles", "family_name": "Kemp", "institution": null}]}