{"title": "Probabilistic Inference in Human Sensorimotor Processing", "book": "Advances in Neural Information Processing Systems", "page_first": 1327, "page_last": 1334, "abstract": "", "full_text": "Probabilistic Inference in Human Sensorimotor\n\nProcessing\n\nKonrad P. K\u00a8ording \u0003\nInstitute of Neurology\n\nUCL London\n\nLondon WC1N 3BG,UK\n\nkonrad@koerding.com\n\nDaniel M. Wolpert (cid:1)\nInstitute of Neurology\n\nUCL London\n\nLondon WC1N 3BG,UK\n\nwolpert@ion.ucl.ac.uk\n\nAbstract\n\nWhen we learn a new motor skill, we have to contend with both the vari-\nability inherent in our sensors and the task. The sensory uncertainty can\nbe reduced by using information about the distribution of previously ex-\nperienced tasks. Here we impose a distribution on a novel sensorimotor\ntask and manipulate the variability of the sensory feedback. We show that\nsubjects internally represent both the distribution of the task as well as\ntheir sensory uncertainty. Moreover, they combine these two sources of\ninformation in a way that is qualitatively predicted by optimal Bayesian\nprocessing. We further analyze if the subjects can represent multimodal\ndistributions such as mixtures of Gaussians. The results show that the\nCNS employs probabilistic models during sensorimotor learning even\nwhen the priors are multimodal.\n\n1 Introduction\n\nReal world motor tasks are inherently uncertain. For example, when we try to play an\napproaching tennis ball, our vision of the ball does not provide perfect information about\nits velocity. Due to this sensory uncertainty we can only generate an estimate of the ball\u2019s\nvelocity. This uncertainty can be reduced by taking into account information that is avail-\nable on a longer time scale: not all velocities are a priori equally probable. For example,\nvery fast and very slow balls may be experienced less often than medium paced balls. Over\nthe course of a match there will be a probability distribution of velocities. Bayesian the-\nory [1-2] tells us that to make an optimal estimate of the velocity of a given ball, this a\npriori information about the distribution of velocities should be combined with the evi-\ndence provided by sensory feedback. This combination process requires prior knowledge,\nhow probable each possible velocity is, and knowledge of the uncertainty inherent in the\nsensory estimate of velocity. As the degree of uncertainty in the feedback increases, for\nexample when playing in fog or at dusk, an optimal system should increasingly depend on\nprior knowledge. Here we examine whether subjects represent the probability distribution\nof a task and if this can be appropriately combined with an estimate of sensory uncer-\n\n\u0003www.koerding.com\n(cid:1)www.wolpertlab.com\n\n\ftainty. Moreover, we examine whether subjects can represent priors that have multimodal\ndistributions.\n\n2 Experiment 1: Gaussian Prior\n\nTo examine whether subjects can represent a prior distribution of a task and integrate it with\na measure of their sensory uncertainty we examined performance on a reaching task. The\nperceived position of the hand is displaced relative to the real position of the hand. This\ndisplacement or shift is drawn randomly from an underlying probability distribution and\nsubjects have to estimate this shift to perform well on the task. By examining where sub-\njects reached while manipulating the reliability of their visual feedback we distinguished\nbetween several models of sensorimotor learning.\n\n2.1 Methods\n\nTen subjects made reaching movement on a table to a visual target with their right index\n\ufb01nger in a virtual reality setup (for details of the set-up see [6]). An Optotrak 3020 mea-\nsured the position of their \ufb01nger and a projection/mirror system prevented direct view of\ntheir arm and allowed us to generate a cursor representing their \ufb01nger position which was\ndisplayed in the plane of the movement (Figure 1A). As the \ufb01nger moved from the starting\ncircle, the cursor was extinguished and shifted laterally from the true \ufb01nger location by an\namount (cid:0)\b\u0006\t(cid:3) which was drawn each trial from a Gaussian distribution:\n\n\u0004\u0004(cid:0)\b\u0006\t(cid:3)\u0005 (cid:2)\n\n(cid:3)\n\n\u0004(cid:4)(cid:2)(cid:3)\u0004\u0006(cid:5)\u0003\u0006\n\n(cid:4)\n\n\u0004(cid:0)\b\u0006\t(cid:4) (cid:0)(cid:5)(cid:6)\u0007\b\u0005(cid:2)\n\n(cid:2)(cid:8)(cid:2)\n\n\u0004\u0006(cid:6)\u0003\u0006\n\n(1)\n\nwhere (cid:0)(cid:7)(cid:5)\u0007\b (cid:2) (cid:3)(cid:5)\u0001 and (cid:3)\u0004\u0006(cid:5)\u0003\u0006 (cid:2) (cid:5)(cid:7)(cid:6)(cid:5)\u0001 (Figure 1B). Halfway to the target (10 cm), vi-\nsual feedback was brie\ufb02y provided for 100 ms either clearly ( (cid:3) (cid:0)) or with different degrees\nof blur ( (cid:3)\u0005 and (cid:3)\u0004), or withheld ( (cid:3)(cid:2)). On each trial one of the 4 types of feedback\n((cid:3)(cid:0)(cid:8) (cid:3)\u0005 (cid:8) (cid:3)\u0004(cid:8) (cid:3)(cid:2)) was selected randomly, with the relative frequencies of (3, 1, 1, 1) re-\nspectively. The ((cid:3)(cid:0)) feedback was a small white sphere. The ((cid:3)\u0005 ) feedback was 25 small\ntranslucent spheres, distributed as a 2 dimensional Gaussian with a standard deviation of 1\ncm, giving a cloud type impression. The ((cid:3) \u0004) feedback was analogous but with a standard\ndeviation of 2 cm. No feedback was provided in the ((cid:3) (cid:2)) case. After another 10 cm of\nmovement the trial \ufb01nished and feedback of the \ufb01nal cursor location was only provided in\nthe ((cid:3)(cid:0)) condition. The experiment consisted of 2000 trials for each subject. Subjects were\ninstructed to take into account what they see at the midpoint and get as close to the target\nas possible and that the cursor is always there even if it is not displayed.\n\n2.2 Results: Trajectories in the Presence of Uncertainty\n\nSubjects were trained for 1000 trials on the task to ensure that they experienced many\nsamples (cid:0)\b\u0006\t(cid:3) drawn from the underlying distribution \u0004\u0004(cid:0) \b\u0006\t(cid:3)\u0005. After this period, when\nfeedback was withheld ((cid:3)(cid:2)), subjects pointed 0.97\u0006 0.06 cm (mean\u0006 se across subjects)\nto the left of the target showing that they had learned the average shift of 1 cm experienced\nover the trials. Subsequently, we examined the relationship between visual feedback and\nthe location (cid:0)(cid:3)\u0007\b(cid:5)\u0001(cid:12)\b(cid:3) subjects pointed to. On trials in which feedback was provided, there\nwas compensation during the second half of the movement. Figure 1A shows typical \ufb01nger\nand cursor paths for two trials, (cid:3)(cid:2) and (cid:3)(cid:0), in which (cid:0)\b\u0006\t(cid:3) (cid:2) (cid:4)(cid:5)\u0001. The visual feedback\nmidway through the movement provides information about the lateral shift on the current\ntrial and allows for a correction for the current lateral shift. However, the visual system\nis not perfect and we expect some uncertainty in the sensed lateral shift (cid:0) \u0007(cid:3)\u0002\u0007(cid:3)(cid:7). The\ndistribution of sensed shifts over a large number of trials is expected to have a Gaussian\n\n\fA\n\nestimated lateral shift\nxestimate\n\nTarget\n\nfinger\npath\n(not \nvisible)\n\nm\nc\n1\n\n \n\n1 cm\n\ncursor\npath\n\nFeedback:\n\nB\n\nC\n\nD\n\nE\n\n \n\n\u03c30\n\u03c3M\n\u03c3L\n\u03c38\n\nPrior\n\nN(1,\u03c3p=0.5) cm\n\nlateral shift xtrue [cm]\n\nEvidence\n\nsensed lateral shift xsensed [cm]\n\nProbabilistic Model\n\ny\nt\ni\nl\ni\n\nb\na\nb\no\nr\np\n\n)\ne\nu\nr\nt\nx\n(\np\n\n)\ne\nu\nr\nt\nx\n|\nd\ne\ns\nn\ne\ns\nx\n(\np\n\ny\nt\ni\nl\ni\n\nb\na\nb\no\nr\np\n\n)\nd\ne\ns\nn\ne\ns\nx\n|\ne\nu\nr\nt\nx\n(\np\n\ny\nt\ni\nl\ni\n\nb\na\nb\no\nr\np\n\nlateral shift xtrue [cm]\n\nCompensation \n\nModel\n\n1\n\n0\n\n-1\n\nProbabilistic \n\nModel\n\nMapping\nModel\n\n1\n\n0\n\n-1\n\nlateral shift xtrue [cm]\n\nlateral shift xtrue [cm]\n\nlateral shift xtrue [cm]\n\n>\n\n]\n\nm\nc\n[\n\ne\n\nt\n\na\nm\n\ni\nt\ns\ne\nx\n-\ne\nu\nr\nt\nx\n\nn\no\n\ni\nt\n\ni\n\na\nv\ne\nd\n\n \nl\n\na\nr\ne\n\nt\n\nlateral shift xtrue\n(e.g.2cm)\n\n1\n\n0\n\n-1\n\na\n\nl\n\n<\n\nFigure 1: The experiment and models. A) Subjects are required to place the cursor on\nthe target, thereby compensating for the lateral displacement. The \ufb01nger paths illustrate\ntypical trajectories at the end of the experiment when the lateral shift was 2 cm (the colors\ncorrespond to two of the feedback conditions). B) The experimentally imposed prior distri-\nbution of lateral shifts is Gaussian with a mean of 1 cm. C) A schematic of the probability\ndistribution of visually sensed shifts under clear and the two blurred feedback conditions\n(colors as in panel A) for a trial in which the true lateral shift is 2 cm. D) The estimate of\nthe lateral shift for an optimal observer that combines the prior with the evidence. E) The\naverage lateral deviation from the target as a function of the true lateral shift for the models.\nLeft: the full compensation model. Middle the Bayesian probabilistic model. Right: the\nmapping model (see text for details).\n\ndistribution centered on (cid:0)\b\u0006\t(cid:3) with a standard deviation (cid:3)\u0007(cid:3)\u0002\u0007(cid:3)(cid:7) that depends on the acuity\nof the system.\n\n\u0004\u0004(cid:0)\u0007(cid:3)\u0002\u0007(cid:3)(cid:7)(cid:2)(cid:0)\b\u0006\t(cid:3)\u0005 (cid:2)\n\n(cid:3)\n\n\u0004(cid:4)(cid:2)(cid:3)\u0007(cid:3)\u0002\u0007(cid:3)(cid:7)\n\n(cid:4)\n\n\u0004(cid:0)\b\u0006\t(cid:4) (cid:0)\u0007(cid:4)\u0002\u0007(cid:4)(cid:5)\u0005(cid:2)\n\n(cid:2)(cid:8)(cid:2)\n\n\u0007(cid:4)\u0002\u0007(cid:4)(cid:5)\n\n(2)\n\nAs the blur increases we expect (cid:3)\u0007(cid:3)\u0002\u0007(cid:3)(cid:7) to increase (Figure 1C).\n\n2.3 Computational Models and Predictions\n\nThere are several computational models which subjects could use to determine the compen-\nsation needed to reach the target based on the sensed location of the \ufb01nger midway through\nthe movement. To analyze the subjects performance we plot the average lateral devia-\ntion (cid:3)(cid:0)\b\u0006\t(cid:3)  (cid:0)(cid:3)\u0007\b(cid:5)\u0001(cid:12)\b(cid:3)(cid:7)(cid:5) in a set of bins of as a function of the true shift (cid:0) \b\u0006\t(cid:3). Because\nfeedback is not biased this term approximates (cid:3)(cid:0) \u0007(cid:3)\u0002\u0007(cid:3)(cid:7)  (cid:0)(cid:3)\u0007\b(cid:5)\u0001(cid:12)\b(cid:3)(cid:7)(cid:5). Three competing\ncomputational models are able to predict such a graph.\n1) Compensation model. Subjects could compensate for the sensed lateral shift (cid:0) \u0007(cid:3)\u0002\u0007(cid:3)(cid:7)\nand thus use (cid:0)(cid:3)\u0007\b(cid:5)\u0001(cid:12)\b(cid:3)(cid:7) (cid:2) (cid:0)\u0007(cid:3)\u0002\u0007(cid:3)(cid:7). The average lateral deviation should thus be\n(cid:3)(cid:0)\b\u0006\t(cid:3)  (cid:0)(cid:3)\u0007\b(cid:5)\u0001(cid:12)\b(cid:3)(cid:7)(cid:5) (cid:2) (cid:5) (Figure 1E, left panel). In this model, increasing the uncertainty\nof the feedback (cid:3)\u0007(cid:3)\u0002\u0007(cid:3)(cid:7) (by increasing the blur) affects the variability of the pointing but\nnot the average location. Errors arise from variability in the visual feedback and the means\nsquared error (MSE) for this strategy (ignoring motor variability) is (cid:3) (cid:1)\n\u0007(cid:3)\u0002\u0007(cid:3)(cid:7). Crucially this\nmodel does not require subjects to estimate their visual uncertainty nor the distribution of\n\n\fshifts.\n\n2) Bayesian model. Subjects could optimally use prior information about the distribution\nand the uncertainty of the visual feedback to estimate the lateral shift. They have to estimate\n(cid:0)\b\u0006\t(cid:3) given (cid:0)\u0007(cid:3)\u0002\u0007(cid:3)(cid:7). Using Bayes rule we can obtain the posterior distribution, that is the\nprobability of a shift (cid:0)\b\u0006\t(cid:3) given the evidence (cid:0)\u0007(cid:3)\u0002\u0007(cid:3)(cid:7),\n\n\u0004\u0004(cid:0)\b\u0006\t(cid:3)(cid:2)(cid:0)\u0007(cid:3)\u0002\u0007(cid:3)(cid:7)\u0005 (cid:2)\n\n\u0004\u0004(cid:0)\b\u0006\t(cid:3)\u0005\u0004\u0004(cid:0)\u0007(cid:3)\u0002\u0007(cid:3)(cid:7)(cid:2)(cid:0)\b\u0006\t(cid:3)\u0005\n\n\u0004\u0004(cid:0)\u0007(cid:3)\u0002\u0007(cid:3)(cid:7)\u0005\n\n(3)\n\nIf subjects choose the most likely shift they also minimize their mean squared error (MSE).\nWe can determine this optimal estimate (cid:0)(cid:3)\u0007\b(cid:5)\u0001(cid:12)\b(cid:3)(cid:7) by differentiating (3) after inserting (1)\nand (2). This optimal estimate is a weighted sum between the mean of the prior and the\nsensed feedback position:\n\n(cid:0)(cid:3)\u0007\b(cid:5)\u0001(cid:12)\b(cid:3)(cid:7) (cid:2)\n\n(cid:3)(cid:1)\n\n\u0007(cid:3)\u0002\u0007(cid:3)(cid:7)\n\n(cid:3)(cid:1)\n\u0007(cid:3)\u0002\u0007(cid:3)(cid:7) \u0007 (cid:3)(cid:1)\n\n\u0004\u0006(cid:5)\u0003\u0006\n\n(cid:0)(cid:7)(cid:5)\u0007\b \u0007\n\n(cid:3)(cid:1)\n\n\u0004\u0006(cid:5)\u0003\u0006\n\n(cid:3)(cid:1)\n\u0007(cid:3)\u0002\u0007(cid:3)(cid:7) \u0007 (cid:3)(cid:1)\n\n\u0004\u0006(cid:5)\u0003\u0006\n\n(cid:0)\u0007(cid:3)\u0002\u0007(cid:3)(cid:7)\n\n(4)\n\nThe average lateral deviation (cid:3)(cid:0) \b\u0006\t(cid:3)  (cid:0)(cid:3)\u0007\b(cid:5)\u0001(cid:12)\b(cid:3)(cid:7)(cid:5) is thus linearly dependent to (cid:0)\b\u0006\t(cid:3) and\nthe slope increases with increasing uncertainty (Figure 1E middle panel).\nThe MSE depends on two factors, the width of the prior (cid:3) \u0004\u0006(cid:5)\u0003\u0006 and the uncertainty in the\nvisual feedback (cid:3)\u0007(cid:3)\u0002\u0007(cid:3)(cid:7). Calculating the MSE for the above optimal choice we obtain:\n\n\u0005 (cid:10)(cid:11) (cid:2)\n\n(cid:3)(cid:1)\n\n\u0004\u0006(cid:5)\u0003\u0006\n(cid:3)(cid:1)\n\u0004\u0006(cid:5)\u0003\u0006 \u0007 (cid:3)(cid:1)\n\n\u0007(cid:3)\u0002\u0007(cid:3)(cid:7)\n\n(cid:3)(cid:1)\n\n\u0007(cid:3)\u0002\u0007(cid:3)(cid:7)\n\n(5)\n\nwhich is always less than the MSE for model 1. As we increase the blur, and thus the degree\nof uncertainty, the estimate of the shift moves away from the visually sensed displacement\n(cid:0)\u0007(cid:3)\u0002\u0007(cid:3)(cid:7) towards the mean of the prior distribution (cid:0) (cid:7)(cid:5)\u0007\b (Figure 1D). Such a computational\nstrategy thus allows subjects to minimize the MSE at the target.\n\n3) Mapping model. A third computational strategy is to learn a mapping from the sensed\nshift (cid:0)\u0007(cid:3)\u0002\u0007(cid:3)(cid:7) to the optimal lateral shift (cid:0)(cid:3)\u0007\b(cid:5)\u0001(cid:12)\b(cid:3)(cid:7). By minimizing the average error over\nmany trials the subjects could achieve a combination similar to model 2 but without any\nrepresentation of the prior distribution or the visual uncertainty. However, to learn such a\nmapping requires visual feedback and knowledge of the error at the end of the movement.\nIn our experiment we only revealed the shifted position of the \ufb01nger at the end of the\nmovement of the clear feedback trials ((cid:3) (cid:0)). Therefore, if subjects learn a mapping, they can\nonly do so for these trials and apply the same mapping to the blurred conditions ((cid:3) \u0005 , (cid:3)\u0004).\nTherefore, this model predicts that the average lateral shift (cid:3)(cid:0) \b\u0006\t(cid:3)  (cid:0)(cid:3)\u0007\b(cid:5)\u0001(cid:12)\b(cid:3)(cid:7)(cid:5) should be\nindependent of the degree of blur (Figure 1E right panel)\n\n2.3.1 Results: Lateral Deviation\nGraphs of (cid:3)(cid:0)\b\u0006\t(cid:3)  (cid:0)(cid:3)\u0007\b(cid:5)\u0001(cid:12)\b(cid:3)(cid:7)(cid:5) against (cid:0)\b\u0006\t(cid:3) are shown for a representative subject in Fig-\nure 2A. The slope increases with increasing uncertainty and is, therefore, incompatible with\nmodels 1 and 3 but is predicted by model 2. Moreover, this transition from using feedback\nto using prior information occurs gradually with increasing uncertainty as also predicted\nby this Bayesian model. These effects are consistent over all the subjects tested. The slope\nincreases with increasing uncertainty in the visual feedback (Figure 2B). Depending on the\nuncertainty of the feedback, subjects thus combine prior knowledge of the distribution of\nshifts with new evidence to generate the optimal compensatory movement.\n\nUsing Bayesian theory we can furthermore infer the degree of uncertainty from the errors\nthe subjects made. Given the width of the prior (cid:3) \u0004\u0006(cid:5)\u0003\u0006 (cid:2) (cid:5)(cid:7)(cid:6)(cid:5)\u0001 and the result in (4) we can\n\n\fA\n\n \n\n1\n\n]\n\nm\nc\n[\n\n\u03c30\n\n>\n\nn\no\n\ni\nt\n\ni\n\na\nv\ne\nd\n\n \nl\n\na\nr\ne\n\nt\n\ne\n\nt\n\na\nm\n\ni\nt\ns\ne\nx\n-\ne\nu\nr\nt\nx\n\na\n\nl\n\n<\n\n1\n\n]\n\nm\nc\n[\n\n\u03c3M\n\n>\n\n \n\nn\no\n\ni\nt\n\ni\n\na\nv\ne\nd\n\n \nl\n\na\nr\ne\n\nt\n\ne\n\nt\n\na\nm\n\ni\nt\ns\ne\nx\n-\ne\nu\nr\nt\nx\n\n0\n\na\n\nl\n\n<\n\n-1\n0\n\n2\nlateral shift xtrue [cm]\n\n1\n\n1\n\n]\n\nm\nc\n[\n\n\u03c3L\n\n>\n\n \n\nn\no\n\ni\nt\n\ni\n\na\nv\ne\nd\n\n \nl\n\na\nr\ne\n\nt\n\ne\n\nt\n\na\nm\n\ni\nt\ns\ne\nx\n-\ne\nu\nr\nt\nx\n\na\n\nl\n\n<\n\n-1\n\n1\n\n]\n\nm\nc\n[\n\n0\n2\nlateral shift xtrue [cm]\n\n\u03c38\n\n>\n\n \n\nn\no\n\ni\nt\n\ni\n\na\nv\ne\nd\n\n \nl\n\na\nr\ne\n\nt\n\ne\n\nt\n\na\nm\n\ni\nt\ns\ne\nx\n-\ne\nu\nr\nt\nx\n\na\n\nl\n\n<\n\n***\n\n**\n\n***\n\n\u03c30\n\n\u03c3M\n\n\u03c3L \u03c38\n\nB\n\nC\n\n1\n\ne\np\no\ns\n\nl\n\n0\n\n1\n\nr\no\ni\nr\np\n\n \n\nd\ne\nr\nr\ne\n\nf\n\nn\n\ni\n\n]\n\nU\nA\n\n[\n\n-1\n0\n\n1\n\n2\nlateral shift xtrue [cm]\n\n-1\n\n1\n\n0\n2\nlateral shift xtrue [cm]\n\n0\n-0.5\n\n1\n\n2.5\n\nlateral shift xtrue [cm]\n\nFigure 2: Results with color codes as in Figure 1. A) The average lateral deviation of the\ncursor at the end of the trial as a function of the imposed lateral shift for a typical subject.\nErrorbars denote the s.e.m. The horizontal dotted lines indicate the prediction from the\nfull compensation model and sloped line for a model that ignores sensory feedback on the\ncurrent trial and corrects only for the mean over all trials. B) The slopes for the optimal\nlinear \ufb01ts are shown for the full population of subjects. The stars indicate the signi\ufb01cance\nindicated by the paired t-test. C) The inferred priors and the real prior (red) for each subjects\nand condition.\n\nestimate the uncertainty (cid:3)\u0007(cid:3)\u0002\u0007(cid:3)(cid:7) from Fig 2A. For the three levels of imposed uncertainty,\n(cid:3)(cid:0), (cid:3)\u0005 and (cid:3)\u0004, we \ufb01nd that the subjects uncertainty (cid:3) \u0007(cid:3)\u0002\u0007(cid:3)(cid:7) are 0.36\u00060.1, 0.67\u00060.3,\n0.8\u00060.2 cm (mean\u0006sd across subjects), respectively. Furthermore we have developed a\nnovel technique to infer the priors used by the subjects. An obvious choice of (cid:0) (cid:3)\u0007\b(cid:5)\u0001(cid:12)\b(cid:3)\nis the maximum of the posterior \u0004\u0004(cid:0) \b\u0006\t(cid:3)(cid:2)(cid:0)\u0007(cid:3)\u0002\u0007(cid:3)(cid:7)\u0005. The derivative of this posterior with\nrespect to (cid:0)\b\u0006\t(cid:3) must vanish at the optimal (cid:0)(cid:3)\u0007\b(cid:5)\u0001(cid:12)\b(cid:3). This allows us to estimate the prior\nused by each subject. Taking derivatives of (3) after inserting (2) and setting to zero we\nget:\n\n(cid:12)\u0004\u0004(cid:0)\b\u0006\t(cid:3)\u0005\n\n(cid:12)(cid:0)\b\u0006\t(cid:3)\n\n(cid:3)\n\n\u0004\u0004(cid:0)\b\u0006\t(cid:3)\u0005(cid:0)(cid:0)(cid:0)(cid:0)(cid:14)(cid:4)\u0007\b(cid:6)\u0001(cid:13)\b(cid:4)\n\n(cid:2)\n\n(cid:0)\u0007(cid:3)\u0002\u0007(cid:3)(cid:7)  (cid:0)(cid:3)\u0007\b(cid:5)\u0001(cid:12)\b(cid:3)(cid:7)\n\n(cid:3)(cid:1)\n\n\u0007(cid:3)\u0002\u0007(cid:3)(cid:7)\n\n(6)\n\nWe assume that (cid:0)(cid:3)\u0007\b(cid:5)\u0001(cid:12)\b(cid:3) has a narrow peak around (cid:0) \b\u0006\t(cid:3) and thus approximate it by (cid:0) \b\u0006\t(cid:3).\nWe insert the (cid:3)\u0007(cid:3)\u0002\u0007(cid:3)(cid:7) obtained in (4), affecting the scaling of the integral but not its form.\nThe average of (cid:0)\u0007(cid:3)\u0002\u0007(cid:3)(cid:7) across many trials is the imposed shift (cid:0)\b\u0006\t(cid:3). Therefore the right\nhand side is measured in the experiment and the left hand side approximates the derivative\nof \u0003(cid:10) \u0004\u0004(cid:0)\b\u0006\t(cid:3)\u0005. Since \u0004 must approach zero for both very small and very large (cid:0) \b\u0006\t(cid:3), we\nsubtract the mean of the right hand side before integrating numerically to obtain an estimate\nthe prior \u0004\u0004(cid:0)\b\u0006\t(cid:3)\u0005. Figure 2C shows the priors inferred for each subject and condition. This\nshows that the real prior (red line) was reliably learned by each subject.\n\n3 Experiment 2: Mixture of Gaussians Priors\n\nThe second experiment was designed to examine whether subjects are able to represent\nmore complicated priors such as mixtures of Gaussians and if they can utilize such prior\nknowledge.\n\n\f3.1 Methods\n\n12 additional subjects participated in an experiment similar to Experiment 1 with the fol-\nlowing changes. The experiments lasted for twice as many trials run on two consecutive\ndays with 2000 trials performed on each day. Feedback midway through the movement\nwas always blurred (spheres distributed as a two dimensional Gaussian with (cid:3) (cid:2) (cid:11)(cid:5)\u0001) and\nfeedback at the end of the movement was provided on every trial. The prior distribution\nwas a mixture of Gaussians ( Figure 3A,D). One group of 6 subjects was exposed to:\n\n\u0004\u0004(cid:0)\b\u0006\t(cid:3)\u0005 (cid:2)\n\n(cid:3)\n\n(cid:4)\u0004(cid:4)(cid:2)(cid:3)\u0004\u0006(cid:5)\u0003\u0006 (cid:1)(cid:4)\n\n\u0004(cid:0)\b\u0006\t(cid:4) (cid:0)(cid:5)(cid:6)\u0007\b\u0005(cid:2)\n\n(cid:2)(cid:8)(cid:2)\n\n\u0004\u0006(cid:6)\u0003\u0006\n\n\u0007 (cid:4)\n\n\u0004(cid:0)\b\u0006\t(cid:4) \u0007(cid:0)(cid:5)(cid:6)\u0007\b\u0005(cid:2)\n\n(cid:2)(cid:8)(cid:2)\n\n\u0004\u0006(cid:6)\u0003\u0006 (cid:2)\n\n(7)\n\nwhere (cid:0)(cid:7)(cid:5)\u0007\b (cid:2) (cid:4)(cid:5)\u0001 is half the distance between the two peaks of the Gaussians. (cid:3) \u0004\u0006(cid:5)\u0003\u0006 is\nthe width of each Gaussian which is set to 0.5 cm. Another group of 6 subjects experienced\n\n\u0004\u0004(cid:0)\b\u0006\t(cid:3)\u0005 (cid:2)\n\n(cid:3)\n\n(cid:3)(cid:5)\u0004(cid:4)(cid:2)(cid:3)\u0004\u0006(cid:5)\u0003\u0006 (cid:1)(cid:4)\n\n\u0004(cid:0)\b\u0006\t(cid:4) (cid:0)(cid:5)(cid:6)\u0007\b\u0005(cid:2)\n\n(cid:2)(cid:8)(cid:2)\n\n\u0004\u0006(cid:6)\u0003\u0006\n\n\u0007 (cid:4)\n\n\u0004(cid:0)\b\u0006\t(cid:4) \u0007(cid:0)(cid:5)(cid:6)\u0007\b\u0005(cid:2)\n\n(cid:2)(cid:8)(cid:2)\n\n\u0004\u0006(cid:6)\u0003\u0006\n\n\u0007 (cid:12)(cid:4)\n\n(cid:2)(cid:8)(cid:2)\n\n(cid:0)(cid:2)\n\n\u0004\u0006(cid:6)\u0003\u0006(cid:2)\n\n(8)\n\nIn this case we set (cid:0)(cid:7)(cid:5)\u0007\b (cid:2) (cid:4)\u0004(cid:6) so that the variance is identical to the two Gaussians case.\n(cid:3)\u0004\u0006(cid:5)\u0003\u0006 is still 0.5 cm.\nTo estimate the priors learned by the subjects we \ufb01tted and compared two models. The\n\ufb01rst assumed that subjects learned a single Gaussian distribution and the second assumed\nthat subjects learned a mixture of Gaussians and we tuned the position of the Gaussians to\nminimizes the MSE between predicted and actual data.\n\np(xtrue)\n\nB\n\n \n\nsingle subject\n\n1\n\n]\n\nm\nc\n[\n\nall subjects\n\n1\n\n]\n\nm\nc\n[\n\n0\n\ni\nt\n\ni\n\na\nv\ne\nd\n\n \nl\n\n>\n\nn\no\n\ni\nt\n\ni\n\na\nv\ne\nd\n\n \nl\n\na\nr\ne\n\ne\n\nt\n\na\nm\n\ni\nt\ns\ne\nx\n-\ne\nu\nr\nt\nx\n\nt\n\n-1\n<\n\na\n\nl\n\nC\n\n \n\nn\no\n\n>\n\n0\n\ne\n\nt\n\na\nm\n\ni\nt\ns\ne\nx\n-\ne\nu\nr\nt\nx\n\na\nr\ne\n\nt\n\n-1\n<\n\na\n\nl\n\nA\n\n1\n\ny\nc\nn\ne\nu\nq\ne\nr\nf\n \ne\nv\ni\nt\na\ne\nr\n\nl\n\n0\n\nD\n\n1\n\ny\nc\nn\ne\nu\nq\ne\nr\nf\n \ne\nv\ni\nt\na\ne\nr\n\nl\n\n0\n\n-2\nlateral shift xtrue [cm]\n\n0\n\n2\n\np(xtrue)\n\nE\n\n \n\n-2\nlateral shift xtrue [cm]\n\n0\n\n2\n\nsingle subject\n\n3\n\n]\n\nm\nc\n[\n\n>\n\nn\no\n\ni\nt\n\ni\n\na\nv\ne\nd\n\n \nl\n\na\nr\ne\n\nt\n\n0\n\ne\n\nt\n\na\nm\n\ni\nt\ns\ne\nx\n-\ne\nu\nr\nt\nx\n\na\n\nl\n\n<\n\n0\n\n-2\nlateral shift xtrue [cm]\n\n2\n\nall subjects\n\n3\n\n]\n\nm\nc\n[\n\nF\n\n \n\n>\n\nn\no\n\ni\nt\n\ni\n\na\nv\ne\nd\n\n \nl\n\na\nr\ne\n\nt\n\n0\n\ne\n\nt\n\na\nm\n\ni\nt\ns\ne\nx\n-\ne\nu\nr\nt\nx\n\na\n\nl\n\n<\n\n\u22124\n\n0\n\n4\nlateral shift xtrue [cm]\n\n-3\n\n\u22124\n\n0\n\n4\nlateral shift xtrue [cm]\n\n-3\n\n\u22124\n\n0\n\n4\n\nlateral shift xtrue [cm]\n\nFigure 3: A\u0005 The used distribution of (cid:0) \b\u0006\t(cid:3) as mixture of Gaussians model. B\u0005 The per-\nformance of an arbitrarily chosen subject is shown together with a \ufb01t from the ignore prior\nmodel (dotted line), the Gaussian model (dashed line) and the Bayesian Mixture of Gaus-\nsians model (solid line) C\u0005 the average response over all subjects is shown D-F\u0005 shows the\nsame as A-C\u0005 for the Three Gaussian Distribution\n\n\f3.2 Results: Two Gaussians Distribution\n\nThe resulting response graphs (Figure 3B,C) show clear nonlinear effects. Fitting the\n(cid:3)\u0007(cid:3)\u0002\u0007(cid:3)(cid:7) and (cid:0)(cid:7)(cid:5)\u0007\b to a two component Mixture of Gaussians model led to an average error\nover all 6 subjects of 0.14\u00060.01 cm compared to an average error obtained for a single\nGaussian of 0.19\u00060.02 cm for the two Gaussians model. The difference, is signi\ufb01cant\nat \u0004 (cid:13) (cid:5)(cid:7)(cid:5)(cid:3). The mixture model of the prior is thus better able to explain the data than\nthe model that assumes that people can just represent one Gaussian. One of the subjects\ncompensated least for the feedback and his data was well \ufb01t by a single Gaussian. After\nremoving this subject from the dataset we could \ufb01t the width of the distribution (cid:0) (cid:7)(cid:5)\u0007\b and\nobtained 2.4\u00060.4 cm, close to the real value of the probability density function of 2 cm.\n3.3 Results: Three Gaussians Distribution\n\nThe resulting response graphs (Figure 3E,F) again show clear nonlinear effects. Fitting the\n(cid:3)\u0007(cid:3)\u0002\u0007(cid:3)(cid:7) and (cid:0)(cid:7)(cid:5)\u0007\b of the three Gaussians model (Figure 3E) led to an average error over\nall subjects of 0.21\u00060.02 cm instead of an error from a single Gaussians of 0.25\u00060.02\ncm. The \ufb01tted distance (cid:0)(cid:7)(cid:5)\u0007\b however was 2.0\u00060.4 cm, signi\ufb01cantly smaller than the real\ndistance.\n\nThis result shows that subjects can not fully learn this more complicated distribution but\nrather just learn some of its properties. This could be due to several effects. First, large\nvalues of (cid:0)\b\u0006\t(cid:3) are experienced only rarely. Second, it could be that subjects use a simpler\nmodel such as a generalized Gaussian (the family of distribution that also the Laplacian\ndistribution belongs to) or that they use a mixture of only a few Gaussians model. Third,\nsubjects could have a prior over priors that makes a mixture of three Gaussians model very\nunlikely. Learning such a mixture would therefore be expected to take far longer.\n\n3.4 Results: Evolution of the Subjects Performance\n\nA\n\nC\n\n5\n\n]\n\nm\nc\n[\n \nr\no\nr\nr\ne\n\n \n\ne\ng\na\nr\ne\nv\na\n\n0\n\n0\n\n20\n\ntrial \n\n4000\n\ni\n\nd\ne\nn\na\np\nx\ne\n\nl\n\n \n\ne\nc\nn\na\ni\nr\na\nv\n \nl\n\na\nn\no\n\ni\nt\ni\n\nd\nd\na\n\n]\n\n%\n\n[\n \nl\n\ne\nd\no\nm\n\n \nl\nl\n\nu\nF\n \ny\nb\n\n0\n\n1\n\n2\n\n3\n\n5\n\n4\nblocks of 500 trials \n\n6\n\n7\n\n8\n\nB\n\n]\n\nm\nc\n[\n\n2\n>\n\n \n\nn\no\n\ni\nt\n\ni\n\na\nv\ne\nd\n\n \nl\n\na\nr\ne\n\nt\n\ne\n\nt\n\na\nm\n\ni\nt\ns\ne\nx\n-\ne\nu\nr\nt\nx\n\na\n\n-2\n<\n\nl\n\n01-500\n\n-2\n\n0\n\n2\n\nlateral shift xtrue [cm]\n\n2\n\n-2\n\n2\n\n-2\n\n2\n\n-2\n\n1001-1500\n\n-2\n\n0\n\n2\n\n3001-3500\n\n-2\n\n0\n\n2\n\n3001-3500\n\n-2\n\n0\n\n2\n\n2\n\n-2\n\n2\n\n-2\n\n2\n\n-2\n\n2\n\n-2\n\n501-1000\n\n-2\n\n0\n\n2\n\n1501-2000\n\n-2\n\n0\n\n2\n\n2501-3000\n\n-2\n\n0\n\n2\n\n3501-4000\n\n-2\n\n0\n\n2\n\nFigure 4: A\u0005 The mean error over the 6 subjects is shown as a function of the trial number\nB\u0005 The average lateral deviation as a function of the shift and the trial number C\u0005 The\nadditional variance explained by the full model is plotted as a function of the trial number\n\nAs a next step we wanted to analyze how the behaviour of the subjects changes over the\n\n\fcourse of training. During the process of training the average error over batches of 500\nsubsequent trials decreased from 1.97 cm to 0.84 cm (Figure 4A). What change leads to\nthis decrease?\n\nTo address this we plot the evolution of the lateral deviation graph, as a function of the\ntrial number (Figure 4B). Subjects initially exhibit a slope of about 1 and approximately\nlinear behaviour. This indicates that initially they are using a narrow Gaussian prior. In\nother words they rely on the prior belief that their hand will not be displaced and ignore\nthe feedback. Only later during training do they show behaviour that is consistent with a\nbimodal Gaussians distribution.\n\nIn Figure 4C we plot the percentage of additional variance explained by the full model\nwhen compared to the Gaussian model averaged over the population. It seems that in par-\nticular after trial 2000, the trial after which people enjoy a nights rest, does the explanatory\npower of the full model improve. It could be that subjects need a consolidation period to\nadequately learn the distribution. Such improvements in learning contingent upon sleep\nhave also been observed in visual learning [7].\n\n4 Conclusion\n\nWe have shown that a prior is used by humans to determine appropriate motor commands\nand that it is combined with an estimate of sensory uncertainty. Such a Bayesian view of\nsensorimotor learning is consistent with neurophysiological studies that show that the brain\nrepresents the degree of uncertainty when estimating rewards [8-10] and with psychophys-\nical studies addressing the timing of movements [11]. Not only do people represent the\nuncertainty and combine this with prior information, they are also able to represent and uti-\nlize complicated nongaussian priors. Optimally using a priori knowledge might be key to\nwinning a tennis match. Tennis professionals spend a great deal of time studying their op-\nponent before playing an important match - ensuring that they start the match with correct\na priori knowledge.\n\nAcknowledgments\n\nWe like to thank Zoubin Ghahramani for inspiring discussions and the Wellcome Trust for\n\ufb01nancial support. We also like to thank James Ingram for technical support.\n\nReferences\n\n[1] Cox, R.T. (1946) American Journal of Physics 17, 1\n\n[2] Bernardo, J.M. & Smith, A.F.M. (1994) Bayesian theory. John Wiley\n\n[3] Berrou, C., Glavieux, A. & Thitimajshima, P. (1993) Proc. ICC\u201993 Geneva, Switzerland 1064\n\n[4] Simoncelli, E.P. & Adelson, E.H. (1996) Proc. 3rd International Conference on Image Processing\nLausanne, Switzerland\n\n[5] Weiss, Y., Simoncelli, E.P. & Adelson, E.H. (2002) Nature Neuroscience 5, 598\n\n[6] Goodbody, W. & Wolpert, D. (1998) Journal of Neurophysiology 79,1825\n\n[7] Stickgold, R., James, L. & Hobson, J.A. (2000) Nature 3 ,1237\n\n[8] Fiorillo, C.D., Tobler, P.N. & Schultz, W. (2003) Science 299, 1898\n\n[9] Basso, M.A. & Wurt, R.H. (1998) Journal of Neuroscience 18, 7519\n\n[10] Platt M.L. (1999) Nature 400, 233\n\n[11] Carpenter, R.H. & Williams, M.L. Nature 377, 59\n\n\f", "award": [], "sourceid": 2423, "authors": [{"given_name": "Konrad", "family_name": "K\u00f6rding", "institution": null}, {"given_name": "Daniel", "family_name": "Wolpert", "institution": null}]}