{"title": "A Minimal Intervention Principle for Coordinated Movement", "book": "Advances in Neural Information Processing Systems", "page_first": 27, "page_last": 34, "abstract": "", "full_text": "A Minimal Intervention Principle for\n\nCoordinated Movement\n\nEmanuel Todorov\n\nDepartment of Cognitive Science\nUniversity of California, San Diego\ntodorov@cogsci.ucsd.edu\n\nMichael I. Jordan\n\nComputer Science and Statistics\nUniversity of California, Berkeley\njordan@cs.berkeley.edu\n\nAbstract\n\nBehavioral goals are achieved reliably and repeatedly with movements\nrarely reproducible in their detail. Here we offer an explanation: we show\nthat not only are variability and goal achievement compatible, but indeed\nthat allowing variability in redundant dimensions is the optimal control\nstrategy in the face of uncertainty. The optimal feedback control laws for\ntypical motor tasks obey a \u201cminimal intervention\u201d principle: deviations\nfrom the average trajectory are only corrected when they interfere with\nthe task goals. The resulting behavior exhibits task-constrained variabil-\nity, as well as synergetic coupling among actuators\u2014which is another\nunexplained empirical phenomenon.\n\n1 Introduction\n\nBoth the dif\ufb01culty and the fascination of the motor coordination problem lie in the ap-\nparent con\ufb02ict between two fundamental properties of the motor system:\nthe ability to\naccomplish its goal reliably and repeatedly, and the fact that it does so with variable move-\nments [1]. More precisely, trial-to-trial \ufb02uctuations in individual degrees of freedom are on\naverage larger than \ufb02uctuations in task-relevant movement parameters\u2014motor variability\nis constrained to a redundant or \u201cuncontrolled\u201d manifold [16] rather than being suppressed\naltogether. This pattern has now been observed in a long list of behaviors [1, 6, 16, 14].\nIn concordance with such naturally occurring variability, experimentally induced perturba-\ntions [1, 3, 12] are compensated in a way that maintains task performance rather than a\nspeci\ufb01c stereotypical movement pattern.\n\nThis body of evidence is fundamentally incompatible with standard models of motor co-\nordination that enforce a strict separation between trajectory planning and trajectory exe-\ncution [2, 8, 17, 10]. In such serial planning/execution models, the role of the planning\nstage is to resolve the redundancy inherent in the musculo-skeletal system, by replacing\nthe behavioral goal (achievable via in\ufb01nitely many movement trajectories) with a speci\ufb01c\n\u201cdesired trajectory.\u201d Accurate execution of the desired trajectory guarantees achievement\nof the goal, and can be implemented with relatively simple trajectory-tracking algorithms.\nWhile this approach is computationally viable (and often used in engineering), the numer-\nous observations of task-constrained variability and goal-directed corrections indicate that\nthe online execution mechanisms are able to distinguish, and selectively enforce, the details\nthat are crucial for the achievement of the goal. This would be impossible if the behavioral\n\n\fgoal were replaced with a speci\ufb01c trajectory.\n\nInstead, these observations imply a very different control scheme, one which pursues the\nbehavioral goal more directly. Efforts to delineate such a control scheme have led to the\nidea of motor synergies, or high-level \u201ccontrol knobs,\u201d that have invariant and predictable\neffects on the task-relevant movement parameters despite variability in individual degrees\nof freedom [9, 11]. But the computational underpinnings of such an approach\u2014how the\nsynergies appropriate for a given task and plant can be constructed, what control scheme is\ncapable of utilizing them, and why the motor system should prefer such a control scheme\nin the \ufb01rst place\u2014remain unclear. This general form of hierarchical control implies corre-\nlations among the control signals sent to multiple actuators (i.e., synergetic coupling) and\na corresponding reduction in control space dimesionality. Such phenonema have indeed\nbeen observed [4, 18], but the relationship to the hypothetical functional synergies remains\nto be established.\n\nIn this paper we aim to resolve the apparent con\ufb02ict at the heart of the motor coordina-\ntion problem, and clarify the relationship between variability, task goals, and motor syn-\nergies. We treat motor coordination within the framework of stochastic optimal control,\nand postulate that the motor system approximates the best possible control scheme for a\ngiven task. Such a control scheme will generally take the form of a feedback control law.\nWhenever the task allows redundant solutions, the initial state of the plant is uncertain, the\nconsequences of the control signals are uncertain, and the movement duration exceeds the\nshortest sensory-motor delay, optimal performance is achieved by a feedback control law\nthat resolves redundancy moment-by-moment\u2014using all available information to choose\nthe most advantageous course of action under the present circumstances. By postponing\nall decisions regarding movement details until the last possible moment, this control law\ntakes advantage of the opportunities for more successful task completion that are con-\nstantly being created by unpredictable \ufb02uctuations away from the average trajectory. Such\nexploitation of redundancy not only results in higher performance, but also gives rise to\ntask-constrained variability and motor synergies\u2014the phenomena we seek to explain.\n\nThe present paper is related to a recent publication targeted at a neuroscience audience\n[14]. Here we provide a number of technical results missing from [14], and emphasize the\naspects of our work that are most likely to be of interest to the computational modeling\ncommunity.\n\n2 The Minimal Intervention principle\n\nOur general explanation of the above phenomena follows from an intuitive property of op-\ntimal feedback controllers which we call the \u201cminimal intervention\u201d principle: deviations\nfrom the average trajectory are corrected only when they interfere with task performance.\n\nIf this principle holds, and the noise perturbs the system in all directions, the interplay of\nthe noise and control processes will result in variability which is larger in task-irrelevant\ndirections. At the same time, the fact that certain deviations are not being corrected im-\nplies that the corresponding control subspace is not being used\u2014which is the phenomenon\ntypically interpreted as evidence for motor synergies [4, 18].\n\nWhy should the minimum intervention principle hold? An optimal feedback controller has\nnothing to gain from correcting task-irrelevant deviations, because its only concern is task\nperformance and by de\ufb01nition such deviations do not interfere with performance. On the\nother hand, generating a corrective control signal can be detrimental, because: 1) the noise\nin the motor system is known to be multiplicative [13] and therefore could increase; 2) the\ncost being minimized most likely includes a control-dependent effort penalty which could\nalso increase.\n\n\fWe now formalize the notions of \u201credundancy\u201d and \u201ccorrection,\u201d and show that for a sur-\nprisingly general class of systems they are indeed related\u2014as our intuition suggests.\n\n2.1 Local analysis of a general class of optimal control problems\n\nRedundancy is not easy to de\ufb01ne. Consider the task of reaching, which requires the \ufb01nger-\n, all arm con\ufb01gurations for\nthis geo-\nmetric approach is insuf\ufb01cient to de\ufb01ne redundancy. Therefore we follow a more general\napproach.\n\ntip to be at a speci\ufb01ed target at some point in time\nwhich the \ufb01ngertip is at the target are redundant. But at times different from\n\n. At time\n\nConsider a system with state\n\n, control\n\n, instantaneous scalar cost\n\n, and dynamics\n\n\u0001\u0003\u0002\u0005\u0004\u0007\u0006\t\b\u000b\n\r\f\n\u0001 \u001f\"!#\u0002\u0005\u0004\u0015\u0014\u0016\u0001$\u0014\u0016\u000e%\u0006\n\n\u000e\u000f\u0002\u0010\u0004\u0007\u0006\t\b\u0011\n\r\u0012\n\u001e.-\n\u0004'&)(*\u0002+\u0004\u0015\u0014\u0007\u0001$\u0014\u0019\u000e,\u0006\n\ntion, de\ufb01ned as\n\nis multidimensional standard Brownian motion. Control signals are\n\n. The analysis below heavily relies on properties of the optimal cost-to-go func-\n\ngenerated by a feedback control law, which can be any mapping of the form\n\n\u0002\u0010\u0004\u0015\u0014\u0016\u0001\u0017\u0002\u0005\u0004\u0007\u0006\u0018\u0014\u0016\u000e\u000f\u0002\u0010\u0004\u0007\u0006\u0019\u0006\u001b\u001a\u001d\u001c\nwhere -\n\u0002\u0010\u0004\u0007\u0006\u001d\b/\n\r0\n\u0002\u0010\u0004\u0015\u0014\u0016\u0001\u0017\u0002\u0005\u0004\u0007\u0006\u0016\u0006\n\u0002\u0005NO\u0014\u0007\u0001\u0003\u0002PNF\u0006\u0018\u0014\n\u0002+NO\u0014\u0007\u0001\u0003\u0002PNF\u0006\u0007\u0006\u0007\u0006\nwhere the minimum is achieved by the optimal control law3\n\u0002\u0005\u0004\u0015\u0014\u0007\u0001\u0003\u0002\u0010\u0004\u0007\u0006\u0019\u0006\nQR\u0001\n\u0001@\u0006\n\u0002UQR\u00017\u0006\u0017\u001f/\u001c\n\n\u0002\u0005\u0004\u0015\u0014\u0007\u00017\u00068\u001f:9\u0017;=<\n>@?\u0010ACBCA\nDFEHG\n\u0001\t\u0002\u0010\u0004\u0007\u0006\n5 be the change in the optimal cost-to-go4\n\u0002PQR\u00017\u0006S\u001f\n\u0001\t&\u001dQR\u00017\u00068T\nQR\u0001\nfunction\u0013 and optimal cost-to-go4\n5 are identical.\n\nSuppose that in a given task the system of interest (driven by the optimal control law)\ngenerates an average trajectory\nbe the deviation form the\naverage trajectory at time\nthe deviation\n. Now we are ready to de\ufb01ne\nredundancy: the deviation\n. Note that our de\ufb01nition\nreduces to the intuitive geometric de\ufb01nition at the end of the movement, where the cost\n\nTo de\ufb01ne the notion of \u201ccorrection,\u201d we need to separate the passive and active dynamics:\n\n5 due to\n\n. On a given trial, let\n\n\u000e1\u0002\u0005\u0004\u0007\u00062\u001f\n\nis redundant iff\n\nDJILK\n\nQR\u0001\n\n. Let\n\n; i.e.,\n\n465\n\n?\u0005A\n\n.\n\n.\n\nand\n\n\u0002\u0010\u0004\u0015\u0014\n\ndue to the control\n\n\u00011&XQR\u00017\u0006\n\nis given [7] by the minimum\n\n, we obviously need to know\n\nIn order to relate the quantities\n\n\u0001@Z\"\u001f[W\u000b\u0002+\u0004\u0015\u0014\n\ncan now\n. The corrective action of the control\n\nThe (in\ufb01nitesimal) expected change in\nbe identi\ufb01ed:\nsignal is naturally de\ufb01ned as\n\n!#\u0002\u0010\u0004\u0015\u0014\u0007\u0001$\u0014\u0019\u000e,\u0006#\u001f\u000bVS\u0002\u0005\u0004\u0015\u0014\u0007\u00017\u00067&LW\u000b\u0002\u0010\u0004\u0015\u0014\u0016\u00017\u00066\u000e\n\u000e*\u001f\n\u00011&LQR\u00017\u0006\n\u0002\u0010\u0004\u0015\u0014\n\u0001\t&LQR\u00017\u0006\n\u0014cQd\u00017e\n\\^]O_\u0016_'\u0002PQR\u00017\u00068\u001fa`\u0016TbY\n\u0002UQR\u00017\u0006\n\\^]O_\u0016_'\u0002UQd\u00017\u0006\nsomething about the optimal control law3\n5 . For problems in the above general form, the\noptimal control law3\n\u0002\u0005\u0004\u0015\u0014\u0007\u0001\u0003\u0002\u0010\u0004\u0007\u0006\u0016\u0006\n\u0002\u0010\u0004\u0015\u0014\u0007\u00017\u0006@&lk\n_\u0019g$9\u0017;=<\n\u0002h\u0004\u0015\u0014\u0007\u0001$\u0014\u0019\u000e,\u0006@&)!#\u0002\u0005\u0004\u0015\u0014\u0016\u0001$\u0014\u0016\u000e%\u0006ji\n\u0002\u0010\u0004\u0015\u0014\u0016\u00017\u00066(*\u0002\u0010\u0004\u0015\u0014\u0016\u0001$\u0014\u0016\u000e%\u0006\u0007u\n\\^prqs(*\u0002+\u0004\u0015\u0014\u0007\u0001$\u0014\u0016\u000e%\u0006ji\nmon\nwhere4\nand4\n\u0002\u0010\u0004\u0015\u0014\u0007\u00017\u0006\n\u0002\u0005\u0004\u0015\u0014\u0007\u00017\u0006\ntion4\nGtG\n\u0002\u0010\u0004\u0015\u0014\u0016\u00017\u0006\n(\u001d\u0002+\u0004\u0015\u0014\u0007\u0001$\u0014\u0019\u000e,\u0006v\u001f\n\u0002\u0010\u0004\u0015\u0014\u0007\u0001$\u0014\u0019\u000e,\u0006v\u001f\n\nThe matrix notation means that the\n. Note that the latter\nformulation is still very general, and can represent realistic musculo-skeletal dynamics and\nmotor tasks.\n\nare the gradient and Hessian of the optimal cost-to-go func-\n. To be able to minimize this expression explicitly, we will restrict the class of\n\nproblems to\n\ncolumn of\n\n465\nGtG\n\ni\u0083\u0082\n\n465\n\n\u0002\u0010\u0004\u0015\u0014\u0016\u00017\u00066\u000e\nwyxHz\n\u0080\u0081\u0002\u0010\u0004\u0015\u0014\u0016\u00017\u00067&lk\n\u0084c\u0085\u0005\u0086\n\n{s{|{\n\u0002+\u0004\u0015\u0014\u0007\u00017\u0006}\u000e\nxy\u0087\n\nis\n\n\u0002\u0005\u0004\u0015\u0014\u0016\u00017\u0006}\u000e\u007f~\n\u0002+\u0004\u0015\u0014\u0007\u00017\u0006}\u000e\n\n\u0013\n\u001e\n\u001e\n3\nM\n\u0013\n3\n\u001e\nN\n5\n\u0004\nQ\n4\nQ\n4\n5\n4\n5\n\u0002\n4\n5\n\u0002\nQ\n4\n5\n\u0001\n3\n5\nY\n3\n5\n\u0001\nZ\nQ\n4\n5\n5\nf\nZ\n\u0013\nG\n_\nf\n5\nG\n5\n5\nx\n0\n\u0013\nm\n\u000e\n(\n\fUsing the fact1 that\ninating terms that do not depend on\nbecomes\n\n, the expression that has to be minimized w.r.t\n\n, and elim-\n\n\\|p\r\u0002\u0005\u0004\u0007\u0006R\u0006S\u001f\n\u0002+\u0004\u0015\u0014\u0007\u00017\u0006\nx\u001b\u0087\n\u0011\u0013\u0012\n\nGtG\n\n\\|p\r\u0002\b\u0006\t\u0004\n\u0002\u0010\u0004\u0015\u0014\u0016\u00017\u0006\nx\u001b\u0087\n\n\u0002+\u0004\u0015\u0014\u0007\u00017\u0006\u000f\u000e\n\nWe now return to the relationship between \u201credundancy\u201d and \u201ccorrection.\u201d The time in-\ndex\nwill be suppressed for clarity. We expand the optimal cost-to-go to second order:\n, also expand its gradient\n, and approximate all other quantities\n. The effect of the control signal becomes\n. Substituting in the above de\ufb01ni-\n\nas being constant in a small neighborhood of\n\n\u0001@\u0006\n\n\u0002\u0010\u0004\u0015\u0014\u0016\u00017\u0006\nQd\u0001\n\n\u0087\u0003\u0002\n\n\u001f\u0001\n(S(\n\u0002\u0010\u0004\u0015\u0014\u0007\u00017\u0006@&lk\n\nW\u0011\u0002\u0010\u0004\u0015\u0014\u0007\u00017\u0006\n\nTherefore the optimal control law is\n\ntions yields\n\nT\u0017\u0016\n\u0002+\u0004\u0015\u0014\u0007\u00017\u0006\r\u001f\n\u0001\u0003&XQR\u00017\u0006\u001a\u0019\n\u00011&XQR\u00017\u00068&\u000bQR\u0001\nto \ufb01rst order: 4\n\u00011&XQd\u00017\u0006\u0007\u0019\n\u00017\u0006$&\n\u0001,\u0006\u001c\u0016\n\u0001'Z\u001b\u0019\nW\u000b\u0002\n\u00017\u0006@&\n\u0001@\u0006\n\u00017\u0006\nTyW\u000b\u0002\n4}5\n4}5\n\u0002+QR\u00017\u0006\u001d\u0019\n`+QR\u0001$\u0014\n4}5\n_\u0019_'\u0002+QR\u00017\u0006\u001d\u0019\n`+QR\u0001$\u0014\n\\|]\n\u0002+QR\u00017\u0006\n\\^]O_\u0016_7\u0002UQR\u00017\u0006\n\u00017\u00066QR\u0001\n\u0002UQR\u00017\u0006\nW\u000b\u0002\n\u0001,\u0006\n\nwhere the weighted dot-product notation\n\n\u0001@\u0006\u0081&\nsian 4\nGtG\n\u0001@\u0006\u001c\u0016\nW\u000b\u0002\n\nGtG\n\u00017\u0006\n\u00017\u0006\n\nand\n\nand\n\n\u0087\r\u0002\nW\u000b\u0002\u0010\u0004\u0015\u0014\u0016\u00017\u0006\nGtG\n\n\u000e,\u000e\n\u0002\u0005\u0004\u0015\u0014\u0007\u00017\u00067&\f\u000b\n\u0002\u0010\u0004\u0015\u0014\u0016\u00017\u0006\u0013\u0018\n\u0001,\u0006\r&\u0011QR\u0001\n\u0001@\u0006\nQd\u0001\nGtG\n\u00017\u00066QR\u00017\u0006\nGtG\n465\n\u00017\u00067&\n\u00017\u0006\u0083QR\u00017e\nGtG\n465\n\u00017\u0006\u0083QR\u00017e\u000f\u001e\n\u00017\u00067&\nGtG\n`\u0010\u0001$\u0014&%@e('\n\\^]\n\n_\u0019_'\u0002PQd\u00017\u0006\n\nand\n\nstands for\n\nD \u001f\"!\n\n.\n\nD$#\n\ni*)\n\nThus both\n\ncorrective action. Furthermore,\n\n\u2014which can happen for in\ufb01nitely many\n\nare dot-products of the same two vectors. When\nwhen the Hes-\nis singular\u2014the deviation is redundant and the optimal controller takes no\nare positively correlated because\nis a positive semi-de\ufb01nite matrix2. Thus the optimal controller re-\nsists single-trial deviations that take the system to more costly states, and magni\ufb01es devia-\ntions to less costly states.\n\nQR\u0001\n\nThis analysis con\ufb01rms the minimal intervention principle to be a very general property\nof optimal feedback controllers, explaining why variability patterns elongated in task-\nirrelevant dimensions (as well as synergetic actuator coupling) have been observed in such\na wide range of experiments involving different actuators and behavioral goals.\n\n2.2 Linear-Quadratic-Gaussian (LQG) simulations\n\nThe local analysis above is very general, but it leaves a few questions open: i) what happens\nis not small; ii) how does the optimal cost-to-go (which de\ufb01nes\nwhen the deviation\nredundancy) relate to the cost function (which de\ufb01nes the task); iii) what is the distribution\nof states resulting from the sequence of optimal control signals? To address such questions\n(and also build models of speci\ufb01c motor control experiments) we need to focus on a class of\ncontrol problems for which the optimal control law can actually be found. To that end, we\nhave modi\ufb01ed [15] the extensively studied LQG framework to include the multiplicative\ncontrol noise characteristic of the motor system. The control problems studied here and in\n\nQR\u0001\n\n1De\ufb01ning the unit vector +-, as having a.\n1325436\n4@?\n2ED\n, .\n,$1\n\nin all other positions, we can write\n\n,$<*+(=\n\nis also positive\n\nsemi-de\ufb01nite.\n\n234\n\n\n\n@BADCE>\n\nFHGJIK@MLON\u0016L\n\n\u0001\u0005\u0004\n\n\u0001\u0005\u0004\n\n576\u00148:9<;\n\n\u0002\u0001\n\n,.-0/2143\n\n\u0003\u0002\n\nTU>\n\nVW@EFMXO@EY\u0005IH@ML\nN\u0016L\n\n[H\\\n\n^4_\n\n^4`bac^\n\n6HZ48\n\nPDQ\n\n\b\n\t\f\u000b\u000e\r\u0010\u000f\u0012\u0011\u0014\u0013\u0005\u0015\u0016\u000b\u000e\r\u0018\u0017\u000e\u0019\u001b\u001a\nFigure 2: Simulations of motor control tasks \u2013 see text.\n\n3 Applications to motor coordination\n\nWe have used the modi\ufb01ed LQG framework to model a wide range of speci\ufb01c motor control\ntasks [14, 15], and always found that optimal feedback controllers generate variability that\nis elongated in redundant dimensions. Here we illustrate two such models. The \ufb01rst model\n(Figure 2, Bimanual Tasks) includes two 1D point masses with positions X1 and X2, each\ndriven with a force actuator whose output is a noisy second-order low-pass \ufb01ltered version\nof the corresponding control signal. The feedback contains noisy position, velocity, and\nforce information\u2014delayed by 50 msec (by augmenting the system state with a sequence\nof recent sensor readings). The \u201c Difference\u201d task requires the two points to start moving\n20cm apart, and stop at identical but unspeci\ufb01ed locations. The covariance of the \ufb01nal\nstate is elongated in the task-irrelevant dimension: the two points always stop close to each\nother, but the \ufb01nal location can vary substantially from trial to trial. A related phenomenon\nhas been observed in the more complex bimanual task of inserting a pointer in a cup [6].\nWe now modify the task: in \u201cSum,\u201d the two points start at the same location and have\nto stop so that the midpoint between them is at zero. Note that the state covariance is\nreoriented accordingly. We also illustrate a Via Point task, where a 2D point mass has to\npass through a sequence of two intermediate targets and stop at a \ufb01nal target (tracing an\nS-shaped curve). Variability is minimal at the via points. Furthermore, when one via point\nis made smaller (i.e., the weight of the corresponding positional constraint is increased),\nthe variability decreases at that point. Due to space limitations, we refer the reader to [14]\nfor details of the models. In [14] we also report a via point experiment that closely matches\nthe predicted effect.\n\n4 Multi-attribute costs and desired trajectory tracking\n\nAs we stated earlier, replacing the task goal with a desired trajectory (which achieves the\ngoal if executed precisely) is generally suboptimal. A number of examples of such subop-\ntimality are provided in [14]. Here we present a more general view of desired trajectory\ntracking which clari\ufb01es its relationship to optimal control.\n\nDesired trajectory tracking can be incorporated in the present framework by using a modi-\n\ufb01ed cost, one that speci\ufb01es a desired state at each point in time, and penalizes the deviations\nfrom that state. Such a modi\ufb01ed cost would normally include the original task cost (e.g.,\nthe terms that specify the desired terminal state), but also a large number of additional\nterms that do not need to be minimized in order to accomplish the actual task. This raises\nthe question: what happens to the expected values of the terms in the original cost, when\nwe attempt to minimize other costs simultaneously? Intuitively, one would expect the orig-\n\n\n\u0001\n\u0006\n\u0007\n\u001c\n\u001d\n\u001e\n\u001f\n \n\u001f\n\u001d\n!\n\"\n#\n$\n\"\n%\n\u001f\n\"\n!\n&\n'\n(\n&\n)\n*\n+\n3\n;\nR\nS\n]\n]\n\final costs to increase (relative to the costs obtained by the task-optimal controller). The\ngeometric argument below formalizes these ideas, and con\ufb01rms our intuition.\n\n\u0003\u000b\n\n\f\u000e\n\n\u0010\u0012\u0011\u0014\u0013\u0016\u0015\n\n\u0002\u0001\n\ncosts, and\n\n\u001b\u001a\n\u0087\r\u0002\n\nz\u000e\u001c\n\n\b \u001f\n\n. Consider a weight vector\n\nand its cor-\n\ncontrol law can achieve a smaller total expected cost, and so\n\nConsider a family of optimal control problems parameterized by the vector\n\nwe can de\ufb01ne the inverse mapping\nthe weight manifold\n\n, as illustrated in Figure 3.\n\nfrom the expected component cost manifold\n\nis locally smooth and invertible. Then\nto\n\nis\nis an optimal control law for the problem de\ufb01ned by the weight vector\n\n, with\nare different component\nare the corresponding non-negative weights. Without loss of generality\nlies in the\nbe an optimal control law3, and\n; i.e.,\n\n, i.e., the weight vector\n\n. Here \u0013\n\u0002\u0005\u0004\u0015\u0014\u0007\u0001$\u0014\u0019\u000e,\u0006\n\u0002\u0010\u0004\u0015\u0014\u0007\u00017\u0006\n\u0002)\u00171\u0006\n\ncost functions \u0013\u0019\u0018\n\u0002\u0010\u0004\u0015\u0014\u0016\u0001$\u0014\u0016\u000e%\u0006:\u001f\nwe can assume that \n\u0087\u001d\u001c\u0016\u001e\npositive quadrant of the unit sphere. Let 3\nbe the vector of expected component costs achieved by 3\n\u0002#\u00171\u0006\"\b%$&!\n?)(\n\u0002+\u0004\u0015\u0014\u0007\u0001\u0003\u0002\u0005\u0004\u0007\u0006\u0018\u0014\n\u0002#\u00171\u0006H\u001f\n\u0002\u0010\u0004\u0015\u0014\u0016\u0001\u0017\u0002+\u0004\u0007\u0006\u0007\u0006\u0016\u0006\nEyG\nresponding\"\n, such that the mapping \"\n\u00171\u0006\n\u0017a\u0002\nFrom the de\ufb01nitions of\u0013\nand '\n, the total expected cost achieved by3\nSince3\n\u0006\u0018\u0014\n. Therefore, if we construct the\u001e\nfor all \"\u000b/\n\b2$\ncontains \"\n\u0017a\u0002\n3)\u0002\n4\u000f\u0002\n41\u0002\n\u0006#\u001f5\u0017a\u0002\nLet \"\n\u000276%\u0006R\b8$\n\bX\n\n4\u000f\u0002#6%\u0006\u001b\u001f94\u000f\u0002\n\u0017a\u0002#6$\u0006\u0081\u001f:\u0017\n\u0002+\u001cO\u0006\u0081\u001f\n\u0002#6%\u0006\u0016\u0006\nwe obtain the tangent \"<; to the curve \"\nat \"\n6\u001d\u001f\n\u000276,\u0006\n4#\u0014\n\u001f\u007f\u001c\n4#\u0014\n; i.e., the tangent \"\n4#\u0014\neH\u001aX\u001c\nwithout \" crossing the hyperplane\n\">;\ne<+*\u001c\n\u001f=\u0017\nGHG\n\nbe a parametric curve that passes through the point of interest \"\n\u000276$\u0006\ne$&\n\u001fl\u001c\n; cannot turn away\ne<+\u001d\u001c\n\n3If we assume that the optimal control law is unique, all inequalities below become strict.\n4For a general 2D manifold\n\nand is orthogonal to\nnot containing the origin. Thus\nnon-negative curvature, and the unit vector\n\nthat\nhas to lie in the half-space\nhas\n,\nsatis\ufb01es 4\n\nThe non-negative curvature of\nfrom the normal\n, we have\n\n.\n\n`#\u0017\ne,+ -.\u0017a\u0002\n3)\u0002\nat point \"\nat \"\n\n.\n, no other\n\n\u0006\u0018\u0014\n\"0/\u00161\n\u0006\u0018\u0014\n\nembedded in\n\n, the mapping\n\non the unit sphere\n\nis known as the Gauss map, and plays an important role in surface\n\n, the entire manifold\n\nis tangent to the manifold\n\n. Differentiating the latter equality once again yields\n\n. By differentiating \"\n\nis normal to\n\n@\u000eA\n\n?CBED\n\nthat satis\ufb01es\ndifferential geometry.\n\n2CI\n\n:\nat\n, we have\n.\n\ndimensional hyperplane\n\n`#\u0017a\u0002\n\nwhich is normal to\n\n. Therefore\n\n, and since\n\n. De\ufb01ne\n\nand\n\nimplies\n\n. Since\n\n\u000276%\u0006\u0016\u0006\n\n.\n\n,\n\nGHG\n\n\u0003\n\u0004\n\u0005\n\u0006\n\u0007\n\u0003\n\u0007\n\b\n\u0003\n\u0006\n\t\n\b\n\u0006\n\u000f\n\u0017\n\u0087\n\u0013\n\u0087\n\u0087\n\u001c\n\u0087\n\u0087\n\u001f\nk\n\u0017\n!\n\n\u001a\n\u0018\n\"\n\n\u001a\n\u0018\n'\n\u0087\nD\nI\nK\n*\n\u0013\n\u0087\n3\n\u0018\n\u001e\n\u0004\n\u0017\n\u001f\n\"\n\u0002\n\"\n\u0006\n$\n\u001f\n\u0018\n\u0087\n\u0018\n\u0002\n\"\n\"\ne\n\u0018\n\u0017\n\"\n\"\n\"\nT\nk\n\"\n\u0006\n\"\n\u0006\n$\n\"\n\u0006\n$\n$\n\"\n\u0006\n$\n\"\n\"\n\u0006\n6\n\"\n\"\n\"\n\u0002\n\"\n\u001c\n4\n$\n`\n\"\n;\ne\n`\n\"\n;\n;\n`\n4\n;\n\u0014\n\"\n;\ne\n$\n`\n\"\n;\n;\n4\n3\n`\n4\n;\n\u0014\n\"\n;\n4\n`\n\u0017\n;\n\u0014\n?\nD\nF\nJ\nJ\n\fThe above result means that whenever we change the weight vector\n\n, the corresponding\nof expected component costs achieved by the (new) optimal control law will\nalong a great circle\ndecreases and all\n\nchange in an \u201copposite\u201d direction. More precisely, suppose we vary\nthat passes through one of the corners of\n\nvector \"\n\n, so that\n\n\u0002#\u00171\u0006\nincrease. Then the component cost'\n\n, say\n\n\u0002#\u00171\u0006\n\nwill increase.\n\n\u0014\u0016\u001c\n\n\u0014\u0001\u0002\u0001^\u0014\u0019\u001cO\u0006\n\nReferences\n\n[1] Bernstein, N.I. The Coordination and Regulation of Movements. Pergamon Press,\n\n(1967).\n\n[2] Bizzi, E., Accornero, N., Chapple, W. & Hogan, N. Posture control and trajectory\n\nformation during arm movement. J Neurosci 4, 2738-44 (1984).\n\n[3] Cole, K.J. & Abbs, J.H. Kinematic and electromyographic responses to perturbation\n\nof a rapid grasp. J Neurophysiol 57, 1498-510 (1987).\n\n[4] D\u2019Avella, A. & Bizzi, E. Low dimensionality of supraspinally induced force \ufb01elds.\n\nPNAS 95, 7711-7714 (1998).\n\n[5] Davis, M.H.A. & Vinter, R. Stochastic Modelling and Control. Chapman and Hall,\n\n(1985).\n\n[6] Domkin D., Laczko, J., Jaric, S., Johansson, H., & Latash, M. Structure of joint\n\nvariability in bimanual pointing tasks. Exp Brain Res 143, 11-23 (2002).\n\n[7] Fleming, W. and Soner, H. (1993). Controlled Markov Processes and Viscosity Solu-\n\ntions. Applications of Mathematics, Springer-Verlag, Berlin.\n\n[8] Flash, T. & Hogan, N. The coordination of arm movements: an experimentally con-\n\n\ufb01rmed mathematical model. J Neuroscience 5, 1688-1703 (1985).\n\n[9] Gelfand, I., Gur\ufb01nkel, V., Tsetlin, M. & Shik, M. In Models of the structural-\nfunctional organization of certain biological systems. Gelfand, I., Gur\ufb01nkel, V.,\nFomin, S. & Tsetlin, M. (eds.) MIT Press, 1971.\n\n[10] Harris, C.M. & Wolpert, D.M. Signal-dependent noise determines motor planning.\n\nNature 394, 780-784 (1998).\n\n[11] Hinton, G.E. Parallel computations for controlling an arm. Journal of Motor Behavior\n\n16, 171-194 (1984).\n\n[12] Robertson, E.M. & Miall, R.C. Multi-joint limbs permit a \ufb02exible response to unpre-\n\ndictable events. Exp Brain Res 117, 148-52 (1997).\n\n[13] Sutton, G.G. & Sykes, K. The variation of hand tremor with force in healthy subjects.\n\nJournal of Physiology 191(3), 699-711 (1967).\n\n[14] Todorov, E. & Jordan, M. Optimal feedback control as a theory of motor coordination.\n\nNature Neuroscience, 5(11), 1226-1235 (2002).\n\n[15] Todorov, E. Optimal feedback control under signal-dependent noise: Methodology\nfor modeling biological movement. Neural Computation, under review. Available at\nhttp://cogsci.ucsd.edu/\u02dctodorov. (2002).\n\n[16] Scholz, J.P. & Schoner, G. The uncontrolled manifold concept: Identifying control\n\nvariables for a functional task. Exp Brain Res 126, 289-306 (1999).\n\n[17] Uno, Y., Kawato, M. & Suzuki, R. Formation and control of optimal trajectory in\nhuman multijoint arm movement: Minimum torque-change model. Biological Cyber-\nnetics 61, 89-101 (1989).\n\n[18] Santello, M. & Soechting, J.F. Force synergies for multi\ufb01ngered grasping. Exp Brain\n\nRes 133, 457-67 (2000).\n\n\u0017\n\u0017\n\u001f\n\u0002\nk\n\u001c\nz\n\u001c\n\u0087\n\u0003\n\u0002\nz\nz\n\f", "award": [], "sourceid": 2195, "authors": [{"given_name": "Emanuel", "family_name": "Todorov", "institution": null}, {"given_name": "Michael", "family_name": "Jordan", "institution": null}]}