{"title": "Optimal Unsupervised Motor Learning Predicts the Internal Representation of Barn Owl Head Movements", "book": "Advances in Neural Information Processing Systems", "page_first": 614, "page_last": 621, "abstract": null, "full_text": "Optimal Unsupervised Motor Learning \nPredicts the Internal Representation of \n\nBarn Owl Head Movements \n\nTerence D. Sanger \n\nJet Propulsion Laboratory \n\nMS 303-310 \n\n4800 Oak Grove Drive \nPasadena, CA 91109 \n\nAbstract \n\n(Masino and Knudsen 1990) showed some remarkable results which \nsuggest that head motion in the barn owl is controlled by distinct \ncircuits coding for the horizontal and vertical components of move(cid:173)\nment. This implies the existence of a set of orthogonal internal co(cid:173)\nordinates that are related to meaningful coordinates of the external \nworld. No coherent computational theory has yet been proposed \nto explain this finding. I have proposed a simple model which pro(cid:173)\nvides a framework for a theory of low-level motor learning. I show \nthat the theory predicts the observed microstimulation results in \nthe barn owl. The model rests on the concept of \"Optimal U n(cid:173)\nsupervised Motor Learning\", which provides a set of criteria that \npredict optimal internal representations. I describe two iterative \nNeural Network algorithms which find the optimal solution and \ndemonstrate possible mechanisms for the development of internal \nrepresentations in animals. \n\n1 \n\nINTRODUCTION \n\nIn the sensory domain, many algorithms for unsupervised learning have been pro(cid:173)\nposed. These algorithms learn depending on statistical properties of the input \ndata, and often can be used to find useful \"intermediate\" sensory representations \n\n614 \n\n\fBam Owl Head Movements \n\n615 \n\nu \n\ny \n\np \n\nz \n\nFigure 1: Structure of Optimal Unsupervised Motor Learning. z is a reduced-order \ninternal representation between sensory data y and motor commands u. P is the \nplant and G and N are adaptive sensory and motor networks. A desired value \nof z produces a motor command u = N z resulting in a new intermediate value \nz = GPNz. \n\nby extracting important features from the environment (Kohonen 1982, Sanger \n1989, Linsker 1989, Becker 1992, for example). An extension of these ideas to the \ndomain of motor control has been proposed in (Sanger 1993). This work defined the \nconcept of \"Optimal Unsupervised Motor Learning\" as a method for determining \noptimal internal representations for movement. These representations are intended \nto model the important controllable components of the sensory environment, and \nneural networks are capable of learning the computations necessary to gain control \nof these components. \n\nIn order to use this theory as a model for biological systems, we need methods to \ninfer the form of biological internal representations so that these representations \ncan be compared to those predicted by the theory. Discrepancies between the \npredictions and results may be due either to incorrect assumptions in the model, or \nto constraints on biological systems which prevent them from achieving optimality. \nIn either case, such discrepancies can lead to improvements in the model and are \nthus important for our understanding of the computations involved. On the other \nhand, if the model succeeds in making qualitative predictions of biological responses, \nthen we can claim that the biological system possesses the optimality properties of \nthe model, although it is unlikely to perform its computations in exactly the same \nmanner. \n\n2 BARN OWL EXPERIMENTS \n\nA relevant set of experiments was performed by (Masino and Knudsen 1990) in \nthe barn owl. These experiments involved microstimulation of sites in the optic \ntectum responsible for head movement. By studying the responses to stimulation \nat different sites separated by short or long time intervals, it was possible to infer the \nexistence of distinct \"channels\" for head movement which could be made refractory \nby prior stimulation. These channels were oriented in the horizontal and vertical \ndirections in external coordinates, despite the fact that the neck musculature of the \nbarn owl is sufficiently complex that such orientations appear unrelated to any set \n\n\f616 \n\nSanger \n\nof natural motor coordinates. This result raises two related questions. First, why \nare the two channels orthogonal with respect to external Cartesian coordinates, and \nsecond, why are they oriented horizontally and vertically? \n\nThe theory of Optimal Unsupervised Motor Learning described below provides a \nmodel which attempts to answer both questions. It automatically develops orthogo(cid:173)\nnal internal coordinates since such coordinates can be used to minimize redundancy \nin the internal representation and simplify computation of motor commands. The \nselection of the internal coordinates will be based on the statistics of the components \nof the sensory data which are controllable, so that if horizontal and vertical move(cid:173)\nments are distinguished in the environment then these components will determine \nthe orientation of intermediate channels. We can hypothesize that the horizontal \nand vertical directions are distinguished in the owl by their relation to sensory in(cid:173)\nformation generated from physical properties of the environment such as gravity or \nsymmetry properties of the owl's head. In the simulation below, I show that reason(cid:173)\nable assumptions on such symmetry properties are sufficient to guarantee horizontal \nand vertical orientations of the intermediate coordinate system. \n\n3 OPTIMAL UNSUPERVISED MOTOR LEARNING \n\nOptimal Unsupervised Motor Learning (OUML) attempts to invert the dynamics of \nan unknown plant while maintaining control of the most important modes (Sanger \n1993). Figure 1 shows the general structure of the control loop, where the plant P \nmaps motor commands u into sensory outputs y = Pu, the adaptive sensory trans(cid:173)\nformation G maps sensory data y into a reduced order intermediate representation \nz = Gy, and the adaptive motor transformation N maps desired values of z into the \nmotor commands u = N z which achieve them. Let z = G P N z be the value of the \nintermediate variables after movement, and f) = P NGy be the resulting value of the \nsensory variables. For any chosen value of z we want z = z, so that we successfully \ncontrol the intermediate variables. \n\nIn (Sanger 1993) it was proposed that we want to choose z to have lower dimen(cid:173)\nsionality than y and to represent only the coordinates which are most important \nfor controlling the desired behavior. Thus, in general, f) =/; y and Ily - f)1I is the \nperformance error. OUML can then be described as \n\n1. Minimize the movement error 1If) - yll \n2. Subject to accurate control z = z. \n\nThese criteria lead to a choice of internal representation that maximizes the loop \ngain through the plant. \n\nTheorem 1: \n(Sanger 1993) For any sensory mapping G there exists a motor \nmapping N such t~at z = z, and [; _ E[lIy - f)1I] is mi1!.imized when G is chosen to \nminimize E[lly - G-1Gyll]' where G-l is such that GG-l = I. \nThe function G is an arbitrary right inverse of G, and this function determines the \nasymptotic values of the unobserved modes. In other words, since G in general is \ndimensionality-reducing, z = Gy will not respond to all the modes in y so that \ndissimilar states may project to identical intermediate control variables z. The \n\n\fBarn Owl Head Movements \n\n617 \n\nPlant 1 \n\nII Motor \n\nSensory \n\nEigenvectors of E[yy'l ] \nLinear \nRBF \nEigenvectors of basis function outputs \nPolynomial Polynomial Eigenvectors of basis function outputs \n\nLinear \nLinear \n\nFigure 2: Special cases of Theorem 1. If the plant inverse is linear or can be \napproximated using a sum of radial basis functions or a polynomial, then simple \nclosed-form solutions exist for the optimal sensory network and the motor network \nonly needs to be linear or polynomial. \n\nfunction a- 1 G is a projection operator that determines the resulting plant output \nfJ for any desired value of y. Unsupervised motor learning is \"optimal\" when the \nprojection surface determined by a- 1G is the best approximation to the statistical \ndensity of desired values of y. \n\nWithout detailed knowledge of the plant, it may be difficult to find the general \nsolution described by the theorem. Fortunately, there are several important special \ncases in which simple closed-form solutions exist. These cases are summarized \nin figure 2 and are determined by the class of functions to which the plant inverse \nbelongs. If the plant inverse can be approximated as a sum of radial basis functions, \nthen the motor network need only be linear and the optimal sensory network is given \nby the eigenvectors of the autocorrelation matrix of the basis function outputs (as \nin (Sanger 1991a)). If the plant inverse can be approximated as a polynomial over \na set of basis functions (as in (Sanger 1991b)), then the motor network needs to be \na polynomial, and again the optimal sensory network is given by the eigenvectors \nof the autocorrelation matrix of the basis function outputs. \n\nSince the model of the barn owl proposed below has a linear inverse we are interested \nin the linear case, so we know that the mappings Nand G need only be linear and \nthat the optimal value of G is given by the eigenvectors of the autocorrelation matrix \nof the plant outputs y. In fact, it can be shown that the optimal Nand G are given \nby the matrices ofleft and right singular vectors of the plant inverse (Sanger 1993). \n\nAlthough several algorithms for iterative computation of eigenvectors exist, until \nrecently there were no iterative algorithms for finding the left and right singular \nvectors. \nI have developed two such algorithms, called the \"Double Generalized \nHebbian Algorithm\" (DGHA) and the \"Orthogonal Asymmetric Encoder\" (OAE). \n(These algorithms are described in detail elsewhere in this volume.) DGHA is \ndescribed by: \n\n!J..G \n!J..NT \n\nr(zyT - LT[zzT]G) \nr(zuT - LT[zzT]NT ) \n\nwhile OAE is described by: \n\n!J..G \n!J..NT \n\nr(zyT - LT[zzT]G) \nr( Gy - LT[GGT]z)uT \n\nwhere LT[ ] is an operator that sets the above diagonal elements of its matrix \nargument to zero, y = Pu, z = Gy, z = NT u, and r is a learning rate constant. \n\n\f618 \n\nSanger \n\nNeck Muscles \n\nMovement Sensors \n\ne \n\nu \n\nMotor Transform \n\nSensory Transform \n\ny \n\nN \n\nz \n\nFigure 3: Owl model, and simulation results. The \"Sensory Transform\" box shows \nthe orientation tuning of the learned internal representation. \n\n4 SIMULATION \n\nI use OUML to simulate the owl head movement experiments described in (Masino \nand Knudsen 1990), and I predict the form of the internal motor representation. I \nassume a simple model for the owl head using two sets of muscles which are not \naligned with either the horizontal or the vertical direction (see the upper left block \nof figure 3). This model is an extreme oversimplification of the large number of \nmuscle groups present in the barn owl neck, but it will serve to illustrate the case \nof muscles which do not distinguish the horizontal and vertical directions. \n\nI assume that during learning the owl gives essentially random commands to the \nmuscles, but that the physics of head movement result in a slight predominance of \neither vertical or horizontal motion. This assumption comes from the symmetry \nproperties of the owl head, for which it is reasonable to expect that the axes of \nrotational symmetry lie in the coronal, sagittal, and transverse planes, and that \nthe moments of inertia about these axes are not equal. I model sensory receptors \nusing a set of 12 oriented directionally-tuned units, each with a half-bandwidth at \nhalf-height of 15 degrees (see the upper right block of figure 3). Together, the Neck \nMuscles and Movement Sensors (the two upper blocks of figure 3) form the model \nof the plant which transforms motor commands u into sensory outputs y. Although \nthis plant is nonlinear, it can be shown to have an approximately linear inverse on \n\n\fBarn Owl Head Movements \n\n619 \n\nDesired Direction \n\nFigure 4: Unsupervised Motor Learning successfully controls the owl head simula(cid:173)\ntion. \n\nits range. \n\nThe sensory units are connected through an adaptive linear network G to three \nintermediate units which will become the internal coordinate system z. The three \nintermediate units are then connected back to the motor outputs through a motor \nnetwork N so that desired sensory states can be mapped onto the motor commands \nnecessary to produce them. The sensory to intermediate and intermediate to motor \nmappings were allowed to adapt to 1000 random head movements, with learning \ncontrolled by DGHA. \n\n5 RESULTS \n\nAfter learning, the first intermediate unit responded to the existence of a motion, \nand did not indicate its direction. The second and third units became broadly \ntuned to orthogonal directions. Over many repeated learning sessions starting from \nrandom initial conditions, it was found that the intermediate units were always \naligned with the horizontal and vertical axes and never with the diagonal motor \naxes. The resulting orientation tuning from a typical session is shown in the lower \nright box of figure 3. \n\nNote that these units are much more broadly tuned than the movement sensors \n(the half-bandwidth at half-height is 45 degrees). The orientation of the internal \nchannels is determined by the assumed symmetry properties of the owl head. This \ninformation is available to the owl as sensory data, and OUML allows it to determine \nthe motor representation. The system has successfully inverted the plant, as shown \nin figure 4. \n\n(Masino and Knudsen 1990) investigated the intermediate representations in the \nowl by taking advantage of the refractory period of the internal channels. It was \nfound that if two electrical stimuli which at long latency tended to move the owl's \nhead in directions located in adjacent quadrants were instead presented at short \nlatency, the second head movement would be aligned with either the horizontal or \nvertical axis. Figure 5 shows the general form of the experimental results, which \nare consistent with the hypothesis that there are four independent channels coding \n\n\f620 \n\nSanger \n\nMove 1 \n\nMove 2a \n\nMove 2b \n\nMove 1 \n\nMove 2a \niliL Move 2b \n\nLong Interval \n\nShort Interval \n\nFigure 5: Schematic description of the owl head movement experiment. At long \ninterstimulus intervals (lSI), moves 2a and 2b move up and to the right, but at \nshort lSI the rightward channel is refractory from move 1 and thus moves 2a and \n2b only have an upward component. \n\n---\nI \nI or \n11 I \n\"\". -- .. \n\n\u2022\u2022 \n\n.. \n\n... \n\n'10 \n\n0' \n\nh. \n\na. \n\n~\"\"\"'tfII., \n\nFigure 6: Movements align with the vertical axis as the lSI shortens. a. Owl \ndata (reprinted with permission from (Masino and Knudsen 1990\u00bb. h. Simulation \nresults. \n\nthe direction of head movement, and that the first movement makes either the \nleft, right, up, or down channels refractory. As the interstimulus interval (lSI) is \nshortened, the alignment of the second movement with the horizontal or vertical \naxis becomes more pronounced. This is shown in figure 6a for the barn owl and 6b \nfor the simulation. If we stimulate sites that move in many different directions, we \nfind that at short latency the second movement always aligns with the horizontal \nor vertical axis, as shown in figure 7a for the owl and figure 7b for the simulation. \n\n6 CONCLUSION \n\nOptimal Unsupervised Motor Learning provides a model for adaptation in low-level \nmotor systems. It predicts the development of orthogonal intermediate representa(cid:173)\ntions whose orientation is determined by the statistics of the controllable compo(cid:173)\nnents of the sensory environment. The existence of iterative neural algorithms for \nboth linear and nonlinear plants allows simulation of biological systems, and I have \n\n\f.... \n\n\u2022 \n; \n\nI \n\n~ \n\ni \n\u00a7~ \n\na. \n\nI.ONG -\n., .. \" .. \n\n\"TEaVAL \n\nSHORT -\n\n.,--\"--\n\nINTERVAL \n\nBarn Owl Head Movements \n\n621 \n\nh. \n\nFigure 7: At long lSI, the second movement can occur in many directions, but \nat short lSI will tend to align with the horizontal or vertical axis. a. Owl data \n(reprinted with permission from (Masino and Knudsen 1990)). h. Simulation re(cid:173)\nsults. \n\nshown that the optimal internal representation predicts the horizontal and vertical \nalignment of the internal channels for barn owl head movement. \n\nAcknowledgements \n\nThanks are due to Tom Masino for helpful discussions as well as for allowing re(cid:173)\nproduction of the figures from (Masino and Knudsen 1990). This report describes \nresearch done within the -laboratory of Dr. Emilio Bizzi in the department of Brain \nand Cognitive Sciences at MIT. The author was supported during this work by a \nNational Defense. Science and Engineering Graduate Fellowship, and by NIH grants \n5R37 AR26710 and 5ROINS09343 to Dr. Bizzi. \n\nReferences \n\nBecker S., 1992, An Information-Theoretic Unsupervised Learning Algorithm for \nNeural Networks, PhD thesis, Univ. Toronto Dept. Computer Science. \nKohonen T., 1982, Self-organized formation of topologically correct feature maps, \nBiological Cybernetics, 43:59-69. \nLinsker R., 1989, How to generate ordered maps by maximizing the mutual infor(cid:173)\nmation between input and output signals, Neural Computation, 1:402-411. \nMasino T ., Knudsen E. I., 1990, Horizontal and vertical components of head move(cid:173)\nment are controlled by distinct neural circuits in the barn owl, Nature, 345:434-437. \nSanger T. D., 1989, Optimal unsupervised learning in a single-layer linear feedfor(cid:173)\nward neural network, Neural Networks, 2:459-473. \nSanger T. D., 1991a, Optimal hidden units for two-layer nonlinear feedforward \nneural networks, International Journal of Pattern Recognition and Artificial Intel(cid:173)\nligence, 5(4):545-561, Also appears in C. H. Chen, ed., Neural Networks in Pattern \nRecognition and Their Applications, World Scientific, 1991, pp. 43-59. \nSanger T. D., 1991b, A tree-structured adaptive network for function approximation \nin high dimensional spaces, IEEE Trans. Neural Networks, 2(2):285-293. \nSanger T. D., 1993, Optimal unsupervised motor learning, IEEE Trans. Neural \nNetworks, in press. \n\n\f", "award": [], "sourceid": 782, "authors": [{"given_name": "Terence", "family_name": "Sanger", "institution": null}]}