{"title": "Four-legged Walking Gait Control Using a Neuromorphic Chip Interfaced to a Support Vector Learning Algorithm", "book": "Advances in Neural Information Processing Systems", "page_first": 741, "page_last": 747, "abstract": null, "full_text": "Four-Iegged Walking Gait Control Using a \nNeuromorphic Chip Interfaced to a Support \n\nVector Learning Algorithm \n\nSusanne Still \n\nNEC Research Institute \n\nBernhard Scholkopf \n\nMicrosoft Research Institute \n\n4 Independence Way, Princeton NJ 08540, USA \n\n1 Guildhall Street, Cambridge, UK \n\nsasa@research.nj.nec.com \n\nbsc@scientist.com \n\nKlaus Hepp \n\nInstitute of Theoretical Physics \n\nETH Zurich, Switzerland \n\nRodney J. Douglas \n\nInstitute of Neuroinformatics \nETHIUNI Zurich, Switzerland \n\nAbstract \n\nTo control the walking gaits of a four-legged robot we present a novel \nneuromorphic VLSI chip that coordinates the relative phasing of the \nrobot's legs similar to how spinal Central Pattern Generators are believed \nto control vertebrate locomotion [3]. The chip controls the leg move(cid:173)\nments by driving motors with time varying voltages which are the out(cid:173)\nputs of a small network of coupled oscillators. The characteristics of the \nchip's output voltages depend on a set of input parameters. The rela(cid:173)\ntionship between input parameters and output voltages can be computed \nanalytically for an idealized system. In practice, however, this ideal re(cid:173)\nlationship is only approximately true due to transistor mismatch and off(cid:173)\nsets. Fine tuning of the chip's input parameters is done automatically by \nthe robotic system, using an unsupervised Support Vector (SV) learning \nalgorithm introduced recently [7]. The learning requires only that the \ndescription of the desired output is given. The machine learns from (un(cid:173)\nlabeled) examples how to set the parameters to the chip in order to obtain \na desired motor behavior. \n\n1 Introduction \nModem robots still lag far behind animals in their capability for legged locomotion. Four(cid:173)\nlegged animals use distinct walking gaits [1], resulting for example in reduction of energy \nconsumption at high speeds [5]. Similarly, the use of different gaits can allow legged robots \nto adjust their walking behavior not only for speed but also to the terrain they encounter. \nCoordinating the rhythmic movement patterns necessary for locomotion is a difficult task \ninvolving a large number of mechanical degrees of freedom (DOF) and input from many \nsensors, and considerable advantages may be gained by emulating control architectures \nfound in animals. Neuroscientists have found increasingly strong evidence during the past \ncentury to support the hypothesis that centers in the nervous system, called Central Pattern \nGenerators (CPGs), generate rhythmic output responsible for coordinating the large number \nof muscles needed for locomotion [2]. CPGs are influenced by signals from the brain stem \nand the cerebellum, brain structures in which locomotive adaptation is believed to take \nplace [11]. This architecture greatly simplifies the control problem for the brain. The \nbrain only needs to set the general level of activity, leaving it to the CPG to coordinate \n\n\fA r-------------------------------~ \n\nSV Learning Algorithm \n\nParameter setting program Data acquisition program \n\nB \n\nBrain \n\n\"-\n\nInput port \n\nt \nt 1 \n\n/' \n~ \nDAC \nI \n\nj \nMotors \n\nl'--__ w_a_lki_\u00b7_ng, G_ a_i_t c_o--.n_tr_ol_c_h_ip __ Jl I \n\nI \n\nSensors \n\nMuscles \n\nSensors \n\nFigure 1: A: Sketch of the control architecture of the robot. Thick arrows indicate the \nlearning loop. B: Sketch of simplified control architecture for locomotion of vertebrates. \n\nthe complex pattern of muscle activity required to generate locomotion [3]. We make \nuse of these biological findings by implementing a similar control architecture to control a \nwalking machine. A neuromorphic Gait Controller (GC) chip produces time varying output \nvoltages that control the movement of the robot's legs. Their frequency, phase relationships \nand duty cycles determine the resulting walking behavior (step frequency, walking gait \nand direction of motion) and depend on a small set of control parameters. For an ideal \nsystem, the relationship between control parameters and output voltages can be determined \nanalytically, but deviations of the chip from ideal have to be compensated for by other \nmeans. Since the goal here is that the resulting machine works autonomously, we propose \na learning procedure to solve this problem. The robot is given the specifications of the \ndesired movement sequence and explores parameter combinations in the input parameter \nspace of its GC chip, keeping only those leading to a movement that is correct within some \ntolerance. It then locates the region in input parameter space that contains most of these \nparameter combinations using an algorithm [7] that extends SV learning to unlabelled data. \n\n2 The robotic system \nThe robotic system consists of (i) a body with one degree of freedom per leg l and a po(cid:173)\ntentiometer attached to each motor that serves as a sensor providing information about the \nangular displacement of the leg, (ii) the neuromorphic Gait Controller (GC) chip and (iii) \na PC on which algorithms are run to (a) acquire data from chip and sensors, (b) set the \nchip's input parameters and (c) implement the learning algorithm. The control architecture \nis inspired by the architecture which is used for locomotion in vertebrates (see Fig. 1 and \nSec. 1). Like in biology, the existence of the GC chip considerably simplifies the control \ntask. The computer only needs to set the input parameters to the GC chip, leaving the \nchip to coordinate the pattern of motor movements necessary to generate locomotion of the \nrobot. The circuitry on the GC chip is based on an analysis [8] of circuits originating from \nM. W. Tilden (e.g. [4]). The GC chip contains five oscillators which can be inter-connected \nin different ways. The chip can be used in three reasonable configurations, one of which, \na chain of four oscillators (Fig. 2), is used in the present work. Each oscillator (see Fig. 3) \nconsists of two similar sub-circuits. In the following, subscripts i E {I, .. ,4} will denote \nthe oscillator identity and subscripts k E {l, r} will denote one of the two sub-circuits \nwithin an oscillator. Here l stands for the left side of the oscillator circuit and r for the right \nside. Each sub-circuit has a capacitor connecting an input node to a node Vi,k, to which the \ninput node of an inverter is connected. The output node of the inverter is called Vout,i,k. \n\nlThe robot's aluminum body is 12 cm long and 6.2 cm wide. It has four DC motors which drive \naluminum legs attached at right angles to the plane of the body. Each leg ends in a foot that contains \na small electromagnet which is activated during the stance phase of the leg and deactivated during \nthe swing phase. Leg and foot together have a length of 6.5 cm. The robot walks on a metal floor so \nthat the electromagnet increases the friction during the stance phase. For further details see [7]. \n\n\fLeft Front Leg ~ \n\nOUT0\n\nBIAS \n\nRight Front Leg \n\nBIA~ \n~ OUT \n\nBIAS \n\nLeft Hind Leg \n\nRight Front Leg \n\nFigure 2: Sketch of the configuration in which the chip is used: a chain of four coupled \noscillators. Each of the thin lines is connected to a pad. The round symbols stand for \noscillators numbered corresponding to the text. The thick arrows stand for the transmission \ngates which couple the oscillators (see circuit diagram in Fig. 3). The arrows that lead to \nthe four legs represent the outputs of the oscillators. \n\nFinally, a n-FET transistor with gate voltage Vb,i,k is connected between Vi,k and ground. \nAn oscillator is obtained by connecting the input node of one sub-circuit to the output node \nof the other and vice versa. The output voltages of a single oscillator are two mirror-image \nstep functions at Vout,i,l and Vout,i,r' These voltages control the stepping movements of \none leg. Two oscillators, j and j + 1 (j E {I, '\" 3}), are coupled with two transmission \ngates. One is connected between Vout,j,l and Vi+l,l. The current that flows through it de(cid:173)\npends on the bias voltage Vi\"j j+1,1' Likewise, the other transmission gate connects Vout,j,r \nand Vi+1,r and has the bias voltage Vb,j j+1 ,r' Note that the coupling is asymmetric, af(cid:173)\nfecting only oscillator j + 1. The voltages at the input nodes to the inverters of oscillator j \nare not affected by the coupling since the inverters act as impedance buffers. \nThe chip's output is characterized by the frequency (common to all oscillators), the four \nduty cycles of the oscillators and three phase lags between oscillators. The phase lags \ndetermine which gait the robot adopts. The duty cycles of the oscillators set the ratio be(cid:173)\ntween stance and swing phase of the legs. Certain combinations of duty cycles differing \nfrom 50% make the robot turn [8]. For a set of constant input parameters, {Vb,i,r, Vb,i,l, \nVb,j j+1,r, Vb,j j+l,l}, a rhythmic output is produced with oscillation period P, duty cycles \nDi and phase shifts \u00a2j, where i E {I, \",4} and j E {I, '\" 3}. Analysis of the resulting cir(cid:173)\ncuit reveals [8] how the output characteristics of the chip depend on the input parameters. \nAssume that all transistors on the chip are identical and that the peak voltages at node V1,1 \nand at node V1,r are identical and equal to Vmax . For a certain range of input parameters, \nthe period of the oscillators is given by the period of the first oscillator in the chain (called \nthe master oscillator) \n\n. . !L . . !L \n\nP = C(Vmax - vth)( e - kTK.Vb,l ,1 + e - kTK.Vb,l,r)/ Ion \n\n(1) \nwhere C = 5.159 X 10- 10 F is the capacitance and Ion is the drain source leakage current \nof the n-FET. The threshold voltage of the inverter, vth = 1. 345V, is calculated from \nthe process parameters [9]. Vmax = 3.23V, Ion = 2.2095 X 10- 16 A and K. = 0.6202 \nare estimated with a least squares fit to the data (Fig. 4a). T is the temperature, k the \nBoltzmann constant and q the electron charge. Let the duty cycle be defined as the fraction \nof the period during which Vout,i,l is high. The master oscillator's duty cycle is \n\nD1 = 1/[1 + ePrK.(Vb,l,r - Vb,l ,I)] \n\n(2) \nA very simple requirement for the controller is to produce a symmetric waveform for \nstraight forward locomotion. For this, all oscillators must have a duty cycle of 112 \n(=50%) [8]. This can be implemented by a symmetric circuit (identical control volt(cid:173)\nages on both right and left side: Vb,j j+1,1 = Vb ,j j+l,r =: Vb,j j+l Vj E {I, '\" 3} and \nVb,i,l = Vb,i,r =: Vb,i Vi E {I, \",4}). For simplicity, let Vb,i = Vb Vi. Then the phase lag \nbetween oscillators j and j + 1 is given by (compare Fig. 4b) \n\n\u00a2j = - + \n\n1 \n2 \n\nkT /2q(Vmax - vth) \n) \n..!L \n\n( \n\n/3e- kT K. Vb,jj+l+Vb -1 \n\n['Y(vth) - p,(vth)ePrK.(Vb,j j+1 + Vb)] \n) \n'Y(Vo)-p,(Vo)ekTK.Vb,jj+l+Vb \n\n..!L \n\n( \n\nIn \n\n(3) \n\n\fOSCILLA TOR j \n\nTransmission \nGate \n\nVout,j+l,r \n\nOSCILLA TOR j+ 1 \n\nFigure 3: Two oscillators are coupled through a transmission gate. The gate voltage on the \nn-FET of each transmission gate is set to be the complementary voltage of the p-FET of \nthe same transmission gate by the circuits which are drawn next to the transmission gates. \nThese circuits are controlled by the bias voltages Vb,i HI,I and Vb,i i+1 ,r and copy the \nvoltages Vb,j+1,1 and Vb,j+1 ,r, to nodes 2 and 4, respectively, while the voltages at nodes \n1 and 3 are (Vdd - Vb ,HI,I) and (Vdd - Vb,HI,r), respectively. The symbols on the right \ncorrespond to the symbols in Fig. 2. \n\nwhere Va = 0.1 V and \n\n(3 = lop ePr\"-Vdd / Ion; \n\n')'(V) = (Ion + lop ePr V) ePr\"-Vdd; \n\nJL(V) = Ion ePr V \n\n3 Learning \nIn theory, all duty cycles should be 112 for the symmetric circuit. In the real system, the \nduty cycle changes with the phase lag (see Fig. 4c) due to transistor mismatch. Thus, to \nobtain a duty cycle of 112, Vb ,i HI,1 might have to differfrom Vb,j HI,r . Parameter points \nwhich lead to both a desired phase lag and a duty cycle of 112 lie in a two dimensional \nspace spanned by Vb,j HI,I and Vb,j j+1,r. These parameters are learned2 . First, a subset \nX in this input parameter space is chosen according to the estimate given by (3). X is \nscanned and at each point, the output characteristics of the GC chip are determined. If they \nmatch the desired output characteristics (within specified tolerances), this point is added to \nthe training data set V eX. After the scan is completed, the training data is transformed \n: X -+ F , into a feature space F such that if x, y EX, the dot \nby a feature map <1> \n\n2The desired duty cycle of 1/2 is an example, leading to forward locomotion for of the test robot. \n\nIn the same way, any other value for the duty cycle can be learned (for examples see [8]) \n\n\f8 \n'\" \n;;: 6 .... \n\n12 r-,----,----,----.,----.,----.,----, \n-.j .. ---.l.----.l.---.. l .. -----\n.... j ...... + ...... j .... ~ ..... ; .... ; .... . \n.j .... j ..... \n: \n---_._( \u00b7-t------)_\u00b7_---+------:----\u00b7':'\u00b7-----\n4 \n2 ... j ............................. ..... . \no \n\n\u00b7\u00b7\u00b7\u00b7\u00b7t\u00b7\u00b7\u00b7\u00b7\u00b7j \n: \n: \n\n, \n: \n\n: \n\ni \n\n0.4 \n\n0.5 \n\n0.6 \n\n; \n0.3 \n\n, \n\n.. ~ ....... ,. \n\n, \n\u00b7\u00b7\u00b7\u00b7r\u00b7\u00b7 \n\nI \n[.. \n0.9 t\u00b7\u00b7 \n~.-... -.! ...... -{ .. -.... + ... -... ~ ..... -.! ....... ~ .... . \n0.8 f\u00b7\u00b7\u00b7\u00b7\u00b7+\u00b7\u00b7\u00b7\u00b7\u00b7+\u00b7\u00b7\u00b7\u00b7+\u00b7\u00b7\u00b7\u00b7\u00b7f-\u00b7\u00b7\u00b7\u00b7\u00b7\u00b7f\u00b7\u00b7\u00b7\u00b7\u00b7 \n\n;--.-.--j----.-.~-----.-+.----- ' \n\n~~ o. 7 ~:::::::l::::::t::::::t:::::::~:::::-+-\u00b7 \u00b7\u00b7\u00b7\u00b7r::::: \n; \n\n0.6 t\u00b7\u00b7\u00b7\u00b7\u00b7\u00b7:\u00b7\u00b7\u00b7\u00b7\u00b7\u00b7\u00b7;\u00b7 \n0.5 ..... ' ----'-----'--'---'---'----'----' \n\n--~ .... -.- . \n\n4.22 \n\n4.24 \n\n4.26 \n\n4.28 \n\n(b) \n\nVb,12/V \n\nA \n\n0.5 ~c:::;---,--,--,........,---,---,---,--, \n\nN::;~~'~~;I;! \n\n:::::::::::::r::r:::T:::r\"::;::::;::::;::::' \n0.2 ..... : ... 1...1....1...1....1 ::r:::::::r:: \n0.1 :::::;:::::::)::::::::::::::; .... ] ..... ( .. ] ..... \n00.5 ' 0.6 0.7 0.8 0.9 \n(c) \n\n<1>1 \n\n1 \n\nFigure 4: (a): Oscillation period P (points = data, 3% error) as a function of the bias \nvoltage Vb i 1 = Vb i r =: Vb. Vb i 1 = Vb i r implies that the duty cycle of the oscillation is \n112 (see (2\u00bb. The ~s~iIIation peri~d follo~s (1) (solid line). (b): Phase lag between the first \ntwo oscillators in a chain of 4 oscillators. Function given in (3) (solid line) and data points. \nVmax and Ion are as determined in (a), lop, as estimated by data fitting is 1.56 x 10- 19 A. \n(c): Duty cycle of the second oscillator in a chain of 4 oscillators as a function of the phase \nlag between oscillators 1 and 2. \n\nproduct ( \n\n4.255 \n\n4.3 \n\n4.285 \n\n~ 4.27 \n> \n\n4.255 \n\n( .. \n\u00b7 . \n\u00b7 . . . \n\u00b7 . \n\u00b7 ... \n.... \n0 \n\n-:. \n\n4.24'-:--~c::---,-':--c-----:--\"-=----,-' \n\n4.28 \n\n4.295 \n\n4.325 \n\n4.34 \n\n4.31 \nVb,12,r \n\n4.24 '-:--~c::---,-':--c-----:--\"-=----,-' \n\n4.28 \n\n4.295 \n\n4.325 \n\n4.34 \n\n4.31 \nVb,12,r \n\nFigure 5: Result of learning values of the bias voltages Vb,12 ,1 and Vb,12,r which lead to \nqh = 0.75 and D2 = 0.5. Correctly classified (stars) and misclassified (crosses) training \ndata (left) and test data (right). Outlined regions: learned by algorithm from training data. \nThe training data is obtained from one scan of the displayed rectangular region X. The test \ndata is a set of points obtained from three scans. \n\n5 Discussion \nWe have introduced a novel neuromorphic chip for inter-leg coordination of a walking \nmachine. This chip successfully controls a four-legged robot. A Support Vector algorithm \nenables tile robotic system to learn a desired movement sequence. We have demonstrated \nthis here using the walk gait as an example. Other gaits have also been learned [8]. The \narchitecture we used reduced the learning of a complex motor behavior to a classification \ntask. The classifier we used requires only a few examples, making the learning efficient, \nand it can handle noisy data, making the learning robust. \nThe chip need not be interfaced to a computer; it can control the robot without any need \nof software once the input parameters of the chip are known. Note, that the chips bias \nvoltages can also be changed by simple sensors in a direct way, enabling the robot to adapt \nits behavior according to sensory information. This point is elaborated in [8]. \nHowever, the chip-computer interface creates a hybrid system in which the complex \nmovement pattern required to make a four legged machine locomote is controlled by the \nchip, while algorithms run on the computer can focus on more demanding tasks. This \narchitecture enables the robotic system to exploit the motor abilities it has due to the GC \nchip - independent of tile particular physical shape of the robot. \nThe hybrid system could also be useful for the development of a second generation of neu(cid:173)\nromorphic motor control chips, able to solve more complex tasks. Furthermore, the control \ncircuit could easily be extended to the control of six (or more) legs simply by addition of \ntwo (or more) oscillators, without increasing drastically in complexity, as the number of \ncontrol parameters is small and scales linearly with the number of oscillators. Similarly, \nthe circuit could be expanded to control n-jointed legs if each of the four oscillators \nbecomes itself the master of a chain of n oscillators. Finally, the learning procedure intro(cid:173)\nduced here could be used as a general method for fine tuning of neuromorphic a VLSI chips. \n\nAcknowledgments \nS. S. is grateful to the late Misha Mahowald for inspiring discussions and indebted to \nMark W. Tilden for discussing his circuits. We thank Adrian M. Whatley for useful \ncomments and technical assistance. For helpful discussions we thank William Bialek, \nGert Cauwenberghs, Giacomo Indiveri, Shih-Chii Liu, John C. Platt, Alex 1. Smola, John \nShawe-Taylor and Robert C. Williamson. S. S. was supported by CSEM, Neuchiitel, the \nPhysics Department of ETH Zurich and tile SPP of tile Swiss National Science Foundation. \n\n\fLF() \n\nRF (o) \n\nLH (+) \n\nRH ( O) \n\n+ .. + : \n\n.(cid:173)\n\n.\u2022 +-\no 00 \n\n0 - 00 \n\n0 \n\n+ \n\n0 \n+ ___ c \n\n\u2022 \n\n0 \n\no \n\n0 . , \n\n+ \n\n0 \n\n+ \n+ \n\n,~, ~-7.,,~~~'~O~,,~~~~~ __ ~~ \n\ntme l s \n\n! ~ lIJIlI IllI [ III! JlIIIlIlllllllII It !IIlH H! !I III! I1mIlU H II HI \n\nUJ .... 0 \n\n3 \ntime / s \n\nFigure 6: Left: Control voltages (upper plot) and angular displacements of the legs as \nmeasured by potentiometers that are attached to the motors (lower plot) as a function of \ntime shown for a cycle of the rhythmic movement. The four legs are distinguished by the \nabbreviations: left front (LF; dots), right front (RF; circles), left hind (LH; crosses) and \nright hind (RH; stars). The legs move in succession with a phase shift of 90\u00b0 : LF, RH, RF, \nand finally LH, a typical walk sequence [1]. Note that the data is acquired with a limited \nsampling rate. Thus the duty cycles of the control voltages appear to deviate from 50 %. \nHowever, the data on the right proves that the duty cycles are sufficiently close to 50 % to \ncause the robot to walk forward in a straight line, as desired. Right: Position of the robot's \ncenter of gravity as a function of time. Upper plot: x-coordinate, lower plot: y-coordinate. \nErrors are due mainly to the extension of the images of the LEDs in the image frames \nobtained from the CCD camera. The y-coordinate is constant within the error. This shows \nthat the robot moves forward on a straight line. The robot moves at roughly 4.7 cm S-l . \n\nReferences \n[1] R. McN. Alexander, The Gaits of Bipedal and Quadrupedal Animals. Intl. 1. Robotics Research, \n1984,3, pp. 49-59 \n[2] F. De1comyn, Neural Basis of Rhythmic Behaviour in Animals. Science, 1980, 210, pp. 492-498 \n[3] S. Grillner, 1981, Control oflocomotion in bipeds, tetrapods and fish. In: Handbook of Physiology \nII, M. D. Bethesda (ed.), Am. Physiol. Soc., pp. 1179-1236; S. Grillner, 1998, Vertebrate Locomotion \n- A Lamprey Perspective. In: Neuronal Mechanisms for Generating Locomotor Activity, O. Kiehn \net. aI. (eds.), New York Academy of Science \n[4] B. Hasslacher & M. W. Tilden, Living Machines. Robotics and Autonomous Systems: The Biology \nand Technology of Intelligent Autonomous Agents, 1995, L. Steels (ed.), Elsevier; \nS. Still & M. W. Tilden Controller for a four legged walking machine. In: Neuromorphic Systems, \n1998, L. S. Smith & A. Hamilton (eds.), World Scientific \n[5] D. F. Hoyt & R. C. Taylor, Gait and the energetics of locomotion in horses. Nature , 1981 , 292, \npp.239-240 \n[6] J. Mercer, Functions of positive and negative type and their connection with the theory of integral \nequations. Phil. Trans. Roy. Soc. London A , 1909, 209, pp. 415-446 \n[7] B. Scholkopf, J. C. Platt, J. Shawe-Taylor, A. J. Smola and R. C. Williamson, Estimating the \nSupport of a High-Dimensional Distribution. Technical Report, Microsoft Research, 1999, MSR(cid:173)\nTR-99-87, Redmond, WA, To appear in Neural Computation. \n[8] S. Still, Walking Gait Control for Four-legged Robots, PhD Thesis, ETH Ziirich, 2000 \n[9] N. H. E. Weste & K. Eshraghian, Principles of CMOS VLSI Design, 1993, Addison Wesley \n[10] R. J. Vanderbei , LOQO User's Manual - Version 3.10, Technical Report, SOR-97-08, Princeton \nUniversity, Statistics and Operations Research, 1997 \n[11] D. Yanagihara, M. Udo, I. Kondo and T. Yoshida, Neuroscience Research, 1993, 18, pp. 241-\n244 \n\n\f", "award": [], "sourceid": 1824, "authors": [{"given_name": "Susanne", "family_name": "Still", "institution": null}, {"given_name": "Bernhard", "family_name": "Sch\u00f6lkopf", "institution": null}, {"given_name": "Klaus", "family_name": "Hepp", "institution": null}, {"given_name": "Rodney", "family_name": "Douglas", "institution": null}]}