{"title": "Delay Compensation with Dynamical Synapses", "book": "Advances in Neural Information Processing Systems", "page_first": 1088, "page_last": 1096, "abstract": "Time delay is pervasive in neural information processing. To achieve real-time tracking, it is critical to compensate the transmission and processing delays in a neural system. In the present study we show that dynamical synapses with short-term depression can enhance the mobility of a continuous attractor network to the extent that the system tracks time-varying stimuli in a timely manner. The state of the network can either track the instantaneous position of a moving stimulus perfectly (with zero-lag) or lead it with an effectively constant time, in agreement with experiments on the head-direction systems in rodents. The parameter regions for delayed, perfect and anticipative tracking correspond to network states that are static, ready-to-move and spontaneously moving, respectively, demonstrating the strong correlation between tracking performance and the intrinsic dynamics of the network. We also find that when the speed of the stimulus coincides with the natural speed of the network state, the delay becomes effectively independent of the stimulus amplitude.", "full_text": "Delay Compensation with Dynamical Synapses\n\nC. C. Alan Fung, K. Y. Michael Wong\n\nHong Kong University of Science and Technology, Hong Kong, China\n\nalanfung@ust.hk, phkywong@ust.hk\n\nState Key Laboratory of Cognitive Neuroscience and Learning,\n\nBeijing Normal University, Beijing 100875, China\n\nSi Wu\n\nwusi@bnu.edu.cn\n\nAbstract\n\nTime delay is pervasive in neural information processing. To achieve real-time\ntracking, it is critical to compensate the transmission and processing delays in a\nneural system. In the present study we show that dynamical synapses with short-\nterm depression can enhance the mobility of a continuous attractor network to the\nextent that the system tracks time-varying stimuli in a timely manner. The state\nof the network can either track the instantaneous position of a moving stimulus\nperfectly (with zero-lag) or lead it with an effectively constant time, in agreement\nwith experiments on the head-direction systems in rodents. The parameter regions\nfor delayed, perfect and anticipative tracking correspond to network states that are\nstatic, ready-to-move and spontaneously moving, respectively, demonstrating the\nstrong correlation between tracking performance and the intrinsic dynamics of the\nnetwork. We also \ufb01nd that when the speed of the stimulus coincides with the\nnatural speed of the network state, the delay becomes effectively independent of\nthe stimulus amplitude.\n\n1 Introduction\n\nTime delay is pervasive in neural information processing. Its occurrence is due to the time for signals\nto transmit in the neural pathways, e.g., 50-80 ms for electrical signals to propagate from the retina\nto the primary visual cortex [13], and the time for neurons responding to inputs, which is in the\norder of 10-20 ms. Delay is also inevitable for neural information processing. For a neural system\ncarrying out computations in the temporal domain, such as speech recognition and motor control,\ninput information needs to be integrated over time, which necessarily incur delays.\nTo achieve real-time tracking of fast moving objects, it is critical for a neural system to compensate\nfor the delay; otherwise, the object position perceived by the neural system will lag behind the\ntrue object position considerably. A natural way to compensate for delays is to predict the future\nposition of the moving stimulus. Experimental \ufb01ndings suggested that delay compensations are\nwidely adopted in neural systems. A remarkable example is the head-direction (HD) systems in\nrodents, which encode the head direction of a rodent in the horizontal plane relative to a static\nenvironment [14, 17]. It was found that when the head of a rodent is moving continuously in space,\nthe direction perceived by the HD neurons in the postsubicular cortex has nearly zero-lag with\nrespect to the instantaneous position of the rodent head [18]. More interestingly, in the anterior\ndorsal thalamic nucleus, the HD neurons perceive the future direction of the rodent head, leading the\ncurrent position by a constant time [3]. The similar anticipative behavior is also observed in the eye-\nposition neurons when animals make saccadic eye movement, the so-called saccadic remapping [16].\nIn human psychophysical experiments, the classic \ufb02ash-lag effect also supports the notion of delay\n\n1\n\n\fFigure 1: (a) Pro\ufb01les of u (x; t) and Iext (x; t) in the absence of STD, where the center of mass of\nthe stimulus is moving with constant velocity v = 0:02a=(cid:28)s. As shown, the pro\ufb01le of u (x; t) is\nalmost Gaussian. (b) The centers of mass of u (x; t) and Iext (x; t) as functions of time. Parameters:\n(cid:26) = 128=2(cid:25), a = 0:5, J0 =\n\np\n2(cid:25)a and (cid:26)J0A = 1:0.\n\ncompensation [12].\nIn the experiment, a \ufb02ash is perceived to lag behind a moving object, even\nthough they are physically aligned. The underlying cause is that the visual system predicts the\nfuture position of the continuously moving object, but is unable do so for the unpredictable \ufb02ash.\nDepending on the available information, the brain may employ different strategies for delay com-\npensation. In the case of self-motion, such as an animal rotating its head actively or performing\nsaccadic eye movements, the motor command responsible for the motion can serve as a cue for\ndelay compensation. It was suggested that an efference copy of the motor command, called corol-\nlary discharge, is sent to the corresponding internal representation system prior to the motion [18].\nFor the head rotation, the advanced time can be up to 20 ms; for the saccadic eye movement, the\nadvanced time is about 70 ms. In the case of tracking an external moving stimulus, the neural sys-\ntem has to rely on the moving speed of the stimulus for prediction. Asymmetric neural interactions\nhave been proposed to drive the network states to catch up with changes in head directions [22] or\npositions [4]. These may be achieved by the so-called conjunctive cells projecting neural signals\nbetween successive modules in forward directions [10]. To explain the \ufb02ash-lag effect, Nijhawan et\nal. proposed a dynamical routing mechanism to compensate the transmission delay in the visual sys-\ntem, in which retinal neurons dynamically choose a pathway according to the speed of the stimulus,\nand transmit the signal directly to the future position in the cortex [13].\nIn this study we propose a novel mechanism of how a neural system compensates for the processing\ndelay. By the processing delay, we mean the time consumed by a neural system in response to\nexternal inputs. The proposed mechanism does not require corollary discharge, or efforts of choosing\nsignal pathways, or speci\ufb01c network structures such as asymmetric interactions or conjunctive cells.\nIt is based on the short-term depression (STD) of synapses, the inherent and ubiquitous nature that\nthe synaptic ef\ufb01cacy of a neuron is reduced after \ufb01ring due to the depletion of neurotransmitters [11].\nIt has been found that STD enhances the mobility of the states of neural networks [21, 9, 6]. The\nunderlying mechanism is that neurotransmitters become depleted in the active region of the network\nstates compared with the neighboring regions, thus increasing the likelihood of the locally active\nnetwork state to shift to its neighboring positions when it is tracking a continuously shifting stimulus.\nWhen STD is suf\ufb01ciently strong, the tracking state of the network can even overtake the moving\nstimulus, demonstrating its potential for generating predictions.\n\n2 The Model\n\nWe consider continuous attractor neural networks (CANNs) as the internal representation models\nfor continuous stimuli [7, 2, 15]. A CANN holds a continuous family of bump-shaped stationary\nstates, which form a subspace in which the neural system is neutrally stable [20]. This property\nendows the neural system the capacity of tracking time-varying stimuli smoothly.\nConsider a continuous stimulus x being encoded by a neural ensemble. The variable x may represent\nthe orientation, the head direction, or the spatial location of an object. Neurons with preferred stimuli\nx produce the maximum response when an external stimulus is present at x. Their preferred stimuli\n\n2\n\n-202x-z(t)00.20.40.6u(x,t)u(x,t)Iext(x,t)00.020.040.06Iext(a)050100t/\u03c4s012z(t), z0(t)Iext(x,t)u(x,t)(b)\fare uniformly distributed in the space (cid:0)1 < x < 1. In the continuum limit, the dynamics of\nthe neural ensemble can be described by a CANN. We denote as u(x; t) the population-averaged\nsynaptic current to the neurons at position x and time t. The dynamics of u(x; t) is determined\nby the external input, the lateral interactions among the neurons, and its relaxation towards zero\nresponse. It is given by\n\n\u222b\n\n(cid:28)s\n\n@u (x; t)\n\n@t\n\n= Iext (x; t) + (cid:26)\n\n\u2032\n\ndx\n\nJ (x; x\n\n\u2032\n\n) p (x\n\n\u2032\n\n\u2032\n; t) r (x\n\n; t) (cid:0) u (x; t) ;\n\n(1)\n\nwhere (cid:28)s is the synaptic time constant, which is typically in the order of 1 to 5 ms, Iext(x; t) the\n\u2032, and\nexternal input, (cid:26) the density of neurons, J(x; x\nr(x; t) is the \ufb01ring rate of the neurons. The variable p(x; t) represents the fraction of available\nneurotransmitters, which evolves according to [6, 19]\n\n) the coupling between neurons at x and x\n\n\u2032\n\n(cid:28)d\n\n@p (x; t)\n\n@t\n\n= 1 (cid:0) p (x; t) (cid:0) (cid:28)d(cid:12)p (x; t) r (x; t) ;\n\n(2)\n\nwhere (cid:28)d is the STD time scale, which is typically of the order of 102 ms. In this work, we choose\n(cid:28)d = 50(cid:28)s. The STD effect is controlled by the parameter (cid:12), which can be considered as the fraction\nof total neurotransmitters consumed per spike.\nThe actual forms of J(x; x\nconvenience of analysis, we choose them to be\n\n) and r(x; t) depend on the details of the neural dynamics. Here, for the\n\n\u2032\n\nJ (x; x\n\n\u2032\n\n) =\n\np\nJ0\n2(cid:25)\na\n\nexp\n\nr (x; t) = (cid:2)[u(x; t)]\n\n]\n\n)2\n\n;\n\n[\n(cid:0) (x (cid:0) x\n\u2032\n\u222b\n\n2a2\n\nu (x; t)2\n\ndx\u2032u (x\u2032; t)2 ;\n\n1 + k(cid:26)\n\n(3)\n\n(4)\n\n\u2032\n\n) is translationally invariant in the space x, since it is a function of (x(cid:0)x\n\nwhere J0 and a control the magnitude and range of the neuronal excitatory interactions respectively.\n), which is essential\nJ(x; x\nfor the network state to be neutrally stable. In the expression for the \ufb01ring rate, (cid:2) is the step func-\ntion. Here, the stabilizing effect of inhibitory interactions is achieved by the divisive normalization\noperation in Eq. (4).\np\nLet us consider \ufb01rst the case without STD by setting (cid:12) = 0. Hence, p (x; t) = 1 in Eq. (1). For\nk (cid:20) kc (cid:17) (cid:26)J 2\n2(cid:25)a), the network holds a continuous family of Gaussian-shaped stationary\nstates when I ext(x; t) = 0. These stationary states are\n\n0 =(8\n\n\u2032\n\n(cid:22)u (x) = (cid:22)u0 exp\n\n(5)\nwhere (cid:22)u is the rescaled variable (cid:22)u (cid:17) (cid:26)J0u, and (cid:22)u0 is the rescaled bump height. The parameter z,\ni.e., the center of the bump, is a free parameter, implying that the stationary state of the network can\nbe located anywhere in the space x.\nNext, we consider the case that the network receives a moving input,\n\n4a2\n\n:\n\n[\n(cid:0) (x (cid:0) z)2\n\n]\n\n]\n\n[\n\n(cid:0) (x (cid:0) z0(t))2\n\n4a2\n\nIext(x; t) = A exp\n\n;\n\n(6)\n\nwhere A is the magnitude of the input and z0 the stimulus position.\nWithout loss of generality, we consider the stimulus position at time t = 0 to be z0 = 0, and the\nstimulus moves at a constant speed thereafter, i.e., z0 = vt for t (cid:21) 0. Let s (cid:17) z(t) (cid:0) z0(t) be the\ndisplacement between the network state and the stimulus position. It has been shown that without\nSTD, the steady value of the displacement is determined by [5]\n\n(\n\n)\n\nv = (cid:0) As\n(cid:28)s\n\nexp\n\n(cid:0) s2\n8a2\n\n:\n\n(7)\n\nNote that s has the opposite sign of v, implying that the network state always trails behind the\nstimulus (see Fig. 1(a)). This is due to the response delay of the network relative to the input.\n\n3\n\n\f3 Tracking in the Presence of STD\n\nThe analysis of tracking in the presence of STD is more involved. Motivated by the nearly Gaussian-\nshaped pro\ufb01le of the network states, we adopt a perturbation approach to solve the network dynam-\nics [5]. The key idea is to expand the network states as linear combinations of a set of orthonormal\nbasis functions corresponding to different distortion modes of the bump, that is,\n\nun (t)  n (x (cid:0) z) ;\npn (t) \u03d5n (x (cid:0) z) ;\n]\n[\n(\n(cid:0) (x (cid:0) z)2\n]\n[\n(\n(cid:0) (x (cid:0) z)2\n\n)\nx (cid:0) zp\n)\n2a\nx (cid:0) z\na\n\nexp\n\nexp\n\n4a2\n\n2a2\n\n:\n\n(8)\n\n(9)\n\n(10)\n\n(11)\n\n;\n\n\u2211\n\u2211\n\nn\n\nn\n\nu (x; t) =\n1 (cid:0) p (x; t) =\n\nwhere the basis functions are\n\n n (x (cid:0) z) =\n\n\u03d5n (x (cid:0) z) =\n\n1\u221ap\n1\u221ap\n\nHn\n\n2(cid:25)a2nn!\n\nHn\n\n(cid:25)a2nn!\n\nHere, Hn is the nth-order Hermite polynomial function.  n (x (cid:0) z) and \u03d5n (x (cid:0) z) have clear phys-\nical meanings. For instance, for n = 1; 2; 3; 4, they corresponds to, respectively, the height, the\nposition, the width and the skewness changes of the Gaussian bump. Depending on the approxima-\ntion precision, we can take the above expansions up to a proper order, and substituting them into\nEqs. (1) and (2) to solve the network dynamics analytically.\nResults obtained from the 11th order perturbation are shown in Fig. 2(a) for three representa-\ntive cases. They depend on the rescaled inhibition (cid:22)k (cid:17) k=kc and the rescaled STD strength\n(cid:22)(cid:12) (cid:17) (cid:28)d(cid:12)=((cid:26)2J 2\n0 ). When STD is weak, the tracking state lags behind the stimulus. When the\nSTD strength increases to a critical value (cid:22)(cid:12)perfect, s becomes effectively zero in a rather broad range\nof stimulus velocity, achieving perfect tracking. When the STD strength is above the critical value,\nthe tracking state leads the stimulus.\nHence delay compensation in a tracking task can be implemented at two different levels. The \ufb01rst\none is perfect tracking, in which the tracking state has zero-lag with respect to the true stimulus\nposition independent of the stimulus speed. The second one is anticipative tracking, in which the\ntracking state leads by a constant time (cid:28)ant relative to the stimulus position, that is, the tracking state\nis at the position the stimulus will travel to at a later time (cid:28)ant. To achieve a constant anticipation\ntime, it requires the leading displacement to increase with the stimulus velocity proportionally, i.e.,\ns = v(cid:28)ant. Both forms of delay compensation have been observed in the head-direction systems of\nrodents, and may serve different functional purposes.\n\n3.1 Prefect Tracking\n\nTo analyze the parameter regime for perfect tracking, it is instructive to consider the 1st order per-\nturbation of the network dynamics, i.e.,\n\nu [x (cid:0) z (t)] = u0 (t) exp\n\np [x (cid:0) z (t)] = 1 (cid:0) p0 (t) exp\n\n]\n\n+ p1 (t)\n\n[\n\n]\n\nexp\n\n[\n(cid:0) (x (cid:0) z (t))2\n\n2a2\n\nx (cid:0) z (t)\n\na\n\n(12)\n\n]\n\n:\n\n(13)\n\n]\n\n[\n(cid:0) (x (cid:0) z (t))2\n[\n(cid:0) (x (cid:0) z (t))2\n\n4a2\n\n;\n\n2a2\n\n4\n\n\fFigure 2: (a) The dependence of the displacement between the bump and the stimulus on the velocity\nof the moving stimulus for different values of (cid:22)(cid:12). Parameters: (cid:22)k = 0:4 and (cid:22)A = 1:8. (b) The\ndependence of (cid:22)(cid:12)perfect on (cid:22)k with (cid:22)A = 1:0. Symbols: simulations. Solid line: the predicted curve\nof (cid:22)(cid:12)perfect. Dashed line: the boundary separating the static and metastatic phases according to the\n1st order perturbation [6]. Inset: the dependence of (cid:22)(cid:12)perfect on (cid:22)A. Symbols: simulations. Lines:\ntheoretical prediction according to the 1st order perturbation.\n\nSubstituting them into Eqs. (1) and (2) and utilizing the orthogonality of the basis functions, we get\n(see Supplementary Material)\n\n(14)\n\n(15)\n\n(16)\n\n(17)\n\n\u221a\n\n)\n\n(\n)3=2\n1 (cid:0) p0\n(\n\n2\n7\n(cid:22)(cid:12) (cid:22)u2\n0\nB\n\n[\n\n1 +\n\np1 +\n\n(\n1 (cid:0) p0\n\n(cid:22)(cid:12) (cid:22)u2\n0\nB\n\n2\n3\n\n=\n\n=\n\np\n(cid:22)u2\n0\nB\n\n(cid:22)u0\nB\n\n2\n\n(\n[\n\n=\n\n(cid:28)s\n(cid:28)d\n= (cid:0) (cid:28)s\n(cid:28)d\n\n(cid:28)s\n\nd(cid:22)u0\ndt\n\n(cid:28)s\n2a\n\ndz\ndt\n\n(cid:28)s\n\ndp0\ndt\n\n(cid:28)s\np0\n\ndp1\ndt\n\n4\n7\n\n(cid:22)A\n2(cid:22)u0\n\n(\n)\n(cid:0) (cid:22)u0 + (cid:22)Ae\nvt (cid:0) z\n)\n]\n\u221a\n]\n)3=2\n(cid:0) p0\n\n2\n3\n\na\n\n(cid:0) (vt(cid:0)z)2\n\n8a2\n\n;\n\n(cid:0) (vt(cid:0)z)2\n\n8a2\n\ne\n\n;\n\n(cid:0) (cid:28)sp1\n2a\n\ndz\ndt\n\n;\n\np1\np0\n\n+\n\n(cid:28)s\na\n\ndz\ndt\n\n:\n\nAt the steady state, d(cid:22)u0=dt = dp0=dt = dp1=dt = 0, and dz=dt = v. Furthermore, for a suf\ufb01ciently\nsmall displacements, i.e., jsj=a \u226a 1, one can approximate (cid:22)A exp[(cid:0)(vt (cid:0) z)2=(8a2)] (cid:25) (cid:22)A and\n(cid:22)A[(vt (cid:0) z)=a] exp[(cid:0)(vt (cid:0) z)2=(8a2)] (cid:25) (cid:0) (cid:22)As=a. Solving the above equations, we \ufb01nd that s=a\ncan be expressed in terms of the variables (cid:22)u0= (cid:22)A, (cid:28)s=(cid:28)d and v(cid:28)d=a. When v(cid:28)d=a \u226a 1, the rescaled\ndisplacement s=a can be approximated by a power series expansion of the rescaled velocity v(cid:28)d=a.\nSince the displacement reverses sign when the velocity reverses, s=a is an odd function of v(cid:28)d=a.\nThis means that s=a (cid:25) c1(v(cid:28)d=a) + c3(v(cid:28)d=a)3. For perfect tracking in the low velocity limit, we\nhave c1 = 0 and \ufb01nd\n\n(\n\n)3\n\n;\n\n(18)\n\ns\na\n\n= (cid:0) C\n2\n\n(cid:22)u0\n(cid:22)A\n\n(cid:28)s\n(cid:28)d\n\nv(cid:28)d\na\n\nwhere C is a parameter less than 1 (the detailed expression can be found in Supplementary Material).\nFor the network tracking a moving stimulus, the input magnitude cannot be too small. This means\nthat (cid:22)u0= (cid:22)A is not a large number. Therefore, for tracking speeds up to v(cid:28)d=a (cid:24) 1, the displacement\ns is very small and can be regarded as zero effectively (see Fig. 2(a)). The velocity range in which\nthe tracking is effectively perfect is rather broad, since it scales as ((cid:28)d=(cid:28)s)1=3 \u226b 1.\nEquation (18) is valid when (cid:22)(cid:12) takes a particular value. This ields an extimate of (cid:22)(cid:12)perfect in the 1st\norder perturbation. Its expression is derived in Supplementary Material and plotted in Fig. 2(b).\nFor reference, we also plot the boundary that separates the metastatic phase above it from the static\nphase below, as reported in the study of intrinsic properties of CANNs with STD in [6]. In the static\nphase, the bump is stable at any position, whereas in the metastatic phase, the static bump starts to\nmove spontanaeously once it is pushed. Hence we say that the phase boundary is in a ready-to-move\nstate. Fig. 2(b) shows that (cid:22)(cid:12)perfect is just above the phase boundary. Indeed, when (cid:22)A approaches\n0, the expression of (cid:22)(cid:12)perfect reduces to the value of (cid:22)(cid:12) along the phase boundary for the 1st order\n\n5\n\n-2-1012\u03c4dv/a-0.2-0.100.10.20.30.4s/a\u03b2 = 0\u03b2 = 0.0035\u03b2 = 0.022(a)00.20.40.60.8k~00.0050.01\u03b2~perfect0.511.5A~00.0020.0040.0060.008k~ = 0.6k~ = 0.5k~ = 0.4k~ = 0.3k~ = 0.7(b)\fFigure 3: (a) The anticipatory time as a function of the speed of the stimulus. Different sets of\nparameters may correspond to different levels of anticipatory behavior. Parameter: (cid:22)k = 0:4. The\nnumerical scales are estimated from parameters in [8]. (b) The contours of constant anticipatory\ntime in the space of rescaled inhibition (cid:22)k and the rescaled STD strength (cid:22)(cid:12) in the limit of very small\nstimulus speed. Dashed line: boundary separating the static and metastatic phases. Dotted line:\nboundary separating the existence and non-existence phases of bumps. Calculations are done using\n11th order perturbation.\n\nperturbation. The inset of Fig. 2(b)) con\ufb01rms that (cid:22)(cid:12)perfect does not change signi\ufb01cantly with (cid:22)A for\ndifferent values of (cid:22)k. This implies that the network with (cid:22)(cid:12) = (cid:22)(cid:12)perfect exhibits effectively perfect\ntracking performance because it is intrinsically in a ready-to-move state.\n\n3.2 Anticipative Tracking\n\nWe further explore the network dynamics when the STD strength is higher than that for achieving\nperfect tracking. By solving the network dynamics with the perturbation expansion up to the 11th\norder, we obtain the relation between the displacement s and the stimulus speed v. The solid curve in\nFig. 2(a) shows that for strong STD, s increases linearly with v over a broad range of v. This implies\nthat the network achieves a constant anticipatory time (cid:28)ant over a broad range of the stimulus speed.\nTo gain insights into how the anticipation time depends on the stimulus speed, we consider the\nregime of small displacements.\nIn this regime, the rescaled displacement s=a can be approxi-\nmated by a power series expansion of the rescaled velocity v(cid:28)d=a, leading to s=a = c1(v(cid:28)d=a) +\nc3(v(cid:28)d=a)3. The coef\ufb01cients c1and c3 are determined such that the anticipation time in the limit\nv = 0 should be (cid:28)ant(0) = s=v, and that s=a reaches a maximum when v = vmax. This yields the\nresult\n\n[\n\n]\n\ns\na\n\n=\n\n(cid:28)ant (0)\n\n(cid:28)d\n\nv(cid:28)d\na\n\nHence the anticipatory time is given by\n\n(cid:0) 1\n3\n\n(cid:28)ant (v) = (cid:28)ant (0)\n\n)3\n\nv(cid:28)d\na\n\na\n\n)2(\n)\n\n(\n(\n1 (cid:0) v2\n3v2\n\nvmax(cid:28)d\n\n:\n\nmax\n\n:\n\n(19)\n\n(20)\n\nThis shows that the anticipation time is effectively constant in a wide range of stimulus velocities, as\nshown in Fig. 3(a). Even for v = 0:5vmax, the anticipation time is only reduced from its maximum\nby 9%.\nThe contours of anticipatory times for slowly moving stimuli are shown in Fig. 3(b). Hence the\nregion of anticipative behavior effectively coincides with the metastatic phase, as indicated by the\nregion above the phase line (dashed) in Fig. 2(b). In summary, there is a direct correspondence\nbetween delayed, perfect, and anticipative tracking on one hand, and the static, ready-to-move, and\nspontaneously moving beahviors on the other. This demonstrates the strong correlation between the\ntracking performance and the intrinsic behaviors of the CANN.\np\nWe compare the prediction of the model with experimental data. In a typical HD experiment of\n2, and the anticipation time drops from 20 ms at v = 0\nrodents [8], (cid:28)s = 1 ms, a = 28:5 degree=\nto 15 ms at v = 360 degree/s. Substituting into Eq. (19) and assuming (cid:28)d = 50(cid:28)s, these parameters\nyield a slope of 0:41 at the origin and the maximum lead at vmax(cid:28)d=a = 1:03. This result can be\n\n6\n\n00.20.40.60.81\u03c4dv/a00.20.40.60.811.2\u03c4ant/\u03c4d\u03b2 = 0.022, A = 1.8\u03b2 = 0.030, A = 1.5\u03b2 = 0.030, A = 1.00100200300400Angular velocity (degree/sec)1020304050Anticipatory time (ms)(a)00.20.40.60.81k00.0050.010.0150.020.0250.03\u03b2\u03c4ant = 0.2\u03c4d\u03c4ant = 0.1\u03c4d\u03c4ant = 0.02\u03c4d(b)Static\fcompared favorably with the curve of (cid:22)(cid:12) = 0:022 in Fig. 2(a), where the slope at the origin is 0:45\nand the maximum lead is located at vmax(cid:28)d=a = 1:01. Based on these parameters, the lowest curve\nplotted in Fig. 3(a) is consistent with the real data in Fig. 4 of [8].\n\nFigure 4: Con\ufb02uence points at natural\nspeeds. There are six curves in two\ngroups with different sets of parame-\nters. Curves in one group intersect at\nthe con\ufb02uence point with the natural\nspeed at the corresponding value of (cid:22)(cid:12).\nSymbols: simulations. Thin lines: pre-\ndiction of the displacement-velocity re-\nlation by 11th order perturbation. L1:\nnatural speed at (cid:22)(cid:12) = 0:005. L2: nat-\nural speed at (cid:22)(cid:12) = 0:01. L3: the line\nfor natural tracking at high (cid:22)A limit. Pa-\nrameter: (cid:22)k = 0:3.\n\n3.3 Natural Tracking\n\nFor strong enough STD, a CANN holds spontaneously moving bump states. The speed of the\nspontaneously moving bump is an intrinsic property of the network depending only on the network\nparameters. We call this the natural speed of the network, denoted as vnatural. An interesting issue is\nthe tracking performance of the network when the stimulus is moving at its natural speed.\nTwo sets of curves corresponding to two values of (cid:22)(cid:12) are shown in Fig. 4, when the stimulus am-\nplitude (cid:22)A is suf\ufb01ciently strong. The lines L1 and L2 indicate the corresponding natural speeds of\nthe system for these values of (cid:22)(cid:12). Remarkably, we obtain a con\ufb02uence point of these curves at the\nnatural speed. This point is referred to as the natural tracking point. It has the important property\nthat the lag is independent of the stimulus amplitude. This independence of s from (cid:22)A persists in the\nasymptotic limit of large (cid:22)A. In this limit, s approaches (cid:0)vnatural(cid:28)s , corresponding to a delay time\nof (cid:28)s, showing that the response is limited by the synaptic time scale in this limit. This asymptotic\nlimit is described by the line L3 and is identical for all values of (cid:22)k and (cid:22)(cid:12). Hence the invariant point\nfor natural tracking is given by (v; s) = (vnatural;(cid:0)vnatural(cid:28)s) for all values of (cid:22)k and (cid:22)(cid:12).\nWe also consider natural tracking in the weak (cid:22)A limit. Again we \ufb01nd a con\ufb02uence point of the\ndisplacement curves at the natural speed, but the delay time (and in some cases the anticipation\ntime) depends on the value of (cid:22)k. For example, at (cid:22)k = 0:3, the natural tracking point traces out an\neffectively linear curve in the space of v and s when (cid:22)(cid:12) increases, with a slope equal to 0:8(cid:28)s. This\nshows that the delay time is 0:8(cid:28)s, effectively independent of (cid:22)(cid:12) at (cid:22)k = 0:3. Since the delay time is\ndifferent from the value of (cid:28)s applicable in the strong (cid:22)A limit, the natural tracking point is slowly\ndrifting from the weak to the strong (cid:22)A limit. However, the magnitude of the natural time delay\nremains of the order of (cid:28)s. This is con\ufb01rmed by the analysis of the dynamical equations when the\nstimulus speed is vnatural + (cid:14)v in the weak (cid:22)A limit.\n\n3.4 Extension to other CANNs\n\nTo investigate whether the delay compensation behavior and the prediction of the natural tracking\npoint are general features of CANN models, we consider a network with Mexican-hat couplings.\nWe replace J(x; x\n\n) in Eq. (1) by\n\n\u2032\n\n[\n\n(\n\n(cid:0)\n\n1\n2\n\n\u2032\n\nx (cid:0) x\n2a\n\n]\n\n)2\n\n]\n\n[\n(cid:0) (x (cid:0) x\n\n2a2\n\n\u2032\n\n)2\n\nexp\n\nJ MH (x; x\n\n\u2032\n\n) = J0\n\nand r (x; t) in Eqs. (1) and (2) by\n\n;\n\n(21)\n\n(22)\n\nr (x; t) = (cid:2) [u (x; t)]\n\nu (x; t)2\n\n1 + u (x; t)2\n\n7\n\n00.511.522.5\u03c4dv/a-0.4-0.200.20.4s/a\u03b2 = 0.005, A = 1\u03b2 = 0.005, A = 2\u03b2 = 0.005, A = 4\u03b2 = 0.010, A = 1\u03b2 = 0.010, A = 2\u03b2 = 0.010, A = 4L1L2L3\fFigure 5: (a) The dependence of anticipatory time on the stimulus speed in the Mexican-hat model.\nParameter: (cid:12) = 0:003. (b) Natural speed of the network as a function of (cid:12). (c) Plot of s against v.\nThere is a con\ufb02uence point at the natural speed of the system. L1: the natural speed of the system\nat (cid:12) = 0:0011. Common parameters: (cid:26) = 128= (2(cid:25)) ; J0 = 0:5 and a = 0:5.\n\nFig. 5 shows that the network exhibits the same behaviors as the model in Eqs. (1) and (2). As\nshown in Fig. 5(a), the anticipatory times are effectively constant and similar in magnitude in the\nrange of stimulus speed comparable to experimental settings. In Fig. 5(b), the natural speed of the\nbump is zero for (cid:12) less than a critical value. As (cid:12) increases, the natural speed increases from zero.\nIn Fig. 5(c), the displacement s is plotted as a function of the stimulus speed v. The invariance\nof the displacement at the natural speed, independent of the stimulus amplitude, also appears in\nthe Mexican-hat model. The con\ufb02uence point of the family of curves is close to the natural speed.\nFurthermore, the displacement at the natural tracking point increases with the natural speed.\n\n4 Conclusions\n\nIn the present study we have investigated a simple mechanism of how processing delays can be com-\npensated in neural information processing. The mechanism is based on the intrinsic dynamics of a\nneural circuit, utilizing the STD property of neuronal synapses. The latter induces translational in-\nstability of neural activities in a CANN and enhances the mobility of the network states in response\nto external inputs. We found that for strong STD, the neural system can track moving stimuli with\neither zero-lag or a lead of a constant time. The conditions for perfect and anticipative tracking hold\nfor a wide range of stimulus speeds, making them applicable in practice. By choosing biologically\nplausible parameters, our model successfully justi\ufb01es the experimentally observed delay compen-\nsation behaviors. We also made an interesting prediction in the network dynamics, that is, when\nthe speed of the stimulus coincides with the natural speed of the network state, the delay becomes\neffectively independent of the stimulus amplitude. We also studied more than one kind of CANN\nmodels to con\ufb01rm the generality of our results.\nCompared with other delay compensation strategies relying on corollary discharge or dynamical\nrouting, the mechanism we propose here is fully dependent on the intrinsic dynamics of the network,\nnamely, the network automatically \u201cadjusts\u201d its tracking speed according to the input information.\nThere exists strong correlations between tracking performance and the intrinsic dynamics of the\nnetwork. The parameter regions for delayed, perfect and anticipative tracking correspond to network\nstates being static, ready-to-move and spontaneously moving, respectively. It has been suggested the\nanticipative response of HD neurons in anterior dorsal thalamus is due to the corollary discharge of\nmotor neurons responsible for moving the head. However, experimental studies revealed that when\nrats were moved passively (and hence no corollary discharge is available), either by hand or by a\nchart, the anticipative response of HD neurons still exists and has an even larger leading time [1].\nOur model provides a possible mechanism to describe this phenomenon.\n\nAcknowledgement\n\nThis work is supported by the Research Grants Council of Hong Kong (grant number 605010) and\nthe National Foundation of Natural Science of China (No.91132702, No.31221003).\n\n8\n\n00.20.40.60.81\u03c4dv/a-0.200.20.40.60.811.2\u03c4ant/\u03c4dA = 0.3A = 0.4A = 0.50100200300400Angular velocity (degree/sec)01020304050Anticipatory time (ms)(a)00.0010.0020.0030.004\u03b200.511.52vnatural\u03c4d/a(b)00.20.40.60.8v\u03c4d/a-0.08-0.0400.04s/aA = 0.1A = 0.2A = 0.3L1(c)\fReferences\n[1] J. P. Bassett, M. B. Zugaro, G. M. Muir, E. J. Golob, R. U. Muller and J. S. Taube. Passive\nMovements of the Head Do Not Abolish Anticipatory Firing Properties of Head Direction\nCells. J. Neurophysiol. 93, 1304-1316 (2005).\n\n[2] R. Ben-Yishai, R. Lev. Bar-Or, and H. Sompolinsky. Theory of orientation tuning in visual\n\ncortex. Proc. Natl. Acad. Sci. U.S.A. 92, 3844-3848 (1995).\n\n[3] H. T. Blair and P. E. Sharp. Anticipatory head direction signals in anterior thalamus: evidence\nfor a thalamocortical circuit that integrates angular head motion to compute head direction. J.\nNeurosci. 15, 6260-6270 (1995).\n\n[4] M. C. Fuhs and D. S. Touretzky. J. Neurosci. 26, A Spin Glass Model of Path Integration in\n\nRat Medial Entorhinal Cortex . 4266-4276 (2006).\n\n[5] C. C. A. Fung, K. Y. Wong and S. Wu. Neural Comput. Moving Bump in a Continuous Mani-\nfold: A Comprehensive Study of the Tracking Dynamics of Continuous Attractor Neural Net-\nworks. 22, 752-792 (2010).\n\n[6] C. C. A. Fung, K. Y. M. Wong, H. Wang and S. Wu. Dynamical Synapses Enhance Neural\nInformation Processing: Gracefulness, Accuracy and Mobility. Neural Comput. 24, 1147-1185\n(2012).\n\n[7] A. P. Georgopoulos, J. T. Lurito, M. Petrides, Mental rotation of the neuronal population vector.\n\nA. B. Schwartz, and J. T. Massey, Science 243, 234-236 (1989).\n\n[8] J. P. Goodridge and D. S. Touretzky. Modeling attractor deformation in the rodent head direc-\n\ntion system. J. Neurophysiol.83, 3402-3410 (2000).\n\n[9] Z. P. Kilpatrick and P. C. Bressloff. Effects of synaptic depression and adaptation on spatiotem-\n\nporal dynamics of an excitatory neuronal network. Physica D 239, 547-560 (2010).\n\n[10] B. L. McNaughton, F. P. Battaglia, O. Jensen, E. I. Moser and M.-B. Moser. Path integration\n\nand the neural basis of the \u2018cognitive map\u2019. Nature Rev. Neurosci. 7, 663-678 (2006).\n\n[11] H. Markram and M. Tsodyks. Redistribution of synaptic ef\ufb01cacy between neocortical pyrami-\n\ndal neurons. Nature 382, 807-810, 1996.\n\n[12] R. Nijhawan. Motion extrapolation in catching. Nature 370, 256-257 (1994).\n[13] R. Nijhawan and S. Wu. Phil. Compensating time delays with neural predictions: are predic-\n\ntions sensory or motor? Trans. R. Soc. A 367, 1063-1078 (2009).\n\n[14] J. O\u2019Keefe and J. Dostrovsky. The hippocampus as a spatial map: preliminary evidence from\n\nunit activity in the freely moving rat. Brain Res. 34, 171-175 (1971).\n\n[15] A. Samsonovich and B. L. McNaughton. Path integration and cognitive mappping in a contin-\n\nuous attractor neural network model. J. Neurosci. 17, 5900-5920 (1997).\n\n[16] M. A. Sommer and R. H. Wurtz. In\ufb02uence of the thalamus on spatial visual processing in\n\nfrontal cortex. Nature 444, 374-377 (2006).\n\n[17] J. S. Taube, R. U. Muller and J. B. Ranck Jr. Head-direction cells recorded from the postsubicu-\nlum in freely moving rats. I. Description and quantitative analysis. J. Neurosci. 10, 420-435\n(1990).\n\n[18] J. S. Taube and R. I. Muller. Comparisons of head direction cell activity in the postsubiculum\n\nand anterior thalamus of freely moving rats. Hippocampus 8, 87-108 (1998).\n\n[19] M. Tsodyks, K. Pawelzik, and H. Markram. Neural Networks with Dynamic Synapses. Neural\n\nComput. 10, 821-835 (1998).\n\n[20] S. Wu and S. Amari. Computing with Continuous Attractors: Stability and Online Aspects no\n\naccess. Neural Comput. 17, 2215-2239 (2005).\n\n[21] L. C. York and M. C. W. van Rossum. Recurrent networks with short term synaptic depression.\n\nJ. Comput. Neurosci 27, 707-620 (2009).\n\n[22] K. Zhang. Representation of spatial orientation bythe intrinsic dynamics of the head-direction\n\ncell ensemble: a theory. J. Neurosci. 16, 2112-2126 (1996).\n\n9\n\n\f", "award": [], "sourceid": 526, "authors": [{"given_name": "Chi", "family_name": "Fung", "institution": null}, {"given_name": "K.", "family_name": "Wong", "institution": null}, {"given_name": "Si", "family_name": "Wu", "institution": null}]}