{"title": "Analog VLSI Circuits for Attention-Based, Visual Tracking", "book": "Advances in Neural Information Processing Systems", "page_first": 706, "page_last": 712, "abstract": null, "full_text": "Analog VLSI Circuits for \n\nAttention-Based, Visual Tracking \n\nTimothy K. Horiuchi \n\nTonia G. Morris \n\nComputation and Neural Systems \nCalifornia Institute of Technology \n\nElectrical and Computer Engineering \n\nGeorgia Institute of Technology \n\nPasadena, CA 91125 \n\ntimmer@klab.caltech.edu \n\nAtlanta, GA, 30332-0250 \ntmorris@eecom.gatech.edu \n\nChristof Koch \n\nStephen P. DeWeerth \n\nComputation and Neural Systems \nCalifornia Institute of Technology \n\nElectrical and Computer Engineering \n\nGeorgia Institute of Technology \n\nPasadena, CA 91125 \n\nAtlanta, GA, 30332-0250 \n\nAbstract \n\nA one-dimensional visual tracking chip has been implemented us(cid:173)\ning neuromorphic, analog VLSI techniques to model selective visual \nattention in the control of saccadic and smooth pursuit eye move(cid:173)\nments. The chip incorporates focal-plane processing to compute \nimage saliency and a winner-take-all circuit to select a feature for \ntracking. The target position and direction of motion are reported \nas the target moves across the array. We demonstrate its function(cid:173)\nality in a closed-loop system which performs saccadic and smooth \npursuit tracking movements using a one-dimensional mechanical \neye. \n\n1 \n\nIntroduction \n\nTracking a moving object on a cluttered background is a difficult task. When more \nthan one target is in the field of view, a decision must be made to determine which \ntarget to track and what its movement characteristics are. If motion information is \nbeing computed in parallel across the visual field, as is believed to occur in the mid(cid:173)\ndle temporal area (MT) of primates, some mechanism must exist to preferentially \nextract the activity of the neurons associated with the target at the appropriate \n\n\fAnalog VLSI Circuits for Attention-Based, VISual Tracking \n\n707 \n\nPhotoreceptors \n\nTemporal \nDerivative \n\nSpatial \nDerivative \n\nDirection \nof Motion \n\nHysteretic \nwinner-take-all \n\n......... - - - -......... - - - -......... - - - Position \n\n-\n---~~---~----~-. Sa~ade \n\nST Target \n\nTrigger \n\nFigure 1: System Block Diagram: P = adaptive photoreceptor circuit, TD = tem(cid:173)\nporal derivative circuit, SD = spatial derivative, DM = direction of motion, HYS \nWTA = hysteretic winner-take-all, P2V = position to voltage, ST = saccade trig(cid:173)\nger. The TD and SD are summed to form the saliency map from which the WTA \nfinds the maximum. The output of the WTA steers the direction-of-motion infor(cid:173)\nmation onto a common output line. Saccades are triggered when the selected pixel \nis outside a specified window located at the center of the array. \n\ntime. Selective visual attention is believed to be this mechanism. \n\nIn recent years, many studies have indicated that selective visual attention is in(cid:173)\nvolved in the generation of saccadic [10] [7] [12] [15] and smooth pursuit eye move(cid:173)\nments [9] [6] [16]. These studies have shown that attentional enhancement occurs \nat the target location just before a saccade as well as at the target location during \nsmooth pursuit. In the case of saccades, attempts to dissociate attention from the \ntarget location has been shown to disrupt the accuracy or latency. \n\nKoch and Ullman [11] have proposed a model for attentional selection based on the \nformation of a saliency map by combining the activity of elementary feature maps \nin a topographic manner. The most salient locations are where activity from many \ndifferent feature maps coincide or at locations where activity from a preferentially(cid:173)\nweighted feature map, such as temporal change, occurs. A winner-take-all (WTA) \nmechanism, acting as the center of the attentional \"spotlight,\" selects the location \nwith the highest saliency. \n\nPrevious work on analog VLSI-based, neuromorphic, hardware simulation of visual \ntracking include a one-dimensional, saccadic eye movement system triggered by \ntemporal change [8] and a two-dimensional, smooth pursuit system driven by visual \nmotion detectors [5]. Neither system has a mechanism for figure-ground discrimi(cid:173)\nnation of the target. In addition to this overt form of attentional shifting, covert \n\n\f708 \n\nT. Horiuchi, T. G. Morris, C. Koch and S. P. DeWeerth \n\ng \n0 \n.t::. \nC. \n> \nE \n~ \n6 \n0 \nC/) \n\nE \nCD t: \n::J \n0 \n0 \nf0-\nE \nCD t: \n::J \n0 \n~ \n0 \n\n10.5 V \n\n,t it \u2022\u2022 \n\nF? ' \n\n\"~....-r, .\u2022. ,,..,..,. \n\n4 \n\n7 \n\n10 \n\n13 \n\n16 \n\n19 \n\n22 \n\nPixel Position \n\nVphoto \n\nI Spatial \nDerivative \n\nI \n\nI Temporal I \n\nDerivative \n\nDirection \nof Motion \n\nFigure 2: Example stimulus - Traces from top to bottom: Photoreceptor voltage, \nabsolute value of the spatial derivative, absolute value of the temporal derivative, \nand direction-of-motion. The stimulus is a high-contrast, expanding bar, which \nprovides two edges moving in opposite directions. The signed, temporal and spatial \nderivative signals are used to compute the direction-of-motion shown in the bottom \ntrace. \n\nattentional shifts have been modeled using analog VLSI circuits [4] [14], based on \nthe Koch and Ullman model. These circuits demonstrate the use of delayed, tran(cid:173)\nsient inhibition at the selected location to model covert attentional scanning. In \nthis paper we describe an analog VLSI implementation of an attention-based, visual \ntracking architecture which combines much of this previous work. Using a hardware \nmodel of the primate oculomotor system [8], we then demonstrate the use of the \ntracking chip for both saccadic and smooth pursuit eye movements. \n\n2 System Description \n\nThe computational goal of this chip is the selection of a target, based on a given \nmeasure of saliency, and the extraction of its retinal position and direction of mo(cid:173)\ntion. Figure 1 shows a block diagram of the computation. The first few stages of \nprocessing compute simple feature maps which drive the WTA-based selection of \na target to track. The circuits at the selected location signal their position and \nthe computed direction-of-motion. This information is used by an external saccadic \nand smooth pursuit eye movement system to drive the eye. The saccadic system \nuses the position information to foveate the target and the smooth pursuit system \nuses the motion information to match the speed of the target. \n\nAdaptive photoreceptors [2] (at the top of Figure 1) transduce the incoming pattern \nof light into an array of voltages. The temporal (TD) and spatial (SD) derivatives \nare computed from these voltages and are used to generate the saliency map and \ndirection of motion. Figure 2 shows an example stimulus and the computed features. \nThe saliency map is formed by summing the absolute-value of each derivative (ITDI \n+ ISD I) and the direction-of-motion (DM) signal is a normalized product of the two \n\n\fAnalog VLSI Circuits for Attention-Based, VISual Tracking \n\n709 \n\n3.5 \n\n3.0 \n\n2.5 \n\n2.0 \n\n1.5 \n\nTarget Position \n\n1 . 0+---r-~---+---r--~--+---r-~---+---r--~ \n-0.1 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 \n\nTime (seconds) \n\nFigure 3: Extracting the target's direction of motion: The WTA output voltage is \nused to switch the DM current onto a common current sensing line. The output \nof this signal is seen in the top trace. The zero-motion level is indicated by the \nflat line shown at 2.9 volts. The lower trace shows the target's position from the \nposition-to-voltage encoding circuits. The target's position and direction of motion \nare used to drive saccades and smooth pursuit eye movements during tracking. \n\nd \u00b7 \u00b7 TD\u00b7SD \nerlvatIves. ITDI+lsDI \n\nIn the saliency map, the temporal and spatial derivatives can be differentially \nweighted to emphasize moving targets over stationary targets. The saliency map \nprovides the input to a winner-take-all (WTA) computation which finds the max(cid:173)\nimum in this map. Spatially-distributed hysteresis is incorporated in this winner(cid:173)\ntake-all computation [4] by adding a fixed current to the winner's input node and \nits neighbors. This distributed hysteresis is motivated by the following two ideas: \n1) once a target has been selected it should continue to be tracked even if another \nequally interesting target comes along, and 2) targets will typically move continu(cid:173)\nously across the array. Hysteresis reduces oscillation of the winning status in the \ncase where two or more inputs are very close to the winning input level and the lo(cid:173)\ncal distribution of hysteresis allows the winning status to freely shift to neighboring \npixels rather than to another location further away. \n\nThe WTA output signal is used to drive three different circuits: the position-to(cid:173)\nvoltage (P2V) circuit [3], the DM-current-steering circuit (see Figure 3), and the \nsaccadic triggering (ST) circuit. The only circuits that are active are those at the \nwinning pixel locations. The P2V circuit drives the common position output line \nto a voltage representing it's position in the array, the DM-steering circuit puts \nthe local DM circuit's current onto the common motion output line, and the ST \ncircuit drives a position-specific current onto a common line to be compared against \nan externally-set threshold value. By creating a \"V\" shaped profile of ST currents \ncentered on the array, winning pixels away from the center will exceed the threshold \n\n\f710 \n\nT. Horiuch~ T. G. Mo\"is, C. Koch and S. P. DeWeerth \n\n2.6 \n\n\"2 \no 2.7 \nE \n(/) o \nQ. \na; \n.~ \nQ. \n-; 2.5 \nN \n'0 > 2.4 \n\n2.1+----+----+---~----~--_4----~--~~--~ \n40 \n\n30 \n\n20 \n\n35 \n\no \n\n5 \n\n10 \n\n15 \n\n25 \n\nTime (msec) \n\nFigure 4: Position vs. time traces for the passage of a strong edge across the array \nat five different speeds. The speeds shown correspond to 327, 548, 1042, 1578,2294 \npixels/sec. \n\nand send saccade requests off-chip. Figure 3 shows the DM and P2V outputs for \nan oscillating target. \n\nTo test the speed of the tracking circuit, a single edge was passed in front of the array \nat varying speeds. Figure 4 shows some of these results. The power consumption \nof the chip (23 pixels and support circuits, not including the pads) varies between \n0.35 m Wand 0.60 m W at a supply voltage of 5 volts. This measurement was taken \nwith no clock signal driving the scanners since this is not essential to the operation \nof the circuit. \n\n3 System Integration \n\nThe tracking chip has been mounted on a neuromorphic, hardware model of the \nprimate oculomotor system [8] and is being used to track moving visual targets. \nThe visual target is mounted to a swinging apparatus to generate an oscillating \nmotion. Figure 5 shows the behavior of the system when the retinal target position \nis used to drive re-centering saccades and the target direction of motion is used \ndrive smooth pursuit. Saccades are triggered when the selected pixel is outside a \nspecified window centered on the array and the input to the smooth pursuit system is \nsuppressed during saccades. The smooth pursuit system mathematically integrates \nretinal motion to match the eye velocity to the target velocity. \n\n4 Acknowledgements \n\nT. H. is supported by an Office of Naval Research AASERT grant and by the NSF \nCenter for Neuromorphic Systems Engineering at Caltech. T . M. is supported by \nthe Georgia Tech Research Institute. \n\n\fAnalog VLSI Circuits for Attention-Based, VISual Tracking \n\n711 \n\n40 \n\nAnalog VLSI \n\nHuman Subject \n\n\u00b7200 \n\n~ __ --~--~~~ __ --~--~~oo \n0.0 \n\n2.0 \n\n0.5 \n\n2.5 \n\n3.0 \n\n1.0 \n\n1.5 \n\n3.S \n\nFrom Collewijn and Tamminga, 1984 \n\nTime (seconds) \n\nFigure 5: Saccades and Smooth Pursuit: In this example, a swinging target is \ntracked over a few cycles. Re-centering saccades are triggered when the target \nleaves a specified window centered on the array. For comparison, on the right, we \nshow human data for the same task [1]. \n\nReferences \n\n[1] H. Collewijn and E. Tamminga, \"Human smooth and saccadic eye movements \nduring voluntary pursuit of different target motions on different backgrounds\" \nJ. Physiol., VoL 351, pp. 217-250. (1984) \n\n[2] T. Delbriick, Ph.D. Thesis, Computation and Neural Systems Program Califor(cid:173)\n\nnia Institute of Technology (1993) \n\n[3] S. P. DeWeerth, \"Analog VLSI Circuits for Stimulus Localization and Centroid \n\nComputation\" Inti. 1. Compo Vis. 8(3), pp. 191-202. (1992) \n\n[4] S. P. DeWeerth and T. G. Morris, \"CMOS Current Mode Winner-Take-All \nwith Distributed Hysteresis\" Electronics Letters, Vol. 31, No. 13, pp. 1051-1053. \n(1995) \n\n[5] R. Etienne-Cummings, J. Van der Spiegel, and P. Mueller \"A Visual Smooth \nPursuit Tracking Chip\" Advances in Neural Information Processing Systems 8 \n(1996) \n\n[6] V. Ferrara and S. Lisberger, \"Attention and Target Selection for Smooth Pursuit \n\nEye Movements\" J. Neurosci., 15(11), pp. 7472-7484, (1995) \n\n[7] J. Hoffman and B. Subramaniam, \"The Role of Visual Attention in Saccadic \n\nEye Movements\" Perception and Psychophysics, 57(6), pp. 787-795, (1995) \n\n[8] T . Horiuchi, B. Bishofberger, and C. Koch, \"An Analog VLSI Saccadic System\" \nAdvances in Neural Information Processing Systems 6, Morgan Kaufmann, pp. \n582-589, (1994) \n\n[9] B. Khurana, and E. Kowler, \"Shared Attentional Control of Smooth Eye Move(cid:173)\n\nment and Perception\" Vision Research, 27(9), pp. 1603-1618, (1987) \n\n\f712 \n\nT. Horiuchi, T. G. Morris, C. Koch and S. P. DeWeerth \n\nAnalog VLSI \n\nMonkey \n\nTarget Position \n\n.............. \n\n10 \n\n5 \n\n0 \n\nCi) \nCl \nQ) \n\nQ) \n\n-5 \n\n:s \nC> c: -10 \n\u00ab \n\n-15 \n\nEye Position \n\n100 msec saccadic delay added \n\n-20 -I---+--+---+-----1--+--_+_---.,r-----+---+----i \n3.8 \n\n3.4 \n\n2.9 \n\n3.0 \n\n2.8 \n\n3.2 \n\n3.3 \n\n3.1 \n\n3.5 \n\n3.6 \n\n3.7 \n\nnme (seconds) \n\nFrom Lisberger et aI., 1987 \n\nFigure 6: Step-Ramp Experiment: In this experiment, the target jumps from the \nfixation point to a new location and begins moving with constant velocity. On the \nleft, the analog VLSI system tracks the target. For comparison, on the right, we \nshow data from a monkey performing the same task [13]. \n\n[10] E. Kowler, E. Anderson, B. Dosher, E. Blaser, \"The Role of Attention in the \n\nProgramming of Saccades\" Vision Research, 35(13), pp. 1897-1916, (1995) \n\n[11] C. Koch and S. Ullman, \"Shifts in selective visual attention: towards the un(cid:173)\n\nderlying neural circuitry\" Human Neurobiology, 4:219-227, (1985) \n\n[12] R. Rafal, P. Calabresi, C. Brennan, and T. Scioltio, \"Saccade Preparation \nInhibits Reorienting to Recently Attended Locations\" 1. Exp. Psych: Hum. \nPercep. and Perf., 15, pp. 673-685, (1989) \n\n[13] S. G. Lisberger, E. J. Morris, and L. Tychsen, \"Visual motion processing and \nsensory-motor integration for smooth pursuit eye movements.\" In Ann. Rev. \nNeurosci., Cowan et al., editors. Vol. 10, pp. 97-129, (1987) \n\n[14] T. G. Morris and S. P. DeWeerth, \"Analog VLSI Circuits for Covert Atten(cid:173)\n\ntional Shifts\" Proc. 5th Inti. Conf. on Microelectronics for Neural Networks and \nFuzzy Systems - MicroNeur096, Feb 12-14, 1996. Lausanne, Switzerland, IEEE \nComputer Society Press, Los Alamitos, CA, pp. 30-37, (1996) \n\n[15] S. Shimojo, Y. Tanaka, O. Hikosaka, and S. Miyauchi, \"Vision, Attention, \nand Action - inhibition and facilitation in sensory motor links revealed by the \nreaction time and the line-motion.\" In Attention and Performance XVI, T. Inui \n& J. L. McClelland, editors. MIT Press, (1995) \n\n[16] W. J. Tam and H. Ono, \"Fixation Disengagement and Eye-Movement Latency\" \n\nPerception and Psychophysics, 56(3) pp. 251-260, (1994) \n\n\f", "award": [], "sourceid": 1286, "authors": [{"given_name": "Timothy", "family_name": "Horiuchi", "institution": null}, {"given_name": "Tonia", "family_name": "Morris", "institution": null}, {"given_name": "Christof", "family_name": "Koch", "institution": null}, {"given_name": "Stephen", "family_name": "DeWeerth", "institution": null}]}