{"title": "Using hippocampal 'place cells' for navigation, exploiting phase coding", "book": "Advances in Neural Information Processing Systems", "page_first": 929, "page_last": 936, "abstract": "", "full_text": "Using hippocampal 'place cells' for \nnavigation, exploiting phase coding \n\nNeil Burgess, John O'Keefe and Michael Recce \nDepartment of Anatomy, University College London, \n\nLondon WC1E 6BT, England. \n(e-mail: n.burgess<Ducl.ac . uk) \n\nAbstract \n\nA model of the hippocampus as a central element in rat naviga(cid:173)\ntion is presented. Simulations show both the behaviour of single \ncells and the resultant navigation of the rat. These are compared \nwith single unit recordings and behavioural data. The firing of \nCAl place cells is simulated as the (artificial) rat moves in an en(cid:173)\nvironment. This is the input for a neuronal network whose output, \nat each theta (0) cycle, is the next direction of travel for the rat. \nCells are characterised by the number of spikes fired and the time \nof firing with respect to hippocampal 0 rhythm. 'Learning' occurs \nin 'on-off' synapses that are switched on by simultaneous pre- and \npost-synaptic activity. The simulated rat navigates successfully to \ngoals encountered one or more times during exploration in open \nfields. One minute of random exploration of a 1m2 environment \nallows navigation to a newly-presented goal from novel starting po(cid:173)\nsitions. A limited number of obstacles can be successfully avoided. \n\n1 Background \n\nExperiments have shown the hippocampus to be crucial to the spatial memory and \nnavigational ability of the rat (O'Keefe & Nadel, 1978). Single unit recordings in \nfreely moving rats have revealed 'place cells' in fields CA3 and CAl of the hip(cid:173)\npocampus whose firing is restricted to small portions of the rat's environment (the \ncorresponding 'place fields') (O'Keefe & Dostrovsky, 1971), see Fig. 1a. In addi(cid:173)\ntion cells have been found in the dorsal pre-subiculum whose primary behavioural \n\n929 \n\n\f930 \n\nBurgess, O'Keefe, and Reece \n\na \n\nI \n\nII \n\n\u2022 IIII \n\nII \n\nb \n\n360\u00b7 \nPhase \n\nA \n\nB \n\n1 \n\nTheta \n[mV) \n\n\u00b71 \n\nTime [s] \n\nFigure 1: a) A typical CAl place field, max. rate (over 18) is 13.6 spikes/so b) One \nsecond of the EEG () rhythm is shown in C, as the rat runs through a place field. \nA shows the times of firing of the place cell. Vertical ticks immediately above and \nbelow the EEG mark the positive to negative zero-crossings of the EEG, which we \ndefine as 00 (or 360 0 ) of phase. B shows the phase of () at which each spike was \nfired (O'Keefe & Recce, 1992). \n\ncorrelate is 'head-direction' (Taube et aI., 1990). Both are suggestive of navigation. \n\nTemporal as well as spatial aspects of the electrophysiology of the hippocampal \nregion are significant for a model. The hippocampal EEG '() rhythm' is best char(cid:173)\nacterised as a sinusoid of frequency 7 - 12H z and occurs whenever the rat is making \ndisplacement movements. Recently place cell firing has been found to have a sys(cid:173)\ntematic phase relationship to the local EEG (O'Keefe & Recce, 1992), see \u00a73.1 and \nFig. lb. Finally, the () rhythm has been found to modulate long-term potentiation \nof synapses in the hippocampus (Pavlides et al., 1988). \n\n2 \n\nIntroduction \n\nWe are designing a model that is consistent with both the data from single unit \nrecording and the behavioural data that are relevant to spatial memory and navi(cid:173)\ngation in the rat. As a first step this paper examines a simple navigational strategy \nthat could be implemented in a physiologically plausible way to enable navigation \nto previously encountered reward sites from novel starting positions. We assume \nthe firing properties of CAl place cells, which form the input for our system. \n\nThe simplest map-based strategies (as opposed to route-following ones) are based \non defining a surface over the whole environment, on which gradient ascent leads to \nthe goal (e.g. delayed reinforcement or temporal difference learning). These tend \nto have the problem that, to build up this surface, the goal must be reached many \ntimes, from different points in the environment (by which time the rat has died of \nold age). Further, a new surface must be computed if the goal is moved. Specific \nproblems are raised by the properties of rats' navigation: (i) the position of CAl \nplace fields is independent of goal position (Speakman & O'Keefe, 1990); (ii) high \nfiring rates in place cells are restricted to limited portions of the environment; (iii) \nrats are able to navigate after a brief exploration of the environment, and (iv) can \ntake novel short-cuts or detours (Tolman, 1948). \n\n\fUsing hippocampal 'place cells' for navigation, exploiting phase coding \n\n931 \n\nTo overcome these problems we propose that a more diffuse representation of posi(cid:173)\ntion is rapidly built up downstream of CAl, by cells with larger firing fields than in \nCAL The patterns of activation of this group of cells, at two different locations in \nthe environment, have a correlation that decreases with the separation of the two \nlocations (but never reaches zero, as is the case with small place fields). Thus the \noverlap between t.he pattern of activity at any moment and the pattern of activity \nat the goal location would be a measure of nearness to the goal. We refer to these \ncells as 'subicular' cells because the subiculum seems a likely site for them, given \nsingle unit recordings (Barnes et al., 1990) showing spatially consistent firing over \nlarge parts of the environment. \n\nWe show that the output of these subicular cells is sufficient to enable navigation \nin our model. In addition the model requires: (i) 'goal' cells (see Fig. 4a) that \nfire when a goal is encountered, allowing synaptic connections from subicular cells \nto be switched on, (ii) phase-coded place cell firing, (iii) 'head-direction' cells, and \n(iv) synaptic change that is modulated by the phase of the EEG. The relative \nfiring rates of groups of goal cells code for the direction of objects encountered \nduring exploration, in the same way that cens in primate motor cortex code for the \ndirection of arm movements (Georgopoulos et al., 1988). \n\n3 The model \n\nIn our simulation a rat is in constant motion (speed 30cm/ s) in a square environment \nof size L x L (L ~ 150cm). Food or obstacles can be placed in the environment \nat any time. The rat is aware of any objects within 6cm (whisker length) of its \nposition. It bounces off any obstacles (or the edge of the environment) with which \nit collides. The f) frequency is taken to be 10Hz (period O.ls) and we model each \nf) cycle as having 5 different phases. Thus the smallest timestep (at which synaptic \nconnections and cell firing rates are updated) is 0.02s. The rat is either 'exploring' \n(its current direction is a random variable within 30 0 of its previous direction), or \n'searching' (its current direction is determined by the goal cells, see below). Synaptic \nand cell update rules are the same during searching or exploring. \n\n3.1 The phase of CAl place cell firing \n\nWhen a rat on a linear track runs through a place field, the place cell fires at \nsuccessively earlier phases of the EEG f) rhythm . A cell that fires at phase 3600 \nwhen the rat enters the place field may fire as much as 355 0 earlier in the f) cycle \nwhen exiting the field (O'Keefe & Recce, 1992), see Fig. lb. \n\nSimulations below involve 484 CAl place cells with place field centres spread evenly \non a grid over the whole environment. The place fields are circular, with diameters \n0.25L, 0.35L or Oo4L (as place fields appear to scale with the size of an environment; \nMuller & Kubie, 1987). The fraction of cells active during any O.ls interval is thus \n7r(0.125 2 + 0.175 2 + 0.2 2 )/3 = 9%. When the rat is in a cell's place field it fires 1 to \n3 spikes depending on its distance from the field centre, see Fig. 2b. \n\nWhen the (simulated) rat first enters a place field the cell fires 1 spike at phase \n360 0 of the f) rhythm; as the rat moves through the place field, its phase of firing \nshifts backwards by 72 0 every time the number of spikes fired by the cell changes \n\n\f932 \n\nBurgess, O'Keefe, and Reece \n\na \n\nc \n\nEJlElm \u2022 \u2022 \n360\u00b7 288\u00b7 216\u00b7 144\u00b7 72\u00b7 \n\n0.0 \n\n0.2 \n\n0.4 \n\n0.6 \n\n0.8 \n\nFigure 2: a) Firing rate map of a typical place cell in the model (max. rate 11.6 \nspikes/s); b) Model of a place field; the numbers indicate the number of spikes fired \nby the place cell when the rat is in each ring. c) The phase at which spikes would \nbe fired during all possible straight trajectories of the rat through the place field \nfrom left to right. d) The total number of spikes fired in the model of CAl versus \ntime, the phase of firing of one place cell (as the rat runs through the centre of the \nfield) is indicated be vertical ticks above the graph. \n\n(i.e. each time it crosses a line in Fig. 2b). Thus each theta cycle is divided into \n5 timesteps. No shift results from passing through the edge of the field, whereas a \nshift of 288 0 (0.08s) results from passing through the middle of the field, see Fig. \n2c. The consequences for the model in terms of which place cells fire at different \nphases within one () cycle are shown in Fig. 3. The cells that are active at phase \n360 0 have place fields centred ahead of the position of the rat (i.e. place fields that \nthe rat is entering), those active at phase 00 have place fields centred behind the \nrat. If the rat is simultaneously leaving field A and entering field B then cell A fires \nbefore cell B, having shifted backwards by up to 0.08s. The total number of spikes \nfired at each phase as the rat moves about implies that the envelope of all the spikes \nfired in CAl oscillates with the () frequency. Fig. 2d shows the shift in the firing of \none cell compared to the envelope (cf. Fig. 1b). \n\n3.2 Subicular cells \n\nWe simulate 6 groups of 80 cells (480 in total); each subicular cell receives one \nsynaptic connection from a random 5% of the CAl cells. These connections are \neither on or off (1 or 0). At each timestep (0.02s) the 10 cells in each group with \nthe greatest excitatory input from CAl fire between 1 and 5 spikes (depending on \ntheir relative excitation). Fig. 3c shows a typical subicular firing rate map. The \nconsequences of phase coding in CAl (Figs. 3a and b) remain in these subicular \ncells as they are driven by CAl: the net firing field of all cells active at phase 360 0 \nof () is peaked ahead of the rat. \n\n\fUsing hippocampal 'place cells' for navigation, exploiting phase coding \n\n933 \n\na \n\nb \n\nFigure 3: Net firing rate map of all the place cells that were active at the 360 0 \n( a) \nand 72 0 (b) phases of e as the rat ran through the centre of the environment from \nleft to right. c) Firing rate map of a typical 'subicular' cell in the model; max. rate \n(over LOs) is 46.4 spikes/so Barnes et al. (1990) found max. firing rates (over O.ls) \nof 80 spikes/s (mean 7 spikes/s) in the subiculum. \n\na \n\nN SEW \n\no 0 0 0 \n\nGoal cells \n\nb \n\nSubicular cells \n<? q q 9 6x80 (480) \n....\u2022.... \n\nonloff synapses \n5% connectivity \n\nPlace cells \n000000000000 22x22 (484) \n\nFigure 4: a) Connections and units in the model; interneurons shown between the \nsubicular cells indicate competitive dynamics, but are not simulated explicitly. b) \nThe trajectory of 90 seconds of 'exploration' in the central 126 x 126cm2 of the \nenvironment. The rat is shown in the bottom left hand corner, to scale. \n\n3.2.1 Learning \n\nThe connections are initialised such that each subicular cell receives on average \none 'on' connection. Subsequently a synaptic connection can be switched on only \nduring phases 180 0 to 360 0 of e. A synapse becomes switched on if the pre-synaptic \ncell is active, and the post-synaptic cell is above a threshold activity (4 spikes), in \nthe same timestep (0.02s). Hence a subicular firing field is rapidly built up during \nexploration, as a superposition of CAl place fields, see Fig 3c. \n\n3.3 Goal cells \n\nThe correlation between the patterns of activity of the subicular cells at two differ(cid:173)\nent locations in the environment decreases with the separation of the two locations. \nThus if synaptic connections to a goal cell were switched on when the rat encoun(cid:173)\ntered food then a firing rate map of the goal cell would resemble a cone covering \nthe entire environment, peaked at the food site, i.e. the firing rate would indicate \n\n\f934 \n\nBurgess, O'Keefe, and Reece a_ \n\nc \n\nFigure 5: Goal cell firing fields, a) West, b) East, of 'food' encountered at the centre \nof the environment. c) Trajectories to a goal from 8 novel starting positions. All \nfigures refer to encountering food immediately after the exploration in Fig. 4b. \nNotice that much of the environment was never visited during exploration. \n\nthe closeness of the food during subsequent movement of the rat. The scheme we \nactually use involves groups of goal cells continuously estimating the distance to 4 \npoints displaced from the goal site in 4 different directions. \n\nNotice that when a freely moving rat encounters an interesting object a fair amount \nof 'local investigation' takes place (sniffing, rearing, looking around and local explo(cid:173)\nration). During the local investigation of a small object the rat crosses the location \nof the object in many different directions. We postulate groups of goal cells that \nbecome excited strongly enough to induce synaptic change in connections from \nsubicular cells whenever the rat encounters a specific piece of food and is heading in \na particular direction. This supposes the joint action of an object classifier and of \nhead-direction cells; head-direction cells corresponding to different directions being \nconnected to different goal cells. Since synaptic change occurs only at the 180 0 to \n360 0 phases of 8, and the net firing rate map of all the subicular cells that are active \nat phase 360 0 during any 8 cycle is peaked ahead of the rat, goal cells have firing \nfields that are peaked a little bit away from the goal position. For example, goal \ncells whose subicular connections are changed when the rat is heading east have \nfiring rate fields that are peaked to the east of the goal location, see Fig. 5. \n\nLocal investigation of a food site is modelled by the rat moving 12cm to the north, \nsouth, east and west and occurs whenever food is encountered. Navigation is re(cid:173)\nstricted to the central 126 x 126cm2 portion of the 150 x 150cm2 environment (over \nwhich firing rate maps are shown) to leave room for this. There are 4 goal cells \nfor every piece of food found in the environment, (GC-Ilorth, GC.-South, GC_east, \nGC_west), see Fig. 4a. Initially the connections from all subicular cells are off; they \nare switched on if the subicular cell is active and the rat is at the particular piece of \nfood, travelling in the right direction. When the rat is searching, goal cells simply \nfire a number of spikes (in each 0.025 timestep) that is proportional to their net \nexcitatory input from the subicular cells. \n\n3.4 Maps and navigation \n\nWhen the rat is to the north of the food, GC-Ilorth fires at a higher rate than \nGC..south. We take the firing rate of GC_north to be a 'vote' that the rat is north \n\n\fUsing hippocampal 'place cells' for navigation, exploiting phase coding \n\n935 \n\na \n\nb \n\nc \n\n.. --_ .. _ .. --- ...... , \n\n\",., ; \n:' ~* \nf ~~ \n: + \n-- \". \n\nFigure 6: a) Trajectory of rat with alternating goals. b) an obstacle is interposed; \nthe rat collides with the obstacle on the first run, but learns to avoid the collision site \nin the 2 subsequent runs. c) Successive predictions of goal (box) and obstacle (cross) \npositions generated as the rat ran from one goal site to the other; the predicted \npositions get more accurate as the rat gets closer to the object in question. \n\nof the goal. Similarly the firing rate of GCJlouth is a vote that the rat is south \nof the goal: the resultant direction (the vector sum of directions north, south, east \nand west, weighted by the firing rates of the corresponding cells) is an estimate \nof the direction of the rat from the food (cf.Georgopoulos et al., 1988). Since the \nfiring rate maps of the 4 goal cells are peaked quite close to the food location, their \nnet firing rate increases as the food is approached, i.e. it is an estimation of how \nclose the food is. Thus the firing rates of the 4 goal cells associated with a piece of \nfood can be used to predict its approximate position relative to the rat (e.g. 70cm \nnortheast), as the rat moves about the environment (see Fig. 6c). \n\nWe use groups of goal cells to code for the locations at which the rat encountered \nany objects (obstacles or food), as described above. A new group of goal cells is \nrecruited every time the rat encounters a new object, or a new (6cm) part of an \nextended object. The output of the system acts as a map for the rat, telling it \nwhere everything is relative to itself, as it moves around. The process of navigation \nis to decide which way to go, given the information in the map. When there are \nno obstacles in the environment, navigation corresponds to moving in the direction \nindicated by the group of goal cells corresponding to a particular piece of food. \nWhen the environment includes many obstacles the task of navigation is much \nharder, and there is not enough clear behavioural data to guide modelling. \n\nWe do not model navigation at a neuronal level, although we wish to examine the \nnavigation that would result from a simple reading of the 'map' provided by our \nmodel. The rules used to direct the simulated rat are as follows: (i) every 0.18 the \ndirection and distance to the goal (one of the pieces of food) are estimated; (ii) \nthe direction and distance to all locations at which an obstacle was encountered \nare estimated; (iii) obstacle locations are classified as 'in-the-way' if (a) estimated \nto be within 45\u00b0 of the goal direction, (b) closer than the goal and (c) less than \nL/2 away; (iv) the current direction of the rat becomes the vector sum of the goal \ndirection (weighted by the net firing rate of the corresponding 4 goal cells) minus \nthe directions to any in-the-way obstacles (weighted by the net firing rate of the \n'0 bstacle cells' and by the similarity of the obstacle and goal directions). \n\n\f936 \n\nBurgess, O'Keefe, and Reece \n\n4 Performance \n\nThe model achieves latent learning (i.e. the map is constructed independently of \nknowledge of the goal, see e.g. Tolman, 1948). A piece of food encountered only \nonce, after exploration, can be returned to, see Fig. 5c. Notice that a large part \nof the environment was never visited during exploration (Fig. 4b). Navigation is \nequally good after exploration in an environment containing food/obstacles from the \nbeginning. If the food is encountered only during the earliest stages of exploration \n(before a stable subicular representation is built up) then performance is worse. \nMultiple goals and a small number of obstacles can be accommodated, see Fig. 6. \nNotice that searching also acts as exploration, and that synaptic connections can \nbe switched at any time: all learning is incremental, but saturates when all the \nrelevant synapses have been switched on. Performance does not depend crucially \non the parameter values, used although it is worse with fewer cells, and smaller \nenvironments require less exploration before reliable navigation is possible (e.g. 60s \nfor a 1m2 box). Quantitative analysis will appear in a longer paper. \n\nReferences \n\nBarnes C A, McNaughton B L, Mizumori S J Y, Leonard B W & Lin L-H (1990) 'Com(cid:173)\nparison of spatial and temporal characteristics of neuronal activity in sequential stages of \nhippocampal processing', Progreu in Brain Re6earch 83 287-300. \n\nGeorgopoulos A P, Kettner R E & Schwartz A B (1988) 'Primate motor cortex and free \narm movements to visual targets in three-dimensional space. II. Coding of the direction \nof movement by a neuronal population', J. Neur06d. 8 2928-2937. \n\nMuller R U & Kubie J L (1987) 'The effects of changes in the environment on the spatial \nfiring of hippocampal complex-spike cells', J. Neur06d. 7 1951-1968. \n\nO'Keefe J & Dostrovsky J (1971) 'The hippocampus as a spatial map: preliminary evidence \nfrom unit activity in the freely moving rat', BrainRe6. 34 171-175. \n\nO'Keefe J & Nadel L (1978) The hippocampu6 a6 a cognitive map, Clarendon Press, Oxford. \n\nO'Keefe J & Reece M (1992) 'Phase relationship between hippocampal place units and \nthe EEG theta rhythm', Hippocampu!, to be published. \n\nPavlides C, Greenstein Y J, Grudman M & Winson J (1988) 'Long-term potentiation in \nthe dentate gyrus is induced preferentially on the positive phase of O-rhythm', Brain Re6. \n439 383-387. \n\nSpeakman A S & O'Keefe J (1990) 'Hippocampal complex spike cells do not change their \nplace fields if the goal is moved within a cue controlled environment', European Journal \nof Neuro!cience 2 544-555. \n\nTaube J S, Muller R U & Ranck J B Jr (1990) 'Head-direction cells recorded from the \npostsubiculum in freely moving rats. I. Description & quantitative analysis', J. Neur06ci. \n10 420-435. \n\nTolman E C (1948) 'Cognitive Maps in rats and men', P6ychological Review 55 189-208. \n\n\f", "award": [], "sourceid": 645, "authors": [{"given_name": "Neil", "family_name": "Burgess", "institution": null}, {"given_name": "John", "family_name": "O'Keefe", "institution": null}, {"given_name": "Michael", "family_name": "Recce", "institution": null}]}