{"title": "A Recurrent Model of Orientation Maps with Simple and Complex Cells", "book": "Advances in Neural Information Processing Systems", "page_first": 995, "page_last": 1002, "abstract": "", "full_text": "A Recurrent Model of Orientation Maps \n\nwith Simple and Complex Cells \n\n \n\n \n \n\nDepartment of Bioengineering \n\nUniversity of Pennsylvania \n\nPaul Merolla and Kwabena Boahen \n\nPhiladelphia, PA 19104 \n\n \n{pmerolla,boahen} @seas.upenn.edu \n\nAbstract \n\nthat utilizes \n\nWe describe a neuromorphic chip \ntransistor \nheterogeneity, introduced by the fabrication process, to generate \norientation maps similar to those imaged in vivo. Our model \nconsists of a recurrent network of excitatory and inhibitory cells in \nparallel with a push-pull stage. Similar to a previous model the \nrecurrent network displays hotspots of activity that give rise to \nvisual feature maps. Unlike previous work, however, the map for \norientation does not depend on the sign of contrast. Instead, sign-\nindependent cells driven by both ON and OFF channels anchor the \nmap, while push-pull interactions give rise to sign-preserving cells. \nThese two groups of orientation-selective cells are similar to \ncomplex and simple cells observed in V1. \n\n1 Orientation Maps \n\nNeurons in visual areas 1 and 2 (V1 and V2) are selectively tuned for a number of \nvisual features, the most pronounced feature being orientation. Orientation \npreference of individual cells varies across the two-dimensional surface of the \ncortex in a stereotyped manner, as revealed by electrophysiology [1] and optical \nimaging studies [2]. The origin of these preferred orientation (PO) maps is debated, \nbut experiments demonstrate that they exist in the absence of visual experience [3]. \nTo the dismay of advocates of Hebbian learning, these results suggest that the initial \nappearance of PO maps rely on neural mechanisms oblivious to input correlations. \nHere, we propose a model that accounts for observed PO maps based on innate noise \nin neuron thresholds and synaptic currents. The network is implemented in silicon \nwhere heterogeneity is as ubiquitous as it is in biology. \n\n2 Patterned Activity Model \n\nErnst et al. have previously described a 2D rate model that can account for the \norigin of visual maps [4]. Individual units in their network receive isotropic \nfeedforward input from the geniculate and recurrent connections from neighboring \n\n\f \n\nunits in a Mexican hat profile, described by short-range excitation and long-range \ninhibition. If the recurrent connections are sufficiently strong, hotspots of activity \n(or \u2018bumps\u2019) form periodically across space. In a homogeneous network, these \nbumps of activity are equally stable at any position in the network and are free to \nwander. \nIntroducing random jitter to the Mexican hat connectivity profiles breaks the \nsymmetry and reduces the number of stable states for the bumps. Subsequently, the \nbumps are pinned down at the locations that maximize their net local recurrent \nfeedback. In this regime, moving gratings are able to shift the bumps away from \ntheir stability points such that the responses of the network resemble PO maps. \nTherefore, the recurrent network, given an ample amount of noise, can innately \ngenerate its own orientation specificity without the need for specific hardwired \nconnections or visually driven learning rules. \n\n2.1 Criticisms of the Bump model \n\nWe might posit that the brain uses a similar opportunistic model to derive and \norganize its feature maps \u2013 but the parallels between the primary visual cortex and \nthe Ernst et al. bump model are unconvincing. For instance, the units in their model \nrepresent the collective activity of a column, reducing the network dynamics to a \nfiring-rate approximation. But this simplification ignores the rich temporal \ndynamics of spiking networks, which are known to affect bump stability. More \nfundamentally, there is no role for functionally distinct neuron types. \nThe primary criticism of the Ernst et al.\u2019s bump model is that its input only consists \nof a luminance channel, and it is not obvious how to replace this channel with ON \nand OFF rectified channels to account for simple and complex cells. One possibility \nwould be to segregate ON-driven and OFF-driven cells (referred to as simple cells in \nthis paper) into two distinct recurrent networks. Because each network would have \nits own innate noise profile, bumps would form independently. Consequently, there \nis no guarantee that ON-driven maps would line up with OFF-driven maps, which \nwould result in conflicting orientation signals when these simple cells converge onto \nsign-independent (complex) cells. \n\n2.2 Simple Cells Solve a Complex Problem \n\nTo ensure that both ON-driven and OFF-driven simple cells have the same \norientation maps, both ON and OFF bumps must be computed in the same recurrent \nnetwork so that they are subjected to the same noise profile. We achieve this by \nbuilding our recurrent network out of cells that are sign-independent; that is both \nON and OFF channels drive the network. These cells exhibit complex cell-like \nbehavior (and are referred to as complex cells in this paper) because they are \nmodulated at double the spatial frequency of a sinusoidal grating input. The simple \ncells subsequently derive their responses from two separate signals: an orientation \nselective feedback signal from the complex cells indicating the presence of either an \nON or an OFF bump, and an ON\u2013OFF selection signal that chooses the appropriate \nresponse flavor. \nFigure 1 left illustrates the formation of bumps (highlighted cells) by a recurrent \nnetwork with a Mexican hat connectivity profile. Extending the Ernst et al. model, \nthese complex bumps seed simple bumps when driven by a grating. Simple bumps \nthat match the sign of the input survive, whereas out-of-phase bumps are \nextinguished (faded cells) by push-pull inhibition. \nFigure 1 right shows the local connections within a microcircuit. An EXC \n(excitatory) cell receives excitatory input from both ON and OFF channels, and \n\n\f \n\nprojects to other EXC (not shown) and INH (inhibitory) cells. The INH cell projects \nback in a reciprocal configuration to EXC cells. The divergence is indicated in left. \nON-driven and OFF-driven simple cells receive input in a push-pull configuration \n(i.e., ON cells are excited by ON inputs and inhibited by OFF inputs, and vise-versa), \nwhile additionally receiving input from the EXC\u2013INH recurrent network. In this \nmodel, we \ninhibitory \nconnections, despite the fact that geniculate input is strictly excitatory. This \nsimplification, while anatomically incorrect, yields a more efficient implementation \nthat is functionally equivalent. \n \n\nimplement our push-pull circuit using monosynaptic \n\nLuminance\n\nleft\n\nON & OFF Input\n\nON Input\n\nOFF Input\n\nright\n\nDivergence\n\ns\nl\nl\ne\nC\n \nx\ne\nl\np\nm\no\nC\n\ns\nl\nl\ne\nC\n \ne\nl\np\nm\nS\n\ni\n\nEXC\n\nINH\n\nON\n\nOFF\n\nEXC\n\nINH\n\nOFF\n\nSpace\n\n \nFigure 1: left, Complex and simple cell responses to a sinusoidal grating input. \nLuminance is transformed into ON (green) and OFF (red) pathways by retinal \nprocessing. Complex cells form a recurrent network through excitatory and \ninhibitory projections (yellow and blue lines, respectively), and clusters of activity \noccur at twice the spatial frequency of the grating. ON input activates ON-driven \nsimple cells (bright green) and suppresses OFF-driven simple cells (faded red), and \nvise-versa. right, The bump model\u2019s local microcircuit: circles represent neurons, \ncurved lines represent axon arbors that end in excitatory synapses (v shape) or \ninhibitory synapses (open circles). For simplicity, inhibitory interneurons were \nomitted in our push-pull circuit. \n \n\n2.3 Mathematical Description \n\n, \nThe neurons in our network follow the equation \n\u2022 is the temporal derivative of the membrane \nwhere C is membrane capacitance, V\nvoltage, \u03b4(\u00b7) is the Dirac delta function, which resets the membrane at the times tn \nwhen it crosses threshold, Isyn is synaptic current from the network, and Ileak is a \nconstant leak current. Neurons receive synaptic current of the form:\n\n\u2211= \u2212 \u2202 \u2212\n\nCV\n\nt\n(\n\nt\n\n)n\n\n\u2212\n\n+\n\n\u2212\n\nI\n\nKCa\n\nleak\n\nI\n\nsyn\n\nI\n\n \n\nn\n\n\u2022\n\nI\nI\n\nO\nN\nsyn\nO\nFF\nsyn\n\n=\n=\n\nw I\n+ O\nN\nw I\n+ O\n\nFF\n\nw I\n\u2212\n\u2212\nw I\n\u2212\n\u2212\n\nO\n\nFF\n\nO\nN\n\n+\n+\n\nw I\nEE E\nw I\nEE E\n\nXC\n\nXC\n\n\u2212\n\u2212\n\nw I\nEI\nw I\nEI\n\nI\nNH\n\nI\nNH\n\n,\n,\n\nO\nN\n\n+\n\nI\n\nO\n\nFF\n\n)\n\n+\n\nw I\nEE E\n\nXC\n\n\u2212\n\nw I\nEI\n\nI\nNH\n\n+\n\nI\n\nI\nI\n\nE\nXC\nsyn\nI\nNH\nsyn\n\n=\n=\n\n+\n\nw I\n(\nw I\nIE E\n\nXC\n\nback\n\n,\n\n \n\n\f \n\nwhere w+ is the excitatory synaptic strength for ON and OFF input synapses, w- is the \nstrength of the push-pull inhibition, wEE is the synaptic strength for EXC cell \nprojections to other EXC cells, wEI\n is the strength of INH cell projections to EXC \ncells, wIE is the strength of EXC cell projections to INH cells, Iback is a constant input \ncurrent, and I{ON,OFF,EXC,INH} account for all impinging synapses from each of the four \ncell types. These terms are calculated for cell i using an arbor function that consists \nof a spatial weighting J(r) and a post-synaptic current waveform \u03b1(t): \n, where k spans all cells of a given type and n indexes their spike \n\u2211\n,k n\n, with \u03c3 \ntimes. The spatial weighting function is described by \nas the space constant. The current waveform, which is non-zero for t>0, convolves \n, where \u03c4c is \na 1 t function with a decaying exponential: \nthe decay-rate, and \u03c4e is the time constant of the exponential. Finally, we model \nspike-rate adaptation with a calcium-dependent potassium-channel (KCa), which \nintegrates Ca triggered by spikes at times tn with a gain K and a time constant \u03c4k, as \ndescribed by KCa\n\nk\u03c4 . \n\n\u03c4 \u03b1\n0\n\nt\u03b1\n\u22c5\n\u2212\n\nt\n(\n\nk \u03c3\n)\n\n\u2212 \u2212\n\ni\n\n\u03b1\n\nt\n( )\n\nexp(\n\nexp(\n\nt\n\nexp(\n\nJ i\n(\n\nJ i\n(\n\n\u2212\n\nt\n\n\u03c4\ne\n\n)\n\n\u2212\n\nt\n\n\u2212\n\nk\n\n)\n\n\u2212\n\nk\n\n)\n\n=\n\nt\n\n(\n\nk\n\nn )\n\n\u2212\n\n1\n\n)\n\n\u2217\n\n+\n\nc\n\n=\n\nI\n\nK\n\n\u2211=\nn\n\nn\n\n)\n\n3 Silicon Implementation \n\nWe implemented our model in silicon using the TSMC (Taiwan Semiconductor \nManufacturing Company) 0.25\u00b5m 5-metal layer CMOS process. The final chip \nconsists of a 2-D core of 48x48 pixels, surrounded by asynchronous digital circuitry \nthat transmits and receives spikes in real-time. Neurons that reach threshold within \nthe array are encoded as address-events and sent off-chip, and concurrently, \nincoming address-events are sent to their appropriate synapse locations. This \ninterface is compatible with other spike-based chips that use address-events [5]. \nThe fabricated bump chip has close to 460,000 transistors packed in 10 mm2 of \nsilicon area for a total of 9,216 neurons. \n\n3.1 Circuit Design \n\nOur neural circuit was morphed into hardware using four building blocks. Figure 2 \nshows the transistor implementation for synapses, axonal arbors (diffuser), KCa \nanalogs, and neurons. The circuits are designed to operate in the subthreshold \nregion (except for the spiking mechanism of the neuron). Noise is not purposely \ndesigned into the circuits. Instead, random variations from the fabrication process \nintroduce significant deviations in I-V curves of theoretically identical MOS \ntransistors. \nThe function of the synapse circuit is to convert a brief voltage pulse (neuron spike) \ninto a postsynaptic current with biologically realistic temporal dynamics. Our \nsynapse achieves this by cascading a current-mirror integrator with a log-domain \nlow-pass filter. The current-mirror integrator has a current impulse response that \ndecays as 1 (with a decay rate set by the voltage \u03c4\nc and an amplitude set by A). \nThis time-extended current pulse is fed into a log-domain low-pass filter (equivalent \nto a current-domain RC circuit) that imposes a rise-time on the post-synaptic current \nset by \u03c4e. ON and OFF input synapses receive presynaptic spikes from the off-chip \nlink, whereas EXC and INH synapses receive presynaptic spikes from local on-chip \nneurons. \n \n\nt\n\n\f \n\nSynapse\n\nDiffuser\n\nJe\n\nA\n\nIr\n\nIg\n\nKCa Analog\n\nNeuron\n\nVmem\n\nVspk\n\nJc\n\nJk\n\nK\n\n \nFigure 2: Transistor implementations are shown for a synapse, diffuser, KCa analog, \nand neuron (simplified), with circuit insignias in the top-left of each box. The \ncircuits they interact with are indicated (e.g. the neuron receives synaptic current \nfrom the diffuser as well as adaptation current from the KCa analog; the neuron in \nturn drives the KCa analog). The far right shows layout for one pixel of the bump \nchip (vertical dimension is 83\u00b5m, horizontal is 30 \u00b5m). \n \nThe diffuser circuit models axonal arbors that project to a local region of space with \nan exponential weighting. Analogous to resistive divider networks, diffusers [6] \nefficiently distribute synaptic currents to multiple targets. We use four diffusers to \nimplement axonal projections for: the ON pathway, which excites ON and EXC cells \nand inhibits OFF cells; the OFF pathway, which excites OFF and EXC cells and \ninhibits ON cells; the EXC cells, which excite all cell types; and the INH cells, which \ninhibits EXC, ON, and OFF cells. Each diffuser node connects to its six neighbors \nthrough transistors that have a pseudo-conductance set by \u03c3r, and to its target site \nthrough a pseudo-conductance set by \u03c3g; the space-constant of the exponential \nsynaptic decay is set by \u03c3r and \u03c3g\u2019s relative levels. \nThe neuron circuit integrates diffuser currents on its membrane capacitance. \nDiffusers either directly inject current (excitatory), or siphon off current (inhibitory) \nthrough a current-mirror. Spikes are generated by an inverter with positive \nfeedback (modified from [7]), and the membrane is subsequently reset by the spike \nsignal. We model a calcium concentration in the cell with a KCa analog. K \ncontrols the amount of calcium that enters the cell per spike; the concentration \ndecays exponentially with a time constant set by \u03c4k. Elevated charge levels activate \na KCa-like current that throttles the spike-rate of the neuron. \n\n3.2 Experimental Setup \n\nOur setup uses either a silicon retina [8] or a National Instruments DIO (digital \ninput\u2013output) card as input to the bump chip. This allows us to test our V1 model \nwith \nthe experimental paradigm of \n\nreal-time visual stimuli, similar \n\nto \n\n\f \n\nelectrophysiologists. More specifically, the setup uses an address-event link [5] to \nestablish virtual point-to-point connectivity between ON or OFF ganglion cells from \nthe retina chip (or DIO card) with ON or OFF synapses on the bump chip. Both the \ninput activity and the output activity of the bump chip is displayed in real-time \nusing receiver chips, which integrate incoming spikes and displays their rates as \npixel intensities on a monitor. A logic analyzer is used to capture spike output from \nthe bump chip so it can be further analyzed. \nWe investigated responses of the bump chip to gratings moving in sixteen different \ndirections, both qualitatively and quantitatively. For the qualitative aspect, we \ncreated a PO map by taking each cell\u2019s average activity for each stimulus direction \nand computing the vector sum. To obtain a quantitative measure, we looked at the \nnormalized vector magnitude (NVM), which reveals the sharpness of a cell\u2019s tuning. \nThe NVM is calculated by dividing the vector sum by the magnitude sum for each \ncell. The NVM is 0 if a cell responds equally to all orientations, and 1 if a cell\u2019s \norientation selectivity is perfect such that it only responds at a single orientation. \n\n4 Results \n\nWe presented sixteen moving gratings to the network, with directions ranging from \n0 to 360 degrees. The spatial frequency of the grating is tuned to match the size of \nthe average bump, and the temporal frequency is 1 Hz. Figure 3a shows a resulting \nPO map for directions from 180 to 360 degrees, looking at the inhibitory cell \npopulation (the data looks similar for other cell types). Black contours represent \nstable bump regions, or equivalently, the regions that exceed a prescribed threshold \n(90 spikes) for all directions. The PO map from the bump chip reveals structure that \nresembles data from real cortex. Nearby cells tend to prefer similar orientations \nexcept at fractures. There are even regions that are similar to pinwheels (delimited \nby a white rectangle). \nA PO is a useful tool to describe a network\u2019s selectivity, but it only paints part of \nthe picture. So we have additionally computed a NVM map and a NVM histogram, \nshown in Figure 3b and 3c respectively. The NVM map shows that cells with sharp \nselectivity tend to cluster, particularly around the edge of the bumps. The histogram \nalso reveals that the distribution of cell selectivity across the network varies \nconsiderably, skewed towards broadly tuned cells. \nWe also looked at spike rasters from different cell-types to gain insight into their \nphase relationship with the stimulus. In particular, we present recordings for the \nsite indicated by the arrow (see Figure 3a) for gratings moving in eight directions \nranging from 0 to 360 degrees in 45-degree increments (this location was chosen \nbecause it is in the vicinity of a pinwheel, is reasonably selective, and shows \nconsiderable modulation in its firing rate). Figure 4 shows the luminance of the \nstimulus (bottom sinusoids), ON- (cyan) and OFF-input (magenta) spike trains, and \nthe resulting spike trains from EXC (yellow), INH (blue), ON- (green), and OFF-\ndriven (red) cell types for each of the eight directions. The center polar plot \nsummarizes the orientation selectivity for each cell-type by showing the normalized \nnumber of spikes for each stimulus. Data is shown for one period. \nEven though all cells-types are selective for the same orientation (regardless of \ngrating direction), complex cell responses tend to be phase-insensitive while the \nsimple cell responses are modulated at the fundamental frequency. It is worth \nnoting that the simple cells have sharper orientation selectivity compared to the \ncomplex cells. This trend is characteristic of our data. \n \n\n\f \n\n0.1\n\n0.2\n\n0.3\n\n0.4\n\n0.5\n\n0.6\n\n0.7\n\n0.8\n\n0.9\n\n1 \n\n300\n\n250\n\n200\n\n150\n\n100\n\n50\n\n20 \n\n40 \n\n60 \n\n80 \n\n100\n\n120\n\n140\n\n160\n\n180\n\n0\n\n0\n\n0.1\n\n0.2\n\n0.3\n\n0.4\n\n0.5\n\n0.6\n\n0.7\n\n0.8\n\n0.9\n\n1\n\n \n\nFigure 3: (a) PO map for the inhibitory cell population stimulated with eight \ndifferent directions from 180 to 360 degrees (black represents no activity, contours \ndelineate regions that exceed 90 spikes for all stimuli). Normalized vector \nmagnitude (NVM) data is presented as (b) a map and (c) a histogram. \n\n \nFigure 4: Spike rasters and polar plot for 8 directions ranging from 0 to 360 degrees. \nEach set of spike rasters represent from bottom to top, ON- (cyan) and OFF-input \n(magenta), INH (yellow), EXC (blue), and ON- (green) and OFF-driven (red). The \nstimulus period is 1 sec. \n\n\f \n\n5 Discussion \n\nWe have implemented a large-scale network of spiking neurons in a silicon chip that \nis based on layer 4 of the visual cortex. The initial testing of the network reveals a \nPO map, inherited from innate chip heterogeneities, resembling cortical maps. Our \nmicrocircuit proposes a novel function for complex-like cells; that is they create a \nsign-independent orientation selective signal, which through a push-pull circuit \ncreates sharply tuned simple cells with the same orientation preference. \nRecently, Ringach et al. surveyed orientation selectivity in the macaque [9]. They \nobserved that, in a population of V1 neurons (N=308) the distribution of orientation \nselectivity is quite broad, having a median NVM of 0.39. We have measured \nmedian NVM\u2019s ranging from 0.25 to 0.32. Additionally, Ringach et al. found a \nnegative correlation between spontaneous firing rate and NVM. This is consistent \nwith our model because cells closer to the center of the bump have higher firing \nrates and broader tuning. \nWhile the results from the bump chip are promising, our maps are less consistent \nand noisier than the maps Ernst et al. have reported. We believe this is because our \nnetwork is tuned to operate in a fluid state where bumps come on, travel a short \ndistance and disappear (motivated by cortical imaging studies). But excessive \nfluidity can cause non-dominant bumps to briefly appear and adversely shift the PO \nmaps. We are currently investigating the role of lateral connections between bumps \nas a means to suppress these spontaneous shifts. \nThe neural mechanisms that underlie the orientation selectivity of V1 neurons are \nstill highly debated. This may be because neuron responses are not only shaped by \nfeedforward inputs, but are also influenced at the network level. If modeling is \ngoing to be a useful guide for electrophysiologists, we must model at the network \nlevel while retaining cell level detail. Our results demonstrate that a spike-based \nneuromorphic system is well suited to model layer 4 of the visual cortex. The same \napproach may be used to build large-scale models of other cortical regions. \n\nReferences \n \n1. Hubel, D. and T. Wiesel, Receptive firelds, binocular interaction and functional \n\narchitecture in the cat's visual cortex. J. Physiol, 1962. 160: p. 106-154. \n\n2. Blasdel, G.G., Orientation selectivity, preference, and continuity in monkey striate cortex. \n\nJ Neurosci, 1992. 12(8): p. 3139-61. \n\n3. Crair, M.C., D.C. Gillespie, and M.P. Stryker, The role of visual experience in the \ndevelopment of columns in cat visual cortex. Science, 1998. 279(5350): p. 566-70. \n\n4. Ernst, U.A., et al., Intracortical origin of visual maps. Nat Neurosci, 2001. 4(4): p. 431-6. \n5. Boahen, K., Point-to-Point Connectivity. IEEE Transactions on Circuits & Systems II, \n\n2000. vol 47 no 5: p. 416-434. \n\nNIPS91. 1992: IEEE. \n\n6. Boahen, K. and Andreou. A contrast sensitive silicon retina with reciprocal synapses. in \n\n7. Culurciello, E., R. Etienne-Cummings, and K. Boahen, A Biomorphic Digital Image \n\nSensor. IEEE Journal of Solid State Circuits, 2003. vol 38 no 2: p. 281-294. \n\n8. Zaghloul, K., A silicon implementation of a novel model for retinal processing, in \n\nNeuroscience. 2002, UPENN: Philadelphia. \n\n9. Ringach, D.L., R.M. Shapley, and M.J. Hawken, Orientation selectivity in macaque V1: \n\ndiversity and laminar dependence. J Neurosci, 2002. 22(13): p. 5639-51. \n\n \n\n\f", "award": [], "sourceid": 2472, "authors": [{"given_name": "Paul", "family_name": "Merolla", "institution": null}, {"given_name": "Kwabena", "family_name": "Boahen", "institution": null}]}