{"title": "Recurrent networks of coupled Winner-Take-All oscillators for solving constraint satisfaction problems", "book": "Advances in Neural Information Processing Systems", "page_first": 719, "page_last": 727, "abstract": "We present a recurrent neuronal network, modeled as a continuous-time dynamical system, that can solve constraint satisfaction problems. Discrete variables are represented by coupled Winner-Take-All (WTA) networks, and their values are encoded in localized patterns of oscillations that are learned by the recurrent weights in these networks. Constraints over the variables are encoded in the network connectivity. Although there are no sources of noise, the network can escape from local optima in its search for solutions that satisfy all constraints by modifying the effective network connectivity through oscillations. If there is no solution that satisfies all constraints, the network state changes in a pseudo-random manner and its trajectory approximates a sampling procedure that selects a variable assignment with a probability that increases with the fraction of constraints satisfied by this assignment. External evidence, or input to the network, can force variables to specific values. When new inputs are applied, the network re-evaluates the entire set of variables in its search for the states that satisfy the maximum number of constraints, while being consistent with the external input. Our results demonstrate that the proposed network architecture can perform a deterministic search for the optimal solution to problems with non-convex cost functions. The network is inspired by canonical microcircuit models of the cortex and suggests possible dynamical mechanisms to solve constraint satisfaction problems that can be present in biological networks, or implemented in neuromorphic electronic circuits.", "full_text": "Recurrent networks of coupled Winner-Take-All\n\noscillators for solving constraint satisfaction problems\n\nHesham Mostafa, Lorenz K. M\u00a8uller, and Giacomo Indiveri\n\nInstitute for Neuroinformatics\n\nUniversity of Zurich and ETH Zurich\n\n{hesham,lorenz,giacomo}@ini.uzh.ch\n\nAbstract\n\nWe present a recurrent neuronal network, modeled as a continuous-time dynami-\ncal system, that can solve constraint satisfaction problems. Discrete variables are\nrepresented by coupled Winner-Take-All (WTA) networks, and their values are en-\ncoded in localized patterns of oscillations that are learned by the recurrent weights\nin these networks. Constraints over the variables are encoded in the network con-\nnectivity. Although there are no sources of noise, the network can escape from\nlocal optima in its search for solutions that satisfy all constraints by modifying\nthe effective network connectivity through oscillations. If there is no solution that\nsatis\ufb01es all constraints, the network state changes in a seemingly random manner\nand its trajectory approximates a sampling procedure that selects a variable assign-\nment with a probability that increases with the fraction of constraints satis\ufb01ed by\nthis assignment. External evidence, or input to the network, can force variables to\nspeci\ufb01c values. When new inputs are applied, the network re-evaluates the entire\nset of variables in its search for states that satisfy the maximum number of con-\nstraints, while being consistent with the external input. Our results demonstrate\nthat the proposed network architecture can perform a deterministic search for the\noptimal solution to problems with non-convex cost functions. The network is\ninspired by canonical microcircuit models of the cortex and suggests possible dy-\nnamical mechanisms to solve constraint satisfaction problems that can be present\nin biological networks, or implemented in neuromorphic electronic circuits.\n\n1\n\nIntroduction\n\nThe brain is able to integrate noisy and partial information from both sensory inputs and internal\nstates to construct a consistent interpretation of the actual state of the environment. Consistency\namong different interpretations is likely to be inferred according to an internal model constructed\nfrom prior experience [1]. If we assume that a consistent interpretation is speci\ufb01ed by a proper con-\n\ufb01guration of discrete variables, then it is possible to build an internal model by providing a set of\nconstraints on the con\ufb01gurations that these variables are allowed to take. Searching for consistent\ninterpretations under this internal model is equivalent to solving a max-constraint satisfaction prob-\nlem (max-CSP). In this paper, we propose a recurrent neural network architecture with cortically\ninspired connectivity that can represent such an internal model, and we show that the network dy-\nnamics solve max-CSPs by searching for the optimal variable assignment that satis\ufb01es the maximum\nnumber of constraints, while being consistent with external evidence.\nAlthough there are many ef\ufb01cient algorithmic approaches to solving max-CSPs, it is still not clear\nhow these algorithms can be implemented as biologically realistic dynamical systems. In particular,\na challenging problem in systems whose dynamics embody a search for the optimal solution of a\nmax-CSP is escaping from local optima. One possible approach is to formulate a stochastic neural\nnetwork that samples from a probability distribution in which the correct solutions have higher\n\n1\n\n\fprobability [2]. However, the stochastic network will continuously explore the solution space and\nwill not stabilize at fully consistent solutions. Another possible solution is to use simulated annealing\ntechniques [3]. Simulated annealing techniques, however, cannot be easily mapped to plausible\nbiological neural circuits due to the cooling schedule used to control the exploratory aspect of the\nsearch process. An alternative deterministic dynamical systems approach for solving combinatorial\noptimization problems is to formulate a quadratic cost function for the problem and construct a\nHop\ufb01eld network whose Lyapunov function is this cost function [4]. Considerable parameter tuning\nis needed to get such networks to converge to good solutions and to avoid local optima [5]. The\naddition of noise [6] or the inclusion of an initial chaotic exploratory phase [7] in Hop\ufb01eld networks\npartially mitigate the problem of getting stuck in local optima.\nThe recurrent neural network we propose does not need a noise source to carry out the search pro-\ncess. Its deterministic dynamics directly realize a form of \u201cusable computation\u201d [8] that is suitable\nfor solving max-CSPs. The form of computation implemented is distributed and \u201cexecutive-free\u201d [9]\nin the sense that there is no central controller managing the dynamics or the \ufb02ow of information. The\nnetwork is cortically inspired as it is composed of coupled Winner-Take-All (WTA) circuits. The\nWTA circuit is a possible cortical circuit motif [10] as its dynamics can explain the ampli\ufb01cation\nof genico-cortical inputs that was observed in intracellular recordings in cat visual cortex [11]. In\naddition to elucidating possible computational mechanisms in the brain, implementing \u201cusable com-\nputation\u201d with the dynamics of a neural network holds a number of advantages over conventional\ndigital computation, including massive parallelism and fault tolerance.\nIn particular, by follow-\ning such dynamical systems approach, we can exploit the rich behavior of physical devices such\nas transistors to directly emulate these dynamics, and obtain more dense and power ef\ufb01cient com-\nputation [12]. For example, the network proposed could be implemented using low-power analog\ncurrent-mode WTA circuits [13], or by appropriately coupling silicon neurons in neuromorphic Very\nLarge Scale Integration (VLSI) chips [14].\nIn the next section we describe the architecture of the proposed network and the models that we use\nfor the network elements. Section 3 contains simulation results showing how the proposed network\narchitecture solves a number of max-CSPs with binary variables. We discuss the network dynamics\nin Section 4 and present our conclusions in Section 5.\n\n2 Network Architecture\n\nThe basic building block of the proposed network is the WTA circuit in which multiple excitatory\npopulations are competing through a common inhibitory population as shown in Fig. 1a. When the\nexcitatory populations of the WTA network receive inputs of different amplitudes, their activity will\nincrease and be ampli\ufb01ed due to the recurrent excitatory connections. This will in turn activate the\ninhibitory population which will suppress activity in the excitatory populations until an equilibrium\nis reached. Typically, the excitatory population that receives the strongest external input is the only\none that remains active (the network has selected a winner). By properly tuning the connection\nstrengths, it is possible to con\ufb01gure the network so that it settles into a stable state of activity (or an\nattractor) that persists after input removal [15].\n\n2.1 Neuronal and Synaptic Dynamics\n\nThe network that we propose is a population-level, rate-based network. Each population is modeled\nas a linear threshold unit (LTU) which has the following dynamics:\n\n(cid:88)\n\n\u03c4i \u02d9xi(t) + xi(t) = max(0,\n\nwji(t)xj(t) \u2212 Ti)\n\n(1)\n\nj\n\nwhere xi(t) is the average \ufb01ring rate in population i, wji(t) is the connection weight from population\nj to population i, and \u03c4i and Ti are the time constant and the threshold of population i respectively.\nThe steady state population activity in eq. 1 is a good approximation of the steady state average\n\ufb01ring rate in a population of integrate and \ufb01re neurons receiving noisy, uncorrelated inputs [16].\nFor a step increase in mean input, the actual average \ufb01ring rate in a population settles into a steady\nstate after a number of transient modes have died out [17] but in eq. 1, we assume the \ufb01ring rate\napproaches steady state only through \ufb01rst order dynamics.\n\n2\n\n\f(a)\n\n(b)\n\n(c)\n\n(d)\n\nFigure 1: (a) A single WTA network. (b) Three coupled WTA circuits form the network representa-\ntion of a single binary variable. Circles labeled A,B,C, and D are excitatory populations. Red circles\non the right are inhibitory populations. (c) Simulation results of the network in (b) showing activity\nin the four excitatory populations. Shaded rectangles indicate the time intervals in which the state of\nthe oscillator can be changed by external input. (d) Switching the state of the oscillator. The bottom\nplot shows the activity of the A and B populations. External input is applied to the A population in\nthe time intervals denoted by the shaded rectangles. While the \ufb01rst input has no effect, the second\ninput is applied at the right time and triggers a change in the variable/oscillator state. The top plot\nshows time evolution of the weights WA and WB.\n\nThe plastic connections in the proposed network obey a learning rule analogous to the Bienenstock-\nCooper-Munro (BCM) rule [18]:\n\n(cid:18) (w(t) \u2212 wmin)[vth \u2212 v(t)]\u2212\n\n(cid:19)\n\n(wmax \u2212 w(t))[v(t) \u2212 vth]+\n\n+\n\n\u03c4pot\n\n\u03c4dep\n\n\u02d9w(t) = Ku(t)\n\n(2)\nwhere [x]+ = max(0, x), and [x]\u2212 = min(0, x). w(t) is the connection weight, and u(t) and v(t)\nare the activities of the source and target populations respectively. The parameters wmin and wmax\nare soft bounds on the weight, \u03c4dep and \u03c4pot are the depression and potentiation time constants\nrespectively, vth is a threshold on the activity of the target population that delimits the transition\nbetween potentiation and depression, and K is a term that controls the overall speed of learning\nor the plasticity rate. The learning rule captures the dependence of potentiation and depression\ninduction on the postsynaptic \ufb01ring rate [19].\n\n2.2 Variable Representation\n\nPoint attractor states in WTA networks like the one shown in Fig. 1a are computationally useful\nas they enable the network to disambiguate the inputs to the excitatory populations by making a\ncategorical choice based on the relative strengths of these inputs. Point attractor dominated dynamics\npromote noise robustness at the expense of reduced input sensitivity: external input has to be large\nto move the network state out of the basin of attraction of one point attractor, and into the basin of\nattraction of another.\nIn this work, instead of using distinct point attractors to represent different variable values, we\nuse limit cycle attractors. To obtain limit cycle attractors, we asymmetrically couple a number of\nWTA circuits to form a loop as shown in Fig. 1b. This has the effect of destroying the \ufb01xed point\n\n3\n\ninputstimulusnetworkactivityduringstimuluspresentationnetworkactivityafterstimulusremovalexcitatory populationinhibitory populationABCDWAWB\ufb01xed connectionplastic connectionAraBCDRate(Hz)Rate(Hz)Rate(Hz)2004060200406080200406080023140231402314Time(s)02040600.040.050.0601234560123456Rate(Hz)Time(s)WeightABWAWB\fattractors in each WTA stage. As a consequence, persistent activity can no longer appear in a single\nWTA stage if there is no input. If we apply a short input pulse to the bottom WTA stage of Fig. 1b,\nwe start oscillatory activity and we observe the following sequence of events: (1) the activity in\nthe bottom WTA stage ramps up due to recurrent excitation, and when it is high enough it begins\nactivating the middle WTA stage; (2) activity in the middle WTA stage ramps up and as activity in\nthe inhibitory population of this stage rises, it shuts down the bottom stage activity; activity in the\nmiddle WTA stage keeps on increasing until it activates the top stage; (3) activity in the top WTA\nstage increases, shuts down the middle stage, and provides input back into the bottom stage via the\nplastic connections. As a consequence, a bump of activity continuously jumps from one WTA stage\nto the next. Since the stages are connected in a loop, the network will exhibit oscillatory activity.\nThere are two stable limit cycles that the network trajectory can follow. The limit cycle chosen by the\nnetwork depends on the outcome of the winner selection process in the bottom WTA stage. The limit\ncycles are stable as the weak coupling between the stages leaves the signal restoration properties of\nthe destroyed attractors intact allowing activity in each WTA stage to be restored to a point close\nto that of the destroyed attractor. The winner selection process takes place at the beginning of each\noscillation period in the bottom WTA stage. In the absence of external input, the dynamics of the\nwinner selection process in the bottom stage will favor the population that receives the stronger\nprojection weight from D. These projection weights obey the plasticity rule given by eq. 2.\nThe oscillatory network in Fig. 1b can represent one binary variable whose value is encoded in the\nidentity of the winning population in the bottom WTA stage, which determines the limit cycle the\nnetwork follows. The identity of the winning population is a re\ufb02ection of the relative strengths of WA\nand WB. More than two values can be encoded by increasing the number of excitatory populations\nin the bottom WTA stage. Fig. 1c shows the simulation results of the network in Fig. 1b when the\nweight WB is larger than WA. This is expressed by a limit cycle in which populations B,C, and D\nare periodically activated.\nDuring the winner selection process in the bottom WTA stage, the WTA circuit is very sensitive to\nexternal input, which can bias the competition towards a particular limit cycle. Once the winner\nselection process is complete, i.e, activity in the winning population has ramped up to a high level,\nthe WTA circuit is relatively insensitive to external input. This is illustrated in Fig. 1d, where input\nis applied in two different intervals. The \ufb01rst external input to population A arrives after the winner,\nB, has already been selected so it is ineffective. A second external input having the same strength\nand duration as the \ufb01rst input arrives during the winner selection phase and biases the competition\ntowards A. As soon as A wins, the plasticity rule in eq. 2 causes WA to potentiate and WB to depress\nso that activity in the network continues to follow the new limit cycle even after the input is removed.\n\n2.3 Constraint Representation\n\nEach variable, as represented by the network in Fig. 1b, is a multi-stable oscillator. Pair-wise con-\nstraints can be implemented by coupling the excitatory populations of the bottom WTA stages of\ntwo variables. Fig 2a shows the implementation of a constraint that requires two variables to be\nunequal, i.e., one variable should oscillate in the cycle involving the A population, and the other\nin the cycle involving the B population. Variable X1 will maximally affect X2 when the activity\npeak in the bottom WTA stage of X1 coincides with the winner selection interval of X2 and vice\nversa. The coupling of the middle and top WTA stages of the two variables in Fig. 2a is not related\nto the constraint, but it is there to prevent coupled variables in large networks from phase locking.\nWe explain why this is important in the next section. We de\ufb01ne the zero phase point of a variable as\nthe point at which activity in the winning excitatory population in the bottom WTA stage reaches a\npeak and we assume the phase changes linearly during an oscillation period (from one peak to the\nnext). The phase difference between two coupled variables determines the direction and strength of\nmutual in\ufb02uence. This can be seen in Fig. 2b. Initially the constraint is violated as both variables\nare oscillating in the A cycle. X1 gradually begins to lead X2 until at a particular phase difference,\ninput from X1 is able to bias the competition in X2 so that the B population in X2 wins even though\nthe A population is receiving a stronger projection from the D population in X2.\nA constraint involving more than two variables can be implemented by introducing an intermediate\nvariable which will in general have a higher cardinality than the variables in the constraint (the\ncardinality of a variable is re\ufb02ected in the number of excitatory populations in the bottom WTA\nstage; the middle and top WTA stages have the same structure irrespective of cardinality). An\nexample is shown in Fig. 2c where three binary variables are related by an XOR relation and the\n\n4\n\n\f(a)\n\n(b)\n\n(c)\n\nFigure 2: (a) Coupling X1 and X2 to implement the constraint X1 (cid:54)= X2. (b) Activity in the\nA and B populations of X1 and X2 that are coupled as shown in (a).\n(c) Constraint involving\nthree variables: X1 XOR X2 = X3. Only the bottom WTA stages of the four variables and the\ninter-variable connections coupling the bottom WTA stages are shown.\n\nintermediate variable has four possible states. The tertiary XOR constraint has been effectively\nbroken down into three pair-wise constraints. The only states, or oscillatory modes, of X1, X2, and\nX3 that are stable under arbitrary phase relations with the intermediate variable are the states which\nsatisfy the constraint X1 XOR X2 = X3.\n\n3 Solving max-CSPs\n\nFrom simulations, we observe that the phase differences between the variables/oscillators are highly\nirregular in large networks comprised of many variables and constraints. These irregular phase re-\nlations enable the network to search for the optimal solution of a max-CSP. The weight attached\nto a constraint is an analogue quantity that is a function of the phase differences between the vari-\nables in the constraint. The phase differences also determine which of the variables in a violated\nconstraint changes in order to satisfy the constraint (see Fig. 2b). The irregular phase relations result\nin a continuous perturbation of the strengths of the different constraints by modulating the effec-\ntive network connectivity embodying these constraints. This is what allows the network to escape\nfrom the local optima of the underlying max-CSP. At a local optimum, the ongoing perturbation of\nconstraint strengths will eventually lead to a con\ufb01guration that de-emphasizes the currently satis\ufb01ed\nconstraints and emphasizes the unsatis\ufb01ed constraints. The transiently dominant unsatis\ufb01ed con-\nstraints will reassign the values of the variables in their domain and pull the network out of the local\noptimum. The network thus searches for optimal solutions by effectively perturbing the underlying\nmax-CSP. Under this search scheme, states that satisfy all constraints are dynamically stable since\nany perturbation of the strengths of the constraints de\ufb01ning the max-CSP will result in a constraints\ncon\ufb01guration that reinforces the current fully consistent state of the network.\nIn principle, if some variables/oscillators phase-lock, then the weights of the constraint(s) among\nthese variables will not change anymore, which will impact the ability of the network to \ufb01nd good\nsolutions. In practice, however, we see that this happens only in very small networks, and not in\nlarge ones, such as the networks described in the following sections.\n\n3.1 Network Behavior in the Presence of a Fully Consistent Variable Assignment\n\nWe simulated a recurrent neuronal network that represents a CSP that has ten binary variables and\nnine tertiary constraints (see Fig. 3a). Each variable is represented by the network in Fig. 1b. Each\ntertiary constraint is implemented by introducing an intermediate variable and using a coupling\nscheme similar to the one in Fig. 2c. We constructed the problem so that only two variable as-\nsignments are fully consistent. The problem is thus at the boundary between over-constrained and\nunder-constrained problems which makes it dif\ufb01cult for a search algorithm to \ufb01nd the optimum [20].\nWe ran 1000 trials starting from random values for the synaptic weights within each variable (each\nvariable effectively starts with a random value). The network always converges to one of the optimal\nvariable assignments. Fig. 3b shows a histogram of the number of oscillation cycles needed to\nconverge to an optimal solution in the 1000 trials. The number of cycles is averaged over the ten\nvariables as the number of cycles needed to converge to an optimal solution is not the same for\n\n5\n\nABCDABCDX1X2Rate(Hz)Rate(Hz)Time(s)X1:AX1:BX2:AX2:B0204060020406001234560123456\f(a)\n\n(b)\n\n(c)\n\n(d)\n\nFigure 3: Solving a CSP with ten binary variables and nine tertiary constraints. (a) CSP de\ufb01nition.\n(b) Histogram of the number of cycles needed for convergence, averaged over all ten variables,\nin 1000 trials. (c) Evolution of network state in a sample trial. The top plot shows the number\nof constraints violated by the variable assignment decoded from the network state. The bottom\nplot shows the Hamming distance between the decoded variable assignment to each of the two fully\nconsistent solutions. (d) One variable is externally forced to take a value that is incompatible with the\ncurrent fully consistent variable assignment. The search resumes to \ufb01nd a fully consistent variable\nassignment that is compatible with the external input.\n\nall variables. Although the sub-networks representing the variables are identical, each oscillates\nat a different instantaneous frequency due to the non-uniform coupling and switching dynamics.\nFig. 3c shows how the network state evolves in a sample trial. Due to the continuous perturbation\nof the weights caused by the irregular phase relations between the variables/oscillators, the network\nsometimes takes steps that lead to the violation of more constraints. This prevents the network from\ngetting stuck in local optima.\nWe model the arrival of external evidence by activating an additional variable/oscillator that has only\none state, or limit cycle, and which is coupled to one of the original problem variables. External\nevidence in this case is sparse since it only affects one problem variable. External evidence also\ndoes not completely \ufb01x the value of that one problem variable, but rather, the single state \u201cevidence\nvariable\u201d affects the problem variable only at particular phase differences between the two. Fig. 3d\nshows that the network is able to take the external evidence into account by searching for, and \ufb01nally\nsettling into, the only remaining fully consistent state that accommodates the external evidence.\n\n3.2 Network Behavior in the Absence of Fully Consistent Variable Assignments\n\nAs shown in the previous section, if a fully consistent solution exists, the network state will end up\nin that solution and stay there. If no such solution exists, the network will never settle into one vari-\nable assignment, but will keep exploring possible assignments and will spend more time in solutions\nthat satisfy more constraints. This behavior can be interpreted as a sampling process where each\noscillation cycle lets one variable re-sample its current state; at any point in time, the network state\nrepresents a sample from a probability distribution de\ufb01ned over the space of all possible solutions\n\n6\n\nX1 AND X2 = X5X3 AND X4 = X6X5 XOR X6 = X7X5 NAND X7 = X8X6 NOR X8 = X9X4 AND X9 = X10X8 XOR X10 = X1X8 XOR X9 = X2X2 AND X8 = X30500100015002000250030003500400045005000050100150AveragenumberofcyclestoconvergenceCountMean=646Median=436 Average number of cyclesHammingDistancetoconsistentstate1HammingDistancetoconsistentstate2NumberofviolatedconstraintsAverage number of cyclesNumberofviolatedconstraintsonevariableexternally forced to be incompatibleconsistent state 1with consistent state 2 but compatible withHammingDistancetoconsistentstate1HammingDistancetoconsistentstate2\f(a)\n\n(b)\n\nFigure 4: Ising model type problems. Each square indicates a binary variable like in Fig. 1b; solid\nblack lines denote a constraint requiring two variables to be equal, dashed red lines a constraint that\nrequires two variables to be unequal. In both problems, all states violate at least one constraint.\n\n(a)\n\n(b)\n\nFigure 5: Behavior of two networks representing the CSPs in Fig. 4. Red squares are data points (the\ntime the network spent in one particular state), a blue star is the average time spent in states of equal\nenergy and the green line is an exponential \ufb01t to the blue stars. (a) Note that at energies 1 and 2 there\nare two complementary states each that are visited almost equally often. (b) Not all assignments of\nenergy 2 are equally probable in this case (not a \ufb01nite samples artifact, but systematic) as can be seen\nin the bimodal distribution there. This is caused by variables that are part of only one constraint.\n\nto the max-CSP, where more consistent solutions have higher probability. The oscillatory dynamics\nthus give rise to a decentralized, deterministic, and time-continuous sampling process. This sam-\npling analogy is only valid when there are no fully consistent solutions. To illustrate this behavior,\nwe consider two max-CSPs having an Ising-model like structure as shown in Figs. 4a, 4b. We\ndescribe the behavior of two networks that represent the max-CSPs embodied by these two graphs.\nLet E(s) be a function that maps a network state s to the number of constraints it violates; this\nis analogous to an energy function and we will refer to E(s) as the energy of state s. For the\nproblem in Fig. 4a, we observe that the average time the network spends in states with energy E\nis t(E) = c1 exp(\u2212c2E) as can be seen in Fig. 5a. The network spends almost equal times in\ncomplementary states that have low energy. Complementary states are maximally different but the\nnetwork is able to traverse the space of intervening states, which can have higher energy, in order to\nvisit the complementary states almost equally often.\nWe expect the network to spend less time in less consistent states; the higher the number of vio-\nlated constraints, the more rapidly the variable values change because there are more possible phase\nrelations that can emphasize a violated constraint. However, we do not have an analytical explana-\ntion for the good exponential \ufb01t to the energy-time spent relation. We expect a worse \ufb01t for high\nenergies. For example, the network can never go into states where all constraints are violated even\nthough they have \ufb01nite energies.\nFor the problem in Fig. 4b, not all states of equally low energy are equally likely as can be seen in\nFig. 5b. For example, the states of energy 2, where C and D (or K and L) are unequal, are less likely\nthan other assignments of the same energy. This is not surprising. When C is in some state, D has no\nreason to be in a different state (no other variables try to force it to be different from C) apart from\nthe memory in its plastic weights. We expect that this effect becomes small for suf\ufb01ciently densely\nconnected constraint graphs. The exponential \ufb01t to the averages is still very good in Fig. 5b.\n\n7\n\nFGHIKEDCBADKLFEIABCHG0.010.11101001000100001000001e+061234567TimeEnergy (#violated constraints)Problem in Fig.4aDataExponential Fit to AveragesAverage of Equal Energy0.00010.0010.010.11101001000100001000001e+0612345678TimeEnergy (#violated constraints)DataExponential Fit to AveragesAverage of Equal EnergyProblem in Fig.4b\f4 Discussion\n\nOscillations are ubiquitous in cortex. Local \ufb01eld potential measurements as well as intracellu-\nlar recordings point to a plethora of oscillatory dynamics operating in many distinct frequency\nbands [21]. One possible functional role for oscillatory activity is that it rhythmically modulates\nthe sensitivity of neuronal circuits to external in\ufb02uence [22, 23]. Attending to a periodic stimulus\nhas been shown to result in the entrainment of delta-band oscillations (2-4 Hz) so that intervals of\nhigh excitability coincide with relevant events in the stimulus [24]. We have used the idea of os-\ncillatory modulation of sensitivity to construct multi-stable neural oscillators whose state, or limit\ncycle, can be changed by external inputs only in narrow, periodically recurring temporal windows.\nSelection between multiple limit cycles is done through competitive dynamics which are thought to\nunderlie many cognitive processes such as decision making in prefrontal cortex [25].\nExternal input to the network can be interpreted as an additional constraint that immediately affects\nthe search for maximally consistent states. Continuous reformulation of the problem, by adding new\nconstraints, is problematic for any approach that works by having an initial exploratory phase that\nslowly morphs into a greedy search for optimal solutions, as the exploratory phase has to be restarted\nafter a change in the problem. For a biological system that has to deal with a continuously changing\nset of constraints, the search algorithm should not exhibit an exploratory/greedy behavior dichotomy.\nThe search procedure used in the proposed networks does not exhibit this dichotomy. The search is\ndriven solely by the violated constraints. This can be seen in the sampling-like behavior in Fig. 5\nwhere the network spends less time in a state that violates more constraints.\nThe size of the proposed network grows linearly with the number of variables in the problem. CSPs\nare in general NP-complete, hence convergence time of networks embodying CSPs will grow expo-\nnentially (in the worst case) with the size of the problem. We observed that in addition to problem\nsize, time to convergence/solution depends heavily on the density of solutions in the search space.\nWe used the network to solve a graph coloring problem with 17 nodes and 4 colors (each oscilla-\ntor/variable representing a node had 4 possible stable limit cycles). The problem was chosen so that\nthere is an abundance of solutions. This led to a faster convergence to an optimal solution compared\nto the problem in Fig. 3a even though the graph coloring problem had a much larger search space.\n\n5 Conclusions and Future Work\n\nBy combining two basic dynamical mechanisms observed in many brain areas, oscillation and com-\npetition, we constructed a recurrent neuronal network that can solve constraint satisfaction problems.\nThe proposed network deterministically searches for optimal solutions by modulating the effective\nnetwork connectivity through oscillations. This, in turn, perturbs the effective weights of the con-\nstraints. The network can take into account partial external evidence that constrains the values of\nsome variables and extrapolate from this partial evidence to reach states that are maximally consis-\ntent with the external evidence and the internal constraints. For sample problems, we have shown\nempirically that the network searches for, and settles into, a state that satis\ufb01es all constraints if there\nis one, otherwise it explores the space of highly consistent states with a stronger bias towards states\nthat satisfy more constraints. An analytic framework for understanding the search scheme employed\nby the network is a topic for future work.\nThe proposed network exploits its temporal dynamics and analog properties to solve a class of\ncomputationally intensive problems. The WTA modules making up the network can be ef\ufb01ciently\nimplemented using neuromorphic VLSI circuits [26]. The results presented in this work encourage\nthe design of neuromorphic circuits and components that implement the full network in order to\nsolve constraint satisfaction problems in compact and ultra-low power VLSI systems.\n\nAcknowledgments\n\nThis work was supported by the European CHIST-ERA program, via the \u201cPlasticity in NEUral\nMemristive Architectures\u201d (PNEUMA) project and by the European Research council, via the \u201cNeu-\nromorphic Processors\u201d (neuroP) project, under ERC grant number 257219.\n\n8\n\n\fReferences\n[1] Pietro Berkes, Gerg\u02ddo Orb\u00b4an, M\u00b4at\u00b4e Lengyel, and J\u00b4ozsef Fiser. Spontaneous cortical activity reveals hall-\n\nmarks of an optimal internal model of the environment. Science, 331(6013):83\u201387, 2011.\n\n[2] Stefan Habenschuss, Zeno Jonke, and Wolfgang Maass. Stochastic Computations in Cortical Microcircuit\n\nModels. PLoS computational biology, 9:e1003311, 2013.\n\n[3] Scott Kirkpatrick, D. Gelatt Jr., and Mario P Vecchi. Optimization by simulated annealing. science,\n\n220(4598):671\u2013680, 1983.\n\n[4] John J Hop\ufb01eld and David W Tank. neural computation of decisions in optimization problems. Biological\n\ncybernetics, 52(3):141\u2013152, 1985.\n\n[5] Behzad Kamgar-Parsi. Dynamical stability and parameter selection in neural optimization. In Neural\n\nNetworks, 1992. IJCNN., International Joint Conference on, volume 4, pages 566\u2013571. IEEE, 1992.\n\n[6] Kate Smith, Marimuthu Palaniswami, and Mohan Krishnamoorthy. Neural techniques for combinatorial\n\noptimization with applications. Neural Networks, IEEE Transactions on, 9(6):1301\u20131318, 1998.\n\n[7] Luonan Chen and Kazuyuki Aihara. Chaotic simulated annealing by a neural network model with tran-\n\nsient chaos. Neural networks, 8(6):915\u2013930, 1995.\n\n[8] James P Crutch\ufb01eld. Critical computation, phase transitions, and hierarchical learning. Towards the\n\nHarnessing of Chaos, Amsterdam, 1994.\n\n[9] Jerry A Fodor. Information and association. Notre Dame journal of formal logic, 27(3):307\u2013323, 1986.\n[10] Rodney J. Douglas and Kevan A. Martin. Neuronal circuits of the neocortex. Annual review of neuro-\n\nscience, 27(1):419\u2013451, 2004.\n\n[11] R. J. Douglas and K. A. Martin. A functional microcircuit for cat visual cortex. The Journal of Physiology,\n\n440(1):735\u2013769, January 1991.\n\n[12] Carver Mead. Neuromorphic electronic systems. In Proc. IEEE, 78:16291636, 1990.\n[13] J. Lazzaro, S. Ryckebusch, M.A. Mahowald, and C.A. Mead. Winner-take-all networks of O(n) com-\nplexity. In D.S. Touretzky, editor, Advances in neural information processing systems, volume 2, pages\n703\u2013711, San Mateo - CA, 1989. Morgan Kaufmann.\n\n[14] G. Indiveri, B. Linares-Barranco, T.J. Hamilton, A. van Schaik, R. Etienne-Cummings, T. Delbruck, S.-C.\nLiu, P. Dudek, P. H\u00a8a\ufb02iger, S. Renaud, J. Schemmel, G. Cauwenberghs, J. Arthur, K. Hynna, F. Folowosele,\nS. Saighi, T. Serrano-Gotarredona, J. Wijekoon, Y. Wang, and K. Boahen. Neuromorphic silicon neuron\ncircuits. Frontiers in Neuroscience, 5:1\u201323, 2011.\n\n[15] Ueli Rutishauser, Rodney J Douglas, and Jean-Jacques Slotine. Collective stability of networks of winner-\n\ntake-all circuits. Neural computation, 23(3):735\u2013773, 2011.\n\n[16] Stefano Fusi and Maurizio Mattia. Collective Behavior of Networks with Linear (VLSI) Integrate-and-\n\nFire Neurons. Neural Computation, 11(3):633\u2013652, April 1999.\n\n[17] Maurizio Mattia and Paolo Del Giudice. Population dynamics of interacting spiking neurons. Physical\n\nReview E, 66(5):051917+, November 2002.\n\n[18] E.L. Bienenstock, L.N. Cooper, and P.W. Munro. Theory for the development of neuron selectivity:\norientation speci\ufb01city and binocular interaction in visual cortex. The Journal of Neuroscience, 2(1):32\u2013\n48, 1982.\n\n[19] Per J. Sj\u00a8ostr\u00a8om, Gina G. Turrigiano, and Sacha B. Nelson. Rate, Timing, and Cooperativity Jointly\n\nDetermine Cortical Synaptic Plasticity. Neuron, 32(6):1149\u20131164, December 2001.\n\n[20] Bob Kanefsky and W Taylor. Where the really hard problems are. In Proceedings of IJCAI, volume 91,\n\npages 163\u2013169, 1991.\n\n[21] Xiao-Jing Wang. Neurophysiological and computational principles of cortical rhythms in cognition. Phys-\n\niological reviews, 90(3):1195\u20131268, 2010.\n\n[22] Pascal Fries et al. A mechanism for cognitive dynamics: neuronal communication through neuronal\n\ncoherence. Trends in cognitive sciences, 9(10):474\u2013480, 2005.\n\n[23] Thilo Womelsdorf, Jan-Mathijs Schoffelen, Robert Oostenveld, Wolf Singer, Robert Desimone, An-\ndreas K Engel, and Pascal Fries. Modulation of neuronal interactions through neuronal synchronization.\nscience, 316(5831):1609\u20131612, 2007.\n\n[24] Peter Lakatos, George Karmos, Ashesh D Mehta, Istvan Ulbert, and Charles E Schroeder. Entrainment\n\nof neuronal oscillations as a mechanism of attentional selection. science, 320(5872):110\u2013113, 2008.\n\n[25] Xiao-Jing Wang. Decision making in recurrent neuronal circuits. Neuron, 60(2):215 \u2013 234, 2008.\n[26] G. Indiveri. A current-mode hysteretic winner-take-all network, with excitatory and inhibitory coupling.\n\nAnalog Integrated Circuits and Signal Processing, 28(3):279\u2013291, September 2001.\n\n9\n\n\f", "award": [], "sourceid": 418, "authors": [{"given_name": "Hesham", "family_name": "Mostafa", "institution": "ETH Zurich"}, {"given_name": "Lorenz.", "family_name": "Mueller", "institution": "ETH Zurich"}, {"given_name": "Giacomo", "family_name": "Indiveri", "institution": "ETH Zurich"}]}