{"title": "From Stochastic Nonlinear Integrate-and-Fire to Generalized Linear Models", "book": "Advances in Neural Information Processing Systems", "page_first": 1377, "page_last": 1385, "abstract": "Variability in single neuron models is typically implemented either by a stochastic Leaky-Integrate-and-Fire model or by a model of the Generalized Linear Model (GLM) family. We use analytical and numerical methods to relate state-of-the-art models from both schools of thought. First we find the analytical expressions relating the subthreshold voltage from the Adaptive Exponential Integrate-and-Fire model (AdEx) to the Spike-Response Model with escape noise (SRM as an example of a GLM). Then we calculate numerically the link-function that provides the firing probability given a deterministic membrane potential. We find a mathematical expression for this link-function and test the ability of the GLM to predict the firing probability of a neuron receiving complex stimulation. Comparing the prediction performance of various link-functions, we find that a GLM with an exponential link-function provides an excellent approximation to the Adaptive Exponential Integrate-and-Fire with colored-noise input. These results help to understand the relationship between the different approaches to stochastic neuron models.", "full_text": "From Stochastic Nonlinear Integrate-and-Fire to\n\nGeneralized Linear Models\n\nSchool of Computer and Communication Sciences and Brain-Mind Institute\n\nSkander Mensi\n\nEcole Polytechnique Federale de Lausanne\n1015 Lausanne EPFL, SWITZERLAND\n\nskander.mensi@epfl.ch\n\nSchool of Computer and Communication Sciences and Brain-Mind Institute\n\nRichard Naud\n\nEcole Polytechnique Federale de Lausanne\n1015 Lausanne EPFL, SWITZERLAND\n\nrichard.naud@epfl.ch\n\nSchool of Computer and Communication Sciences and Brain-Mind Institute\n\nWulfram Gersnter\n\nEcole Polytechnique Federale de Lausanne\n1015 Lausanne EPFL, SWITZERLAND\n\nwulfram.gerstner@epfl.ch\n\nAbstract\n\nVariability in single neuron models is typically implemented either by a stochastic\nLeaky-Integrate-and-Fire model or by a model of the Generalized Linear Model\n(GLM) family. We use analytical and numerical methods to relate state-of-the-\nart models from both schools of thought. First we \ufb01nd the analytical expressions\nrelating the subthreshold voltage from the Adaptive Exponential Integrate-and-\nFire model (AdEx) to the Spike-Response Model with escape noise (SRM as an\nexample of a GLM). Then we calculate numerically the link-function that pro-\nvides the \ufb01ring probability given a deterministic membrane potential. We \ufb01nd a\nmathematical expression for this link-function and test the ability of the GLM to\npredict the \ufb01ring probability of a neuron receiving complex stimulation. Compar-\ning the prediction performance of various link-functions, we \ufb01nd that a GLM with\nan exponential link-function provides an excellent approximation to the Adaptive\nExponential Integrate-and-Fire with colored-noise input. These results help to un-\nderstand the relationship between the different approaches to stochastic neuron\nmodels.\n\n1 Motivation\n\nWhen it comes to modeling the intrinsic variability in simple neuron models, we can distinguish two\ntraditional approaches. One approach is inspired by the stochastic Leaky Integrate-and-Fire (LIF)\nhypothesis of Stein (1967) [1], where a noise term is added to the system of differential equations\nimplementing the leaky integration to a threshold. There are multiple versions of such a stochastic\nLIF [2]. How the noise affects the \ufb01ring probability is also a function of the parameters of the neuron\nmodel. Therefore, it is important to take into account the re\ufb01nements of simple neuron models\nin terms of subthreshold resonance [3, 4], spike-triggered adaptation [5, 6] and non-linear spike\n\n1\n\n\finitiation [7, 5]. All these improvements are encompassed by the Adaptive Exponential Integrate-\nand-Fire model (AdEx [8, 9]).\nThe other approach is to start with some deterministic dynamics for the the state of the neuron\n(for instance the instantaneous distance from the membrane potential to the threshold) and link the\nprobability intensity of emitting a spike with a non-linear function of the state variable. Under some\nconditions, this type of model is part of a greater class of statistical models called Generalized Linear\nModels (GLM [10]). As a single neuron model, the Spike Response Model (SRM) with escape noise\nis a GLM in which the state variable is explicitly the distance between a deterministic voltage and\nthe threshold. The original SRM could account for subthreshold resonance, refractory effects and\nspike-frequency adaptation [11]. Mathematically similar models were developed independently in\nthe study of the visual system [12] where spike-frequency adaptation has also been modeled [13].\nRecently, this approach has retained increased attention since the probabilistic framework can be\nlinked with the Bayesian theory of neural systems [14] and because Bayesian inference can be\napplied to the population of neurons [15].\nIn this paper, we investigate the similarity and differences between the state-of-the-art GLM and the\nstochastic AdEx. The motivation behind this work is to relate the traditional threshold neuron models\nto Bayesian theory. Our results extend the work of Plesser and Gerstner (2000) [16] since we include\nthe non-linearity for spike initiation and spike-frequency adaptation. We also provide relationships\nbetween the parameters of the AdEx and the equivalent GLM. These precise relationships can be\nused to relate analog implementations of threshold models [17] to the probabilistic models used in\nthe Bayesian approach.\nThe paper is organized as follows: We \ufb01rst describe the expressions relating the SRM state-variable\nto the parameters of the AdEx (Sect. 3.1) in the subthreshold regime. Then, we use numerical\nmethods to \ufb01nd the non-linear link-function that models the \ufb01ring probability (Sect. 3.2). We \ufb01nd\na functional form for the SRM link-function that best describes the \ufb01ring probability of a stochastic\nAdEx. We then compare the performance of this link-function with the often used exponential or\nlinear-recti\ufb01er link-functions (also called half-wave linear recti\ufb01er) in terms of predicting the \ufb01ring\nprobability of an AdEx under complex stimulus (Sect. 3.3). We \ufb01nd that the exponential link-\nfunction yields almost perfect prediction. Finally, we explore the relations between the statistic of\nthe noise and the sharpness of the non-linearity for spike initiation with the parameters of the SRM.\n\n2 Presentation of the Models\n\nIn this section we present the general formula for the stochastic AdEx model (Sect. 2.1) and the\nSRM (Sect 2.2).\n\n2.1 The Stochastic Adaptive Exponential Integrate-and-Fire Model\n\nThe voltage dynamics of the stochastic AdEx is given by:\n\n(cid:18) V \u2212 \u0398\n\n(cid:19)\n\n\u03c4m \u02d9V = El \u2212 V + \u2206T exp\n\u03c4w \u02d9w = a(V \u2212 El) \u2212 w\n\n\u2206T\n\n\u2212 Rw + RI + R\u0001\n\n(1)\n\n(2)\nwhere \u03c4m is the membrane time constant, El the reverse potential, R the membrane resistance, \u0398 is\nthe threshold, \u2206T is the shape factor and I(t) the input current which is chosen to be an Ornstein-\nUhlenbeck process with correlation time-constant of 5 ms. The exponential term \u2206T exp( V \u2212\u0398\n) is\na non-linear function responsible for the emission of spikes and \u0001 is a diffusive white noise with\nstandard deviation \u03c3 (i.e. \u0001 \u223c N (0, \u03c3)). Note that the diffusive white-noise does not imply white\nnoise \ufb02uctuations of the voltage V (t), the probability distribution of V (t) will depend on \u2206T and\n\u0398. The second variable, w, describes the subthreshold as well as the spike-triggered adaptation both\nparametrized by the coupling strength a and the time constant \u03c4w. Each time \u02c6tj the voltage goes to\nin\ufb01nity, we assumed that a spike is emitted. Then the voltage is reset to a \ufb01xed value Vr and w is\nincreased by a constant value b.\n\n\u2206T\n\n2.2 The Generalized Linear Model\n\nIn the SRM, The voltage V (t) is given by the convolution of the injected current I(t) with the\nmembrane \ufb01lter \u03ba(t) plus the additional kernel \u03b7(t) that acts after each spikes (here we split the\n\n2\n\n\fspike-triggered kernel in two \u03b7(t) = \u03b7v(t) + \u03b7w(t) for reasons that will become clear later):\n\n(cid:0)\u03b7v(t \u2212 \u02c6tj) + \u03b7w(t \u2212 \u02c6tj)(cid:1)\n\n(3)\n\nV (t) = El + [\u03ba \u2217 I](t) +\n\n(cid:88)\n\n{\u02c6tj}\n\nThen at each time \u02c6tj a spike is emitted which results in a change of voltage described by \u03b7(t) =\n\u03b7v(t) + \u03b7w(t).\nGiven the deterministic voltage, (Eq. 3) a spike is emitted according to the \ufb01ring intensity \u03bb(V ):\n\n(4)\nwhere f (\u00b7) is an arbitrary function called the link-function. Then the \ufb01ring behavior of the SRM\ndepends on the choice of the link-function and its parameters. The most common link-function used\nto model single neuron activities are the linear-recti\ufb01er and the exponential function.\n\n\u03bb(t) = f (V (t))\n\n3 Mapping\n\nIn order to map the stochastic AdEx to the SRM we follow a two-step procedure. First we derive the\n\ufb01lter \u03ba(t) and the kernels \u03b7v(t) and \u03b7w(t) analytically as a function of AdEx parameters. Second,\nwe derive the link-function of the SRM from the stochastic spike emission of the AdEx.\n\nFigure 1: Mapping of the subthreshold dynamics of an AdEx to an equivalent SRM. A. Mem-\nbrane \ufb01lter \u03ba(t) for three different sets of parameters of the AdEx leading to over-damped, critically\ndamped and under-damped cases (upper, middle and lower panel, respectively). B. Spike-Triggered\n\u03b7(t) (black), \u03b7v(t) (light gray) and \u03b7w (gray) for the three cases. C. Example of voltage trace pro-\nduced when an AdEx is stimulated with a step of colored noise (black). The corresponding voltage\nfrom a SRM stimulated with the same current and where we forced the spikes to match those of the\nAdEx (red). D. Error in the subthreshold voltage (VAdEx \u2212 VGLM) as a function of the mean voltage\nof the AdEx, for the three different cases: over-, critically and under-damped (light gray, gray and\nblack, respectively) with \u2206T = 1 mV. Red line represents the voltage threshold \u0398. E. Root Mean\nSquare Error (RMSE) ratio for the three cases with \u2206T = 1 mV. The RMSE ratio is the RMSE\nbetween the deterministic VSRM and the stochastic VAdEx divided by the RMSE between repetitions\nof the stochastic AdEx voltage. The error bar shows a single standard deviation as the RMSE ratio\nis averaged accross multiple value of \u03c3.\n\n3.1 Subthreshold voltage dynamics\n\nWe start by assuming that the non-linearity for spike initiation does not affect the mean subthreshold\nvoltage of the stochastic AdEx (see Figure 1 D). This assumption is motivated by the small \u2206T\n\n3\n\n\fobserved in in-vitro recordings (from 0.5 to 2 mV [8, 9]) which suggest that the subthreshold dy-\nnamics are mainly linear except very close to \u0398. Also, we expect that the non-linear link-function\nwill capture some of the dynamics due to the non-linearity for spike initiation. Thus it is pos-\nsible to rewrite the deterministic subthreshold part of the AdEx (Eq. 1-2 without \u0001 and without\n\u2206T exp((V \u2212 \u0398)/\u2206T )) using matrices:\n\n\u02d9x = Ax\n\n(cid:20)\u2212 1\n\n\u03c4m\na\n\u03c4w\n\n(cid:21)\n\n\u2212 1\n\u2212 1\n\ngl\u03c4m\n\n\u03c4w\n\n(cid:18)V\n\n(cid:19)\n\nw\n\nwith x =\n\nand A =\n\n(5)\n\n(6)\n\nIn this form, the dynamics of the deterministic AdEx voltage is a damped oscillator with a driving\nforce. Depending on the eigenvalues of A the system could be over-damped, critically damped or\nunder-damped. The \ufb01lter \u03ba(t) of the GLM is given by the impulse response of the system of coupled\ndifferential equations of the AdEx, described by Eq. 5 and 6. In other words, one has to derive the\nresponse of the system when stimulating with a Dirac-delta function. The type of damping gives\nthree different qualitative shapes of the kernel \u03ba(t), which are summarized in Table 3.1 and Figure 1\nA. Since the three different \ufb01lters also affect the nature of the stochastic voltage \ufb02uctuations, we will\nkeep the distinction between over-damped, critically damped and under-damped scenarios through-\nout the paper. This means that our approach is valid for at least 3 types of diffusive voltage-noise\n(i.e. the white noise \u0001 in Eq. 1 \ufb01ltered by 3 different membrane \ufb01lters \u03ba(t)).\nTo complete the description of the deterministic voltage, we need an expression for the spike-\ntriggered kernels. The voltage reset at each spike brings a spike-triggered jump in voltage of mag-\nnitude \u2206 = Vr \u2212 V (\u02c6t). This perturbation is superposed to the current \ufb02uctuations due to I(t) and\ncan be mediated by a Delta-diract pulse of current. Thus we can write the voltage reset kernel by:\n\n\u03b7v(t) =\n\n[\u03b4 \u2217 \u03ba] (t) =\n\n\u2206\n\n\u03ba(0)\n\n\u2206\n\n\u03ba(0)\n\n\u03ba(t)\n\n(7)\n\nwhere \u03b4(t) is the Dirac-delta function. The shape of this kernel depends on \u03ba(t) and can be computed\nfrom Table 3.1 (see Figure 1 B).\nFinally, the AdEx mediates spike-frequency adaptation by the jump of the second variables w. From\nEq. 2 we can see that this produces a current wspike(t) = b exp (\u2212t/\u03c4w) that can cumulate over\nsubsequent spikes. The effect of this current on voltage is then given by the convolution of wspike(t)\nwith the membrane \ufb01lter \u03ba(t). Thus in the SRM framework the spike-frequency adaptation is taken\ninto account by:\n\n\u03b7w(t) = [wspike \u2217 \u03ba](t)\n\n(8)\nAgain the precise form of \u03b7w(t) depends on \u03ba(t) and can be computed from Table 3.1 (see Figure 1\nB).\nAt this point, we would like to verify our assumption that the non-linearity for spike emission can\nbe neglected. Fig. 1 C and D shows that the error between the voltage from Eq. 3 and the voltage\nfrom the stochastic AdEx is generally small. Moreover, we see that the main contribution to the\nvoltage prediction error is due to the mismatch close to the spikes. However the non-linearity for\nspike initiation may change the probability distribution of the voltage \ufb02uctuations, which in turn\nin\ufb02uences the probability of spiking. This will in\ufb02uence the choice of the link-function, as we will\nsee in the next section.\n\n3.2 Spike Generation\n\nUsing \u03ba(t), \u03b7v(t) and \u03b7w(t), we must relate the spiking probability of the stochastic AdEx as a\nfunction of its deterministic voltage. According to [2] the probability of spiking in time bin dt given\nthe deterministic voltage V (t) is given by:\n\np(V ) = prob{spike in [t, t + dt]} = 1 \u2212 exp (\u2212f (V (t))dt)\n\n(9)\nwhere f (\u00b7) gives the \ufb01ring intensity as a function of the deterministic V (t) (Eq. 3). Thus to extract\nthe link-function f we have to compute the probability of spiking given V (t) for our SRM. To do\nso we apply the method proposed by Jolivet et al. (2004) [18], where the probability of spiking is\nsimply given by the distribution of the deterministic voltage estimated at the spike times divided by\nthe distribution of the SRM voltage when there is no spike (see \ufb01gure 2 A). One can numerically\ncompute these two quantities for our models using N repetitions of the same stimulus.\n\n4\n\n\fTable 1: Analytical expressions for the membrane \ufb01lter \u03ba(t) in terms of the parameters of the AdEx\nfor over-, critically-, and under-damped cases.\n\nMembrane Filter: \u03ba(t)\n\nover-damped if:\n\ncritically-damped if:\n\nunder-damped if:\n\n(\u03c4m + \u03c4w)2 > 4\u03c4m\u03c4w(gl+a)\n\ngl\n\n(\u03c4m + \u03c4w)2 = 4\u03c4m\u03c4w(gl+a)\n\ngl\n\n(\u03c4m + \u03c4w)2 < 4\u03c4m\u03c4w(gl+a)\n\ngl\n\n\u03ba(t) = k1e\u03bb1t + k2e\u03bb2t\n\n\u03ba(t) = (\u03b1t + \u03b2)e\u03bbt\n\n\u03ba(t) = (k1 cos (\u03c9t) + k2 sin (\u03c9t)) e\u03bbt\n\n(cid:17)\n(cid:17)\n\n(cid:113)\n(cid:113)\n\n(\u2212(\u03c4m + \u03c4w) +\n\n2\u03c4m\u03c4w\n\n\u03bb1 = 1\n(\u03c4m + \u03c4w)2 \u2212 4 \u03c4m\u03c4w\n\ngl\n\n(gl + a)\n(\u2212(\u03c4m + \u03c4w) \u2212\n\n\u03bb2 = 1\n\n2\u03c4m\u03c4w\n\n(gl + a)\n\ngl\n\n(\u03c4m + \u03c4w)2 \u2212 4 \u03c4m\u03c4w\n\u2212(1+(\u03c4m\u03bb2))\nC\u03c4m(\u03bb1\u2212\u03bb2)\nk2 = 1+(\u03c4m\u03bb1)\nC\u03c4m(\u03bb1\u2212\u03bb2)\n\nk1 =\n\n\u03bb =\n\n\u2212(\u03c4m+\u03c4w)\n\n2\u03c4m\u03c4w\n\n\u03b1 = \u03c4m\u2212\u03c4w\n\n2C\u03c4m\u03c4w\n\n\u03c9 =\n\n\u2212(\u03c4m+\u03c4w)\n\n2\u03c4m\u03c4w\n\n\u03bb =\n\n(cid:115)(cid:12)(cid:12)(cid:12)(cid:12)(cid:16) \u03c4w\u2212\u03c4m\n\n2\u03c4m\u03c4w\n\n(cid:17)2 \u2212 a\n\ngl\u03c4m\u03c4w\n\n(cid:12)(cid:12)(cid:12)(cid:12)\n\n\u03b2 = 1\nC\n\nk1 = 1\nC\n\nk2 =\n\n\u2212(1+\u03c4m\u03bb)\n\nC\u03c9\u03c4m\n\nThe standard deviation \u03c3 of the noise and the parameter \u2206T of the AdEx non-linearity may affect the\nshape of the link-function. We thus extract p(V ) for different \u03c3 and \u2206T (Fig. 2 B). Then using visual\nheuristics and previous knowledge about the potential analytical expression of the link-funtion, we\ntry to \ufb01nd a simple analytical function that captures p(V ) for a large range of combinations of \u03c3\nand \u2206T . We observed that the log(\u2212 log(p)) is close to linear in most studied conditions Fig. 2 B\nsuggesting the following two distributions of p(V ):\n\n(10)\n\n(11)\n\np(V ) = 1 \u2212 exp\n\n\u2212 exp\n\np(V ) = exp\n\n\u2212 exp\n\n(cid:18)\n\n(cid:18) V \u2212 VT\n(cid:19)(cid:19)\n(cid:19)(cid:19)\n\u2212 V \u2212 VT\n\n\u2206V\n\n(cid:18)\n\n\u2206V\n\n(cid:18)\n\n\u22121\ndt\n\nOnce we have p(V ), we can use Eq. 4 to obtain the equivalent SRM link-function, which leads to:\n\nf (V ) =\n\nlog (1 \u2212 p(V ))\n\n(12)\n\nThen the two potential link-functions of the SRM can be derived from Eq. 10 and Eq. 11 (respec-\ntively):\n\n(cid:18) V \u2212 VT\n(cid:18)\n\n\u2206V\n1 \u2212 exp\n\n(cid:19)\n(cid:18)\n\nf (V ) = \u03bb0 exp\n\nf (V ) = \u2212\u03bb0 log\n\n(cid:19)(cid:19)(cid:19)\n\n(cid:18)\n\u2212 V \u2212 VT\n\n\u2206V\n\n(13)\n\n(14)\n\n\u2212 exp\n\nwith \u03bb0 = 1\ndt, VT the threshold of the SRM and \u2206V the sharpness of the link-function (i.e. the\nparameters that governs the degree of the stochasticity). Note that the exact value of \u03bb0 has no\nimportance since it is redundant with VT . Eq. 13 is the standard exponential link-function, but we\ncall Eq. 14 the log-exp-exp link-function.\n\n3.3 Prediction\n\nThe next point is to evaluate the \ufb01t quality of each link-function. To do this, we \ufb01rst estimate the\nparameters VT and \u2206V of the GLM link-function that maximize the likelihood of observing a spike\n\n5\n\n\fFigure 2: SRM link-function. A. Histogram of the SRM voltage at the AdEx \ufb01ring times (red) and\nat non-\ufb01ring times (gray). The ratio of the two distributions gives p(V ) (Eq. 9, dashed lines). Inset,\nzoom to see the voltage histogram evaluated at the \ufb01ring time (red). B. log(\u2212 log(p)) as a function\nof the SRM voltage for three different noise levels \u03c3 = 0.07, 0.14, 0.18 nA (pale gray, gray, black\ndots, respectively) and \u2206T = 1 mV. The line is a linear \ufb01t corresponding to the log-exp-exp link-\nfunction and the dashed line corresponds to a \ufb01t with the exponential link-function. C. Same data\nand labeling scheme as B, but plotting f (V ) according to Eq. 12. The lines are produced with Eq.\n14 with parameters \ufb01tted as described in B. and the dashed lines are produced with Eq. 13. Inset,\nsame plot but on a semi-log(y) axis.\n\n\uf8eb\uf8ed(cid:88)\n\nNLL = \u2212\n\nlog(f (t|I, \u03b8)) \u2212(cid:88)\n\nf (t|I, \u03b8)\n\n\uf8f6\uf8f8\n\ntrain generated with an AdEx. Second we look at the predictive power of the resulting SRM in\nterms of Peri-Stimulus Time Histogram (PSTH). In other words we ask how close the spike trains\ngenerated with a GLM are from the spike train generated with a stochastic AdEx when both models\nare stimulated with the same input current.\nFor any GLM with link-function f (V ) \u2261 f (t|I, \u03b8) and parameters \u03b8 regulating the shape of \u03ba(t),\n\u03b7v(t) and \u03b7w(t), the Negative Log-Likelihood (NLL) of observing a spike-train {\u02c6t} is given by:\n\n(15)\n\n\u02c6t\n\nt\n\nIt has been shown that the negative log-likelihood is convex in the parameters if f is convex and log-\nconcave [19]. It is easy to show that a linear-recti\ufb01er link-function, the exponential link-function\nand the log-exp-exp link-function all satisfy these conditions. This allows ef\ufb01cient estimation of\nthe optimal parameters \u02c6VT and \u02c6\u2206V using a simple gradient descent. One can thus estimate from a\nsingle AdEx spike train the optimal parameters of a given link-function, which is more ef\ufb01cient than\nthe method used in Sect. 3.2.\nThe minimal NLL resulting from the gradient descent gives an estimation of the \ufb01t quality. A better\nestimate of the \ufb01t quality is given by the distance between the PSTHs in response to stimuli not used\nfor parameter \ufb01tting . Let \u03bd1(t) be the PSTH of the AdEx, and \u03bd2(t) be the PSTH of the \ufb01tted SRM,\n\n6\n\n\fFigure 3: PSTH prediction. A. Injected current. B. Voltage traces produced by an AdEx (black) and\nthe equivalent SRM (red), when stimulated with the current in A. C. Raster plot for 20 realizations\nof AdEx (black tick marks) and equivalent SRM (red tick marks). D. PSTH of the AdEx (black)\nand the SRM (red) obtained by averaging 10,000 repetitions. E. Optimal log-likelihood for the three\ncases of the AdEx, using three different link-functions, a linear-recti\ufb01er (light gray), an exponential\nlink-function (gray) and the link-function de\ufb01ned by Eq. 14 (dark gray), these values are obtained\nby averaging over 40 different combinations \u03c3 and \u2206T (see Fig. 4). Error bars are one standard\ndeviation, the stars denote a signi\ufb01cant difference, two-sample t-test with \u03b1 = 0.01. F. same as E.\nbut for Md (Eq. 16).\n\nthen we use Md \u2208 [0, 1] as a measure of match:\n\n2(cid:82) (\u03bd1(t) \u2212 \u03bd2(t))2 dt\n(cid:82) \u03bd1(t)2dt +(cid:82) \u03bd2(t)2dt\n\nMd =\n\n(16)\n\nMd = 1 means that it is impossible to differentiate the SRM from the AdEx in terms of their PSTHs,\nwhereas a Md of 0 means that the two PSTHs are completely different. Thus Md is a normalized\nsimilarity measure between two PSTHs. In practice, Md is estimated from the smoothed (boxcar\naverage of 1 ms half-width) averaged spike train of 1 000 repetitions for each models. We use both\nthe NLL and Md to quantify the \ufb01t quality for each of the three damping cases and each of the three\nlink-functions.\nFigure 3 shows the match between the stochastic AdEx used as a reference and the derived GLM\nwhen both are stimulated with the same input current (Fig. 3 A). The resulting voltage traces are\nalmost identical (Fig. 3 B) and both models predict almost the same spike trains and so the same\nPSTHs (Fig. 3 C and D). More quantitalively, we see on Fig. 3 E and F, that the linear-recti\ufb01er \ufb01ts\nsigni\ufb01cantly worse than both the exponential and log-exp-exp link-functions, both in terms of NLL\nand of Md. The exponential link-function performs as well as the log-exp-exp link-function, with a\nspike train similarity measure Md being almost 1 for both.\nFinally the likelihood-based method described above gives us the opportunity to look at the rela-\ntionship between the AdEx parameters \u03c3 and \u2206T that governs its spike emission and the parameters\nVT and \u2206V of the link-function (Fig. 4). We observe that an increase of the noise level produces a\n\ufb02atter link-function (greater \u2206V ) while an increase in \u2206T also produces an increase in \u2206V and VT\n(note that Fig. 4 shows \u2206V and VT for the exponential link-function only, but equivalent results are\nobtained with the log-exp-exp link-function).\n\n4 Discussion\n\nIn Sect. 3.3 we have shown that it is possible to predict with almost perfect accuracy the PSTH\nof a stochastic AdEx model using an appropriate set of parameters in the SRM. Moreover, since\n\n7\n\n\fFigure 4: In\ufb02uence of the AdEx parameters on the parameters of the exponential link-function. A.\nVT as a function of \u2206T and \u03c3. B. \u2206V as a function of \u2206T and \u03c3.\n\nthe subthreshold voltage of the AdEx also gives a good match with the deterministic voltage of the\nSRM, we expect that the AdEx and the SRM will not differ in higher moments of the spike train\nprobability distributions beyond the PSTH. We therefore conclude that diffusive noise models of the\ntype of Eq. 1-2 are equivalent to GLM of the type of Eq. 3-4. Once combined with similar results on\nother types of stochastic LIF (e.g. correlated noise), we could bridge the gap between the literature\non GLM and the literature on diffusive noise models.\nAnother noteworthy observation pertains to the nature of the link-function. The link-function has\nbeen hypothesized to be a linear-recti\ufb01er, an exponential, a sigmoidal or a Gaussian [16]. We have\nobserved that for the AdEx the link-function follows Eq. 14 that we called the log-exp-exp link-\nfunction. Although the link-function is log-exp-exp for most of the AdEx parameters, the exponen-\ntial link-function gives an equivalently good prediction of the PSTH. This can be explained by the\nfact that the difference between log-exp-exp and exponential link-functions happens mainly at low\nvoltage (i.e. far from the threshold), where the probability of emitting a spike is so low (Figure 2 C,\nuntil -50 mv). Therefore, even if the exponential link-function overestimates the \ufb01ring probability at\nthese low voltages it rarely produces extra spikes. At voltages closer to the threshold, where most of\nthe spikes are emitted, the two link-functions behave almost identically and hence produce the same\nPSTH. The Gaussian link-function can be seen as lying in-between the exponential link-function and\nthe log-exp-exp link-function in Fig. 2. This means that the work of Plesser and Gerstner (2000)\n[16] is in agreement with the results presented here. The importance of the time-derivative of the\nvoltage stressed by Plesser and Gerstner (leading to a two-dimensional link-function f (V, \u02d9V )) was\nnot studied here to remain consistent with the typical usage of GLM in neural systems [14].\nFinally we restricted our study to exponential non-linearity for spike initiation and do not consider\nother cases such as the Quadratic Integrate-and-\ufb01re (QIF, [5]) or other polynomial functional shapes.\nWe overlooked these cases for two reasons. First, there are many evidences that the non-linearity in\nneurons (estimated from in-vitro recordings of Pyramidal neurons) is well approximated by a single\nexponential [9]. Second, the exponential non-linearity of the AdEx only affects the subthreshold\nvoltage at high voltage (close to threshold) and thus can be neglected to derive the \ufb01lters \u03ba(t) and\n\u03b7(t). Polynomial non-linearities on the other hand affect a larger range of the subthreshold voltage\nso that it would be dif\ufb01cult to justify the linearization of subthreshold dynamics essential to the\nmethod presented here.\n\nReferences\n\n[1] R. B. Stein, \u201cSome models of neuronal variability,\u201d Biophys J, vol. 7, no. 1, pp. 37\u201368, 1967.\n[2] W. Gerstner and W. Kistler, Spiking neuron models. Cambridge University Press New York, 2002.\n[3] E. Izhikevich, \u201cResonate-and-\ufb01re neurons,\u201d Neural Networks, vol. 14, no. 883-894, 2001.\n[4] M. J. E. Richardson, N. Brunel, and V. Hakim, \u201cFrom subthreshold to \ufb01ring-rate resonance,\u201d Journal of\n\nNeurophysiology, vol. 89, pp. 2538\u20132554, 2003.\n\n8\n\n\f[5] E. Izhikevich, \u201cSimple model of spiking neurons,\u201d IEEE Transactions on Neural Networks, vol. 14, pp.\n\n1569\u20131572, 2003.\n\n[6] S. Mensi, R. Naud, M. Avermann, C. C. H. Petersen, and W. Gerstner, \u201cParameter extraction and classi\ufb01-\n\ncation of three neuron types reveals two different adaptation mechanisms,\u201d Under review.\n\n[7] N. Fourcaud-Trocme, D. Hansel, C. V. Vreeswijk, and N. Brunel, \u201cHow spike generation mechanisms\ndetermine the neuronal response to \ufb02uctuating inputs,\u201d Journal of Neuroscience, vol. 23, no. 37, pp.\n11 628\u201311 640, 2003.\n\n[8] R. Brette and W. Gerstner, \u201cAdaptive exponential integrate-and-\ufb01re model as an effective description of\n\nneuronal activity,\u201d Journal of Neurophysiology, vol. 94, pp. 3637\u20133642, 2005.\n\n[9] L. Badel, W. Gerstner, and M. Richardson, \u201cDependence of the spike-triggered average voltage on mem-\n\nbrane response properties,\u201d Neurocomputing, vol. 69, pp. 1062\u20131065, 2007.\n\n[10] P. McCullagh and J. A. Nelder, Generalized linear models, 2nd ed. Chapman & Hall/CRC, 1998, vol. 37.\n[11] W. Gerstner, J. van Hemmen, and J. Cowan, \u201cWhat matters in neuronal locking?\u201d Neural computation,\n\nvol. 8, pp. 1653\u20131676, 1996.\n\n[12] D. Hubel and T. Wiesel, \u201cReceptive \ufb01elds and functional architecture of monkey striate cortex,\u201d Journal\n\nof Physiology, vol. 195, pp. 215\u2013243, 1968.\n\n[13] J. Pillow, L. Paninski, V. Uzzell, E. Simoncelli, and E. Chichilnisky, \u201cPrediction and decoding of retinal\nganglion cell responses with a probabilistic spiking model,\u201d Journal of Neuroscience, vol. 25, no. 47, pp.\n11 003\u201311 013, 2005.\n\n[14] K. Doya, S. Ishii, A. Pouget, and R. P. N. Rao, Bayesian brain: Probabilistic approaches to neural coding.\n\nThe MIT Press, 2007.\n\n[15] S. Gerwinn, J. H. Macke, M. Seeger, and M. Bethge, \u201cBayesian inference for spiking neuron models with\n\na sparsity prior,\u201d in Advances in Neural Information Processing Systems, 2007.\n\n[16] H. Plesser and W. Gerstner, \u201cNoise in integrate-and-\ufb01re neurons: From stochastic input to escape rates,\u201d\n\nNeural Computation, vol. 12, pp. 367\u2013384, 2000.\n\n[17] J. Schemmel, J. Fieres, and K. Meier, \u201cWafer-scale integration of analog neural networks,\u201d in Neural\nNetworks, 2008. IJCNN 2008. (IEEE World Congress on Computational Intelligence). IEEE International\nJoint Conference on, june 2008, pp. 431 \u2013438.\n\n[18] R. Jolivet, T. Lewis, and W. Gerstner, \u201cGeneralized integrate-and-\ufb01re models of neuronal activity approx-\nimate spike trains of a detailed model to a high degree of accuracy,\u201d Journal of Neurophysiology, vol. 92,\npp. 959\u2013976, 2004.\n\n[19] L. Paninski, \u201cMaximum likelihood estimation of cascade point-process neural encoding models,\u201d Net-\n\nwork: Computation in Neural Systems, vol. 15, pp. 243\u2013262, 2004.\n\n9\n\n\f", "award": [], "sourceid": 794, "authors": [{"given_name": "Skander", "family_name": "Mensi", "institution": null}, {"given_name": "Richard", "family_name": "Naud", "institution": null}, {"given_name": "Wulfram", "family_name": "Gerstner", "institution": null}]}