{"title": "Scalable Bayesian inference of dendritic voltage via spatiotemporal recurrent state space models", "book": "Advances in Neural Information Processing Systems", "page_first": 10165, "page_last": 10174, "abstract": "Recent advances in optical voltage sensors have brought us closer to a critical goal in cellular neuroscience: imaging the full spatiotemporal voltage on a dendritic tree. However, current sensors and imaging approaches still face significant limitations in SNR and sampling frequency; therefore statistical denoising and interpolation methods remain critical for understanding single-trial spatiotemporal dendritic voltage dynamics. Previous denoising approaches were either based on an inadequate linear voltage model or scaled poorly to large trees. Here we introduce a scalable fully Bayesian approach. We develop a generative nonlinear model that requires few parameters per compartment of the cell but is nonetheless flexible enough to sample realistic spatiotemporal data. The model captures different dynamics in each compartment and leverages biophysical knowledge to constrain intra- and inter-compartmental dynamics. We obtain a full posterior distribution over spatiotemporal voltage via an augmented Gibbs sampling algorithm. The nonlinear smoother model outperforms previously developed linear methods, and scales to much larger systems than previous methods based on sequential Monte Carlo approaches.", "full_text": "Scalable Bayesian inference of dendritic voltage via\n\nspatiotemporal recurrent state space models\n\nRuoxi Sun\u2217\n\nColumbia University\n\nScott W. Linderman\u2217\nStanford University\n\nIan August Kinsella\nColumbia University\n\nLiam Paninski\n\nColumbia University\n\nAbstract\n\nRecent advances in optical voltage sensors have brought us closer to a critical goal\nin cellular neuroscience: imaging the full spatiotemporal voltage on a dendritic tree.\nHowever, current sensors and imaging approaches still face signi\ufb01cant limitations in\nSNR and sampling frequency; therefore statistical denoising methods remain critical\nfor understanding single-trial spatiotemporal dendritic voltage dynamics. Previous\ndenoising approaches were either based on an inadequate linear voltage model or\nscaled poorly to large trees. Here we introduce a scalable fully Bayesian approach.\nWe develop a generative nonlinear model that requires few parameters per dendritic\ncompartment but is nonetheless \ufb02exible enough to sample realistic spatiotemporal\ndata. The model captures potentially di\ufb00erent dynamics in each compartment\nand leverages biophysical knowledge to constrain intra- and inter-compartmental\ndynamics. We obtain a full posterior distribution over spatiotemporal voltage via\nan e\ufb03cient augmented block-Gibbs sampling algorithm. The nonlinear smoother\nmodel outperforms previously developed linear methods, and scales to much larger\nsystems than previous methods based on sequential Monte Carlo approaches.\n\nIntroduction\n\n1\nRecent progress in the development of voltage indicators [1\u20138] has brought us closer to a long-\nstanding goal in cellular neuroscience: imaging the full spatiotemporal voltage on a dendritic tree.\nThese recordings have the potential (pun not intended) to resolve fundamental questions about\nthe computations performed by dendrites \u2014 questions that have remained open for more than a\ncentury [9, 10]. Unfortunately, despite accelerating progress, currently available voltage indicators\nand imaging technologies provide data that is noisy and sparse in time and space. Our goal in this work\nis to take this noisy, sparse data and output Bayesian estimates, with uncertainty, of the spatiotemporal\nvoltage on the tree, at arbitrary resolution.\n\nA number of generic denoisers are available. For example, one previous approach is to run an\nindependent spline smoother on the temporal trace from each pixel [1]. However, this approach\nignores two critical features of the data. First, the data is highly spatiotemporally structured; thus,\nrunning a purely temporal smoother and ignoring spatial information (or vice versa) is suboptimal.\nSecond, the smoothness of voltage data is highly inhomogeneous; for example, action potentials are\nmuch less smooth than are subthreshold voltage dynamics, and it is suboptimal to enforce the same\nlevel of smoothness in these two very di\ufb00erent regimes.\n\nHow can we exploit our strong priors, based on decades of biophysics research, about the highly-\nstructured dynamics governing voltage on the dendritic tree? Our starting point is the cable equa-\ntion [11], the partial di\ufb00erential equation that speci\ufb01es the evolution of membrane potential in space\nand time. If we divide up the tree into N discrete compartments, then letting V (n)\ndenote the voltage\n\nt\n\n\u2217Equal contribution\n\n33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canada.\n\n\fof compartment n at time t, we have\n\nt+\u2206t \u2248 V (n)\nV (n)\n\nt +\n\n\u2206t\nCn\n\n\uf8ee\uf8f0(cid:88)\n\nj\n\nN(cid:88)\n\nn(cid:48)=1\n\nI (n,j)\nt\n\n+\n\n\uf8f9\uf8fb .\n\n)\n\ngnn(cid:48) \u00b7 (V (n\n\nt\n\n(cid:48)\n\n)\n\n\u2212 V (n)\n\nt\n\n(1)\n\n(cid:48)\n\n)\n\nt\n\nt\n\nt\n\nt \u2212 V (n)\n\nEach compartment n has its own membrane capacitance Cn and internal currents I (n,j)\n; j indexes\nmembrane channel types, with j = 0 denoting the current driven by the membrane leak conductance.\nThe currents through each channel type for j > 0 in turn depend on the local voltage V (n)\nplus\nauxiliary channel state variables with nonlinear, voltage-dependent dynamics. The coupling of voltage\nand channel state variables renders the intra-compartment dynamics highly nonlinear.\nAdditional current \ufb02ows between compartments n and n(cid:48) according to the conductance gnn(cid:48) \u2265 0 and\nthe voltage drop V (n\n. The conductances are undirected so that gnn(cid:48) = gn(cid:48)n. The symmetric\nmatrix of conductances G = {gnn(cid:48)} \u2208 RN\u00d7N speci\ufb01es a weighted, undirected tree graph. Nonzero\nentries indicate the strength of connection between two physically coupled compartments: if gnn(cid:48) is\nlarge then voltage di\ufb00erences between compartments n and n(cid:48) are resolved quickly, i.e. their voltages\nbecome more tightly coupled.\nThe Hodgkin-Huxley (HH) model [12] and its generalizations [13] o\ufb00er biophysically detailed models\nof voltage dynamics, but learning and inference pose signi\ufb01cant challenges in the resulting high-\ndimensional nonlinear dynamical system. Two approaches have been pursued in the past. First, we\ncan restrict attention to the subthreshold regime, where the dynamics can be approximated as linear.\nInvoking the central limit theorem leads to a Gaussian approximation on the current noise (due to\nthe sum over j in Eq. 1), resulting in an overall linear-Gaussian model. If the observed data can in\nturn be modeled as a linear function of the voltage plus Gaussian noise, we are left with a classical\nKalman \ufb01lter model. Paninski [14] develops e\ufb03cient methods to scale inference in this Kalman \ufb01lter\nmodel to handle large trees. However, the resulting smoother doesn\u2019t handle spikes well \u2014 it either\nover-smooths spikes or under-smooths subthreshold voltages. The Kalman model su\ufb00ers because it\nassumes voltage has one uniform smoothness level, and as already discussed, this assumption only\nmakes sense in the subthreshold regime.\nAlternatively, we can attempt to perform inference on noisy voltage recordings based on model (1)\ndirectly. There are many compartments, each with a voltage and a collection of channel state variables,\nleading to a very high-dimensional nonlinear dynamical system. For low-dimensional models, like\nsingle compartment models with few channel states, methods like sequential Monte Carlo (SMC) and\napproximate Bayesian computation (ABC) can be applied [15\u201317], but even with recent advances [18\u2013\n20], inference in large scale biophysical models remains di\ufb03cult. The learning problem (i.e. estimating\nthe parameters governing the intra- and inter-compartment dynamics) is even harder: inaccurate state\ninferences lead to errors in parameter estimation, and poor parameter estimates lead to incorrect state\ninferences. Compounding all of this is model misspeci\ufb01cation \u2014 critical parameters such as time\nconstants and voltage-sensitivity functions vary across channels, and there are dozens of types of\nchannels in real cells [13]\u2014and model identi\ufb01ability \u2014 many channel combinations can produce\nsimilar dynamics [21]. Thus performing inference on the multi-compartment biophysical model (1)\ndirectly seems intractable.\nBelow we propose an alternative approach, blending the cable equation model with general purpose\nstatistical models of nonlinear dynamical systems, to enable e\ufb03cient learning and inference of\nspatiotemporal voltage dynamics. Code is available at: https://github.com/SunRuoxi/Voltage_\nSmoothing_with_rSLDS.\n2 Model\nOur basic strategy is as follows. For each compartment n, the biophysical model in (1) involves\npotentially dozens of channel types, each with their own state variables evolving according to\nnonlinear dynamics \u2014 but to infer voltages (not individual currents) we only actually need the sum of\nthe induced currents. We replace this sum with a simpler, low-dimensional e\ufb00ective model. We retain\nthe basic spatial biophysical constraints on voltage dynamics (i.e., leaving the second term in model\n(1) as is), while approximating the nonlinear interactions between voltage and membrane channels\nwith a more tractable model: a recurrent switching linear dynamical system (rSLDS). Figure 1 shows\nhow each compartment is given its own discrete states, continuous states, and voltage, and how these\nvariables interact to produce nonlinear spatiotemporal dynamics.\n\n2\n\n\fFigure 1: Approximating the biophysical model of membrane potential dynamics with a tractable graphical\nmodel. a. We study voltage dynamics on dendritic trees like the one shown here. b. Each compartment of the cell\nis approximated with a recurrent switching linear dynamical system, which has discrete latent states, continuous\nlatent states, a true (unobserved) voltage, and noisy voltage observations. The continuous states and voltage\nfollow piecewise linear dynamics conditioned on the discrete states; marginalizing over the discrete states, we\nobtain nonlinear dynamics within each compartment. The red arrows denote the recurrent dependencies by which\ncontinuous states and voltage modulate discrete transition probabilities. c. Importantly, the inter-compartmental\nvoltage dependencies are linear, as speci\ufb01ed by the cable equation.\n\nt \u2208 RD, a continuous latent state of dimension D. (We\nFor each compartment n, we introduce x(n)\nset D = 1 in all the examples shown below, but higher-dimensional dynamics are possible). Let\nt \u2208 {1, . . . , K} denote a corresponding discrete latent state; as we will see, these will correspond\nz(n)\nto di\ufb00erent phases of the action potential. Given the discrete state, the voltage and continuous latent,\n(cid:34)(cid:32)\n(V (n)\n\n)(cid:62), together follow linear dynamics,\n\n, x(n)\n\nt\n\nt\n\nE\n\nV (n)\nt+\u2206t\nx(n)\nt+\u2206t\n\n(cid:33)(cid:12)(cid:12)(cid:12)(cid:12) z(n)\n(cid:32)\n\n=\n\n(cid:110)\n\n(cid:33)\n\n(cid:111)\n\n)\n\n(cid:48)\n\n(cid:34)\n\nt = k,\n\nV (n\nt\n\nn(cid:48)(cid:54)=n\n\nV (n)\nt\nx(n)\nt\n\n+\n\n\u2206t\nCn\n\n(cid:35)\n(cid:32)\n\u2248(cid:80)\n\n(cid:33)\n\nA(n)\n\nk\n\n(cid:124)\n\nV (n)\nt\nx(n)\nt\n\n(cid:123)(cid:122)\n\n+ b(n)\n\nk\n\n+\n\n(cid:125)\n\nj I (n,j)\n\nt\n\n(cid:32)(cid:80)N\nn(cid:48)=1 gnn(cid:48) \u00b7(cid:16)\n\n(cid:17)\n\n(cid:33)(cid:35)\n\n.\n\n(cid:48)\n\n)\n\n\u2212 V (n)\n\nt\n\nV (n\nt\n0\n\nk \u2208 R(D+1)\u00d7(D+1) and the bias vector b(n)\n\ncompartment currents(cid:80)\n\nk \u2208 RD+1 parameterize linear\nThe dynamics matrix A(n)\ndynamical systems for each discrete state in each compartment. We further assume additive Gaussian\ndynamics noise for each compartment and discrete state with covariance Q(n)\nk . Importantly, the voltage\ndynamics retain the inter-compartmental linear terms from the cable equation, linking connected\ncompartments in the dendritic tree (inset of Fig. 1). On the other hand, the nonlinear summed intra-\nare replaced with a collection of piecewise linear dynamics on voltage\nand continuous states; by switching between these discrete linear dynamics, we can approximate the\nnonlinear dynamics of the original model (since any su\ufb03ciently smooth function can be approximated\nwith a piecewise-linear function).\nTo complete the dynamics model, we must specify the dynamics of the discrete states z(n)\nrSLDS, allowing the discrete states to depend on the preceding voltage and continuous states,\n\n. We use a\n\nj I (n,j)\n\nt\n\nt\n\n(cid:40)\n\n(cid:62)(cid:32)\n\n(cid:33)\n\n(cid:41)\n\np(z(n)\n\nt+\u2206t = k | V (n)\nThe red arrows in Fig. 1 highlight\nby {w(n)\n\nV (n)\nt\nx(n)\nt\nThe linear hyperplanes de\ufb01ned\nk=1 de\ufb01ne a weak partition of the space of voltages and continuous latent states.\n\nthese dependencies.\n\n) \u221d exp\n\nk , d(n)\n\nk }K\n\n+ d(n)\n\n, x(n)\n\nw(n)\n\n(2)\n\nk\n\nk\n\n.\n\nt\n\nt\n\n3\n\nz(n)t+1......y(n)t+1x(n)t+1z(n)tx(n)ty(n)ty(n)Tx(n)Tz(n)Tz(n)1x(n)1y(n)1\u03b8(n)timeV(n)t+1V(n)tV(n)TV(n)1Compartment z()t+1n......y(n)t+1x(n)t+1z()tnx(n)ty(n)ty(n)Tx(n)Tz()Tnz()1nx(n)1y(n)1\u03b8()timeV(n)t+1V(n)tV(n)TV(n)1Compartment parametersdiscretestatescontinuousstatesvoltagenoisyvoltageobservationsnnnnnMulti-compartment neuronsomadendriteslinear dynamics(see inset)V(n)t+1V(n)tV(n)TV(n)1V(n)t+1V(n)tV(n)TV(n)1Inset: linear cable eqn. dynamicstimecompartments........................a.b.c.\fAs the magnitude of the weight vectors increases, the partition becomes more and more deterministic.\nIn the in\ufb01nite limit, the discrete states are fully determined by the voltage and continuous states, and\nthe switching linear dynamical system becomes a piecewise linear dynamical system [22]. We \ufb01nd\nthat these models admit tractable learning and inference algorithms, and that they can provide a good\napproximation to the nonlinear dynamics of membrane potential.\n\nt \u223c N (V (n)\n\nt\n\n2\n\n, \u03c3(n)\n\nk\n\nk }K\n\nk , b(n)\n\nk , w(n)\n\nk , d(n)\n\nk , \u03c3(n)\n\nk , Q(n)\n\nFinally, we observe noisy samples of the voltage y(n)\n) for each compartment,\nwith state-dependent noise. Our goal is to learn the parameters of the multi-compartment\nrSLDS, \u0398 = {{\u03b8(n)}N\nn=1, G}, where \u03b8(n) = {A(n)\nk=1. Given the\nlearned parameters, we seek a Bayesian estimate of the voltage given the noisy observations.\n3 Bayesian learning and inference\nThe recurrent switching linear dynamical system (rSLDS) inherits some of the computational advan-\ntages of the standard switching linear dynamical system (SLDS); namely, the conditional distribution\nof the discrete states given the continuous states is a chain-structured discrete graphical model, and\nmost model parameters admit conjugate updates. However, the additional dependency in (2) breaks the\nlinear Gaussian structure of the conditional distribution on voltage and continuous latent states.\nFollowing previous work [23, 24], we use P\u00f3lya-gamma augmentation [25] to render the model\nconjugate and amenable to an e\ufb03cient Gibbs sampling algorithm. Brie\ufb02y, we introduce augmentation\nvariables \u03c9(n)\ntk for each compartment, discrete state, and time bin. After augmentation, the voltage and\ncontinuous states are rendered conditionally linear and Gaussian; we sample them from their complete\nconditional with standard message passing routines. Moreover, the augmentation variables are\nconditionally independent of one another and P\u00f3lya-gamma distributed; we appeal to fast methods [26]\nfor sampling these. The discrete states retain their chain-structured conditionals, just as in a hidden\nMarkov model. Finally, the model parameters all admit conjugate Gaussian or matrix-normal inverse\nWishart prior distributions as in [23]; we sample from their complete conditionals.\nOne algorithmic choice remains: how to update across compartments? Note that all of the voltages and\ncontinuous latent states are jointly Gaussian given the discrete states and the augmentation variables.\nMoreover, within single compartments, the variables follow a Gaussian chain-structured model (i.e., a\nKalman smoother model). We leverage this structure to develop a block-Gibbs sampling algorithm that\njointly updates each compartment\u2019s voltage and continuous latent trajectories simultaneously, given\nthe discrete states, augmentation variables, and the voltages of neighboring compartments2.\n4 Experimental Results\nWe evaluate the spatiotemporal recurrent SLDS and the corresponding inference algorithms on a\nvariety of semi-synthetic datasets. First, we show that we can model real voltage recordings from\na single compartment. Then, we evaluate performance on multi-compartment models, including a\nsimulated dendritic branch and a real dendritic tree morphology.\n4.1 A single-compartment rSLDS is a useful model of real somatic voltage\nAs a \ufb01rst test of the model and algorithm, we use intracellular voltage traces from the Allen Institute\nfor Brain Science Cell Types Atlas [27, cell id=464212183]. The traces are recorded via patch clamp,\na high signal-to-noise (SNR) method that provides \u201cground truth\u201d measurements. We added arti\ufb01cial\nwhite noise with a standard deviation of 5mV to these recordings in order to test the model\u2019s ability to\nboth learn membrane potential dynamics and smooth noisy data. We \ufb01t the rSLDS to 100ms of data\nsampled at \u2206t = 0.1ms, for a total of 1000 time points. We used three discrete latent states (K = 3)\nand one additional latent dimension (D = 1); already, this simple model is su\ufb03cient to provide a\ngood model of the observed data. Figure 2(a) shows the observed voltage (i.e. the ground truth plus\nwhite noise), and the 95% posterior credible intervals estimated with samples from the posterior under\nthe estimated rSLDS model. The inferred discrete state sequence (c) shows how the rSLDS segments\nthe voltage trajectory into periods of roughly linear accumulation (blue), spike rise (yellow) and spike\nfall (red). These periods are each approximated with linear dynamics in (V, x) space, as shown in\n2In principle, all compartments\u2019 continuous latent states and voltages could be updated at once, given that\nthey are conditionally jointly Gaussian; however, the computational cost would naively scale as O(T (N D)3),\nbased on a Kalman forward-backward sweep. An important direction for future work would be to adapt the fast\napproximations proposed in [14] to implement this joint update more e\ufb03ciently.\n\n4\n\n\fFigure 2: A single-compartment, three-state rSLDS provides an adequate model of real somatic voltage responses.\n(a) Observed voltage (formed by adding noise to the intracellular voltage from a real neuron) and the credible\ninterval (CI) output by a the estimated rSLDS model. (b) Inferred continuous latent state x(t) corresponding\nto this trial. (c) Inferred discrete latent state z(t). (d) Inferred two-dimensional dynamics. The blue state\n(corresponding to z = 0 in (c)) is a \ufb01xed point at the rest potential; the yellow state corresponds to a fast\ndepolarization (the upswing of the spike; z = 2) and the red state the hyperpolarization (the downswing of the\nspike; z = 1), followed by a return to the blue rest state. The thin traces indicate samples from x(t) and V (t)\ngiven the observed noisy voltage data. (e) Generative samples from the learned rSLDS; the di\ufb00erence between\nthese traces and those shown in the previous panel is that these traces are generated using the learned rSLDS\nparameters without conditioning on the observed noisy voltages {yt}, whereas in (d) we show samples from\nthe posterior given {yt}; the fact that the two sets of traces are similar is a useful check that the model \ufb01ts have\nconverged. (f) Voltages sampled from the rSLDS prior, corresponding to traces shown in (e). Note that the simple\nthree-state two-dimensional rSLDS is able to learn to produce reasonably accurate spike shapes and \ufb01ring rates.\n\nFig. 2(d). Moreover, simulating from the learned dynamics yields realistic trajectories in both latent\nspace (e) and in voltage space, as shown in Fig 2(f).\n\nFigure 3 compares the rSLDS to a simpler baseline method. Past work in voltage smoothing has utilized\nthe Kalman smoother, based on an assumption of approximately linear dynamics in the subthreshold\nregime [14]. Notably, the Kalman smoother is a special case of the rSLDS with K = 1 discrete state.\nWe compare performance over a range of noise levels and \ufb01nd that the rSLDS signi\ufb01cantly outperforms\nthe standard Kalman smoother, due largely to the fact that the former can adapt to the drastically\ndi\ufb00erent smoothness of the voltage signal in the spiking versus the subthreshold regimes.\n\n5\n\n604020020V (mV)Observed Voltage & Inferred Continuous Latent State X1Observed voltageCIInferred Continuous Latent State X201020304050607080t (ms)012Inferred Discrete Latent State ZVXInferred Dynamics (rSLDS)VXGenerated States020406080t (ms)502502550V (mV)Samples Drawn from Trained rSLDSSample 1Sample 2Sample 3(a)(b)(c)(d)(e)(f)\fFigure 3: Comparing the Kalman smoother against the rSLDS. Left: example noisy data (constructed by adding\nnoise to real voltage data) and output of the Kalman smoother (top) and rSLDS (bottom). Right: summary of\nMSE as a function of observation noise variance. The rSLDS outperforms the Kalman smoother, because the\nlatter is a linear \ufb01lter with a single homogeneous smoothness level, and therefore it must either under\ufb01t spikes or\nover\ufb01t subthreshold voltage (or both), whereas the rSLDS can enforce di\ufb00erent levels of smoothness in di\ufb00erent\ndynamical regimes (e.g., spiking versus non-spiking).\n\nFigure 4: Voltage smoothing in noisy recordings in a simulated dendritic branch. The branch consists of \ufb01ve\ncompartments connected in a chain. Left: The observed voltage (black line) and the inferred voltage (colored line)\nare shown for each compartment. Right: A sample from the learned multi-compartment rSLDS. The generated\nvoltage traces show that the model has learned to reproduce the nonlinear dynamics of multi-compartment\nmodels, including the interactions between compartments that propagate spikes down the dendritic branch.\n\n4.2 Spatiotemporal denoising with simulated data on real morphologies\nThe single compartment studies a\ufb00ord us ground truth electrical recordings, but obtaining simultaneous\nelectrical recordings from a multiple compartments of a single cell is a signi\ufb01cant technical challenge.\nOptical recordings of \ufb02uorescent voltage indicators o\ufb00er multiple compartments, but at the cost of\nlower temporal resolution and signi\ufb01cantly higher noise. To test our method\u2019s ability to denoise\nmulti-compartment voltage traces, we simulate voltage traces using the simple HH model in lieu of\nground truth electrical recordings3. We start by simulating a single dendritic branch and then move to\na full dendritic tree, using real neuron morphologies.\n\n3Of course more detailed realistic simulations with multiple nonlinear channel types are possible; we leave\nthis for future work. As we will see below, the simple HH model already provides a rich and interesting testbed\nsimulated dataset for the methods presented here.\n\n6\n\n204060801001208060402002040Voltage (mV)Kalman Smoother (noise level = 0.7)ObservedInferredTrue20406080100120t (ms)8060402002040Voltage (mV)rSLDSObservedInferredTrue0.10.20.30.40.50.60.7Noise level (mV)0123MSE (mV/time point)MSEK=3K=1n = 1Inferred - denoisingn = 2n = 3n = 4n = 5-8040040 (mV)Generation-8040040 (mV)-8040040 (mV)-8040040 (mV)-8040040 (mV)\fFigure 5: Spatiotemporal denoising on a large simulated dendritic tree. The 3D morphology of the tree is\ntaken from a real cell in the Allen institute database. (a) Simulated voltage with additive Gaussian noise (25mV\nstd.) on the dendritic tree. (b) True simulated voltage without noise. (c) Denoised estimate of voltage with\nthe multi-compartment rSLDS; note the close match with (b). Colorbar shared between (a), (b), and (c). (d)\nResidual equals observed (a) minus inferred (c). (e) Error equals (b) minus (c). (f) Cartoon \ufb01gure indicates the\nsingle compartment (green dot) and a segment of dendritic tree (red branch) shown in the following panels. (g)\nNoisy observation (black) and posterior credible interval (CI) of inferred voltage (cyan), and true voltage (red)\ncorresponding to dot in (f). (h) The inferred discrete state of the compartment (0: subthreshold; 1: spike fall; 2:\nspike rise). (i, j, k) spatiotemporal representation of the noisy, true, and denoised voltage propagating up the\nbranch shown in (f). (l, m) are the residual and error computed from (i, j, k). See Video link for full details.\n\n7\n\n(a). Observed(b). True(c). inferred15012510075502502550(d). Residue = (a) - (c)6040200204060(e). Error = (b) - (c)6040200204060(f) Dendritic tree1000V (mV)(g) Inferred VObsTrueCI012(h) Inferred Zinferred z(i) ObservedCompartment ID(j) True(k) Inferred(l) Residue = (i) - (k)020406080100t (ms)(m) Error = (j) - (k)150125100755025025505005050050\fDenoising a dendritic branch.\nWe model a single dendritic branch as a \ufb01ve-compartment chain. We simulate membrane potential\naccording to the classical HH model with voltage-gated sodium and potassium channels, a leak current,\nand currents from neighboring compartments in the chain. We inject a constant 35mA current into\nthe \ufb01rst compartment, and induced spikes propagate to downstream compartments according to the\ncable equation. However, we have tuned the inter-compartment conductances gnn(cid:48) such that only\nevery other spike propagates \u2014 a single spike in compartment 1 cannot depolarize compartment 2\nenough to reach the spike threshold, but two spikes can. Moreover, we corrupt the voltage traces with\nadditive Gaussian noise with standard deviation of 8mV. We test the multi-compartment rSLDS\u2019s\nability to mimic the nonlinear dynamics of the true generative process and smooth the noisy voltage\nobservations.\n\nFigure 4 shows the \ufb01ve compartments in the chain. In the left column we show the observed voltage\ntraces (black) and the smoothed voltage (color) traces. Again, the model does an excellent job\ndenoising the data, and is also able to learn a generative model of the rich spatiotemporal dynamics.\nIn particular, the model does not simply learn separate models for each compartment \u2014 it also learns\ninteractions between compartments. This is evident in the traces that are generated by the model,\nas shown in the right column of Fig. 4. The \ufb01rst compartment shows a high \ufb01ring rate, but only\napproximately every other spike propagates to downstream compartments, as in the real data. The\ngeneration is not perfect: some spikes fail to propagate (e.g. the third spike in compartment 3 does\nnot propagate to compartment 4), and we see some spurious discrete state transitions (data not shown).\nNevertheless, these generated samples indicate that the learned dynamical system captures the gross\nstructure of multi-compartmental membrane dynamics. An accurate generative model o\ufb00ers a strong\nprior for smoothing spatiotemporal noisy voltage traces given by optical recordings.\n\nDenoising a full dendritic tree.\nWe have shown that the rSLDS can learn the dynamics of single compartments and dendritic branches\n(i.e. multi-compartment chains), but can these methods scale to full dendritic trees? We test these\nmodels on real three-dimensional dendritic tree [27, cell id=464212183]. (Note that this morphological\ndata only includes the three-dimensional shape, not any voltage recordings.) Of the 2620 compartments\nin the original tree, we retain approximately every \ufb01fth compartment to create a tree with 519\ncompartments (Fig. 5f). We simulate the HH model on this tree to obtain 100ms of data at 10kHz\ntemporal resolution. As above, we add Gaussian white noise (uncorrelated in space and time; standard\ndeviation 25mV).\n\nFig. 5a shows the observed voltage across all spatial compartments for a single time point, and panel\n(b) shows the true underlying voltage. Supplementary Video 1 shows the voltage propagating in time\nthrough the tree. Panel (c) shows the voltage inferred by the multi-compartment rSLDS. The residual\nin panel (d) and the error in (e) show the di\ufb00erence between the inferred voltage and the observed\nand true voltage, respectively. There is a slight spatial correlation in the errors \u2014 speci\ufb01cally, the\ninferred voltage tends to slightly underestimate spike amplitude and overestimate the voltage during\nrecovery \u2014 but the errors are generally small. Panel (g) shows the temporal estimates for the single\ncompartment indicated by the green dot in (f). The posterior credible interval captures the true voltage,\ndespite the high level of noise. Each spike corresponds to a canonical discrete state sequence, as\nin shown in (h). By transitioning between these discrete states, the piecewise linear dynamics aid\nthe model-based denoising. (As in Figure 3, we \ufb01nd that the K = 1 Kalman smoother tends to\nundersmooth the data here; results not shown.) Finally, panels (i-k) show both spatial and temporal\ndynamics of voltage for the dendritic branch highlighted in red in (f). We see the spikes propagating\nalong the compartments in observed, true, and inferred voltage. Again, the residuals (l) and errors\n(m) show small spatiotemporal correlations, but overall good recovery of the voltage from the noisy\nobservations.\n\nFor comparison, previous particle \ufb01ltering-based approaches to voltage smoothing [15] would need\nto infer the trajectories of \u2248 2000 state variables (N = 500 compartments with D = 4 dimensions\neach) in the simplest HH model of this data, and this model would not be able to adapt to modest\nchanges in the dynamical parameters of the channels in each compartment. Scaling such methods\nto this number of state variables is a serious challenge, but the recurrent switching linear dynamical\nsystem approach makes this problem tractable.\n\n8\n\n\f5 Conclusion\nThe advent of new voltage imaging methods presents exciting opportunities to study computation in\ndendritic trees. We have developed new methods to smooth and denoise optical voltage traces in order\nto realize this potential. These methods incorporate biophysical knowledge in the form of constraints\non the form of inter-compartmental dynamics, while allowing for e\ufb00ectively nonparametric learning\nof nonlinear intra-compartmental dynamics. We have illustrated the potential of these methods using\nsemi-synthetic data based on real electrophysiological recordings of membrane potential and actual\nneuron morphologies. The results show promising progress toward the critical goal of denoising\nnoisy spatiotemporal voltage measurements.\nAcknowledgements\nWe gratefully acknowledge support from the ONR and Simons Foundation, and from NIH grants\n1U01NS103489-01, 1U19NS113201, and EB22913.\nReferences\n[1] Daniel R Hochbaum, Yongxin Zhao, Samouil L Farhi, Nathan Klapoetke, Christopher A Werley,\nVikrant Kapoor, Peng Zou, Joel M Kralj, Dougal Maclaurin, Niklas Smedemark-Margulies, et al.\nAll-optical electrophysiology in mammalian neurons using engineered microbial rhodopsins.\nNature methods, 11(8):825, 2014.\n\n[2] Jesse D Marshall, Jin Zhong Li, Yanping Zhang, Yiyang Gong, Fran\u00e7ois St-Pierre, Michael Z\nLin, and Mark J Schnitzer. Cell-type-speci\ufb01c optical recording of membrane voltage dynamics\nin freely moving mice. Cell, 167(6):1650\u20131662, 2016.\n\n[3] Hiroaki Miyazawa, Kanoko Okumura, Kanae Hiyoshi, Kazuhiro Maruyama, Hisaya Kakinuma,\nRyunosuke Amo, Hitoshi Okamoto, Kyo Yamasu, and Sachiko Tsuda. Optical interrogation of\nneuronal circuitry in zebra\ufb01sh using genetically encoded voltage indicators. Scienti\ufb01c reports, 8\n(1):6048, 2018.\n\n[4] Ahmed S Abdelfattah, Takashi Kawashima, Amrita Singh, Ondrej Novak, Hui Liu, Yichun\nShuai, Yi-Chieh Huang, Luke Campagnola, Stephanie C Seeman, Jianing Yu, et al. Bright and\nphotostable chemigenetic indicators for extended in vivo voltage imaging. Science, 365(6454):\n699\u2013704, 2019.\n\n[5] Simon Chamberland, Helen H Yang, Michael M Pan, Stephen W Evans, Sihui Guan, Mariya\nChavarha, Ying Yang, Charleen Salesse, Haodi Wu, Joseph C Wu, et al. Fast two-photon imaging\nof subcellular voltage dynamics in neuronal tissue with genetically encoded indicators. eLife, 6:\ne25690, 2017.\n\n[6] Madhuvanthi Kannan, Ganesh Vasan, Cheng Huang, Simon Haziza, Jin Zhong Li, Hakan Inan,\nMark J Schnitzer, and Vincent A Pieribone. Fast, in vivo voltage imaging using a red \ufb02uorescent\nindicator. Nature methods, 15(12):1108, 2018.\n\n[7] Kiryl D Piatkevich, Seth Bensussen, Hua-an Tseng, Sanaya N Shro\ufb00, Violeta Gisselle Lopez-\nHuerta, Demian Park, Erica E Jung, Or A Shemesh, Christoph Straub, Howard J Gritton, et al.\nPopulation imaging of neural activity in awake behaving mice. Nature, pages 413\u2013417, 2019.\n[8] Yoav Adam, Jeong J Kim, Shan Lou, Yongxin Zhao, Michael E Xie, Daan Brinks, Hao Wu,\nMohammed A Mostajo-Radji, Simon Kheifets, Vicente Parot, et al. Voltage imaging and\noptogenetics reveal behaviour-dependent changes in hippocampal dynamics. Nature, 569(7756):\n413\u2013417, 2009.\n\n[9] S Ramon Cajal. Histologie du syst\u00e8me nerveux de l\u2019Homme et des vert\u00e9br\u00e9s. Maloine (Paris),\n\n2:891\u2013942, 1911.\n\n[10] Greg Stuart, Nelson Spruston, and Michael H\u00e4usser. Dendrites. Oxford University Press, 2016.\n[11] Henry C Tuckwell. Introduction to theoretical neurobiology: Nonlinear and stochastic theories,\n\nvolume 2. Cambridge University Press, 1988.\n\n[12] Allan L Hodgkin and Andrew F Huxley. Currents carried by sodium and potassium ions through\nthe membrane of the giant axon of Loligo. The Journal of physiology, 116(4):449\u2013472, 1952.\n\n9\n\n\f[13] Christof Koch. Biophysics of computation: Information processing in single neurons. Oxford\n\nuniversity press, 2004.\n\n[14] Liam Paninski. Fast Kalman \ufb01ltering on quasilinear dendritic trees. Journal of computational\n\nneuroscience, 28(2):211\u2013228, 2010.\n\n[15] Quentin JM Huys and Liam Paninski. Smoothing of, and parameter estimation from, noisy\n\nbiophysical recordings. PLoS computational biology, 5(5):e1000379, 2009.\n\n[16] Liang Meng, Mark A Kramer, and Uri T Eden. A sequential Monte Carlo approach to estimate\n\nbiophysical neural models from spikes. Journal of neural engineering, 8(6):065006, 2011.\n\n[17] Jan-Matthis Lueckmann, Pedro J Goncalves, Giacomo Bassetto, Kaan \u00d6cal, Marcel Nonnen-\nmacher, and Jakob H Macke. Flexible statistical inference for mechanistic models of neural\ndynamics. In Advances in Neural Information Processing Systems, pages 1289\u20131299, 2017.\n\n[18] Christian Naesseth, Scott Linderman, Rajesh Ranganath, and David Blei. Variational Sequential\nMonte Carlo. In Proceedings of the 21st International Conference on Arti\ufb01cial Intelligence and\nStatistics (AISTATS), pages 968\u2013977, 2018.\n\n[19] Chris J Maddison, John Lawson, George Tucker, Nicolas Heess, Mohammad Norouzi, Andriy\nMnih, Arnaud Doucet, and Yee Teh. Filtering variational objectives. In Advances in Neural\nInformation Processing Systems, pages 6573\u20136583, 2017.\n\n[20] Tuan Anh Le, Maximilian Igl, Tom Rainforth, Tom Jin, and Frank Wood. Auto-encoding\n\nsequential Monte Carlo. International Conference on Learning Representations, 2018.\n\n[21] Timothy O\u2019Leary, Alex H Williams, Jonathan S Caplan, and Eve Marder. Correlations in ion\nchannel expression emerge from homeostatic tuning rules. Proceedings of the National Academy\nof Sciences, 110(28):E2645\u2013E2654, 2013.\n\n[22] Eduardo Sontag. Nonlinear regulation: The piecewise linear approach. IEEE Transactions on\n\nAutomatic Control, 26(2):346\u2013358, 1981.\n\n[23] Scott W. Linderman, Matthew J. Johnson, Andrew C. Miller, Ryan P. Adams, David M. Blei,\nand Liam Paninski. Bayesian learning and inference in recurrent switching linear dynamical\nsystems. In Proceedings of the 20th International Conference on Arti\ufb01cial Intelligence and\nStatistics (AISTATS), 2017.\n\n[24] Josue Nassar, Scott Linderman, Monica Bugallo, and Il Memming Park. Tree-structured recurrent\nswitching linear dynamical systems for multi-scale modeling. International Conference on\nLearning Representations, 2019.\n\n[25] Nicholas G Polson, James G Scott, and Jesse Windle. Bayesian inference for logistic models\nusing P\u00f3lya-gamma latent variables. Journal of the American Statistical Association, 108(504):\n1339\u20131349, 2013.\n\n[26] Jesse Windle, Nicholas G Polson, and James G Scott. Sampling P\u00f3lya-gamma random variates:\n\nalternate and approximate techniques. arXiv preprint arXiv:1405.0506, 2014.\n\n[27] Allen Institute for Brain Science. Allen Cell Types Database, 2015. URL https://celltypes.\n\nbrain-map.org/.\n\n10\n\n\f", "award": [], "sourceid": 5367, "authors": [{"given_name": "Ruoxi", "family_name": "Sun", "institution": "Columbia University"}, {"given_name": "Scott", "family_name": "Linderman", "institution": "Columbia University"}, {"given_name": "Ian", "family_name": "Kinsella", "institution": "Columbia University"}, {"given_name": "Liam", "family_name": "Paninski", "institution": "Columbia University"}]}