Summary and Contributions: This study develops an approach for inferring whether neurons are functionally connected (binary connection weights), from the observation of the stimulation of small neural ensembles.
Strengths: Soundness of the claims: With one exception (see Weaknesses), the claims are appropriately backed by empirical evaluation. Significance: The results suggest that the developed approach is a solid step towards applications in real neuroscience datasets, given the quality of the inference and the scalability in terms of numbers of inferred connections. Novelty: The novelty of the study is two fold: (1) it uses the framework of Group Testing for network inference; (2) it frames Group Testing as a Bayesian Inference problem. Relevance to the NeurIPS community: The methodology and results presented are of interest to the neuroscience community as well as the Bayesian Inference community and will inspire applications to real neuroscience data, and further method development.
Weaknesses: Soundness of the claims: The discussion about the relation between the proposed approach and previous network inference approaches (such as Aitchison et al. 2017) does not make full justice to these. E.g. the proposed approach is said to have multiple advantages compared to previous approaches (fewer statistical assumptions, ...), but without a proper quantitative comparison with previous methods, such claims should be toned down. Indeed, it could be that approaches relying for instance on generative models have better performance in particular real data settings. So, unless the authors compare their approach with these other methods in terms of specificity, sensitivity and scalability, I would recommend adding a sentence or two safeguarding the possibility that other approaches might be better in certain contexts.
Correctness: The claims and methodology are correct.
Clarity: The paper is clearly written, and provides enough information for an expert reader to understand all the steps to reproduce the results.
Relation to Prior Work: The manuscript clearly puts its contribution in the context of previous work in the field.
Reproducibility: Yes
Additional Feedback: -- After the rebuttal -- The authors' response addressed my comments to satisfaction, and I therefore maintain my support for the (clear) acceptance of this paper.
Summary and Contributions: This paper presents an approach to the problem of inferring a functional network across many neurons using noisy group testing. The authors formulate the connections across a population of neurons as a binary network that encodes the presence or absence of functional (not necessarily synaptic) connections between pairs of neurons, with a noisy Bernoulli observation model to capture neurons occasionally not being activated even when neurons they are functionally connected to are stimulated. Inference over the connections is initially formulated as a maximum likelihood problem, which can be rewritten as an integer optimization problem. The authors further extend this formulation by relaxing the variables to restricted continuous values and reformulating the problem as approximate Bayesian inference to infer the posterior probability of each connection. A dual decomposition algorithm is presented for solving the problem, which can be adapted to perform online inference in the setting where an experimenter might want to update the posterior probabilities as new tests are performed or adaptively select tests based on the current network estimate. Experiments on simulated data show the ability of this approach to correctly infer connections as more tests are performed for a range of assumed values of the false positive and negative rates. They also perform experiments that demonstrates the comparable performance of both online and batch inference approaches.
Strengths: The derivation of the problem formulation and algorithm for inferring posterior connection probabilities appears to be sound. In particular, the Bayesian interpretation/formulation that allows the estimation of uncertainty about network connections is an interesting extension and very useful in practice. The empirical evaluation is strong, with various versions of the method tested on a range of different simulated data scenarios, although with no application to real data (more on this below). The contribution seems novel and the authors clearly delineate the differences between theirs and other approaches to the same problem.
Weaknesses: As mentioned above, the primary limitation is the lack of experiments on real data. Particularly since the authors claim that the method is scalable to large datasets, a demonstration of this on a real dataset that shows plausible estimation of a network with a reasonable amount of computation would really strengthen the work.
Correctness: The method and empirical evaluation appear to be correct.
Clarity: Yes, the paper is overall well written.
Relation to Prior Work: Yes, the authors cite several functional connectivity estimation approaches in their introduction, including other group testing approaches like the one they present. In Section 2, they discuss other optimization methods for solving the exact problem they present and how their approach differs from past work. The comparisons to previous work are elaborated upon in the Discussion, with the highlighted differences of the presented approach being the lack of a specific parametric model, the ability to perform either exact or Bayesian inference with a probabilistic interpretation of the inferred parameters, and scalability to large numbers of neurons.
Reproducibility: Yes
Additional Feedback: UPDATE: After reviewing the authors' response, I believe my concerns have been adequately addressed. While I still think that the lack of real data experiments is the main weakness of the paper (and the authors seem to agree), I believe that the well-explained derivation of their approach as well as the extensive empirical evaluations on simulated data provide enough of a technical contribution to warrant acceptance. I have updated my score accordingly.
Summary and Contributions: Holographic photostimulation techniques have advanced to the stage it is possible to precisely stimulate selected ensembles of neurons. This paper asks how, mathematically, one should use this new experimental ability to efficiently infer functional connectivity between neurons. The algorithmic approach that is developed is based on noisy group testing and the result shows the number of tests (stimulations) grows logarithmically with population size. This is further extended to a streaming setting. Simulations corroborate theoretical findings. Note that the focus is on the decoding algorithm (which is connected to Bayesian inference) rather than the pooling scheme, which is basically a random design. After author response, I realize I missed their use of adaptive pooling designs, which improves my assessment of the paper.
Strengths: * This problem area is certainly relevant to the neuroscience wing of NeurIPS and has seen many previous papers * The mathematical connection to group testing seems novel: previous work in this area seems to have focused on connections to compressed sensing * Implementation of recovery algorithm via GPUs is interesting for a massive pooling setting as here
Weaknesses: * No experiments with real neuronal data. * No consideration of better pooling designs than random pooling * No consideration of inferring connection weights
Correctness: Paper is correct, as far as I can tell.
Clarity: Paper reads well.
Relation to Prior Work: There seems to be no discussion of compressed sensing approaches to a very closely related problem of multineuron stimulation, which is a somewhat glaring omission. See e.g. the papers [Hu, Leonardo, Chklovskii, "Reconstruction of sparse circuits using multi-neuronal excitation (RESCUME)", 2009] and [Fletcher, et al, "Neural reconstruction with approximate message passing (NeuRAMP)", 2011].
Reproducibility: Yes
Additional Feedback:
Summary and Contributions: This paper proposes a novel approach for inferring the functional connectivity in a network of neurons, which involves making an intervention (stimulation) of a selected group of neurons in the network, and testing which neurons changed state in response to the stimulation. The paper claims that the method scales well, primarily due to the group-based nature, especially in the sparse regime. A notable contribution of this work is how it formulates the inference problem for the connectivity: it is first written down in terms of the likelihood of group test results, then as a constrained optimization problem with relaxed variables, as a Bayesian inference problem, then again as an optimization problem for probability distribution in a variational Bayes. The optimization problem is analyzed in a rigorous statistical light. Finally, a simulated test demonstrates that the proposed method can recover a sparse network in a readily scalable way.
Strengths: This paper provides a fresh perspective to the important problem of functional connectivity inference. The subject is relevant to the broad neuroscience community, and also timely, given the increasing access to large-scale neural recordings. This paper will be of great interest to the NeurIPS community. The contribution is clearly novel, and its impact could be significant if (i) the proposed method can be implemented experimentally, with photostimulation or other forms of intervention, and (ii) an easy-to-use software package can be developed. The theoretical development of the inference method is very impressive; I feel that the paper could merit acceptance just for this, even without taking into account the potential practical benefit.
Weaknesses: The practical strength of the method should be tested through actual experimental implementations, which would be a future work. The proposed method requires certain conditions (statistical assumptions), and it may take a large number of stimulations, as discussed in the paper.
Correctness: The formulation of the group testing and the inference problems appear to be correct. I do not have the full expertise to assess the correctness of the optimization theory and the statistics involved, but the construction seems to be grounded on a well-established body of previous works.
Clarity: This paper is very well written; it is with an exceptional clarity that the method sections unfold, given all the technical complexity. The paper uses an effective story-telling style, with a series of goal-challenge-solution blocks. It also provides many insightful comments that helps appreciating the work in a broader context.
Relation to Prior Work: The paper does a great job of putting its contributions in context with the previous works. Such contexts are provided throughout the paper almost every time a modeling choice is made (by mentioning the alternatives); several paragraph in the discussion section is then dedicated to discussing prior works in the two related directions, group testing and functional connectivity inference.
Reproducibility: Yes
Additional Feedback: Great work --- it was a stimulating read. I wonder if the authors are thinking about an experimental implementation of this method; can you comment if you are working on a follow-up? And what would be the potential problems if this method was to be made more broadly available in many neurophysiology labs, and for different brain areas? ======= ** Edit ** The authors response addressed my concern about the experimental validation of the method, the major limitation of this paper as agreed among the reviewers. I remain enthusiastic about this work despite the limitation, and I also look forward to seeing follow-ups in the future.