NeurIPS 2020

Sanity-Checking Pruning Methods: Random Tickets can Win the Jackpot

Meta Review

This paper proposes a series of simple sanity checks to evaluate whether winning lottery tickets depend on information from the training data and whether the layerwise pruning statistics account for improvements of winning tickets. Finding that many pruning methods don't pass these two "sanity checks", the authors propose a series of data-independent pruning ratios which work across a number of settings. Overall, reviewers found the paper to generally be well-executed and the results to be interesting, though there were some concerns about the lack of iterative pruning and a lack of large-scale results. In the rebuttal, the authors described the results of their experiments with iterative pruning and promised to perform a large-scale analysis of their data-independent ratios. Reviewers were generally satisfied with these results, but I would strongly encourage the authors to include these experiments (with plots, which were lacking from the rebuttal) in the final paper. I therefore recommend that this paper should be accepted as a poster.