NeurIPS 2020

Logarithmic Pruning is All You Need

Meta Review

The paper shows that for some a target ReLU network F and a larger (overparametrized) network G, there exists a subnetwork of G of size greater than that of F by a factor logarithmic in all parameters of G except depth (possibly linear) that, without training, is a predictor that is with high probability close to F for all inputs. This paper improves on the the loss in the size of the network found from polynomial factor to a logarithmic. The reviewers found this an important paper studying the strong lottery ticket hypothesis. I recommend it for acceptance.