NeurIPS 2020

Neuron-level Structured Pruning using Polarization Regularizer

Meta Review

This paper proposes a new channel pruning approach, polarization regularization, which attempts to scale some units to 0 and others to some value > 0, in contrast to L1 which pushes everything to 0. Reviewers consistently found the ideas behind this approach interesting and more than one described it as "simple yet effective." However, reviewers had a number of concerns, primarily regarding hyperparameter tuning, comparisons to uniform pruning, and questions about method performance with <50% FLOPs. However, both reviewers found the author rebuttal compelling wrt all of these points, and I agree. I therefore recommend that this paper be accepted.