NeurIPS 2020

Curriculum By Smoothing

Meta Review

The reviews of this paper are positive overall. This paper presents a curriculum based approach that smoothes the features with a Gaussian kernel in order to mitigate the impact of noisy features can have when training a CNN, especially during the initial stages of training. The proposed approach is evaluated on several tasks: transfer learning, generative models, image classification, representation learning, etc. The reviewers appreciated the simplicity and the effectiveness of the proposed approach. A reviewer comments that the proposed approach is 'simple, clearly presented, can easily be added to any architecture' and that 'no hyperparameter tuning is added, which is a clear benefit'. A reviewer observed that 'the code is well-written' and 'easy to use'. The authors submitted a response to the reviewers' comments. After reading the response, updating the reviews, and discussion, the reviewers feel that the authors provide 'extensive results', present 'an ablation study to understand their approach', and that 'the paper has sufficient merit'. The AC decision making was based on constructive comments only. We recommend to take the reviewers' comments and suggestions into account while preparing the final version of the paper. Recommendation for spotlight. Curriculum learning is a highly popular topic in machine learning and beyond. Yet, as of today, before this paper, no method seemed to stand out as both simple and practical. This paper proposes such a method. The proposed method has a high potential for a broad impact because of its striking mathematical and numerical simplicity and its excellent performance on several tasks. Indeed, as R1 says, 'the approach is evaluated empirically on several tasks, showing that it is applicable in a varied scenarios’. Moreover, as R1 and R3 say, ‘the code is well-written and seems easy to use’. All in all, this paper makes an important contribution to the field, with a (finally) simple and clear approach for curriculum learning. Accept.