NeurIPS 2019
Sun Dec 8th through Sat the 14th, 2019 at Vancouver Convention Center
Paper ID:8201
Title:Implicit Regularization of Accelerated Methods in Hilbert Spaces

This paper investigates Nesterov's acceleration technique and early stopping problem for optimization in RKHS for a nonparametric regression problem. It is shown that the acceleration technique induces earlier optimal stopping time to achieve the optimal generalization error (minimax optimal rate). This paper deals with an interesting problem and the analysis is novel. The result is interesting because the acceleration technique might cause instability but the result of this paper gives a proper answer to this issue. The analysis given in this paper will stimulate further researches in that direction.