NIPS 2018
Sun Dec 2nd through Sat the 8th, 2018 at Palais des Congrès de Montréal
Paper ID: 2387 Learning Confidence Sets using Support Vector Machines

Reviewer 1

Interesting paper on an interesting topic. Contains theoretical results, a numerical algorithm, and numerical experiments. Perhaps some additional references on surrogate loss functions and their risk should be mentioned on page 4, line 128.

Reviewer 2

This paper provides an SVM-based set-valued classifier under the framework of classification with confidence''. This framework indeed has a long history in the literature of statistical learning and has recently regained attention due to practical concerns of classification problems such as class imbalance, asymmetric cost of wrong classification, etc. Strength: This paper puts SVM under the framework of classification with confidence. The idea of providing classification confidence set is important and practically useful, but the current methods rely on conditional probability estimation, which is itself a hard problem and may not be necessary for classification. Instead, SVM is one of the most practically successful classification algorithms. Therefore, making SVM available in this new framework will greatly increase the usefulness of both SVM and the framework of classification with confidence. Moreover, this paper clearly explains and motivated the formulation of the new optimization algorithm and provides convincing theoretical justification of the methodology. Questions and Comments: The clarity of the theoretical section can be improved, for example, by providing some more discussions and examples of the assumptions. In particular, in Assumption 3, do $\nu_1$ and $\nu_{-1}$ need to be constants, can they vanish to 0? Also, the analysis focuses on the constrained version but the algorithm is the penalized version. It is known in other cases (such as penalized linear regression) that constrained versions are easier to analyze. Can the author(s) comment on the theoretical property of the penalized version? Finally, the simulation uses unrealistically large sample sizes. Smaller (for example, a few hundred) sample sizes may be more relevant. Writing: There are some missing punctuations, especially after math displays. For example: before line 11, line 81, line 145, line 148, line 160, etc.