+ I really enjoyed reading this paper. It's a well-written work. The technical foundation about polarimetric imaging sounds solid. I cannot find out any serious issue in their image formation model in polarimetric imaging. The first neural network only estimates the surface normal of the reflective glass plane as \alpha and \beta, and then the image formation model allows for polarization-guided separation, resulting in reflection and transmission layers via the second network of a pair of encoder and decoder. These two networks are optimized in an end-to-end manner. + They found that the gradient loss of the estimated reflection and transmission layers is effective. Their ablation study shows that having both the intensity and gradient losses is the most effective option. It outperforms a state-of-the-art method in this field, ReflectNet, with almost 1dB of PSNR. + In addition, the proposed method is applied to a real polarimetric camera, Lucid. The proposed method was also tested with real images, where unpolarized images were obtained by adding a pair of orthogonal polarimetric images. + I don't have any critical comments on this work. Strongly, I would like to recommend accepting this work for NeurIPS. - Figures 3 and 4 are too small to evaluate the qualitative performance of separation. It would be better to have some closeup figures of results.

This paper proposes a deep-learning based reflection separation approach that uses only a pair of unpolarized and polarized images. Their network combines polarization physics in terms of a semi-reflector orientation that defines the polarization effect, assuming planar semi-reflector geometry and no motion in between the image pair. Their method performs favorably over RefletNet [22] that is fine-tuned on their planar, stationary dataset. The approach is original in terms of the three points stated in Q1 above. Especially, using a simple pair of unpolarized and polarized images is a great advantage compared to using three polarized images separated in certain angular differences: The scene may change in time while capturing multiple images, which breaks the physical assumption that would cause critical errors during the process. In addition, controlling the polarizer manually in exact angles is hard and very inconvenient, which brings a need of complicated (possibly expensive) hardware for automatic control, such as the experimental vision camera used in this work. It seems that the proposed method performs slightly better than RefletNet on planar semi-reflector without dynamic motion. However, to be certain they need to improve the experiments further. Please see Q5 for the list of required improvements.

Originality The setting and proposed solution is novel to reflection separation. + The paper proposes a new setting for reflection separation using a pair of polarized and unpolarized images. The setting is easy to capture in practice. Similar to [22], the setting provides polarization cues for this task. + The paper formulates the problem based on the physical model, and presents that the relection separation depends on the plane parameters under the assumption of planar medium. Based on the model, the authors propose to use a network to regress the plane parameters. Quality + The problem formulation and the derivation onto plane parameter estimation is reasonable and technically sound. The estimation of two parameters enables light network and facilitates training. + The proposed method achieves promising results compared with other deep-learning-based methods on the synthetic dataset and real images, which clearly show the benefits of the proposed method. + The authors show ablation study on directly estimating xi and zeta, and gradient loss. The ablation show that the proposed design on these are effective. - It is better to discuss limitations of the proposed method. For exmaple, when the assumption of planar medium does not hold? It is better to discuss and provide examples on this common case. - Is image noise added in the synthetic data generation? It is better to provide algorithm analysis on noise sensitivity, e.g., with Gaussian noise at different levels. - Since the proposed method include a refinement process, it is suggested to provide quantitative and qualitative results before the refinement to show how the method works before refinement network. Clarity The paper is clearly written and provides enough details for reproduction. Significance The setting and formulation are useful and inspire future research along this direction.