NeurIPS 2020

Neural Bridge Sampling for Evaluating Safety-Critical Autonomous Systems

Meta Review

The paper proposes a method for empirical verification of safety system by developing an iterative method for sampling rare and potentially out of bounds system states. The reviewers agree that the strengths are in the novel and important area that is generally little worked in. The method is well founded, rigorously analyzed, and evaluated thoroughly. The methodology presented in the paper is general and applicable to other applications other than the safety, and can generate a lot of follow-up work. In the final version the authors should: - Address R1's concerns about ReLU non-reversibility and show the validity of the bridge-sampling hybrid algorithm empirically or analytically. - Consider positioning the method, and not the application, as the main contribution.