Based on the level of disagreement in the initial reviews -- and due to concerns brought up about novelty of the approach -- I invited a fifth reviewer, with strong experience in the area, to review the paper. This review reinforced the novelty of the approach, and agreed with its usefulness. Overall, this is a paper that should be of interest to the Bayesian Optimization community. However, we strongly suggest that the authors take into account the reviewers' comments on clarity in preparing the final version. In addition to the points mentioned in the rebuttal, I would like to see discussion of the limitations of BART in this context, in particular regarding out-of-sample uncertainty estimates, as brought up by reviewer 5. While we do not expect the authors to have compared against concurrent work, we think it would improve the paper to discuss the related work mentioned by Reviewer 3.