The paper formulates a budgeted Markov decision process (BMDP) able to deal with large search spaces. All reviewers feel the proposed method is novel, interesting and could be an important step in trying to address some existing problems with "modern" RL approaches.