The paper provides some theoretical treatment of prioritized experience replay, and shows how the weighted sampling scheme can be viewed as minimizing a different loss function under the uniform sampling scheme. The main insight here is that since the weights are derived from the loss itself, there is some cancellation that changes the original loss function. The insights are used to derive two new algorithms which perform reasonably well in experiments. The paper is interesting, uses new theoretical insights to derive algorithms with competitive performance. As such, we recommend acceptance.