The paper proposes a hierarchical recurrent state space model that can infer latent temporal structure and perform stochastic state transition hierarchically. The idea of jumpy navigation is quite novel and has potential to be adopted and inspire further research. However, the experimental evaluation is a bit weak, even after the additional experiments in the rebuttal.