Exact Solutions to Time-Dependent MDPs

Part of Advances in Neural Information Processing Systems 13 (NIPS 2000)

Bibtex Metadata Paper


Justin Boyan, Michael Littman


We describe an extension of the Markov decision process model in which a continuous time dimension is included in the state space. This allows for the representation and exact solution of a wide range of problems in which transitions or rewards vary over time. We examine problems based on route planning with public trans(cid:173) portation and telescope observation scheduling.