1. Introduction / Eugene A. Feinberg and Adam Shwartz -- Pt. 1. Finite State and Action Models -- 2.
... Finite State and Action MDPs / Lodewijk Kallenberg -- 3. Bias Optimality / Mark E. Lewis and Martin L. Puterman -- 4. Singular Perturbations of Markov Chains and Decision Processes / Konstantin E. Avrachenkov, Jerzy Filar and Moshe Haviv -- Pt. II. Infinite State Models -- 5. Average Reward Optimization Theory for Denumerable State Spaces / Linn I. Sennott -- 6. Total Reward Criteria / Eugene A. Feinberg -- 7. Mixed Criteria / Eugene A. Feinberg and Adam Shwartz -- 8. Blackwell Optimality / Arie Hordijk and Alexander A. Yushkevich -- 9. Poisson Equation for Countable Markov Chains: Probabilistic Methods and Interpretations / Armand M. Makowski and Adam Shwartz -- 10. Stability, Performance Evaluation, and Optimization / Sean P. Meyn -- 11. Convex Analytic Methods in Markov Decision Processes / Vivek S. Borkar -- 12. Linear Programming Approach / Onesimo Hernandez-Lerma and Jean B. Lasserre -- 13. Invariant Gambling Problems and Markov Decision Processes / Lester E. Dubins, Ashok P. Maitra and William D. Sudderth -- Pt. III. Applications -- 14. Neuro-Dynamic Programming: Overview and Recent Trends / Benjamin Van Roy -- 15. Markov Decision Processes in Finance and Dynamic Options / Manfred Schal -- 16. Applications of Markov Decision Processes in Communication Networks / Eitan Altman -- 17. Water Reservoir Applications of Markov Decision Processes / Bernard F. Lamond and Abdeslem Boukhtouta. Read More