Partially Observed Markov Decision Processes
From Filtering to Controlled Sensing
$108.00 (C)
- Author: Vikram Krishnamurthy, Cornell University/Cornell Tech
- Date Published: March 2016
- availability: Available
- format: Hardback
- isbn: 9781107134607
$
108.00
(C)
Hardback
-
Covering formulation, algorithms, and structural results, and linking theory to real-world applications in controlled sensing (including social learning, adaptive radars and sequential detection), this book focuses on the conceptual foundations of partially observed Markov decision processes (POMDPs). It emphasizes structural results in stochastic dynamic programming, enabling graduate students and researchers in engineering, operations research, and economics to understand the underlying unifying themes without getting weighed down by mathematical technicalities. Bringing together research from across the literature, the book provides an introduction to nonlinear filtering followed by a systematic development of stochastic dynamic programming, lattice programming and reinforcement learning for POMDPs. Questions addressed in the book include: when does a POMDP have a threshold optimal policy? When are myopic policies optimal? How do local and global decision makers interact in adaptive decision making in multi-agent social learning where there is herding and data incest? And how can sophisticated radars and sensors adapt their sensing in real time?
Read more- Links theory to real-world applications in controlled sensing
- Brings together results from across the literature and from multiple different disciplines
- Includes several new topics, such as reciprocal processes, geometric ergodicity of the optimal filter, data incest and social learning
Customer reviews
Not yet reviewed
Be the first to review
Review was not posted due to profanity
×Product details
- Date Published: March 2016
- format: Hardback
- isbn: 9781107134607
- length: 488 pages
- dimensions: 254 x 180 x 25 mm
- weight: 1.1kg
- contains: 47 b/w illus. 5 tables
- availability: Available
Table of Contents
Preface
1. Introduction
Part I. Stochastic Models and Bayesian Filtering:
2. Stochastic state-space models
3. Optimal filtering
4. Algorithms for maximum likelihood parameter estimation
5. Multi-agent sensing: social learning and data incest
Part II. Partially Observed Markov Decision Processes. Models and Algorithms:
6. Fully observed Markov decision processes
7. Partially observed Markov decision processes (POMDPs)
8. POMDPs in controlled sensing and sensor scheduling
Part III. Partially Observed Markov Decision Processes:
9. Structural results for Markov decision processes
10. Structural results for optimal filters
11. Monotonicity of value function for POMPDs
12. Structural results for stopping time POMPDs
13. Stopping time POMPDs for quickest change detection
14. Myopic policy bounds for POMPDs and sensitivity to model parameters
Part IV. Stochastic Approximation and Reinforcement Learning:
15. Stochastic optimization and gradient estimation
16. Reinforcement learning
17. Stochastic approximation algorithms: examples
18. Summary of algorithms for solving POMPDs
Appendix A. Short primer on stochastic simulation
Appendix B. Continuous-time HMM filters
Appendix C. Markov processes
Appendix D. Some limit theorems
Bibliography
Index.-
General Resources
Find resources associated with this title
Type Name Unlocked * Format Size Showing of
This title is supported by one or more locked resources. Access to locked resources is granted exclusively by Cambridge University Press to instructors whose faculty status has been verified. To gain access to locked resources, instructors should sign in to or register for a Cambridge user account.
Please use locked resources responsibly and exercise your professional discretion when choosing how you share these materials with your students. Other instructors may wish to use locked resources for assessment purposes and their usefulness is undermined when the source files (for example, solution manuals or test banks) are shared online or via social networks.
Supplementary resources are subject to copyright. Instructors are permitted to view, print or download these resources for use in their teaching, but may not change them or use them for commercial gain.
If you are having problems accessing these resources please contact [email protected].
Sorry, this resource is locked
Please register or sign in to request access. If you are having problems accessing these resources please email [email protected]
Register Sign in» Proceed
You are now leaving the Cambridge University Press website. Your eBook purchase and download will be completed by our partner www.ebooks.com. Please see the permission section of the www.ebooks.com catalogue page for details of the print & copy limits on our eBooks.
Continue ×Are you sure you want to delete your account?
This cannot be undone.
Thank you for your feedback which will help us improve our service.
If you requested a response, we will make sure to get back to you shortly.
×