References
Dynamic programming (DP) is a fairly powerful and yet general framework that finds its use in many disciplines. Optimal control is not the only one. But in this overview of the literature we deliberately focus on the DP references with optimal control flavour.
Our introductory treatment was based almost exclusively on the (also just introductory) Chapter 6 in [1]. Electronic version of the book is freely available on the author’s webpage.
Comparable introduction is provided in [2]. Although it does not appear to be legally available for free in an electronic form, its reprint by a low-cost publisher makes it an affordable (and recommendable) classic reference. Another classic [3] actually uses dynamic programming as the key technique to derive all those LQ-optimal regulation and tracking results. A few copies of this book are available in the faculty library at NTK. The authors also made an electronic version available for free on their website.
Fairly comprehensive treatment of control-oriented DP is in the two-volume monograph [4] and [5]. It is not available online for free, but the book webpage contains links to other supporting materials including lecture notes. Furthermore, the latest book by the same author [6], which is available for free download, contains a decent introduction to dynamic programming.
Having just referenced a book on reinforcement learning (RL), indeed, this popular concept — or at least some of its flavours — is closely related to dynamic programming. In fact, it offers a way to overcome some of the limitations of dynamic programming. In our introductory lecture we are not covering RL, but an interested student can take advantage of availability of high-quality resources such as the the RL-related books and other resources by D. Bertsekas and another recommendable introduction to RL from control systems perspective [7], which is also available for free download.
The book [8] often regarded as the bible of RL is nice (and freely available for download) but may be rather difficult to read for a control engineer because of major differences in terminology and notation.
Back to top