Give the first-order necessary conditions of optimality for a general optimal control problem for a nonlinear discrete-time system over a finite horizon. Namely, give the general two-point boundary value problem, highlighting the state equation, the co-state equation and a stationarity equation. Do not forget to include general boundary conditions.
Give the first-order necessary conditions of optimality for a linear and time invariant (LTI) discrete-time system and a quadratic cost function over a finite horizon. Namely, give them in the format displaying the state equation, co-state equation and stationarity equation. Show and discuss also two types of boundary conditions.
Give a qualitative characterization of the solution to the fixed final state LQ-optimal control problem over a finite horizon, that is, you do not have to give formulas but you should be able to state among the highlights that the control is open-loop and that reachability of the system is a necessary condition.
Give a qualitative characterization of the solution to the free final state LQ-optimal control problem over a finite horizon, that is, you do not have to give formulas but you should be able to state among the highlights that the control is closed-loop, namely, a time-varying linear state feedback and that the feedback gains can be computed by solving a difference Riccati equation.
Discuss how solution to the free final state LQ problem changes as the horizon is extended to infinity. Emphasize that the optimal solution is given by a constant linear state feedback whose gains are computed by solving a discrete-time algebraic Riccati equation (DARE). What are the conditions under which a stabilizing solution is guaranteed to exist? What are the conditions under which it is guaranteed that there is a unique stabilizing solution of DARE?
Skills (use the knowledge to solve a problem)
Design an LQ-optimal state feedback controller for a discrete-time linear system both for a finite and an infinite horizon, both for regulation and for tracking.