Loop transfer recovery (LTR)

We have learned that LQG controllers can be arbitrarily non-robust (low gain or phase margins). Note that unlike an LQR controller that only relies on the availability of the measurements of the state, an LQG controller relies (through its internal Kalman filter) on knowing the control input as well. The Kalman filter combines this knowledge of the control input with the model of the system to predict the state (and correct this prediction using the measurements of the system output). But if the model is not accurate enough, the estimate may be poor. We can view this situation with an innacurate model as if the control input reported to the Kalman filter was different from the control input actually applied to the system.

One heuristic idea how to mitigate this weakness of the LQG control is to make it similar to the LQR control by reducing its dependence of the accuracy of the knowledge of the control variable. This heuristic idea is formalized under the name or Loop Transfer Recovery (LTR), and it is symbolically illustrated in Fig. 1.

Figure 1: Sketch of the LTR concept: make the Kalman filter rely less on the availability of the control signal

But how can we enforce this behavior upon the LQG controller? We can introduce a fictitious noise \bm w_\mathrm{fict}(t) during the computatonal design (not in the operation!) that enters the system in the same way as the control signal, see Fig. 2. In this way, we model the inaccuracy of the model as a corruption of the control input.

Figure 2: Implementation of the LTR concept: assume during the design that the system is subject to another disturbance acting on it in the same way as the control does

The state equation of the system used for the LQG design is then given by

\begin{aligned} \dot{\bm{x}}(t) &= \mathbf A\bm x(t) + \mathbf B_u\bm u(t) + \mathbf B_w\bm w(t) + \mathbf B_u{\color{red}\bm w_\mathrm{fict}(t)}\\ &= \mathbf A\bm x(t) + \mathbf B_u\bm u(t) + \begin{bmatrix}\mathbf B_w & \mathbf B_u\end{bmatrix}\begin{bmatrix}\bm w(t)\\{\color{red}\bm w_\mathrm{fict}(t)}\end{bmatrix} \end{aligned}

The spectral density of the new fictitious disturbance modelled as white noise is \mathbf S = \begin{bmatrix}\mathbf S_w & 0\\ 0 & {\color{red}S_{w_{\mathrm{fict}}}}\mathbf I\end{bmatrix}

It is this fictitious noise spectral density S_{w_{\mathrm{fict}}} that we can tune to make the Kalman filter rely less on the knowledge of the control signal. By increasing it, we make the control less sensitive to the modelling errors.

Example 1 (LQG/LTR for a robot arm) A single-link robotic arm is to be held vertical. Its model is J\ddot{\theta}(t) = mg\sin\theta(t) + u(t).

Linearizing and substituting some values for the parameters yields \begin{bmatrix} \dot\theta \\ \ddot\theta \end{bmatrix} = \begin{bmatrix} 0 & 1\\ 0.5 & 0 \end{bmatrix} \begin{bmatrix} \theta \\ \dot\theta \end{bmatrix} + \begin{bmatrix} 0 \\ 0.1 \end{bmatrix} u + \begin{bmatrix} 0 \\ 0.1 \end{bmatrix} w, where w is a random disturbance by a torquer circuit with a spectral density S_w = 1.

The angle of the arm is measured, which is modelled using the output equation y(t) = \begin{bmatrix} 1 & 0 \end{bmatrix} \begin{bmatrix} \theta \\ \dot \theta \end{bmatrix} +v, where the measurement noise spectral density is S_v=1.

The cost function that we choose for the LQG design is J = \mathbf E\left\{\theta^2 + 16u^2\right\}.

using ControlSystems
using LinearAlgebra
using MatrixEquations: arec
using Plots
using PrettyTables

# Plant model
A  = [0.0 1.0; 0.5 0.0]
Bu = [0.0; 0.1]
Bw = [0.0; 0.1]
C  = [1.0 0.0]

# Noise spectral densities
Sw = 1.0    # Actual disturbance
Sv = 10.0   # Measurement noise

# LQR gain (fixed across all designs)
Q = Diagonal([1.0, 0.0])
R = 16.0
K = lqr(A, Bu, Q, R)

# Plant state-space model
G = ss(A, Bu, C, 0.0)

# Build open-loop transfer function L(S) = C(s)·G(s) for negative-feedback stability margin analysis.
# Sw_fict = 0 gives pure LQG; increasing Sw_fict drives LTR recovery.
function open_loop(Sw_fict)
    L = kalman(Continuous, A, C, Bw*Sw*Bw' + Bu*Sw_fict*Bu', Sv)    # Kalman filter gain.
    #P, _ = arec(A', C' / Sv * C, Bw_cov)                           # Alternatively, we can solve CARE
    #L = P * C' / Sv                                                #  to compute the Kalman filter gain.
    Klqg  = ss(A - L*C - Bu*K, L, K, zeros(size(K,1), size(C,1)))   # Observer-controller (to be pluggeded into a negative .feedback loop)
    return Klqg*G                                                   # Open-loop transfer function.
end

# Designs to compare
Sw_fict_values = [0.0, 1e4, 1e6]
labels = vcat("LQG", ["LTR (Sw_fict=$(Int(Sw_fict_values[i])))" for i in 2:length(Sw_fict_values)])

# Compute open-loop TFs and collect stability margins
OL_array = []
table_data = Matrix{Any}(undef, length(Sw_fict_values), 5)
for (i, (Sw_fict_value, lab)) in enumerate(zip(Sw_fict_values, labels))
    OL = open_loop(Sw_fict_value)
    push!(OL_array, OL)
    wgm, gm, wpm, pm = margin(OL, allMargins=false)
    table_data[i, :] = [lab, round(20log10(only(gm)), digits=2), round(only(wgm), digits=4),
                             round(only(pm), digits=2),  round(only(wpm), digits=4)]
end
pretty_table(table_data;
    column_labels = ["Design", "GM (dB)", "ωgm (rad/s)", "PM (°)", "ωpm (rad/s)"],
    alignment = [:l, :r, :r, :r, :r])

┌───────────────────────┬─────────┬─────────────┬────────┬─────────────┐
│ Design                │ GM (dB) │ ωgm (rad/s) │ PM (°) │ ωpm (rad/s) │
├───────────────────────┼─────────┼─────────────┼────────┼─────────────┤
│ LQG                   │   -1.16 │        -0.0 │ 370.51 │      0.3804 │
│ LTR (Sw_fict=10000)   │   -2.91 │        -0.0 │ 383.58 │      0.6835 │
│ LTR (Sw_fict=1000000) │   -4.71 │        -0.0 │ 402.19 │      0.9889 │
└───────────────────────┴─────────┴─────────────┴────────┴─────────────┘

We can see that although the GM of the pure LQG design is acceptable, the PM is rather low. By increasing the fictitious noise spectral density, both GM and PM are improving with respect to the pure LQG design.

Bode plots (magnitude and phase) for open-loop transfer functions for two designs (LQG and LTR with the largest fictitious noise spectral density) are shown in Fig. 3. These only confirm/visualize the results in the above table, namely that the LTR improves both gain and phase margin with respect to the pure LQG design.

ω = logrange(1e-3, 1e4, length=500)
p_lqg = marginplot(OL_array[1], ω, title="Stability margins – LQG", lw=2)
p_ltr = marginplot(OL_array[end], ω, title="Stability margins – LTR (Sw_fict=$(Sw_fict_values[end]))", lw=2)
display(plot(p_lqg, p_ltr, layout=(1,2)))

Figure 3: Bode plots for the LQG- and the LTR-controlled robotic arm.

Nyquist plots for the several designs are shown in Fig. 4. We can see that, indeed, with the increasing fictitious noise spectral density, both GM and PM are improving with respect to the pure LQG design.

pn = nyquistplot(OL_array[1], ω, label=labels[1], lw=2, hover=false)
for (OL, lab) in zip(OL_array[2:end], labels[2:end])
    nyquistplot!(pn, OL, ω, label=lab, xlabel="Re", ylabel="Im", lw=2, unit_circle=true, aspect_ratio=:equal, hover=false)
end
display(pn)

┌ Warning: Keyword argument hover not supported with Plots.GRBackend().  Choose from: annotationcolor, annotationfontfamily, annotationfontsize, annotationhalign, annotationrotation, annotations, annotationvalign, arrow, aspect_ratio, axis, background_color, background_color_inside, background_color_outside, background_color_subplot, bar_width, bins, bottom_margin, camera, clims, color_palette, colorbar, colorbar_entry, colorbar_scale, colorbar_title, colorbar_titlefont, colorbar_titlefontcolor, colorbar_titlefontrotation, colorbar_titlefontsize, connections, contour_labels, discrete_values, fill, fill_z, fillalpha, fillcolor, fillrange, fillstyle, flip, fontfamily, fontfamily_subplot, foreground_color, foreground_color_axis, foreground_color_border, foreground_color_grid, foreground_color_subplot, foreground_color_text, formatter, framestyle, grid, gridalpha, gridlinewidth, gridstyle, group, guide, guidefont, guidefontcolor, guidefontfamily, guidefonthalign, guidefontrotation, guidefontsize, guidefontvalign, html_output_format, inset_subplots, label, layout, left_margin, legend_background_color, legend_column, legend_font, legend_font_color, legend_font_family, legend_font_halign, legend_font_pointsize, legend_font_rotation, legend_font_valign, legend_foreground_color, legend_position, legend_title, legend_title_font_color, legend_title_font_family, legend_title_font_pointsize, legend_title_font_rotation, legend_title_font_valigm, levels, lims, line, line_z, linealpha, linecolor, linestyle, linewidth, link, margin, marker_z, markeralpha, markercolor, markershape, markersize, markerstrokealpha, markerstrokecolor, markerstrokewidth, minorgrid, minorgridalpha, minorgridlinewidth, minorgridstyle, minorticks, mirror, normalize, orientation, overwrite_figure, permute, plot_title, plot_titlefontcolor, plot_titlefontfamily, plot_titlefontrotation, plot_titlefontsize, plot_titlelocation, plot_titlevspan, polar, primary, projection, quiver, ribbon, right_margin, rotation, scale, series_annotations, seriesalpha, seriescolor, seriestype, show, show_empty_bins, showaxis, size, smooth, subplot, subplot_index, thickness_scaling, tick_direction, tickfontcolor, tickfontfamily, tickfonthalign, tickfontrotation, tickfontsize, tickfontvalign, ticks, title, titlefontcolor, titlefontfamily, titlefonthalign, titlefontrotation, titlefontsize, titlefontvalign, top_margin, unitformat, weights, widen, window_title, x, xdiscrete_values, xerror, xflip, xforeground_color_axis, xforeground_color_border, xforeground_color_grid, xforeground_color_text, xformatter, xgrid, xgridalpha, xgridlinewidth, xgridstyle, xguide, xguidefontcolor, xguidefontfamily, xguidefonthalign, xguidefontrotation, xguidefontsize, xguidefontvalign, xlims, xlink, xminorgrid, xminorgridalpha, xminorgridlinewidth, xminorgridstyle, xminorticks, xmirror, xrotation, xscale, xshowaxis, xtick_direction, xtickfontcolor, xtickfontfamily, xtickfonthalign, xtickfontrotation, xtickfontsize, xtickfontvalign, xticks, xunitformat, xwiden, y, ydiscrete_values, yerror, yflip, yforeground_color_axis, yforeground_color_border, yforeground_color_grid, yforeground_color_text, yformatter, ygrid, ygridalpha, ygridlinewidth, ygridstyle, yguide, yguidefontcolor, yguidefontfamily, yguidefonthalign, yguidefontrotation, yguidefontsize, yguidefontvalign, ylims, ylink, yminorgrid, yminorgridalpha, yminorgridlinewidth, yminorgridstyle, yminorticks, ymirror, yrotation, yscale, yshowaxis, ytick_direction, ytickfontcolor, ytickfontfamily, ytickfonthalign, ytickfontrotation, ytickfontsize, ytickfontvalign, yticks, yunitformat, ywiden, z, z_order, zdiscrete_values, zerror, zflip, zforeground_color_axis, zforeground_color_border, zforeground_color_grid, zforeground_color_text, zformatter, zgrid, zgridalpha, zgridlinewidth, zgridstyle, zguide, zguidefontcolor, zguidefontfamily, zguidefonthalign, zguidefontrotation, zguidefontsize, zguidefontvalign, zlims, zlink, zminorgrid, zminorgridalpha, zminorgridlinewidth, zminorgridstyle, zminorticks, zmirror, zrotation, zscale, zshowaxis, ztick_direction, ztickfontcolor, ztickfontfamily, ztickfonthalign, ztickfontrotation, ztickfontsize, ztickfontvalign, zticks, zunitformat, zwiden
└ @ Plots ~/.julia/packages/Plots/GIume/src/args.jl:1563

Figure 4: Nyquist plots for the LQG/LTR-controlled robotic arm for increasing values of the fictitious noise spectral density.

However, we can also see in Fig. 4 that since the open loop has one unstable open-loop pole, stability of the closed loop requires exactly one counter-clockwise encirclement of the critical point −1. And this implies in the particular setting that not only \text{GM}^+ > 1 but also \text{GM}^- < 1 must be considered. Unfortunately, it appears that the margin function in ControlSystems.jl currently only returns the positive gain margin \text{GM}^+. Anyway, the conlcusion that the LTR design improves the robustness of the LQG design is still valid.