]> 6.2 The lq controller problem

### 6.2 The lq controller problem

In a Hilbert space H with the scalar product $〈\cdot ,\cdot 〉$ we consider the following feedback system:
 $\left\{\begin{array}{ccc}\hfill ẋ\left(t\right)& \hfill =\hfill & \mathcal{𝒜}x\left(t\right)-\mathcal{ℬ}\mathcal{𝒢}x\left(t\right),\phantom{\rule{1em}{0ex}}t\ge 0\hfill \\ \hfill x\left(0\right)& \hfill =\hfill & {x}_{0}\hfill \\ \hfill y\left(t\right)& \hfill =\hfill & \mathcal{𝒞}x\left(t\right)\hfill \end{array}\right\}$ (6.1)

where $\mathcal{𝒜}:\left(D\left(\mathcal{𝒜}\right)\subset \text{H}\right)\to \text{H}$ is the inﬁnitesimal generator of a ${\text{C}}_{0}$–semigroup ${\left\{S\left(t\right)\right\}}_{t\ge 0}$ on $\text{H}$; $\mathcal{ℬ}\in L\left(\text{U},\text{H}\right)$, $\mathcal{𝒞}\in L\left(\text{H},\text{Y}\right)$ where $\text{U}$, $\text{Y}$ are Hilbert spaces with scalar products ${〈\cdot ,\cdot 〉}_{\text{U}}$, ${〈\cdot ,\cdot 〉}_{\text{Y}}$, respectively; ${x}_{0}\in \text{H}$ is a ﬁxed element of $\text{H}$, $\mathcal{𝒢}\in L\left(\text{H},\text{U}\right)$ is an operator parameter describing the linear feedback $u=-\mathcal{𝒢}x$.

Consider also the set

 $\Gamma =\left\{\mathcal{𝒢}\in L\left(\text{H},\text{U}\right):\phantom{\rule{0ex}{0ex}}{∥y∥}_{{\text{L}}^{2}\left(0,\infty ;\text{Y}\right)}^{2}+{∥u∥}_{{\text{L}}^{2}\left(0,\infty ;\text{U}\right)}^{2}<\infty \phantom{\rule{2em}{0ex}}\forall {x}_{0}\in \text{H}\right\}$ (6.2)

Deﬁnition 6.2.1. The pair $\left(\mathcal{𝒜},\mathcal{ℬ}\right)$ is called stabilizable if the set

 (6.3)

is not empty.

Lemma 6.2.1. Let $\left(\mathcal{𝒜},\mathcal{ℬ}\right)$ be stabilizable. Then

(i)
$\Omega$ is an open set, $\Omega \subset \Gamma$.
(ii)
The mapping $\Omega \ni \mathcal{𝒢}↦\mathcal{ℋ}\left(\mathcal{𝒢}\right)\in \mathcal{𝒮}$ is well deﬁned, where $\mathcal{𝒮}\subset L\left(\text{H}\right)$ denotes a positive cone of all self–adjoint nonnegative operators and $\mathcal{ℋ}\left(\mathcal{𝒢}\right)$ is a unique solution to the Lyapunov operator equation
 $\begin{array}{c}〈\left(\mathcal{𝒜}-\mathcal{ℬ}\mathcal{𝒢}\right){x}_{1},\mathcal{ℋ}{x}_{2}〉+〈{x}_{1},\mathcal{ℋ}\left(\mathcal{𝒜}-\mathcal{ℬ}\mathcal{𝒢}\right){x}_{2}〉=\hfill \\ =-{〈\mathcal{𝒞}{x}_{1},\mathcal{𝒞}{x}_{2}〉}_{\text{Y}}-{〈\mathcal{𝒢}{x}_{1},\mathcal{𝒢}{x}_{2}〉}_{\text{U}}\phantom{\rule{2em}{0ex}}\forall {x}_{1},{x}_{2}\in D\left(\mathcal{𝒜}\right)\hfill \end{array}$ (6.4)

Moreover,

 $〈{x}_{0},\mathcal{ℋ}\left(\mathcal{𝒢}\right){x}_{0}〉={\int }_{0}^{\infty }\left[{∥\mathcal{𝒞}x\left(t\right)∥}_{\text{Y}}^{2}+{∥\mathcal{𝒢}x\left(t\right)∥}_{\text{U}}^{2}\right]dt$ (6.5)
(iii)
For every ${x}_{0}\in \text{H}$, the mapping
$\Omega \ni \mathcal{𝒢}↦{∥y∥}_{{\text{L}}^{2}\left(0,\infty ;\text{Y}\right)}^{2}+{∥u∥}_{{\text{L}}^{2}\left(0,\infty ;\text{U}\right)}^{2}=〈{x}_{0},\mathcal{ℋ}\left(\mathcal{𝒢}\right){x}_{0}〉\in \left[0,\infty \right)$

is continuous.

Proof. (i) Clearly, $\Omega \subset \Gamma$. If $\mathcal{ℋ}\in L\left(\text{H}\right)$ is such that $∥\mathcal{ℋ}∥$ is sufficiently small then by the fundamental perturbation result (see [69, Theorem 1.1, p. 76]) the type of the semigroup generated by $\mathcal{𝒜}-\mathcal{ℬ}\mathcal{𝒢}-\mathcal{ℬ}\mathcal{ℋ}$ is negative provided that the same holds for the semigroup generated by $\mathcal{𝒜}-\mathcal{ℬ}\mathcal{𝒢}$. This establishes (i).

(ii) This follows from Theorem 2.4.1 and Theorem 2.4.2.

(iii) For the proof of (iii) we recall the result from [69, Corollary 1.3, p. 78],

$∥{S}_{\mathcal{𝒢}+\mathcal{ℋ}}\left(t\right)-{S}_{\mathcal{𝒢}}\left(t\right)∥\le M\varphi \left(t\right)\phantom{\rule{2em}{0ex}}\forall t\ge 0,\phantom{\rule{2em}{0ex}}\varphi \left(t\right):={e}^{\left(\omega +M∥\mathcal{ℬ}∥\phantom{\rule{0ex}{0ex}}∥\mathcal{ℋ}∥\right)t}-{e}^{\omega t},\phantom{\rule{1em}{0ex}}t\ge 0$

for some $M\ge 1$, where ${\left\{{S}_{\mathcal{𝒢}+\mathcal{ℋ}}\left(t\right)\right\}}_{t\ge 0}$, ${\left\{{S}_{\mathcal{𝒢}}\left(t\right)\right\}}_{t\ge 0}$ are the semigroups generated by $\mathcal{𝒜}-\mathcal{ℬ}\mathcal{𝒢}-\mathcal{ℬ}\mathcal{ℋ}$ and $\mathcal{𝒜}-\mathcal{ℬ}\mathcal{𝒢}$, respectively and $\omega$ is the type of ${\left\{{S}_{\mathcal{𝒢}}\left(t\right)\right\}}_{t\ge 0}$. But, for $\mathcal{𝒢}\in \Omega$ and sufficiently small $∥\mathcal{ℋ}∥$, the function $\varphi$ belongs to ${\text{L}}^{2}\left(0,\infty \right)$, and its ${\text{L}}^{2}\left(0,\infty \right)$ norm tends to $0$ as $∥\mathcal{ℋ}∥$ tends to $0$. Hence, the mapping $\Omega \ni \mathcal{𝒢}↦\mathcal{𝒞}{S}_{\mathcal{𝒢}}\left(\cdot \right){x}_{0}\in {\text{L}}^{2}\left(0,\infty ;\text{Y}\right)$ is continuous. Only minor modiﬁcations are required to prove that the same holds for the mapping $\Omega \ni \mathcal{𝒢}↦\mathcal{𝒢}{S}_{\mathcal{𝒢}}\left(\cdot \right){x}_{0}\in {\text{L}}^{2}\left(0,\infty ;\text{U}\right)$. □

Deﬁnition 6.2.2. The pair $\left(\mathcal{𝒜},\mathcal{𝒞}\right)$ is called detectable if there exists $\mathcal{𝒬}\in L\left(\text{Y},\text{H}\right)$ such that the semigroup generated by $\mathcal{𝒜}+\mathcal{𝒬}\mathcal{𝒞}$ is EXS.

Lemma 6.2.2. Let $\left(\mathcal{𝒜},\mathcal{ℬ}\right)$ be stabilizable. Assume additionally that the pair $\left(\mathcal{𝒜},\mathcal{𝒞}\right)$ is detectable. Then

(i)
$\Omega =\Gamma$.
(ii)
The mapping
$J:\phantom{\rule{0ex}{0ex}}L\left(\text{H},\text{U}\right)\ni \mathcal{𝒢}↦\left\{\begin{array}{cc}{∥y∥}_{{\text{L}}^{2}\left(0,\infty ;\text{Y}\right)}^{2}+{∥u∥}_{{\text{L}}^{2}\left(0,\infty ;\text{U}\right)}^{2},\hfill & \phantom{\rule{0ex}{0ex}}\mathcal{𝒢}\in \Omega \hfill \\ +\infty ,\hfill & \phantom{\rule{0ex}{0ex}}\mathcal{𝒢}\notin \Omega \hfill \end{array}\right\}\in \left[0,\infty \right]$

is continuous.

Proof. (i) It is sufficient to prove that $\Omega \supset \Gamma$. We take $\mathcal{𝒢}\in \Gamma$ and represent the ﬁrst two lines of (6.1) in the form

$\left\{\begin{array}{ccc}\hfill ẋ\left(t\right)& \hfill =\hfill & \left(\mathcal{𝒜}+\mathcal{𝒬}\mathcal{𝒞}\right)x\left(t\right)-\left(\mathcal{𝒬}\mathcal{𝒞}x\left(t\right)+\mathcal{ℬ}\mathcal{𝒢}x\left(t\right)\right)\hfill \\ \hfill x\left(0\right)& \hfill =\hfill & {x}_{0}\hfill \end{array}\right\}$

with $\mathcal{𝒬}\in L\left(\text{Y},\text{H}\right)$ chosen in such a manner that the semigroup ${\left\{S\left(t\right)\right\}}_{t\ge 0}$ generated by $\mathcal{𝒜}+\mathcal{𝒬}\mathcal{𝒞}$ is EXS. The existence of $\mathcal{𝒬}$ is ensured by the detectability of $\left(\mathcal{𝒜},\mathcal{𝒬}\right)$. Indeed, employing the variation–of–constants formula, we get

$∥x\left(t\right)∥\le ∥S\left(t\right){x}_{0}∥+max\left\{∥\mathcal{𝒬}∥,∥\mathcal{ℬ}∥\right\}{\int }_{0}^{t}∥S\left(t-\tau \right)∥\left[{∥\mathcal{𝒞}x\left(\tau \right)∥}_{\text{Y}}+{∥\mathcal{𝒢}x\left(\tau \right)∥}_{\text{U}}\right]d\tau .$

By deﬁnition of $\Gamma$, $\mathcal{𝒞}x\left(\cdot \right)\in {\text{L}}^{2}\left(0,\infty ;\text{Y}\right)$, $\mathcal{𝒢}x\left(\cdot \right)\in {\text{L}}^{2}\left(0,\infty ;\text{U}\right)$. Hence, from the basic properties of convolution, it follows that $∥x\left(\cdot \right)∥\in {\text{L}}^{2}\left(0,\infty \right)$ for all ${x}_{0}\in \text{H}$. The last property is equivalent to the exponential stability of the semigroup generated by $\mathcal{𝒜}-\mathcal{ℬ}\mathcal{𝒢}$ [69, Theorem 4.1, p. 116], and thus $\mathcal{𝒢}\in \Omega$.

(ii) By (i) we have $J\left(\mathcal{𝒢}\right)=\infty$ on $L\left(\text{H},\text{U}\right)\setminus \Omega$ (we may assume that $L\left(\text{H},\text{U}\right)\setminus \Omega \ne \varnothing$ as otherwise the result to be proved follows from Lemma 6.2.1/(iii)) and, to show the continuity of $J$, it suffices to prove that $J\left(\mathcal{𝒢}\right)$ tends to $\infty$ as $\mathcal{𝒢}$ tends to $\partial \Omega$ from the inside. Take any $R>0$ and let ${\left\{{\mathcal{𝒢}}_{k}\right\}}_{k\in \mathbb{ℕ}}$ be a sequence in $\Omega$ with ${\mathcal{𝒢}}_{k}\to {\mathcal{𝒢}}_{\infty }\in \partial \Omega$ as $k\to \infty$. We claim that, for almost all $k\in \mathbb{ℕ}$, we have $J\left({\mathcal{𝒢}}_{k}\right)\ge R$. Observe that the function

$\left[0,\infty \right)\ni t↦{∥{y}_{\infty }∥}_{{\text{L}}^{2}\left(0,t;\text{Y}\right)}^{2}+{∥{u}_{\infty }∥}_{{\text{L}}^{2}\left(0,t;\text{U}\right)}^{2}={\int }_{0}^{t}\left[{∥\mathcal{𝒞}{x}_{\infty }\left(\tau \right)∥}_{\text{Y}}^{2}+{∥{\mathcal{𝒢}}_{\infty }{x}_{\infty }\left(\tau \right)∥}_{\text{U}}^{2}\right]d\tau$

where ${x}_{\infty }$, ${y}_{\infty }$, ${u}_{\infty }$ denote respectively the state, output, and control functions due to ${\mathcal{𝒢}}_{\infty }$, is nondecreasing and tends to $\infty$ as $t\to \infty$. Hence there exists $T>0$ such that

${\int }_{0}^{T}\left[{∥\mathcal{𝒞}{x}_{\infty }\left(t\right)∥}_{\text{Y}}^{2}+{∥{\mathcal{𝒢}}_{\infty }{x}_{\infty }\left(t\right)∥}_{\text{U}}^{2}\right]dt=2R.$

The mapping $L\left(\text{H},\text{U}\right)\ni \mathcal{𝒢}↦{∥y∥}_{{\text{L}}^{2}\left(0,T;\text{Y}\right)}^{2}+{∥u∥}_{{\text{L}}^{2}\left(0,T;\text{U}\right)}^{2}\in \left[0,\infty \right)$ is continuous. Indeed, from [69, Corollary 1.3, p. 78], we know that

$∥{S}_{\mathcal{𝒢}+\mathcal{ℋ}}\left(t\right)-{S}_{\mathcal{𝒢}}\left(t\right)∥\le M\varphi \left(t\right)\phantom{\rule{2em}{0ex}}\forall t\ge 0$

where ${\left\{{S}_{\mathcal{𝒢}+\mathcal{ℋ}}\left(t\right)\right\}}_{t\ge 0}$, ${\left\{{S}_{\mathcal{𝒢}}\left(t\right)\right\}}_{t\ge 0}$ are the semigroups generated by $\mathcal{𝒜}-\mathcal{ℬ}\mathcal{𝒢}-\mathcal{ℬ}\mathcal{ℋ}$ and $\mathcal{𝒜}-\mathcal{ℬ}\mathcal{𝒢}$ respectively, and $\omega$ is the type of ${\left\{{S}_{\mathcal{𝒢}}\left(t\right)\right\}}_{t\ge 0}$. But the function $\varphi$ belongs to ${\text{L}}^{2}\left(0,T\right)$, and its ${\text{L}}^{2}\left(0,T\right)$ norm tends to $0$ as $∥\mathcal{ℋ}∥$ tends to $0$. Hence the mappings

$\begin{array}{ccc}\hfill L\left(\text{H},\text{U}\right)& \hfill \ni \hfill & \mathcal{𝒢}↦\mathcal{𝒞}{S}_{\mathcal{𝒢}}\left(\cdot \right){x}_{0}\in {\text{L}}^{2}\left(0,T;\text{Y}\right),\hfill \\ \hfill L\left(\text{H},\text{U}\right)& \hfill \ni \hfill & \mathcal{𝒢}↦\mathcal{𝒢}{S}_{\mathcal{𝒢}}\left(\cdot \right){x}_{0}\in {\text{L}}^{2}\left(0,T;\text{U}\right)\hfill \end{array}$

are both continuous.

By the continuity of the mapping $L\left(\text{H},\text{U}\right)\ni \mathcal{𝒢}↦{∥y∥}_{{\text{L}}^{2}\left(0,T;\text{Y}\right)}^{2}+{∥u∥}_{{\text{L}}^{2}\left(0,T;\text{U}\right)}^{2}$ just proved, for any $\varepsilon \in \left(0,R\right]$, we get

$\left|\phantom{\rule{0ex}{0ex}}{∥{y}_{\infty }∥}_{{\text{L}}^{2}\left(0,T;\text{Y}\right)}^{2}+{∥{u}_{\infty }∥}_{{\text{L}}^{2}\left(0,T;\text{U}\right)}^{2}-{∥{y}_{k}∥}_{{\text{L}}^{2}\left(0,T;\text{Y}\right)}^{2}-{∥{u}_{k}∥}_{{\text{L}}^{2}\left(0,T;\text{U}\right)}^{2}\right|\le \varepsilon$

where ${y}_{k}$ and ${u}_{k}$ denote respectively the output and control functions due to ${\mathcal{𝒢}}_{k}$, for almost all $k\in \mathbb{ℕ}$. However, this implies that

$J\left({\mathcal{𝒢}}_{k}\right)={∥{y}_{k}∥}_{{\text{L}}^{2}\left(0,\infty ;\text{Y}\right)}^{2}+{∥{u}_{k}∥}_{{\text{L}}^{2}\left(0,\infty ;\text{U}\right)}^{2}\ge {∥{y}_{k}∥}_{{\text{L}}^{2}\left(0,T;\text{Y}\right)}^{2}+{∥{u}_{k}∥}_{{\text{L}}^{2}\left(0,T;\text{U}\right)}^{2}\ge R$

for almost all $k\in \mathbb{ℕ}$, and the proof is complete. □

Now we formulate the parametric optimization problem which consists in ﬁnding $\mathcal{𝒢}\in \Omega$ such that

 $〈{x}_{0},\mathcal{ℋ}\left(\mathcal{𝒢}\right){x}_{0}〉={min}_{K\in \Omega }〈{x}_{0},\mathcal{ℋ}\left(K\right){x}_{0}〉\phantom{\rule{2em}{0ex}}\forall {x}_{0}\in \text{H}$ (6.6)

Theorem 6.2.1. If $\left(\mathcal{𝒜},\mathcal{ℬ}\right)$ is stabilizable and $\left(\mathcal{𝒜},\mathcal{𝒞}\right)$ is detectable, then the problem (6.6) has a unique solution.

Before starting the proof, let us remark that this is a well–known fundamental result concerning the lq problem (see  and [16, Section 4.4]), reformulated above as a parametric optimization problem. However, a new derivation of this result will be given. The main novelty, besides reformulation, is the simple explicit proof of convergence of the Newton – Kleinman sequence of stabilizing controllers.

Proof. Using (6.4), it is easy to show that, if $\mathcal{𝒢}\in \Omega$, then for each $\mathcal{ℱ}\in L\left(\text{H},\text{U}\right)$ such that $\mathcal{𝒢}+\mathcal{ℱ}\in \Omega$, the operator $\Delta =\mathcal{ℋ}\left(\mathcal{𝒢}+\mathcal{ℱ}\right)-\mathcal{ℋ}\left(\mathcal{𝒢}\right)$ is the unique bounded self–adjoint operator satisfying the operator equation

 $\begin{array}{c}〈\left(\mathcal{𝒜}-\mathcal{ℬ}\mathcal{𝒢}-\mathcal{ℬ}\mathcal{ℱ}\right){x}_{1},\Delta {x}_{2}〉+〈{x}_{1},\Delta \left(\mathcal{𝒜}-\mathcal{ℬ}\mathcal{𝒢}-\mathcal{ℬ}\mathcal{ℱ}\right){x}_{2}〉=〈{x}_{1},\left[\mathcal{ℋ}\left(\mathcal{𝒢}\right)\mathcal{ℬ}-{\mathcal{𝒢}}^{\ast }\right]\mathcal{ℱ}{x}_{2}〉\hfill \\ +〈\left[\mathcal{ℋ}\left(\mathcal{𝒢}\right)\mathcal{ℬ}-{\mathcal{𝒢}}^{\ast }\right]\mathcal{ℱ}{x}_{1},{x}_{2}〉-{〈\mathcal{ℱ}{x}_{1},\mathcal{ℱ}{x}_{2}〉}_{\text{U}}\phantom{\rule{2em}{0ex}}\forall {x}_{1},{x}_{2}\in D\left(\mathcal{𝒜}\right)\hfill \end{array}$ (6.7)

Now we show that the following implication holds:

 $\mathcal{𝒢}\in \Omega ⇒{\mathcal{ℬ}}^{\ast }\mathcal{ℋ}\left(\mathcal{𝒢}\right)\in \Omega$ (6.8)

Suppose for a moment that, contrary to our statement, one has ${\mathcal{ℬ}}^{\ast }\mathcal{ℋ}\left(\mathcal{𝒢}\right)\notin \Omega$. Since $\Omega$ is an open set, there is ${\lambda }_{\partial \Omega }\in \left(0,1\right]$ such that (see Figure 6.1) Figure 6.1: An auxiliary diagram for the proof

${\mathcal{𝒢}}_{\lambda }=\left(1-\lambda \right)\mathcal{𝒢}+\lambda {\mathcal{ℬ}}^{\ast }\mathcal{ℋ}\left(\mathcal{𝒢}\right)\in \Omega \phantom{\rule{1em}{0ex}}\text{for}\phantom{\rule{1em}{0ex}}\lambda \in \left[0,{\lambda }_{\partial \Omega }\right)\phantom{\rule{1em}{0ex}}\text{and}\phantom{\rule{1em}{0ex}}{\mathcal{𝒢}}_{{\lambda }_{\partial \Omega }}\in \partial \Omega .$

Consequently, putting $\mathcal{ℱ}={\mathcal{𝒢}}_{\lambda }-\mathcal{𝒢}=\lambda \left[{\mathcal{ℬ}}^{\ast }\mathcal{ℋ}\left(\mathcal{𝒢}\right)-\mathcal{𝒢}\right]$, $\lambda \in \left[0,{\lambda }_{\partial \Omega }\right)$ in (6.7) we come to a conclusion that $\Delta =\mathcal{ℋ}\left({\mathcal{𝒢}}_{\lambda }\right)-\mathcal{ℋ}\left(\mathcal{𝒢}\right)$ is a unique bounded, self–adjoint operator satisfying an operator equation

$〈\left(\mathcal{𝒜}-\mathcal{ℬ}{\mathcal{𝒢}}_{\lambda }\right){x}_{1},\Delta {x}_{2}〉+〈{x}_{1},\Delta \left(\mathcal{𝒜}-\mathcal{ℬ}{\mathcal{𝒢}}_{\lambda }\right){x}_{2}〉=\left(2\lambda -{\lambda }^{2}\right)〈\left[\mathcal{ℋ}\left(\mathcal{𝒢}\right)\mathcal{ℬ}-{\mathcal{𝒢}}^{\ast }\right]\left[{\mathcal{ℬ}}^{\ast }\mathcal{ℋ}\left(\mathcal{𝒢}\right)-\mathcal{𝒢}\right]{x}_{1},{x}_{2}〉$

for all ${x}_{1},{x}_{2}\in D\left(\mathcal{𝒜}\right)$ and all $\lambda \in \left[0,{\lambda }_{\partial \Omega }\right)$. But $2\lambda -{\lambda }^{2}\ge 0$ for $\lambda \in \left[0,{\lambda }_{\partial \Omega }\right)$, and again by the results of Theorem 2.4.1 and Theorem 2.4.2, $\left(-\Delta \right)\ge 0$ (in the sense of quadratic forms). Hence the function

$\left[0,{\lambda }_{\partial \Omega }\right)\ni \lambda ↦〈{x}_{0},\mathcal{ℋ}\left({\mathcal{𝒢}}_{\lambda }\right){x}_{0}〉={∥{y}_{\lambda }∥}_{{\text{L}}^{2}\left(0,\infty ;\text{Y}\right)}^{2}+{∥{u}_{\lambda }∥}_{{\text{L}}^{2}\left(0,\infty ;\text{U}\right)}^{2}$

is bounded from above by $〈{x}_{0},\mathcal{ℋ}\left(\mathcal{𝒢}\right){x}_{0}〉$, where ${y}_{\lambda }\left(t\right)=\mathcal{𝒞}{x}_{\lambda }\left(t\right)$ and ${u}_{\lambda }\left(t\right)=\mathcal{𝒢}{x}_{\lambda }\left(t\right)$, with ${x}_{\lambda }$ denoting the solution of (6.1) with $\mathcal{𝒢}$ replaced by ${\mathcal{𝒢}}_{\lambda }$. But, from Lemma 6.2.2/(i), it follows that this function takes arbitrarily large values in a sufficiently small neighbourhood of ${\lambda }_{\partial \Omega }$. Hence our claim ${\mathcal{ℬ}}^{\ast }\mathcal{ℋ}\left(\mathcal{𝒢}\right)\notin \Omega$ leads to a contradiction, and thus (6.8) holds. By (6.8), the sequence ${\left\{{\mathcal{𝒢}}_{k}\right\}}_{k\in \mathbb{ℕ}}$ given by

 ${\mathcal{𝒢}}_{k+1}={\mathcal{ℬ}}^{\ast }\mathcal{ℋ}\left({\mathcal{𝒢}}_{k}\right)$ (6.9)

where ${\mathcal{𝒢}}_{1}$ is an arbitrary element of $\Omega$, is well–deﬁned and contained in $\Omega$. Taking $\mathcal{𝒢}={\mathcal{𝒢}}_{k}$, $\mathcal{ℱ}={\mathcal{𝒢}}_{k+1}-{\mathcal{𝒢}}_{k}={\mathcal{ℬ}}^{\ast }\mathcal{ℋ}\left({\mathcal{𝒢}}_{k}\right)-{\mathcal{𝒢}}_{k}$ in (6.7), one obtains

$〈\left[\mathcal{𝒜}-\mathcal{ℬ}{\mathcal{ℬ}}^{\ast }\mathcal{ℋ}\left({\mathcal{𝒢}}_{k}\right)\right]{x}_{1},\Delta {x}_{2}〉+〈{x}_{1},\Delta \left[\mathcal{𝒜}-\mathcal{ℬ}{\mathcal{ℬ}}^{\ast }\mathcal{ℋ}\left({\mathcal{𝒢}}_{k}\right)\right]{x}_{2}〉=$

$=〈{x}_{1}\left[\mathcal{ℋ}\left({\mathcal{𝒢}}_{k}\right)\mathcal{ℬ}-{\mathcal{𝒢}}_{k}^{\ast }\right]\left[{\mathcal{ℬ}}^{\ast }\mathcal{ℋ}\left({\mathcal{𝒢}}_{k}\right)-{\mathcal{𝒢}}_{k}\right],{x}_{2}〉\phantom{\rule{2em}{0ex}}\forall {x}_{1},{x}_{2}\in D\left(\mathcal{𝒜}\right),\phantom{\rule{1em}{0ex}}\forall k\in \mathbb{ℕ}.$

Applying once more the results from Theorem 2.4.1 and Theorem 2.4.2 we get $\left(-\Delta \right)\ge 0$. Thus the sequence of the terms

$〈{x}_{0},\mathcal{ℋ}\left({\mathcal{𝒢}}_{k}\right){x}_{0}〉={∥{y}_{k}∥}_{{\text{L}}^{2}\left(0,\infty ;\text{Y}\right)}^{2}+{∥{u}_{k}∥}_{{\text{L}}^{2}\left(0,\infty ;\text{U}\right)}^{2}$

is nonincreasing and bounded from below. Now, by standard arguments [86, Theorem 4.28, p. 79] there exists ${\mathcal{ℋ}}_{\infty }\in L\left(\text{H}\right)$, with ${\mathcal{ℋ}}_{\infty }={\mathcal{ℋ}}_{\infty }^{\ast }\ge 0$, such that $\mathcal{ℋ}\left({\mathcal{𝒢}}_{k}\right)x\to {\mathcal{ℋ}}_{\infty }x$ as $k\to \infty$, for each $x\in \text{H}$. Since ${\mathcal{ℬ}}^{\ast }\in L\left(\text{H},\text{U}\right)$, we have

 ${\mathcal{𝒢}}_{k+1}x={\mathcal{ℬ}}^{\ast }\mathcal{ℋ}\left({\mathcal{𝒢}}_{k}\right)x\to {\mathcal{ℬ}}^{\ast }{\mathcal{ℋ}}_{\infty }x={\mathcal{𝒢}}_{\infty }x\phantom{\rule{2em}{0ex}}\forall x\in \text{H}$ (6.10)

By virtue of Lemma 6.2.2/(ii),

$〈{x}_{0},\mathcal{ℋ}\left({\mathcal{𝒢}}_{k}\right){x}_{0}〉={∥{y}_{k}∥}_{{\text{L}}^{2}\left(0,\infty ;\text{Y}\right)}^{2}+{∥{u}_{k}∥}_{{\text{L}}^{2}\left(0,\infty ;\text{U}\right)}^{2}\to$

$\to {∥{y}_{\infty }∥}_{{\text{L}}^{2}\left(0,\infty ;\text{Y}\right)}^{2}+{∥{u}_{\infty }∥}_{{\text{L}}^{2}\left(0,\infty ;\text{U}\right)}^{2}=〈{x}_{0},{\mathcal{ℋ}}_{\infty }{x}_{0}〉<\infty .$

Hence ${\mathcal{𝒢}}_{\infty }\in \Omega$. Now we can apply Lemma 6.2.1/(iii) to get

$\begin{array}{cc}\hfill 〈{x}_{0},{\mathcal{ℋ}}_{\infty }{x}_{0}〉=& {∥{y}_{\infty }∥}_{{\text{L}}^{2}\left(0,\infty ;\text{Y}\right)}^{2}+{∥{u}_{\infty }∥}_{{\text{L}}^{2}\left(0,\infty ;\text{U}\right)}^{2}=\hfill \\ \hfill =& {\int }_{0}^{\infty }\left[{∥\mathcal{𝒞}x\left(t\right)∥}_{\text{Y}}^{2}+{∥{\mathcal{𝒢}}_{\infty }x\left(t\right)∥}_{\text{U}}^{2}\right]dt=〈{x}_{0},\mathcal{ℋ}\left({\mathcal{𝒢}}_{\infty }\right){x}_{0}〉\phantom{\rule{2em}{0ex}}\forall {x}_{0}\in \text{H}.\hfill \end{array}$

This means that ${\mathcal{𝒢}}_{\infty }\in \Gamma$ and ${\mathcal{ℋ}}_{\infty }$ satisﬁes (6.4) with $\mathcal{𝒢}={\mathcal{𝒢}}_{\infty }$, i.e.

 $\begin{array}{c}〈\left(\mathcal{𝒜}-\mathcal{ℬ}{\mathcal{𝒢}}_{\infty }\right){x}_{1},{\mathcal{ℋ}}_{\infty }{x}_{2}〉+〈{x}_{1},{\mathcal{ℋ}}_{\infty }\left(\mathcal{𝒜}-\mathcal{ℬ}{\mathcal{𝒢}}_{\infty }\right){x}_{2}〉=\hfill \\ =-{〈\mathcal{𝒞}{x}_{1},\mathcal{𝒞}{x}_{2}〉}_{\text{Y}}-{〈{\mathcal{𝒢}}_{\infty }{x}_{1},{\mathcal{𝒢}}_{\infty }{x}_{2}〉}_{\text{U}}\phantom{\rule{2em}{0ex}}\forall {x}_{1},{x}_{2}\in D\left(\mathcal{𝒜}\right)\hfill \end{array}$ (6.11)

Substituting $\mathcal{𝒢}={\mathcal{𝒢}}_{\infty }$ in (6.7), for any $\mathcal{ℱ}\in L\left(\text{H},\text{U}\right)$ such that ${\mathcal{𝒢}}_{\infty }+\mathcal{ℱ}\in \Omega$, we get

$〈\left(\mathcal{𝒜}-\mathcal{ℬ}{\mathcal{𝒢}}_{\infty }-\mathcal{ℬ}\mathcal{ℱ}\right){x}_{1},\Delta {x}_{2}〉+〈{x}_{1},\Delta \left(\mathcal{𝒜}-\mathcal{ℬ}{\mathcal{𝒢}}_{\infty }-\mathcal{ℬ}\mathcal{ℱ}\right){x}_{2}〉=$

$=-{〈\mathcal{ℱ}{x}_{1},\mathcal{ℱ}{x}_{2}〉}_{\text{U}}\phantom{\rule{2em}{0ex}}\forall {x}_{1},{x}_{2}\in D\left(\mathcal{𝒜}\right).$

Recalling again the results from Theorem 2.4.1 and Theorem 2.4.2 we come to the inequality $\mathcal{ℋ}\left({\mathcal{𝒢}}_{\infty }+\mathcal{ℱ}\right)\ge \mathcal{ℋ}\left({\mathcal{𝒢}}_{\infty }\right)$, and thus ${\mathcal{𝒢}}_{\infty }$ is a solution of (6.6). Moreover, from (6.11) and Theorem 2.4.4 it follows that ${\mathcal{ℋ}}_{\infty }$ is a Hilbert – Schmidt operator (HS–operator) provided that $\mathcal{𝒢}$ and $\mathcal{𝒞}$ are ﬁnite – rank operators. □

Remark 6.2.1. The inﬁnite–dimensional version of the Kleinman algorithm was used for the ﬁrst time in  to prove that (6.11) has a maximal bounded self–adjoint positive solution (being the limit of the Kleinman sequence), provided that $\left(\mathcal{𝒜},\mathcal{ℬ}\right)$ is only stabilizable.

It follows from (6.10) and the closed–loop Lyapunov operator equation that ${\mathcal{ℋ}}_{\infty }$ satisﬁes also the open–loop Lyapunov operator equation

 $〈\mathcal{𝒜}{x}_{1},{\mathcal{ℋ}}_{\infty }{x}_{2}〉+〈{x}_{1},{\mathcal{ℋ}}_{\infty }\mathcal{𝒜}{x}_{2}〉=-{〈\mathcal{𝒞}{x}_{1},\mathcal{𝒞}{x}_{2}〉}_{\text{Y}}+{〈{\mathcal{𝒢}}_{\infty }{x}_{1},{\mathcal{𝒢}}_{\infty }{x}_{2}〉}_{\text{U}}\phantom{\rule{2em}{0ex}}\forall {x}_{1},{x}_{2}\in D\left(\mathcal{𝒜}\right)$ (6.12)