An adaptive partition of unity method for Chebyshev polynomial interpolation

Abstract: For a function that is analytic on and around an interval, Chebyshev polynomial interpolation provides spectral convergence. However, if the function has a singularity close to the interval, the rate of convergence is near one. In these cases splitting the interval and using piecewise interpolation can accelerate convergence. Chebfun includes a splitting mode that finds an optimal splitting through recursive bisection, but the result has no global smoothness unless conditions are imposed explicitly at the breakpoints. An alternative is to split the domain into overlapping intervals and use an infinitely smooth partition of unity to blend the local Chebyshev interpolants. A simple divide-and-conquer algorithm similar to Chebfun's splitting mode can be used to find an overlapping splitting adapted to features of the function. The algorithm implicitly constructs the partition of unity over the subdomains. This technique is applied to explicitly given functions as well as to the solutions of singularly perturbed boundary value problems.

\section{Introduction}
Chebyshev polynomial interpolants provide powerful approximation properties, both in theory and as implemented in practice by the Chebfun software system~\cite{battles2004extension}. Chebfun uses spectral collocation to provide very accurate automatic solutions to differential equations~\cite{driscoll2008chebop}. The method is not fully adaptive, though, since the refinement is limited to the degree of the global interpolant.

 Chebfun includes a \textit{splitting} method that creates piecewise polynomial approximations \cite{pachon2010piecewise}. When splitting is enabled, if a Chebyshev interpolant is unable to represent the function accurately at a specified maximum degree on an interval, the interval is bisected; this process is recursively repeated on the subintervals. Afterwards adjacent subintervals are merged if the new interval allows for a Chebyshev approximation with lower degree. In effect, the method does a binary search for a good splitting location. In \cite{driscoll2014optimal} it was shown that the splitting locations are roughly optimal based on the singularity structure of the function in the complex plane.

A drawback of Chebfun's splitting approach is that the resulting representation does not ensure anything more than $C^0$ continuity. Differentiation of the Chebyshev interpolation polynomial of degree $n$ has norm $O(n^2)$, so a jump in the derivative develops across a splitting point and becomes more pronounced for higher derivatives and larger $n$. In order to solve a boundary-value problem, Chebfun imposes explicit continuity conditions on the solution to augment the discrete problem. This solution works well in 1D but becomes cumbersome in higher dimensions, particularly if refinements are made nonconformingly.

In this paper we explore the use of Chebyshev interpolants on overlapping domains combined using a \emph{partition of unity}. The resulting approximation has the same accuracy as the individual piecewise interpolants. We use compactly supported weight functions that are infinitely differentiable, so the resulting combined interpolant is also infinitely smooth (though not analytic). We also show that the accuracy of the derivative can be bounded by $\Theta(\delta^{-2})$ for an overlap amount $\delta$, revealing an explicit tradeoff between efficiency (smaller overlap and more like Chebfun splitting) and global accuracy of the derivative. Because the global approximation is smooth, there are no matching conditions needed to solve a BVP, and there are standard preconditioners available that should aid with iterative methods for large discretizations. For example, since we split the interval into overlapping domains we could use the restricted additive Schwarz preconditioner \cite{doi:10.1137/S106482759732678X}.

We describe a recursive, adaptive algorithm for creating and applying a partition of unity, modeled on the recursive splitting in Chebfun but merging adjacent subdomains aggressively in order to keep the total node count low. Even though each node of the recursion only combines two adjacent subdomains, we show that the global approximant is also a partition of unity. We demonstrate that the adaptive refinement is able to resolve highly localized features of an explicitly given function and of a solution to a singularly perturbed BVP.

The use of a partition of unity in our approximation affords us some flexibility; we are able to create approximations which are both efficient and infinitely smooth without matching. Partition of unity schemes have been widely used for interpolation \cite{franke1980smooth,mclain1976two,shepard1968two} and solving PDE's \cite{griebel2000particle,safdari2015radial}. In section~\ref{PUM_FORM_SEC} we introduce the partition of unity method, and we discuss the convergence of the method for a simple split on the interval $[-1,1]$ in section~\ref{converge_sec}. We describe our adaptive algorithm in section~\ref{PUM_recurse}. In section~\ref{PUM_BVP_SEC} we explain how to apply our method to solve boundary value problems on an interval and perform some experiments with singularly perturbed problems.

\section{Chebyshev interpolation}
\label{sec_cheb}
We use Chebyshev interpolants for our partition of unity method because they enjoy spectral convergence. Suppose that $f(x)$ is analytic inside a  Bernstein ellipse $E_\rho$ (an ellipse with foci $\pm 1$ and semi-major axis $\rho>1$). We then have Theorem 6 from \cite{trefethen2000spectral}:
\begin{theorem} Suppose $f(z)$ is analytic on and inside the Bernstein ellipse $E_\rho$. Let $p_n$ be the polynomial that interpolates $f(z)$ at $n+1$ Chebyshev points of the second kind. Then there exists a constant $C>0$ such that for all $n>0$,
$$ \left \| f(x)-p_n(x) \right \|_{\infty} \leq C  \rho^{-n}.$$
 \end{theorem}
If $f(x)$ is Lipschitz continuous on $[-1,1]$ then
\begin{equation}
f(x) = \sum_{k=0}^\infty a_k T_k(x), \quad a_k = \frac{2}{\pi} \int_{-1}^1 \frac{f(x) T_k(x)}{\sqrt{1-x^2}} dx,
\end{equation}
where $T_k$ denotes the degree $k$ Chebyshev polynomial (and for $a_0$, we multiply by $\frac{1}{\pi}$ instead of $\frac{2}{\pi}$). Furthermore if $p_n(x)$ is the $n$th degree Chebyshev interpolant then
\begin{equation}
f(x)-p_n(x) = \sum_{k=n+1}^{\infty} a_k \left( T_k(x)-T_m(x)\right),
\end{equation}
where
\begin{equation}
m = \left [ (k+n-1)(\text{mod }2n) - (n-1)\right ],
\end{equation}
implying we can determine the accuracy of the interpolant $p_n(x)$ by inspecting the Chebyshev coefficients \cite{Trefethen2013}. Chebfun's standardChop method determines the minimum required degree by searching for a plateau of low magnitude coefficients \cite{Aurentz:2017:CCS:3034774.2998442}.  For example, Figure~\ref{Coeff_example} shows the first 128 coefficients of $f(x)=\exp \left( \sin \left( \pi x \right) \right)$. We see that all coefficients after the first 46 have magnitude less than $10^{-15}$. In this case, Chebfun determines the ideal degree to be 50.

\begin{figure}[!htb]
\centering
\includegraphics[scale = 0.5]{coeff_plot.eps}
\caption{Chebyshev coefficients for $f(x)=\exp \left( \sin \left( \pi x \right) \right)$.}
\label{Coeff_example}
\end{figure}

\section{Partition of unity formalism}
\label{PUM_FORM_SEC}
 Suppose we have an overlapping covering $\{ \Omega_k \}_{k=1}^N$ on a bounded region $\Omega$. A partition of unity is a collection of real valued functions $\{w_k(x)\}_{k=1}^N$ such that:
\begin{itemize}
\item $w_k(x)$ has support within $\Omega_k$,
\item each $w_k(x)$ is nonnegative,
\item $\forall x \in \Omega, \quad \sum_{k=1}^N w_k(x)=1$.
\end{itemize}
The functions $\{w_k(x)\}_{k=1}^N$ are called the \textit{weights} of the partition. Suppose now that $\Omega=[-1,1]$ and each $\Omega_k$ is an interval. We can use the partition of unity $\{w_k(x)\}_{k=1}^N$ to construct an approximating function. Suppose that for $m \geq 0$ we have a function $f \in C^{m}([-1,1])$, each weight $w_k(x)\in C^{m}([-1,1])$ and for each patch $\Omega_k$ we have an approximation $s_k(x)$ of $f(x)$. Then the function
\begin{equation}
\label{POUAPPROX}
s(x) = \sum_{k=1}^N w_k(x)s_k(x)
\end{equation}
can be used to approximate $f(x)$ and its derivatives \cite{wendland2004scattered}.

\begin{theorem}
\label{PUMCON}
Suppose $f \in C^{m}([-1,1])$ and for each patch $\Omega_k$ we have a function $s_k(x)$ such that
$$ \|f^{(\alpha)}(x)-s_k^{(\alpha)}(x)\|_{L_{\infty}(\Omega_k)} \leq \varepsilon_k(\alpha) $$
for $\alpha \leq m$. Thus for $j\leq m$, if $s(x)$ is the approximation (\ref{POUAPPROX}) then
\begin{equation}
\left \|f^{(j)}(x)- s^{(j)}(x) \right \|_{L_{\infty}(\Omega_k)} \leq \sum_{k=1}^N\sum_{i=0}^j \binom{j}{i} \left \| w_k^{(j-i)}(x) \right \|_{L_{\infty}(\Omega_k)} \epsilon_k(i).
\end{equation}
\end{theorem}
\begin{proof}
Since $\sum_{k=1} w_k(x)=1$, $\sum_{k=1} w_k(x)f(x)=f(x)$. Thus
\begin{equation}
\begin{aligned}
\frac{d^{j}}{d x^j}f(x)-\frac{d^{j}}{d x^j} \sum_{k=1}^N w_k(x)s_k(x) &= \frac{d^{j}}{d x^j} \sum_{k=1}^N w_k(x)(f(x)-s_k(x)) \\
&= \sum_{k=1}^N\sum_{i=0}^j \binom{j}{i} w_k^{(j-i)}(x) \left( f^{(i)}(x)-s_k^{(i)}(x) \right).
\end{aligned}
\end{equation}
The result follows from here by the triangle inequality.
\end{proof}

\section{Convergence analysis}
\label{converge_sec}
In this section we consider a single interval partitioned into two overlapping parts, i.e.
$[-1,t]$,$[-t,1]$, where $t$ is the overlap parameter such that $0<t<1$. For the weights, we use Shepard's method \cite{shepard1968two} based on the compactly supported, infinitely differentiable shape function
\begin{align}
\psi(x) = \begin{cases}
\exp \left(  1 - \frac{1}{1-x^2}\right) & |x| < 1, \\
0 & |x| \geq 1.
\end{cases}
\end{align}
We define support functions
\begin{align}
\psi_{\ell}(x) = \psi \left( \frac{x+1}{1+t} \right) \quad \text{ and } \quad \psi_{r}(x) = \psi \left( \frac{x-1}{1+t} \right) ,
\end{align}
to construct the PU weight functions
\begin{align}
w_{\ell}(x) = \frac{\psi_{\ell}(x)}{\psi_{\ell}(x)+\psi_{r}(x)} \quad \text{and} \quad w_{r}(x) = \frac{\psi_{r}(x)}{\psi_{\ell}(x)+\psi_{r}(x)},
\label{PUW}
\end{align}
where $w_{\ell}(x),w_{r}(x)$ have support on the left and right intervals, respectively.

Suppose that $s_{\ell}(x),s_{r}(x)$ approximate $f(x)$ on $[-1,t]$, $[-t,1]$ respectively and are both infinitely smooth. Let
\begin{equation}
s(x) = w_{\ell}(x)s_{\ell}(x)+w_{r}(x)s_{r}(x),
\label{PUM2}
\end{equation}
where $s(x)$ is the partition of unity approximation. Following Theorem~\ref{PUMCON} we have for $x \in [-1,1]$ that
\begin{equation}
\begin{aligned}
\left | f(x)-s(x) \right | &= \left | w_{\ell}(x) \left( f(x) - s_{\ell}(x) \right) + w_{r}(x) \left( f(x) - s_{r}(x) \right) \right | \\
&\leq w_{\ell}(x) \left | f(x) - s_{\ell}(x) \right | + w_{r}(x) \left | f(x) - s_{r}(x) \right |.
\end{aligned}
\end{equation}
We conclude that
\begin{align}
\left \| f(x)-s(x) \right \|_{L_{\infty}[-1,1]} \leq \max \left( \left \| f(x)-s_{\ell}(x) \right \|_{L_{\infty}[-1,t]} , \left \| f(x)-s_{r}(x) \right \|_{L_{\infty}[-t,1]} \right).
\label{POU_UP}
\end{align}
This implies that the Partition of unity Method (PUM) preserves the accuracy of its local approximants. We also have that $s(x)$ is infinitely smooth. For the first derivative we have
\begin{equation}
\begin{aligned}
\left | f'(x)-s'(x) \right | &\leq \left | w_{\ell}(x) \left( f'(x)-s_{\ell}'(x) \right) \right |+\left | w_{r}(x) \left( f'(x)-s_{r}'(x) \right) \right | \\
&+\left | w_{\ell
}'(x) \left( f(x)-s_{\ell}(x) \right) \right | +\left | w_{r}'(x) \left( f(x)-s_{r}(x) \right) \right |,
\end{aligned}
\end{equation}
giving us
\begin{equation}
\begin{aligned}
\left \| f'(x)-s'(x) \right \|_{L_{\infty}[-1,1]} &\leq \max \left( \left \| f'(x)-s_{\ell}'(x) \right \|_{L_{\infty}[-1,t]} , \left \| f'(x)-s_{r}'(x) \right \|_{L_{\infty}[-t,1]} \right) \\
&+\left \| w_{\ell}'(x) \right \|_{L_{\infty}[-t,t]} \left \| f(x)-s_{\ell}(x)\right \|_{L_{\infty}[-t,t]}\\
&+ \left \| w_{r}'(x) \right \|_{L_{\infty}[-t,t]} \left \| f(x)-s_{r}(x)\right \|_{L_{\infty}[-t,t]},
\end{aligned}
\label{diff_error}
\end{equation}
since the derivatives of the weights have support only on the overlap. For $t \ll 1$, the weights steepen to become nearly step functions. This causes the derivatives of the weights to be large in magnitude, resulting in an increase in the error for the derivative.

Since $w_{\ell}'(x)=-w_{r}'(x)$, from (\ref{diff_error}) we can infer
\begin{equation}
\begin{aligned}
\left \| f'(x)-s'(x) \right \|_{L_{\infty}[-1,1]} \leq \max \left( \left \| f'(x)-s_{\ell}'(x) \right \|_{L_{\infty}[-1,t]} , \left \| f'(x)-s_{r}'(x) \right \|_{L_{\infty}[-t,1]} \right)& \\
+\left \| w_{\ell}'(x) \right \|_{L_{\infty}[-t,t]} \max \left( \left \| f(x)-s_{\ell}(x)\right \|_{L_{\infty}[-t,t]}
, \left \| f(x)-s_{r}(x)\right \|_{L_{\infty}[-t,t]} \right). &
\end{aligned}
\label{diff_error_2}
\end{equation}
We have that $x=0$ is a critical point of $w_{\ell}'(x)$ and for $t<0.4$ it can be shown that the maximum of $\left |w_{\ell}'(x) \right |$ occurs at $x=0$. Since
\begin{equation}
w_{\ell}'(0) = -\frac{(1+t)^2}{t^2 (2+t)^2},
\end{equation}
we can infer that $\left \| w_{\ell}'(x) \right \|_{L_{\infty}[-t,t]} = \Theta(t^{-2})$ as $t\to 0$. The norm of the Chebyshev differentiation operator is $\Theta(n^2)$ (for $n$ nodes), implying that the two terms on the right-hand side of (\ref{diff_error_2}) are balanced if $t^{-2}=\Theta(n^2)$, or equivalently $t = \Theta \left( \frac{1}{n} \right)$. A simple example of a split can be seen in Figure~\ref{ARCTAN}.

\begin{figure}[!htb]
\centering
\includegraphics[scale = 0.5]{PUTANEXAMP.eps}
\caption{Plot of the PU approximation with overlap parameter $t=0.1$ for $f(x)=\arctan \left( x/0.1 \right)$, where the thick lines represent the domains of the left and right approximation. Here $\left \| f(x)-s(x) \right \|_{L_{\infty}[-1,1]}= 2.4 \mathrm{e}{-15}$ and $\left \| f'(x)-s'(x) \right \|_{L_{\infty}[-1,1]}= 1.7 \mathrm{e}{-13}$.}
\label{ARCTAN}
\end{figure}

\section{Recursive algorithm}
\label{PUM_recurse}
In order to allow for adaptation to specific features of $f(x)$, we next describe a recursive bisection algorithm that works similarly to Chebfun's splitting algorithm \cite{driscoll2014optimal} and to that of \cite{tobor2006reconstructing}. Suppose we want to construct a PU approximation $s_{[a,b]}(x)$ on the interval $[a,b]$ using Chebyshev interpolants on the patches. If $f(x)$ can be resolved by a Chebyshev interpolant $s(x)$ of length $n_{\text{max}}$ on $[a,b]$ then
\begin{align}
s_{[a,b]}(x) = s(x).
\end{align}
Otherwise we split the interval into two overlapping domains and blend the results as in (\ref{PUM2}):
\begin{align}
s_{[a,b]}(x) = w_{\ell}(x)
s_{\left [a,a+\delta \right ]}(x)+ w_{r}(x) s_{\left [b-\delta,b\right ]}(x),
\end{align}
where $w_{\ell},w_{r}$ are the PU weight functions defined in (\ref{PUW}) (but defined for $[a,b]$, and $\delta= (1+t) \left( \frac{a+b}{2} \right)$).

We define a binary tree $T$ with each node $\nu$ having the following properties:
\begin{itemize}
\item interval($\nu$):=the domain of the patch
\item \child{0}($\nu$),\child{1}($\nu$):=respective left and right subtrees of $\nu$ (if split)
\item \weight{0}($\nu$),\weight{1}($\nu$):=respective left and right weights of $\nu$ (if split)
\item interpolant($\nu$):=Chebyshev interpolant on interval($\nu$) if $\nu$ is a leaf
\item values($\nu$):=values of the function we are approximating at the Chebyshev points of $\nu$.
\end{itemize}
We define root($T$) as the root node of $T$. In Algorithm~\ref{alg2} we formally describe how we refine our splitting; the merge method is described in section~\ref{Merging_sec}.

\begin{algorithm}[!h]
\caption{splitleaves($\nu$,$n_{\text{max}}$,$t$)}
\label{alg2}
\begin{algorithmic}
\IF{$\nu$ is a leaf and $f(x)$ cannot be resolved by interpolant($\nu$)}
\STATE Define new nodes $\nu_0$, $\nu_1$.
\STATE $[a,b]$:=interval($\nu$)
\STATE $\delta:= \frac{b-a}{2} \left( 1+t \right)$
\STATE interval($\nu_0$):= $[a,a+\delta]$
\STATE interval($\nu_1$):= $[b-\delta,b]$
\FOR{$k=0,1$}
\STATE \child{k}($\nu$) := $\nu_k$
\ENDFOR
\STATE \weight{0}($\nu$),\weight{1}($\nu$):= weights in (\ref{PUW}) defined for $[a,a+\delta]$,$[b-\delta,b]$
\ELSIF{$\nu$ is a leaf and $f(x)$ can be resolved by a Chebyshev interpolant with degree less than $n_{\text{max}}$}
\STATE interpolant($\nu$):=minimum degree interpolant $f(x)$ can be resolved by \par\qquad\ \enspace
as determined by Chebfun
\ELSE
\FOR{$k=0,1$}
\STATE splitleaves(\child{k}($\nu$),$n_{\text{max}}$,$t$)
\ENDFOR
\STATE merge($\nu$,$n_{\text{max}}$)
\ENDIF
\end{algorithmic}
\end{algorithm}

We first initialize the tree $T$ with a single node $\nu$ where interval($\nu$)=$[a,b]$. Next we repeatedly call the splitleaves method until each leaf of $T$ has a Chebyshev interpolant that can resolve $f(x)$ with degree less than $n_{\text{max}}$, as seen in Algorithm~\ref{alg6}. For each leaf $\nu$ of $T$, sample($T$,$f(x)$) sets values($\nu$) using $f(x)$ . For a leaf $\nu$, we determine if a Chebyshev interpolant can resolve $f(x)$ using Chebfun's standardChop method with values($\nu$) (as described in Section~\ref{sec_cheb}). Using $T$ we can evaluate $s_{[a,b]}(x)$ recursively as demonstrated in Algorithm~\ref{alg3}.

\begin{algorithm}[!h]
\caption{$T$=refine($n_{\text{max}}$,$t$,$f(x)$)}
\label{alg6}
\begin{algorithmic}
\STATE Define $T$ as a tree with a single node.
\WHILE{$T$ has unresolved leaves}
\STATE sample($T$,$f(x)$)
\STATE splitleaves(root($T$),$n_{\text{max}}$,$t$)
\ENDWHILE
\end{algorithmic}
\end{algorithm}

\begin{algorithm}[!h]
\caption{v=eval($\nu$,$x$)}
\label{alg3}
\begin{algorithmic}
\IF{$\nu$ is a leaf}
\STATE $p$:=interpolant($\nu$)
\STATE v:= $p(x)$
\ELSE
\STATE $v_0,v_1$:=0

\STATE $w_0$:=\weight{0}($\nu$)
\STATE $w_1$:=\weight{1}($\nu$)
\FOR{$k=0,1$}
\IF{$x \in$ interval(\child{k}($\nu$))}
\STATE $v_k$:=eval(\child{k}($\nu$),$x$)
\ENDIF
\ENDFOR
\STATE v := $w_0(x)v_0 + w_1(x)v_1$
\ENDIF
\end{algorithmic}
\end{algorithm}

As a simple example, we approximate the function $f(x)=\arctan \left( \frac{x-0.25}{0.001} \right)$ with $n_{\max}=128$. In order to resolve to machine precision, a global Chebyshev interpolant on the interval $[-1,1]$ requires 25743 nodes while our method requires 523. Chebfun with non-overlapping splitting requires 381 nodes. Overlapping splittings will typically require more total nodes while offering the benefit of global smoothness. The result can be seen in Figure~\ref{ARCTAN2}.

\begin{figure}[!htb]
\centering
\includegraphics[scale = 0.5]{non_merging_example.eps}
\caption{Plot of the partition of unity approximation with overlap parameter $t=0.1$ for $f(x)=\arctan \left( (x-0.25)/0.001 \right)$, where the solid blue lines represent the patches.}
\label{ARCTAN2}
\end{figure}

We can deduce from (\ref{POU_UP}) that $s_{[a,b]}(x)$ will approximate $f(x)$. Moreover, our method implicitly creates a PU on the leaves of the tree through the product of the weights at each level.

\begin{theorem}
Let an approximation $s_{[a,b]}(x)$ be as in (\ref{PUM2}). Then the tree that represents $s_{[a,b]}(x)$ implicitly defines a PU $\{w_k(x)\}_{k=1}^M$, where $w_k(x)$ has compact support over the $k$th leaf.
\end{theorem}
\begin{proof}
Suppose that on the domain $[a,b]$ we have PU's $\{w_{\ell k}(x)\}_{k=1}^{M_\ell}$, $\{w_{rk}(x)\}_{k=1}^{M_r}$ for the leaves of the left and right child respectively. We claim that
\begin{equation}
\{w_{\ell}(x) w_{\ell k}(x)\}_{k=1}^{M_\ell} \cup \{w_r(x) w_{r k}(x)\}_{k=1}^{M_r}
\label{UNPU}
\end{equation}
forms a PU over the leaves of the tree. We first observe that $w_{\ell}(x)w_{\ell k}(x)$ will have support in $\supp \left( w_{1k}(x) \right)$, the domain of the respective leaf. This is similarly true for $w_{r}(x)w_{r k}(x)$.

Next suppose that $x \in \sup \left( w_{\ell}(x)\right) \cap \, \sup \left( w_{r}(x)\right)^C$. Then $w_{\ell}(x)=1$ and $w_r(x)=0$, so
\begin{equation}
\sum_{k=1}^{M_\ell} w_{\ell}(x) w_{\ell k}(x)+ \sum_{k=1}^{M_r} w_{r}(x) w_{r k}(x) = \sum_{k=1}^{M_\ell} w_{\ell k}(x) = 1,
\end{equation}
since $\{w_{\ell k}(x)\}_{k=1}^{M_\ell}$ is a PU. This is similarly true if $x \in \sup \left( w_{\ell}(x)\right)^C \cap \sup \left( w_{r}(x)\right)$. Finally if $x \in \sup \left( w_{\ell}(x)\right) \cap \sup \left( w_{r}(x)\right)$ then
\begin{equation}
\begin{aligned}
\sum_{k=1}^{M_\ell} w_{\ell}(x) w_{\ell k}(x)+ \sum_{k=1}^{M_r} w_{r}(x) w_{r k}(x) &=  w_{\ell}(x) \sum_{k=1}^{M_\ell}  w_{\ell k}(x)+ w_{r}(x) \sum_{k=1}^{M_r} w_{r k}(x) \\
&= w_{\ell}(x)+w_{r}(x)=1.
\end{aligned}
\end{equation}
Thus by induction, we have that the product of weights through the binary tree for (\ref{PUM2}) implicitly creates a PU over the leaves.
\end{proof}

\subsection{Merging}
\label{Merging_sec}
As we create the tree we opportunistically merge leaves for greater efficiency. If a particular location in the interval requires a great deal of refinement, the recursive splitting essentially performs a binary search for that location (as was noted about Chebfun splitting in \cite{driscoll2014optimal}). The intermediate splits are not necessarily aiding with resolving the function; they are there just to keep the binary tree full. In Chebfun the recursive splitting phase is followed by a merging phase that discards counterproductive splits. We describe a similar merging operation here, but we allow these merges to take place whenever a leaf splits while its sibling does not, in order to keep the number of leaves from unnecessarily growing exponentially.

\begin{figure}[!htb]
\centering
\adjustbox{valign=t}{
\subfloat[Tree before merging. Here ${a_2<b_1}$ and ${a_{22}<b_{21}}$.]{
     \begin{forest}
for tree={circle,draw, l sep=20pt,,scale=0.98}
[ {$\begin{array}{c}
[a,b] \\
s_{[a,b]}(x)
\end{array}$}
    [ {$\begin{array}{c}
 [a,b_1]  \\
s_{[a,b_{1}]}(x) \\
 w_{\ell_1}(x)
\end{array}$},name=left_p]
    [ {{$\begin{array}{c}
    [a_2,b] \\
s_{[a_{2},b]}(x) \\
 w_{r_1}(x)
\end{array}$}}
      [{$\begin{array}{c}
 [a_2,b_{21}] \\
s_{[a_2,b_{21}]}(x) \\
 w_{\ell_2}(x)
\end{array}$},name=right_p]
      [{$\begin{array}{c}
  [a_{22},b] \\
s_{[a_{22},b]}(x) \\
 w_{r_2}(x)
\end{array}$}]
  ]
]
]
\draw[<->,dotted,thick] (right_p) to[out=north west,in=south] (left_p);
\end{forest}
   \label{TRM_1}
 }}\hfill
\adjustbox{valign=t}{
\subfloat[Tree after merging.]{
     \begin{forest}
for tree={circle,draw, l sep=20pt,scale=0.98}
[ {$\begin{array}{c}
[a,b]\\
s_{[a,b]}(x)
\end{array}$}
    [ {$\begin{array}{c}
 [a,b_{21}] \\
s_{[a,b_{21}]}(x) \\
\hat{w}_{\ell_1}(x)
\end{array}$} ]
    [ {$\begin{array}{c}
[a_{22},b] \\
s_{[a_{22},b] }(x) \\
\hat{w}_{r_1}(x)
\end{array}$} ]
]
]
\end{forest}
   \label{TRM_2}
 }}
\caption{An example of how leaves are merged, where each node is labeled with its domain, PU approximation and weight.}
\label{Tree_merge}
\end{figure}

In Figure~\ref{Tree_merge} we illustrate how we merge leaves; the interval $[a,b_{1}]$ is merged with $[a_2,b_{21}]$. Here we decide to merge if $f(x)$ can be resolved with an interpolant with degree less than $n_{\text{max}}$ on the interval $[a,b_{21}]$. For the new tree we define the left weight $\hat{w}_{\ell_1}(x)$ in Figure~\ref{Tree_merge} as
\begin{align}
\hat{w}_{\ell_1}(x) = \begin{cases}
                1 & x<a_{22}, \\
                w_{\ell_2}(x) & \text{ otherwise.}
\end{cases}
\label{pwe}
\end{align}
Since $w_{\ell_2}(x)=1$ for $x<a_{22}$, $\hat{w}_{\ell_1}(x)$ is smooth. For the right weight we use $\hat{w}_{r_1}(x) = w_{r_2}(x)$; these new weights form a PU. The PU approximation
\begin{align}
\hat{s}(x)=w_{\ell 1}(x) s_{[a,b_{1}]}(x)+w_{r 1}(x) s_{[a_2,b_{21}]}(x)
\end{align}
can be used to approximate  $f(x)$ on $[a,b_{21}]$ since $f(x)$ is resolved at the leaves. In this case $s_{[a,b_{21}]}(x)$ is computed from sampling $\hat{s}(x)$. If the degree of $s_{[a,b_{21}]}(x)$ after Chebfun's chopping is less than $n_{\text{max}}$, we decide to merge. We explain in more detail the merging in Algorithm~\ref{alg5}; here extend($w(x)$,$[a,b]$) piecewise extends the weight $w(x)$ in $[a,b]$ as in (\ref{pwe}).  We show the results for merging in Figure~\ref{MERGE_EXAMPLE} with $f(x)=\frac{1}{x-1.001}$.

\begin{algorithm}[!h]
\caption{merge($\nu$,$n_{\text{max}}$)}
\label{alg5}
\begin{algorithmic}
\IF{\child{0}($\nu$) and \child{0}(\child{1}($\nu$)) (child and grandchild of $\nu$) are leaves and both of the intervals of the leaves can be resolved on}
\STATE Define a new leaf $\nu_0$
\STATE $p_0(x)$:=interpolant(\child{0}($\nu$))
\STATE $p_1(x)$:=interpolant(\child{1}(\child{0}($\nu$)))
\STATE $w_0(x)$:= \weight{0}($\nu$)
\STATE $w_1(x)$:= \weight{1}($\nu$)
\STATE $\hat{s}(x)$:=$w_0(x)p_0(x)+w_1(x)p_1(x)$
\IF{$\hat{s}(x)$ can be resolved by a Chebyshev interpolant $p(x)$ with degree less than $n_{\text{max}}$}
\STATE interval($\nu_0$):=interval(\child{0}($\nu$))$\cup$interval(\child{0}(\child{1}($\nu$)))
\STATE interpolant($\nu_0$):=$p(x)$
\STATE points($\nu_0$):=Chebyshev grid of length deg($p(x)$) on $[a_0,b_1]$
\STATE $\hat{w}_0(x)$:=  \weight{0}(\child{1}($\nu$))
\STATE $\hat{w}_1(x)$:=  \weight{1}(\child{1}($\nu$))
\STATE \weight{0}($\nu$):= extend($\hat{w}_0(x)$,interval($\nu_0$))
\STATE \weight{1}($\nu$):= $\hat{w}_1(x)$
\STATE \child{0}($\nu$):= $\nu_0$
\STATE \child{1}($\nu$):= \child{1}(\child{1}($\nu$))
\ENDIF
\ELSIF{\child{1}($\nu$) is a leaf and \child{1}(\child{0}($\nu$)) is a leaf (and exists)}
\STATE inv(merge($\nu$)) (i.e. apply the algorithm, except swap 0 and 1)
\ENDIF
\end{algorithmic}
\end{algorithm}

\begin{figure}[!htb]
\centering
\subfloat[Tree before merging.]{
\includegraphics[scale = 0.35]{PUBEFORE_MERGE.eps}
   \label{MERGEB}
 }
\subfloat[Tree after merging.]{
\includegraphics[scale = 0.35]{PUAFTER_MERGE.eps}
   \label{MERGEA}
 }
\caption{An example of how the PUM with $t=0.08$, $n_{\text{max}}=128$ resolves $f(x)=\frac{1}{x-1.0005}$ without merging (a) and after merging (b).
}
\label{MERGE_EXAMPLE}
\end{figure}

\subsection{Differentiation matrices}
\label{PUM_matrix}

Next we demonstrate how to construct a first derivative matrix; higher derivative matrices can be similarly constructed.  Suppose we have constructed a splitting represented with the tree $T$. For each node $\nu$ of the tree, we add the following methods:
\begin{itemize}
\item points($\nu$):= provides the Chebyshev points of the leaves of $\nu$
\item leafpoints($\nu$):= provides the Chebyshev points of $T$ in interval($\nu$) i.e. \newline $\text{points(root($T$))} \cap \text{interval($\nu$)}$
\item pointindex($\nu$):=gives the index of points($\nu$) with respect to the points of the parent of $\nu$ (if $\nu$ is a child)
\item leafpointindex($\nu$):=gives the index of leafpoints($\nu$) with respect to the leafpoints of the parent of $\nu$ (if $\nu$ is a child).
\end{itemize}

Let $[\alpha,\beta]=\text{interval($\nu$)}$. We want to construct matrices $M,D$ such that
\begin{equation}
\begin{aligned}
M \left . f(x) \right |_{\text{points($\nu$)}} &= \left . s_{[\alpha,\beta]}(x) \right |_{\text{leafpoints($\nu$)}}, \\
D \left . f(x) \right |_{\text{points($\nu$)}} &= \left . \frac{d}{dx} s_{[\alpha,\beta]}(x) \right |_{\text{leafpoints($\nu$)}}.
\end{aligned}
\end{equation}

Let $I_k = \text{interval(\child{k}($\nu$))}$, $w_k(x)=\text{\weight{k}($\nu$)}$ for $k=0,1$. Then
\begin{equation}
\begin{aligned}
\left . s_{[\alpha,\beta]}(x) \right |_{\text{leafpoints($\nu$)}} &= \sum_{k=0}^1 \left . w_k(x) s_{I_k}(x) \right |_{\text{leafpoints($\nu$)}}, \\
\left . \frac{d}{dx} s_{[\alpha,\beta]}(x) \right |_{\text{leafpoints($\nu$)}} &= \sum_{k=0}^1 \left( \left . w_k(x)  \frac{d}{dx} s_{I_k}(x) + \frac{d}{dx} w_k(x)  s_{I_k}(x) \right) \right |_{\text{leafpoints($\nu$)}} .
\end{aligned}
\label{sum_eval}
\end{equation}
Thus we can recursively build up the differentiation matrix through the tree $T$. Due to the support of the weights, for each term in (\ref{sum_eval}) we only need evaluate the approximation $s_{I_k}(x)$ (or its derivative) for $\text{leafpoints($\nu$)} \cap I_k$, i.e. leafpoints(\child{k}($\nu$)). We describe how to construct the differentiation recursively in Algorithm~\ref{alg4}, using MATLAB notation for matrices. At each leaf the interpolation matrix $M$ has entries given by the barycentric interpolation formula based on second-kind Chebyshev points, as produced by the Chebfun command {\tt barymat} \cite{driscoll2015rectangular}.

\begin{algorithm}
\caption{[$M,D$]=diffmatrix($\nu$)}
\label{alg4}
\begin{algorithmic}
\IF{$\nu$ is a leaf}
\STATE $M$:= the Chebyshev barycentric matrix from points($\nu$) to leafpoints($\nu$)
\STATE $D_x$:= Chebyshev differentiation matrix with grid points($\nu$).
\STATE $D$:=$M D_x$.
\ELSE
\STATE $M,D$:=zeros(length(leafpoints($\nu$)),length(points($\nu$)))
\FOR{$k=0,1$}
\STATE [$M_k$,$D_k$]:= diffmatrix(\child{k}($\nu$))
\STATE $M$(leafpointindex(\child{k}($\nu$)),pointindex(\child{k}($\nu$))) = \par\qquad\(\hookrightarrow\)\enspace \text{diag} $\left( \left . w_k \right |_{\text{leafpoints(\child{k}($\nu$))}} \right)$*$M_k$;
\STATE $D$(leafpointindex(\child{k}($\nu$)),pointindex(\child{k}($\nu$))) = \par\qquad\(\hookrightarrow\)\enspace \text{diag} $\left( \left . w_k \right |_{\text{leafpoints(\child{k}($\nu$))}}  \right)$*$D_k$+ \par\qquad\(\hookrightarrow\)\enspace \text{diag} $\left( \frac{d}{dx} \left . w_k \right |_{\text{leafpoints(\child{k}($\nu$))}} \right)$*$M_k$;
\ENDFOR
\ENDIF
\end{algorithmic}
\end{algorithm}

For $x \in [\alpha,\beta]$ we only need to evaluate the local approximations for the patches $x$ belongs to; this implies that the differentiation matrices will be inherently sparse. For example, Figure~\ref{SPARSE_DX} shows the sparsity of the first derivative matrix for the tree generated in Figure~\ref{ARCTAN2}. In this case, we have a sparsity ratio of around 76\%.

\begin{figure}[!htb]
\centering
\includegraphics[scale = 0.5]{diff_sparse2.eps}
\caption{Sparsity of the first derivative matrix for the tree generated for Figure~\ref{ARCTAN2}.}
\label{SPARSE_DX}
\end{figure}

\section{PUM for boundary-value problems}
\label{PUM_BVP_SEC}
Our method can be applied to solve linear and nonlinear boundary-value problems. For instance, consider a simple Poisson problem with zero boundary conditions:
\begin{equation}
\begin{aligned}
&u''(x) = f(x) \text{ for }-1<x<1 \\
&u(-1)=0, u(1)=0.
\end{aligned}
\label{simp_pois}
\end{equation}
Suppose that we have differentiation and interpolation matrices $D_{xx}$ and $M$ from section~\ref{PUM_matrix}, $X$ is the set of Chebyshev points over all the leaves, and that $X_I$, $X_B$ are the respective interior and boundary points of $X$. Let $E_{I}$ and $E_{B}$ be the matrices that map a vector to its subvector for the interior and boundary indices respectively. Let $F$ be the vector of values used for the local interpolants (i.e. if we had only two leaves whose interpolants used values $F_1,F_2$, we set $F = [F_1^T  F_2^T]^T$). In order to find a PUM approximation $s(x)$ that approximates (\ref{simp_pois}) we find $F$ by solving the following linear system:
\begin{equation}
\begin{bmatrix}
E_{I} D_{xx} \\[1mm]
 E_{B} M
\end{bmatrix}
\begin{bmatrix}
E_{I} F \\[1mm]
E_{B} F
\end{bmatrix}
=
\begin{bmatrix}
\left . f \right |_{X_I} \\[1mm]
0 \\
\end{bmatrix}.
\label{PUM_lin_system}
\end{equation}

Algorithm~\ref{BVP_solve} builds an adaptive solution for the BVP. We first construct a PU approximation $s(x)$ by solving the discretized system in (\ref{PUM_lin_system}). Sampling with $s(x)$, we use Algorithm~\ref{alg6} to determine if the solution is refined and split leaves that are determined to be unrefined. Here we also allow merging for a node with resolved left and right leaves (i.e., the left and right leaves can be merged back together).

\begin{algorithm}
\caption{$T$=refineBVP($n_{\text{max}}$,$t$,BVP)}
\label{alg7}
\begin{algorithmic}
\STATE Define $T$ as a tree with a single node with the domain of the BVP.
\WHILE{$T$ has unrefined leaves}
\STATE Find values for the interpolants $F$ of the leaves of $T$ by solving a discretized \par\qquad\ \enspace system defined by the interpolation and differentiation matrices of $T$.
\STATE sample($T$,$F$)
\STATE $s(x) = \text{eval}(\text{root}(T),x)$ (the PU approximation)
\STATE sample($T$,$s(x)$)
\STATE splitleaves(root($T$),$n_{\text{max}}$,$t$)
\ENDWHILE
\end{algorithmic}
\label{BVP_solve}
\end{algorithm}

\subsection{BVP examples}
\label{BVP_SEC}

We solve the stationary Burgers equation on the interval $[0,1]$ with Robin boundary conditions \cite{reyna1995exponentially}:
\begin{equation}
\begin{aligned}
&\nu u''(x)-u(x)u'(x)=0 \\
&\nu u'(0)-\kappa(u(0)-\alpha)=0 \\
&\nu u'(1)+\kappa(u(1)+\alpha)=0
\end{aligned}
\label{PUM_nlin_system}
\end{equation}
which has nontrivial solution
\begin{equation}
u(x)=-\beta \tanh \left( \frac{1}{2} \beta \nu^{-1} \left( x-\frac{1}{2} \right) \right)
\label{true_sol}
\end{equation}
where $\beta$ satisfies
\begin{equation}
-\frac{1}{2} \beta^2 \text{sech}^2 \left( \frac{1}{4} \beta \nu^{-1} \right)+\kappa \left [ \alpha-\beta \text{tanh} \left( \frac{1}{4} \beta \nu^{-1} \right) \right ]=0.
\end{equation}

We choose $\nu=5 \times 10^{-3},\alpha=1$, and $\kappa=2$. We use {\tt fsolve} in MATLAB to solve the BVP, supplying the Jacobian of the discretized nonlinear system. Starting with a linear guess $u(x)=0$, we update the solution from the latest solve (i.e. if the solution $s(x)$ from Algorithm~\ref{BVP_solve} is determined to be unresolved, we use it as the next initial guess). For this problem we set the Chebfun chopping tolerance to $10^{-10}$. Our solution was resolved to the tolerance we set after four nonlinear solves; as seen in Figure~\ref{NLIN_EXAMPLE}, the final approximation had 298 nodes and the absolute error was less than $10^{-4}$ as seen in Figure~\ref{NLIN_EXAMPLE}. On a machine with processor 2.6 GHz Intel Core i5, the solution was found in 1.3 seconds.

\begin{figure}[!htb]
\centering
\subfloat[Solution with subintervals plotted.]{
\includegraphics[scale = 0.35]{non_lin_plot.eps}
   \label{NLIN_EXAMPLEA}
 }
\subfloat[Plot of the error.]{
\includegraphics[scale = 0.35]{non_lin_residual.eps}
   \label{NLIN_EXAMPLEB}
 }
\caption{Numerical solution using the PU method and residual for the BVP (\ref{PUM_nlin_system}) with $\nu=5 \times 10^{-3}$, $t=0.1$, and $n_{\text{max}} = 128$.}
\label{NLIN_EXAMPLE}
\end{figure}

We preformed a similar experiment but instead used global Chebyshev interpolants. We adapt by increasing the degree of the polynomial from $n$ to $\text{floor}(1.5 n)$, starting with $n=128$. We stop when we have a solution that is refined to the tolerance $10^{-10}$ (same as before). Both the solution and residual are in Figure~\ref{GNLIN_EXAMPLE}; here we have the absolute error is higher at 1.8e-2. The solution took 3.2 minutes on the same machine. There are two main reasons why the global solution performs much slower. First, in order to resolve the true solution with the tolerance $10^{-10}$, the global Chebyshev solution requires 766 nodes versus 300 for the PU approximation. Secondly, when adapting with the PUM, if a leaf is determined to be refined, the number of nodes is reduced as dictated in Algorithm~\ref{alg2} and the leaf is not split in further iterations. This keeps the total number of nodes lower while adapting.

\begin{figure}[!htb]
\centering
\subfloat[Solution with subintervals plotted.]{
\includegraphics[scale = 0.35]{globalburger.eps}
   \label{GNLIN_EXAMPLEA}
 }
\subfloat[Plot of the error.]{
\includegraphics[scale = 0.35]{globalburgererr.eps}
   \label{GNLIN_EXAMPLEB}
 }
\caption{Numerical solution using the global Chebyshev method and residual for the BVP (\ref{PUM_nlin_system}) with $\nu=5 \times 10^{-3}$.
}
\label{GNLIN_EXAMPLE}
\end{figure}

\section{Discussion}
Our method offers a simple way to adaptively construct infinitely smooth approximations of functions that are given explicitly or that solve BVPs. By recursively constructing the PU weights with the binary tree, we avoid the need to determine the neighbors of each patch (as would be needed with the standard Shepard's PU weights). While this is not a serious issue in one dimension, the complexity of how the patches overlap increases with higher dimension. For example, in 2D we could build a similar method on a box where we use tensor product Chebyshev approximations. We would refine by splitting the box into two overlapping parts (either in $x$ or $y$) and recursively build a binary tree. We similarly would define partition of unities for each of the splits. If we used infinitely smooth weights at the splits, the 2D PU approximation will be infinitely smooth as well.

Our method leaves room for improvement. For instance, while merging helps reduce the number of nodes, in cases where we have a singularity right above the split the PU method over-resolves in the overlap; this can be seen in Figure~\ref{ARCTAN2}. The source of the problem is that patches may be adjacent in space but not in the tree. This could be resolved by a more robust merging algorithm. Alternatively we could determine an optimal splitting location through a Chebyshev-Pad\'{e} approximation as in \cite{driscoll2014optimal}, but the PU adds a layer of complexity since we must optimize not just for the splitting location but the size of the overlap.

Additionally it is possible to construct weights that are not $C^{\infty}$ but have smaller norms in their derivatives. For instance,
\begin{equation}
\begin{aligned}
w_{\ell}(x) &= \begin{cases}
1 & x \leq -t \\
\frac{1}{4t^3} x^3 - \frac{3}{4 t} x+\frac{1}{2} & -t\leq x \leq t \\
0 & x>t
 \end{cases} \\
 w_{r}(x) &= 1-w_{\ell}(x)
\end{aligned}
\end{equation}
defines a $C^1[-1,1]$ piecewise cubic partition of unity, where $\| w_{\ell}'(x)\|_{\infty} = \frac{3}{4t}$. If a BVP requires higher smoothness, we could similarly construct a higher degree polynomial for the weights.

\bibliographystyle{siamplain}


On the Convergence of Ritz Pairs and Refined Ritz Vectors for Quadratic Eigenvalue Problems

Abstract: For a given subspace, the Rayleigh-Ritz method projects the large quadratic eigenvalue problem (QEP) onto it and produces a small sized dense QEP. Similar to the Rayleigh-Ritz method for the linear eigenvalue problem, the Rayleigh-Ritz method defines the Ritz values and the Ritz vectors of the QEP with respect to the projection subspace. We analyze the convergence of the method when the angle between the subspace and the desired eigenvector converges to zero. We prove that there is a Ritz value that converges to the desired eigenvalue unconditionally but the Ritz vector converges conditionally and may fail to converge. To remedy the drawback of possible non-convergence of the Ritz vector, we propose a refined Ritz vector that is mathematically different from the Ritz vector and is proved to converge unconditionally. We construct examples to illustrate our theory.

\section{Introduction}
\label{intro}

Consider the numerical solution of the large quadratic eigenvalue
problem (QEP)
\begin{eqnarray}
     \mathcal{Q}(\lambda) x \equiv ( \lambda^2 M + \lambda D + K ) x = 0,
     \label{eq:QEP}
\end{eqnarray}
where $\lambda \in \mathcal{C}$, $x \in \mathcal{C}^{n} \backslash \{0\}$,
$M$, $D$ and $K$ are $n \times n$ complex matrices with $M=M^H>0$ Hermitian
positive definite. The scalar $\lambda$ and the nonzero vector $x$
in \eqref{eq:QEP} are called an eigenvalue and a corresponding eigenvector
of the quadratic pencil $\mathcal{Q}(\lambda)$  or $( M,D,K )$, respectively.
The pair $(\lambda, x)$ is called an eigenpair of $( M,D,K )$. Since
$M=M^H>0$ in \eqref{eq:QEP}, $\mathcal{Q}(\lambda)$ has $2n$ finite eigenvalues.

QEP \eqref{eq:QEP} arises in a wide variety of scientific and engineering
applications \cite{BetHigMehSchTis_NLEVP,TissMeer01}.
The theoretical framework for general
matrix polynomials and in particular for quadratic pencils can be
found in books by Lancaster  \cite{Lancaster_66} and more recently by Gohberg,
Lancaster and Rodman \cite{GohLanRod_82}. A good survey of
mathematical properties, perturbation analysis, and a variety of numerical
algorithms for QEPs can be found in the paper by Tisseur and
Meerbergen \cite{TissMeer01}.

In practice, a small number of eigenvalues that are nearest to a target $\tau$
or located in a prescribed region of the complex plane
and the corresponding eigenvectors are often of interest.
To this end, we exploit the shift transformation
$\lambda_{\tau} = \lambda - \tau$ with $\mathrm{det}(\mathcal{Q}(\tau)) \neq 0$
to transform \eqref{eq:QEP} to a new QEP of the form
\begin{eqnarray}
     \mathcal{Q}_{\tau}(\lambda_{\tau})x \equiv (\lambda_{\tau}^2 M_{\tau} +
     \lambda_{\tau} D_{\tau} + K_{\tau})x=0, \label{eq:shift-and-invert QEP}
\end{eqnarray}
where $M_{\tau} = M$, $D_{\tau} = 2 \tau M + D$ and $K_{\tau} = \tau^2 M
+ \tau D + K$ is nonsingular. So, without loss of generality,
throughout the paper, we assume that the eigenvalues to be sought are nonzero.

One kind of classical methods for solving QEP \eqref{eq:QEP} is to reformulate
it as a certain standard (or generalized) eigenvalue problem via a so-called
linearization process and then to apply Krylov subspace based methods
or Jacobi-Davidson type methods to solve the corresponding linear
eigenvalue problem. Most of these methods fall into
the category of the Rayleigh-Ritz method that
is widely used for the computation of partial eigenpairs of a standard
linear eigenvalue problem from a given projection subspace. As is well known,
under the assumption that the angle between a desired eigenvector and
the projection subspace tends to zero, there exists a Ritz value that
converges to the desired eigenvalue unconditionally but its
corresponding Ritz vector may fail to converge; furthermore, when one
is concerned with eigenvectors, one can compute certain refined Ritz vectors
whose convergence is guaranteed
\cite{Jia_97,Jia_02,Jia_05,JiaStewart_99,JiaStewart_01}; see also
\cite{Stewart_MaxAlg_Eig_01}.

Over the years, some reliable numerical methods have been
proposed that are used to solve large and sparse QEPs directly. Based on
certain orthogonal projection conditions, various methods are designed to construct
suitable lower dimensional subspaces. Then, the large QEP is projected onto a
given subspace to produce a small sized dense QEP which can be solved by the
standard QR or QZ algorithm. They fall into the category of
the  Rayleigh-Ritz method, as will be described in the next paragraph.
Methods of this type include the residual inverse iteration method
\cite{Huitfeldt-Ruhe90,Meerbergen01,Neumaier85},
the Jacobi-Davidson method \cite{SBFV:96,Sleijpen-vanderVorst-vanGijzen96},
Krylov subspace type methods \cite{Hoffnung-Li-Ye06,Li-Ye03}, the nonlinear
Arnoldi method \cite{Voss04}, second-order Arnoldi (SOAR) type methods
\cite{Bai-Su05,JiaSun11,Lin-Bao06,Wang-Su-Bai06}, the iterated shift-and-invert
Arnoldi method \cite{Ye06} and the semiorthogonal generalized Arnoldi (SGA)
method \cite{Huang-Li-Li-Lin11}.

Now we describe the  Rayleigh-Ritz method for the QEP.
For a given orthonormal matrix $Q \in \mathcal{C}^{n \times m}\ (m \leq n)$,
the  Rayleigh-Ritz method is to find a scalar $\mu \in \mathcal{C}$ and a
unit length vector $\hat{x} \in \mathcal{C}^{m}$ satisfying the orthogonal projection
condition
\begin{align*}
    (\mu^{2} MQ + \mu DQ + KQ)\hat{x} \ \perp\ \mbox{span}\{Q\},
\end{align*}%
which amounts to solving the projected QEP
\begin{equation}
( \mu^2 \widehat{M} +
\mu \widehat{D} + \widehat{K} ) \hat{x}=0, \label{pqep}
\end{equation}
where
\begin{eqnarray}
    \widehat{M} = Q^H M Q, \quad  \widehat{D} = Q^H D Q, \quad \widehat{K}
    = Q^H K Q. \label{eq:q_RR triple}
\end{eqnarray}
If $(\mu, \hat{x})$ with $\|\hat{x}\|=1$ is an eigenpair of $( \widehat{M},
\widehat{D},\widehat{K} )$, i.e., $ (\mu^{2}\widehat{M} + \mu \widehat{D} +
\widehat{K})\hat{x} = 0$, then $\mu$ and $Q\hat{x}$ are, respectively,
called a  Ritz value  and a corresponding  Ritz vector of $( M,D,K )$
with respect to ${\rm span}\{Q\}$, and $(\mu, Q \hat{x})$ is a
 Ritz pair of $( M,D,K )$. Since $M$ is Hermitian positive definite, so
is $\widehat{M}$ for any given $Q$.  Therefore, we have $2m$ finite  Ritz
values.

For a {\em given} $Q$, the assumption that $M$ is Hermitian
positive definite is a {\em sufficient} condition to ensure the finiteness of
both the eigenvalues and the Ritz values.
Without this assumption, $\widehat{M}$
would possibly be {\em singular} for some given orthonormal $Q$. In this case,
there could be some {\em infinite}  Ritz values, the situation would
become much more complicated, and the  Rayleigh--Ritz method may fail to work.
Indeed, as will be seen, some of our important convergence conclusions
cannot be drawn, e.g., the bound in Theorem~\ref{cor:qritzvalue} may not tend to zero
when the subspace $\mbox{span}\{Q\}$ is sufficiently good.
In contrast, as will be clear, QEP \eqref{eq:QEP}
is mathematically equivalent to some standard linear eigenvalue problem
provided that $M$ is nonsingular; see
\eqref{eq:1stGEP}. It is well known that
the standard Rayleigh--Ritz method for the {\em linear} eigenvalue problem
{\em always} computes {\em finite} Ritz values for {\em any} projection subspace.
Therefore, there are some essential differences between the Rayleigh--Ritz
method for (\ref{eq:QEP}) and the method for the linear eigenvalue problem.
As is expected, it is nontrivial to establish a convergence theory
of the Rayleigh--Ritz method for (\ref{eq:QEP}).
As a key step of our further discussions,
we first assume the finiteness of Ritz values for {\em any} projection
subspace span$\{Q\}$. It is simple to justify that for {\em any} orthonormal
$Q$ the Hermitian positive definiteness of $M$ is sufficient to ensure
that of $\widehat{M}$. Generally, what we need in the paper is to
assume that $\|\widehat{M}^{-1}\|$ is uniformly bounded independently of $Q$.
This assumption is true if $M$ is Hermitian positive definite,
as $\|\widehat{M}^{-1}\|\leq\|M^{-1}\|$ for any orthonormal $Q$.
So, purely for simplicity of presentation, we assume that $M$ is
Hermitian positive definite throughout the paper. Nevertheless, we must
keep it in mind that all the convergence results and claims are true
in this paper provided that $\widehat{M}$ is nonsingular and
$\|\widehat{M}^{-1}\|$ is bounded.

In this paper we study the convergence of the  Ritz
value and the corresponding  Ritz vector, and extend some of the results
in \cite{JiaStewart_99,JiaStewart_01,Stewart_MaxAlg_Eig_01} to the
Rayleigh-Ritz method for (\ref{eq:QEP}). Although a number of
Rayleigh-Ritz procedures with respect to different subspaces have been used,
to our best knowledge, there has been no unified convergence result
and general theory. As will be seen later,
carrying out this task is indeed nontrivial and complicated. We establish some
important results similar to those for the linear
eigenvalue problem. It turns out that there exists a  Ritz value that
converges to the desired eigenvalue unconditionally but
the corresponding  Ritz vector may fail to converge even if the corresponding
projection subspace ${\rm span}\{Q\}$ contains a sufficiently accurate
approximation to the desired eigenvector.
It is thus necessary and significant to replace the
 Ritz vector by a refined  Ritz vector that has residual minimization and
is mathematically different from the  Ritz vector.
We prove that the refined  Ritz vector converges
unconditionally provided that the angles between the desired eigenvector
and the subspaces tend to zero.  All convergence results are nontrivial
generalizations of the known results on the Rayleigh-Ritz method and the
refined Rayleigh--Ritz method for the linear eigenvalue problem in
\cite{JiaStewart_99,JiaStewart_01,Stewart_MaxAlg_Eig_01}.

This paper is organized as follows. In Section~\ref{sec:conv_q_Ritz},
we analyze the convergence for  Ritz values and  Ritz vectors and prove that
the  Ritz value is unconditionally convergent but the associated  Ritz vector
may fail to converge. To remedy this drawback, in
Section~\ref{sec:conv_refined_q_Ritz}, we introduce  a refined  Ritz vector
and prove its unconditional convergence.
Finally, we conclude the paper in Section~\ref{sec:conclusion}.

Throughout this paper, the superscripts $H$ and $T$
denote the conjugate transpose and the transpose of a matrix or vector,
respectively.  $I_n$ is the identity matrix of order $n$. We denote by
$\| \cdot \|$ both Euclidean vector norm and the spectral matrix norm.

\section{Convergence of  Ritz values and  Ritz vectors} \label{sec:conv_q_Ritz}

Throughout the paper, let $(\lambda_1, x_1)$ with $\| x_1 \| = 1$
be a desired eigenpair of $( M,D,K )$ and assume that $\lambda_1$ is simple.
Furthermore, we keep in mind the assumption made in the introduction
that $\lambda_1\neq 0$, which is without loss of generality due to the
equivalence of (\ref{eq:QEP}) and (\ref{eq:shift-and-invert QEP}).

We convert QEP \eqref{eq:QEP} to a generalized eigenvalue problem (GEP)
of the form%
\begin{subequations}\label{eq:1stGEP}
\begin{align}\label{eq:1stGEP_eq}
A \left[ \begin{array}{c}
         \lambda x \\ x
     \end{array} \right] = \lambda B \left[ \begin{array}{c}
         \lambda x \\ x
     \end{array} \right],
\end{align}
or a standard linear eigenvalue problem (LEP) of the form
\begin{align}\label{eq:1stSEP_eq}
B^{-1}A \left[ \begin{array}{c}
         \lambda x \\ x
     \end{array} \right] = \lambda \left[ \begin{array}{c}
         \lambda x \\ x
     \end{array} \right],
\end{align}
where
\begin{align}
A = \left[ \begin{array}{cc} -D & -K \\ I_n & 0 \end{array} \right]\quad \text{and}\quad
B = \left[ \begin{array}{cc} M & 0 \\ 0 & I_n \end{array} \right]. \label{eq:mtx_A_B_1stGEP}
\end{align}
\end{subequations}
So $\lambda_1$ is an eigenvalue of the matrix pencil $( A,B )$ or the matrix
$B^{-1}A$ in \eqref{eq:1stGEP} and $v_1  \equiv \left[ \begin{array}{c}
\lambda_1 x_1 \\ x_1\end{array} \right] / \sqrt{1+|\lambda_1|^2 }$
is its corresponding normalized eigenvector. There
are numerous linearizations of QEP \eqref{eq:QEP}.
We use \eqref{eq:1stGEP} for two reasons. The first is that it is a
very commonly used linearlization in the literature. The second is that we
establish our results in this paper by relating the QEP to such linearization.
Other linearizations are certainly possible and useable, but if then we may
have to make a very different and more complicated
analysis in order to establish the convergence theory of
the Rayleigh-Ritz method and refined Ritz vectors for the QEP.

There are unitary matrices
$[v_1, \ X]$ and $[y_1,\ Y] \in \mathcal{C}^{2n\times 2n}$
with $v_1, y_1 \in \mathcal{C}^{2n}$ such that
\begin{align}
     \begin{bmatrix}
          y_1^H \\ Y^H
     \end{bmatrix} A \begin{bmatrix}
          v_1 & X
     \end{bmatrix} = \begin{bmatrix}
          \alpha & s^H \\ 0 & L
     \end{bmatrix}, \quad \begin{bmatrix}
          y_1^H \\ Y^H
     \end{bmatrix} B \begin{bmatrix}
          v_1 & X
     \end{bmatrix} = \begin{bmatrix}
          \beta & t^H \\ 0 & N
     \end{bmatrix}, \label{eq:RR_AB}
\end{align}%
where  $L, N \in \mathcal{C}^{(2n-1) \times (2n-1)}$ and $\lambda_1 = \alpha \beta^{-1}$.
Since $\lambda_1$ is supposed to be simple, it is not an eigenvalue of $( L,N )$.

For a given orthonormal matrix $Q \in \mathcal{C}^{n \times m}$ with $m \leq n$, define
      \begin{eqnarray}
      W = \left[ \begin{array}{cc} Q & 0 \\ 0 & Q \end{array} \right] \label{eq:mtx_W}
      \end{eqnarray}
and let $[ Q, Q^{\bot}]$ be unitary
with $Q^{\bot} \in \mathcal{C}^{n \times (n-m)}$. From now on, throughout the paper,
let $\theta_1$ be the acute angle between $x_1$ and the projection subspace
${\rm span}\{Q\}$ and
\begin{eqnarray}
       q_1 = Q^H x_1, \quad q_1^{\bot} = ( Q^{\bot} )^H x_1. \label{eq:theta}
\end{eqnarray}
Then it holds that \cite[p. 249, Theorem 2.2]{Stewart_MaxAlg_Eig_01}
\begin{eqnarray}
      \| q_1^{\bot} \| = \sin \theta_1, \quad \| q_1 \| = \sqrt{1-\sin^2\theta_1}
      = \cos \theta_1. \label{eq:q1_norm}
\end{eqnarray}
First of all, we want to show that there is a Ritz value $\mu_1$ that converges
to $\lambda_1$ unconditionally when $\sin \theta_1 \to 0$.
The following perturbation result is needed, which is expressed in terms of
the a priori uncomputable $\tan\theta_1$ and is different from
Theorem 1 in \cite{tisseur00}, which is a backward perturbation result
in terms of the a posteriori
computable residual norm of an approximate eigenpair.

\begin{lemma} \label{thm:conv_thm1}
     With $\lambda_1,\ q_1$ and $\theta_1$ defined as above.
     Let $\widehat{M}, \widehat{D}$ and $\widehat{K}$ be defined
     in \eqref{eq:q_RR triple} and $\widehat{q_1}=q_1/\|q_1\|$.
     Then there are perturbation matrices $\mathcal{E}_{\widehat{M}},
     \mathcal{E}_{\widehat{D}}, \mathcal{E}_{\widehat{K}}\in
     \mathcal{C}^{m\times m}$ with
     \begin{subequations} \label{eq:lowerbdd_eps}
           \begin{eqnarray}
                 && \| \mathcal{E}_{\widehat{M}} \| \leq \frac{1}{3} \left( m_0 +
                 \frac{1}{| \lambda_1 |} d_0 +
                 \frac{1}{| \lambda_1 |^2} k_0 \right) \tan \theta_1,
                 \label{eq:lowerbdd_eM} \\
                 && \| \mathcal{E}_{\widehat{D}} \| \leq \frac{1}{3}
                 \left( | \lambda_1 | m_0 +  d_0 +
                 \frac{1}{| \lambda_1 |} k_0 \right) \tan \theta_1,
                 \label{eq:lowerbdd_eD} \\
                 && \| \mathcal{E}_{\widehat{K}} \| \leq \frac{1}{3}
                 \left( |\lambda_1 |^2 m_0 +
                 | \lambda_1 | d_0 +  k_0 \right) \tan \theta_1,
                 \label{eq:lowerbdd_eK}
           \end{eqnarray}
     \end{subequations}
     such that $(\lambda_1,\widehat{q}_1)$ is an exact eigenpair of the perturbed
     $( \widehat{M} + \mathcal{E}_{\widehat{M}}, \widehat{D} + \mathcal{E}_{\widehat{D}},
     \widehat{K} + \mathcal{E}_{\widehat{K}} )$, where
     \begin{equation}\label{m0d0k0}
     m_0=\|M\|,\ d_0=\|D\|,\ k_0=\|K\|.
     \end{equation}
\end{lemma}
\begin{proof}
      Recalling \eqref{eq:theta} and \eqref{eq:q1_norm},
      since
      \begin{align*}
            0 = \left( \lambda_1^2 M + \lambda_1 D + K \right) x_1 = \left( \lambda_1^2 M +
            \lambda_1 D + K \right) \left[ \begin{array}{cc}
                   Q & Q^{\bot}
            \end{array} \right] \left[ \begin{array}{c}
                   Q^H \\ ( Q^{\bot})^H
            \end{array} \right] x_1,
      \end{align*}
      we obtain
      \begin{eqnarray}
           \lambda_1^2 M Q q_1 + \lambda_1 D Q q_1 + K Q q_1 = - \left( \lambda_1^2 M +
           \lambda_1 D+ K \right) Q^{\bot} q_1^{\bot}. \label{eq:pf_conv_thm1_2}
      \end{eqnarray}
      Pre-multiplying \eqref{eq:pf_conv_thm1_2} by $Q^H$ gives
      \begin{equation}
            r_1 \equiv ( \lambda_1^2 \widehat{M} + \lambda_1 \widehat{D} + \widehat{K} )
            \widehat{q}_1 = - ( \lambda_1^2 Q^H M + \lambda_1 Q^H D + Q^H K ) Q^{\bot}
            \frac{q_1^{\bot}}{\| q_1 \|}. \label{eq:pf_conv_thm1_3}
      \end{equation}
      So, noting from (\ref{eq:q1_norm}) that $\tan\theta_1=\frac{\sin\theta_1}
      {\cos\theta_1}=\frac{\|q_1^{\bot}\|}{\| q_1 \|}$, we have
      $$
      \|r_1\|\leq (|\lambda_1|^2m_0+|\lambda_1|d_0+k_0)\tan\theta_1.
      $$
      Define
      \begin{align*}
            \mathcal{E}_{\widehat{M}} = - \frac{1}{3 \lambda_1^2} r_1 \widehat{q}_1^H, \quad
            \mathcal{E}_{\widehat{D}} = -\frac{1}{3\lambda_1} r_1 \widehat{q}_1^H, \quad
            \mathcal{E}_{\widehat{K}} = - \frac{1}{3} r_1 \widehat{q}_1^H.
      \end{align*}
      By \eqref{eq:pf_conv_thm1_3} it is easily seen that $\| \mathcal{E}_{\widehat{M}} \|$,
      $\| \mathcal{E}_{\widehat{D}} \|$ and $\| \mathcal{E}_{\widehat{K}} \|$
      satisfy \eqref{eq:lowerbdd_eps} and
      \begin{eqnarray*}
           \left[\ \lambda_1^2 ( \widehat{M} + \mathcal{E}_{\widehat{M}} ) + \lambda_1
           ( \widehat{D} + \mathcal{E}_{\widehat{D}} ) + ( \widehat{K} +
           \mathcal{E}_{\widehat{K}} ) \ \right] \widehat{q}_1 = 0,
      \end{eqnarray*}
      which completes the proof.
\end{proof}

We may deduce from this lemma that there exists an eigenvalue $\mu_1$ of
$(\widehat{M},\widehat{D},\widehat{K})$ that converges to $\lambda_1$ as
$\theta_1\rightarrow 0$. However, things are subtle and by no means trivial
here. The difficulty is that, unlike a usual matrix perturbation problem where
matrices are {\em given} and {\em fixed} and perturbations are allowed to
{\em change}, here the matrix triple $(\widehat{M},\widehat{D},\widehat{K})$
and the perturbation triple $(\mathcal{E}_{\widehat{M}},\mathcal{E}_{\widehat{D}},
\mathcal{E}_{\widehat{K}})$ {\em change simultaneously} as $\theta_1\rightarrow 0$.
This means that there may be a
possibility that, as $\theta_1$ changes, the eigenvalue $\lambda_1$ of
$(\widehat{M} +\mathcal{E}_{\widehat{M}},\widehat{D}+\mathcal{E}_{\widehat{D}},
\widehat{K}+\mathcal{E}_{\widehat{K}})$ and
the eigenvalues of $(\widehat{M},\widehat{D},\widehat{K})$
become ill conditioned so swiftly
that no eigenvalue of $(\widehat{M},\widehat{D},\widehat{K})$ converges to
$\lambda_1$ though $\theta_1\rightarrow 0$.

Fortunately, by exploiting a theorem of Elsner \cite{elsner82}
(also see \cite[p.168]{StewartSun90})
we can prove that this cannot happen and there is indeed an eigenvalue $\mu_1$
that converges to the desired $\lambda_1$ provided that $\theta_1\rightarrow 0$.
Elsner's theorem states that, given matrices $C$ and $\tilde{C}$ of order $n$,
for any eigenvalue $\lambda$ of $C$ there is an eigenvalue $\tilde{\lambda}$ of
$\tilde{C}$ such that
$$
|\lambda-\tilde{\lambda} |\leq (\|C\|+\|\tilde{C}\|)^{1-\frac{1}{n}}
\|C-\tilde{C}\|^{\frac{1}{n}}.
$$
For our purpose, define the matrices $\widehat{A}$ and $\widehat{B}$ by
\begin{eqnarray}
     \widehat{A} = \left[ \begin{array}{cc}
           -\widehat{D} & -\widehat{K} \\ I_m & 0
     \end{array} \right], \quad \widehat{B} = \left[ \begin{array}{cc}
           \widehat{M} & 0 \\ 0 & I_m
     \end{array} \right].  \label{eq:mtx_hat_AB}
\end{eqnarray}
Then the eigenvalues $\mu$ of $(\widehat{M},\widehat{D},\widehat{K})$ are equal to
those of $( \widehat{A}, \widehat{B} )$, whose normalized eigenvectors $\hat{v}
\equiv \left[ \begin{array}{c} \mu \hat{x} \\ \hat{x} \end{array}
\right]/\sqrt{1+| \mu|^2}$ with $\hat{x}$ the eigenvectors
associated with the eigenvalues $\mu$ of $(\widehat{M},\widehat{D},\widehat{K})$.
Since $\widehat{M}$ is Hermitian positive definite, so is $\widehat{B}$.
Therefore, all the $\mu$ are the eigenvalues of $\widehat{B}^{-1}\widehat{A}$.
Furthermore, it holds that $\|\widehat{B}^{-1}\|\leq \|B^{-1}\|$
for any given orthonormal $Q$ and Hermitian positive definite $M$.

From Lemma~\ref{thm:conv_thm1},
$\lambda_1$ is an eigenvalue of $(\widehat{A}+\mathcal{E}_{\widehat{A}},
\widehat{B}+\mathcal{E}_{\widehat{B}})$
with the perturbation matrices
\begin{eqnarray*}
     \mathcal{E}_{\widehat{A}} = \left[ \begin{array}{cc}
           -\mathcal{E}_{\widehat{D}} & -\mathcal{E}_{\widehat{K}} \\ 0& 0
     \end{array} \right], \quad \mathcal{E}_{\widehat{B}} =
     \left[ \begin{array}{cc}
           \mathcal{E}_{\widehat{M}} & 0 \\ 0 & 0
     \end{array} \right],
\end{eqnarray*}
i.e., an eigenvalue of $(\widehat{B}+\mathcal{E}_{\widehat{B}})^{-1}
(\widehat{A}+\mathcal{E}_{\widehat{A}})$ if $(\widehat{B}+
\mathcal{E}_{\widehat{B}})^{-1}$ exists. Since $\widehat{B}$ is
Hermitian positive definite and its smallest singular value is bounded by
that of $B$ from below, $ \widehat{B}+\mathcal{E}_{\widehat{B}}$ must be
nonsingular for $\theta_1$ small enough. Moreover,
for $\theta_1\rightarrow 0$, it follows from Lemma~\ref{thm:conv_thm1} that
\begin{equation}\label{boundhatB}
\|(\widehat{B}+
\mathcal{E}_{\widehat{B}})^{-1}\|=\|\widehat{B}^{-1}
+O(\mathcal{E}_{\widehat{B}})\|\rightarrow\|\widehat{B}^{-1}\|\leq \|B^{-1}\|
\end{equation}
is uniformly bounded independent of $\theta_1$.
Since $\|\widehat{A}\|$ is always bounded from above as $\|\widehat{D}\|\leq\|D\|$
and $\|\widehat{K}\|\leq\|K\|$, it follows that
$\|\widehat{B}^{-1}\widehat{A}\|\leq\|\widehat{B}^{-1}\|\|\widehat{A}\|$
is uniformly bounded independent of $\theta_1$. As a result, for
$\theta_1\rightarrow 0$, since $\widehat{A}+\mathcal{E}_{\widehat{A}}
\rightarrow \widehat{A}$, it follows from \eqref{boundhatB}
and Theorem~\ref{thm:conv_thm1} that
$$
\|(\widehat{B}+\mathcal{E}_{\widehat{B}})^{-1}
(\widehat{A}+\mathcal{E}_{\widehat{A}})\|\leq
\|(\widehat{B}+\mathcal{E}_{\widehat{B}})^{-1}\|
\|(\widehat{A}+\mathcal{E}_{\widehat{A}})\|
$$
is uniformly bounded independently of $\theta_1$.

Finally, from Theorem~\ref{thm:conv_thm1} and $(\widehat{B}+
\mathcal{E}_{\widehat{B}})^{-1}=\widehat{B}^{-1}
+O(\mathcal{E}_{\widehat{B}})$, it is easily justified
that
$$
\|\widehat{B}^{-1}\widehat{A}-
(\widehat{B}+\mathcal{E}_{\widehat{B}})^{-1}
(\widehat{A}+\mathcal{E}_{\widehat{A}})\|=O(\sin\theta_1).
$$

Based on Elsner's theorem, we have the following result, which, together
with the above discussions, proves the global unconditional convergence
of  Ritz values when $\theta_1\rightarrow 0$.

\begin{theorem}\label{cor:qritzvalue}
Assume that $\theta_1$ is small enough to make $\widehat{B}+
\mathcal{E}_{\widehat{B}}$ nonsingular. There is a  Ritz value $\mu_1$ such
that
\begin{equation}\label{boundqritzvalue}
|\mu_1-\lambda_1|\leq (\|\widehat{B}^{-1}\widehat{A}\|
+\|(\widehat{B}+\mathcal{E}_{\widehat{B}})^{-1}
(\widehat{A}+\mathcal{E}_{\widehat{A}})\|)^{1-\frac{1}{2m}}
\|\widehat{B}^{-1}\widehat{A}-
(\widehat{B}+\mathcal{E}_{\widehat{B}})^{-1}
(\widehat{A}+\mathcal{E}_{\widehat{A}})\|^{\frac{1}{2m}}.
\end{equation}
\end{theorem}

The theorem indicates that as $\theta_1\rightarrow 0$ there is always a  Ritz
value $\mu_1\rightarrow \lambda_1$ unconditionally.
We should comment that bound \eqref{boundqritzvalue} will in general be a too
pessimistic overestimate and be for the worst case.
If, as usually happens in practice, the condition
number of $\lambda_1$ as an eigenvalue of $(\widehat{B}+
\mathcal{E}_{\widehat{B}})^{-1} (\widehat{A}+\mathcal{E}_{\widehat{A}})$ is
bounded, the convergence will be linear in $\theta_1$, much
better than that predicted by bound \eqref{boundqritzvalue}.

Next, we analyze the convergence of the corresponding  Ritz vector $\tilde{x}_1$.
Based on decomposition \eqref{eq:RR_AB}, we can establish the following result,
which is an analogue of Theorem 3.1 in \cite{ipsen00}
for the standard linear eigenvalue problem. The result will be used when
we prove the unconditional convergence of refined  Ritz vectors to be
introduced in the next section.

\begin{lemma} \label{thm:conv_thm2}
      Let $( \mu_1, \tilde{v}_1  )$ with $\| \tilde{v}_1 \| = 1$ be an
      approximation to $\left( \lambda_1, v_1 \right)$ of the matrix pair
      $( A,B )$ with
      $\| v_1 \| = 1$.
      Let
      \begin{align}
             r = A \tilde{v}_1 - \mu_1 B \tilde{v}_1  \label{eq:residual_AB}
      \end{align}
      be the residual of $(\mu_1, \tilde{v}_1 )$, and define
      ${\rm sep}(\mu_1, ( L,N )) := \| ( L - \mu_1 N)^{-1} \|^{-1}$. Then
      \begin{eqnarray}
            \sin \angle ( v_1, \tilde{v}_1) \leq \frac{\| r \|}{{\rm sep}(\mu_1,
            ( L,N ))}. \label{eq:sin_sep}
      \end{eqnarray}
\end{lemma}
\begin{proof}
     From \eqref{eq:RR_AB}, pre-multiplying \eqref{eq:residual_AB} by $Y^H$ leads to
     \begin{align*}
          Y^H r =& Y^H \left( \alpha y_1 v_1^H + y_1 s^H X^H + Y L X^H \right)
          \tilde{v}_1\\
          &- \mu_1 Y^H \left( \beta y_1 v_1^H + y_1 t^H X^H +
          Y N X^H \right) \tilde{v}_1 \\
                =& \left( L - \mu_1 N \right) X^H \tilde{v}_1.
     \end{align*}
     Therefore, it follows from  $\| X^H \tilde{v}_1 \| =
     \sin \angle ( v_1, \tilde{v}_1)$
     that \eqref{eq:sin_sep} holds.
\end{proof}

In terms of the a posteriori computable residual $r$,
Theorem~\ref{thm:conv_thm2} establishes the relationship
between the eigenvector $v_1$ and its approximation $\tilde{v}_1$
for the generalized eigenvalue problem \eqref{eq:1stGEP}.

Let $(\mu_1, \widetilde{x}_1)$ be the  Ritz pair
approximating the desired the desired eigenpair $(\lambda_1,x_1)$ of $(M,D,K)$,
where $\widetilde{x}_1=Q\hat{x}_1$ and $(\mu_1,\hat{x}_1)$
with $\| \hat{x}_1 \| = 1$ is the eigenpair
of $( \widehat{M}, \widehat{D}, \widehat{K} )$.
In terms of $\theta_1$, we attempt to derive one of our main results,
an a priori bound for the  Ritz vector $\hat{x}_1$ as
an approximation to the eigenvector $x_1$.
Note that $\mu_1$ is an eigenvalue of $( \widehat{A}, \widehat{B} )$ and $\hat{v}_1
\equiv \left[ \begin{array}{c} \mu_{1} \hat{x}_1 \\ \hat{x}_1 \end{array}
\right]/\sqrt{1+| \mu_1|^2}$
is its corresponding normalized eigenvector. Similar to \eqref{eq:RR_AB}, there
are unitary matrices $[\hat{v}_1,\ \widehat{X}]$ and $[\hat{y}_1, \ \widehat{Y}]
\in \mathcal{C}^{2m\times 2m}$
with $\widehat{v}_1,\widehat{y}_1 \in \mathcal{C}^{2m}$ such that
\begin{align}
     \begin{bmatrix}
           \hat{y}_1^H \\ \widehat{Y}^H
     \end{bmatrix} \widehat{A} \begin{bmatrix}
           \hat{v}_1 & \widehat{X}
     \end{bmatrix} = \begin{bmatrix}
           \hat{\alpha} & \hat{s}^H \\ 0 & \widehat{L}
     \end{bmatrix}, \quad \begin{bmatrix}
           \hat{y}_1^H \\ \widehat{Y}^H
     \end{bmatrix} \widehat{B} \begin{bmatrix}
           \hat{v}_1 & \widehat{X}
     \end{bmatrix} = \begin{bmatrix}
           \hat{\beta} & \hat{t}^H \\ 0 & \widehat{N}
     \end{bmatrix}, \label{eq:RR_hatAB}
\end{align}
where $\widehat{L}$, $\widehat{N} \in \mathcal{C}^{(2m-1) \times (2m-1)}$
and  $\mu_1 = \hat{\alpha}\hat{\beta}^{-1}$.
Under the only hypothesis that $\sin\theta_1\rightarrow 0$, it is possible that
there is an eigenvalue of $( \widehat{L}, \widehat{N} )$
that could be arbitrarily near or even equal to $\mu_1$. For a multiple and
derogatory $\mu_1$, that is, $\mu_1$ has
more than one trivial or nontrivial Jordan blocks,
there are more than one $\tilde{x}_1 = Q \hat{x}_1$
to approximate the unique eigenvector $x_1$ of $( M,D,K )$.
If $\mu_1$ is near an eigenvalue of $( \widehat{L}, \widehat{N} )$,
we will get a unique $\tilde{x}_1$, but there is no guarantee
that it converges to $x_1$. It leads us to postulate that
$\tilde{x}_1$ will converge provided that
${\rm sep}(\lambda_1,( \widehat{L}, \widehat{N} ))$ is uniformly away from zero
independent of $\theta_1$, i.e., ${\rm sep}(\lambda_1,( \widehat{L}, \widehat{N} ))>c$
with $c$ a positive constant independent of $\theta_1$.
We will, quantitatively, show that it is indeed the case.
Before proceeding, we need the following lemma.

\begin{lemma} \label{lem:conv_lem3}
     Let $u = \left[ \begin{array}{c} u_2 \\ u_1 \end{array} \right]$
     and $\tilde{u} = \left[ \begin{array}{c} \tilde{u}_2 \\ \tilde{u}_1
     \end{array} \right]$ where $u_i, \tilde{u}_i \in \mathcal{C}^n$ for $i = 1, 2$
     and $\| u_1 \| = \| \tilde{u}_1 \| = 1$.
     Then
     $$
      \sin \angle (u_1, \tilde{u}_1) \leq  \min\{\| u \|,\|\tilde{u}\|\}
      \sin \angle (u, \tilde{u}).
     $$
\end{lemma}
\begin{proof}
Since $\|u_1\|=1$, from the definition of $\sin\angle (u,\tilde{u})$,
we have
\begin{align*}
\sin^2\angle (u,\tilde{u})=&\min_{\alpha}\left\|\frac{u}{\|u\|}-\alpha \tilde u\right\|^2\\
=&\min_{\alpha}\left(\left\|\frac{u_1}{\|u\|}-\alpha \tilde{u}_1\right\|^2+
\left\|\frac{u_2}{\|u\|}-\alpha \tilde{u}_2\right\|^2\right)\\
\geq&\min_{\alpha}\left\|\frac{u_1}{\|u\|}-\alpha \tilde{u}_1\right\|^2\\
=&\frac{1}{\|u\|^2}\min_{\alpha}\|u_1-\alpha\tilde{u}_1\|^2\\
=&\frac{1}{\|u\|^2}\sin^2\angle (u_1,\tilde{u}_1).
\end{align*}
In the same way, we can also prove that
$$
\sin \angle (u_1, \tilde{u}_1) \leq  \|\tilde{u}\| \sin \angle (u, \tilde{u}).
$$
Therefore, the assertion holds.
\end{proof}

\begin{theorem} \label{thm:conv_thm3}
     Let $( \widehat{A}, \widehat{B} )$ be defined in \eqref{eq:mtx_hat_AB} and
     it have decomposition \eqref{eq:RR_hatAB}. Suppose that the  Ritz pair
     $(\mu_1,\tilde{x}_1)$  is used to approximate the desired eigenpair
     $(\lambda_1,x_1)$ with $\| \tilde{x}_1 \| = \| x_1 \| = 1$.
     If {\rm sep}$(\lambda_1, ( \widehat{L}, \widehat{N} )) > 0$, then
     \begin{equation}\label{boundqritzvector}
           \sin \angle ( x_1, \tilde{x}_1) \leq  \sin \theta_1 +
           \frac{ | \lambda_1 |^2 m_0 + |\lambda_1| d_0 + k_0}
           {{\rm sep}(\lambda_1, ( \widehat{L}, \widehat{N} ))} \tan \theta_1,
     \end{equation}
     where $m_0,\ d_0$ and $k_0$ are defined in \eqref{m0d0k0}.
\end{theorem}
\begin{proof}
      By the triangle inequality we have
      \begin{eqnarray}
            \angle( x_1, \tilde{x}_1) \leq \angle(x_1, Q Q^H x_1) +
            \angle( QQ^H x_1, \tilde{x}_1).
             \label{eq:pf_conv_thm4_1}
      \end{eqnarray}
      From \eqref{eq:theta} and \eqref{eq:q1_norm}, we have
      \begin{eqnarray}
            \cos \angle( x_1, Q Q^H x_1) = \frac{| x_1^H Q Q^H x_1|}{\| Q Q^H x_1 \|}
            = \| Q^H x_1 \| =  \cos \theta_1. \label{eq:pf_conv_thm4_0}
      \end{eqnarray}
      Let $\hat{q}_1 = \frac{Q^H x_1}{\| Q^H x_1\|}$. From \eqref{eq:pf_conv_thm4_1}
      and \eqref{eq:pf_conv_thm4_0} we get
      \begin{align}
             \sin \angle(x_1, \tilde{x}_1) \leq& \sin \theta_1 +
             \sin \angle( QQ^H x_1, \tilde{x}_1) = \sin \theta_1 +
             \sin \angle ( Q \hat{q}_1, Q \hat{x}_1) \nonumber \\
             =& \sin \theta_1 + \sin \angle ( \hat{x}_1, \hat{q}_1).
             \label{eq:pf_conv_thm4_2}
      \end{align}
      From \eqref{eq:mtx_hat_AB}, it is easily seen that $( \mu_1, \hat{v}_1 \equiv
      \left[ \begin{array}{c} \mu_1 \hat{x}_1 \\ \hat{x}_1 \end{array} \right] )$ is
      an eigenpair of $( \widehat{A}, \widehat{B})$. So we can regard $(\lambda_1,
      \hat{q} \equiv
      \left[ \begin{array}{c} \lambda_1 \hat{q}_1 \\  \hat{q}_1 \end{array} \right] )$
      as an approximation of $(\mu_1, \hat{v}_1)$. Then the residual of
      $(\lambda_1, \hat{q})$ as an approximate eigenpair of $(\widehat{A},\widehat{B})$
      is
      \begin{align*}
            \hat{r} &= \left[ \begin{array}{cc}
                   - \widehat{D} & -\widehat{K} \\ I_m & 0
            \end{array} \right] \left[ \begin{array}{c}
                   \lambda_1 \hat{q}_1 \\  \hat{q}_1
           \end{array} \right] - \lambda_1 \left[ \begin{array}{cc}
                   \widehat{M} & 0 \\ 0 & I_{m}
           \end{array} \right] \left[ \begin{array}{c}
                   \lambda_1 \hat{q}_1 \\ \hat{q}_1
           \end{array} \right] \nonumber \\
           &= \left[ \begin{array}{c}
                  -\left( \lambda_1^2 \widehat{M} + \lambda_1 \widehat{D} +
                  \widehat{K} \right) \hat{q}_1 \\
                  0
           \end{array} \right] \equiv \left[ \begin{array}{c}
                  -\hat{r}_1 \\ 0
           \end{array} \right].
      \end{align*}
      By \eqref{eq:pf_conv_thm1_3} in the proof of Theorem~\ref{thm:conv_thm1} we have
       \begin{eqnarray}
            \frac{\| \hat{r} \| }{ \| \hat{q} \|} = \frac{\| \hat{r}_1 \|}{\|\hat{q}\|}
            \leq \frac{ | \lambda_1|^2 m_0 + | \lambda_1 | d_0 +
            k_0}{\|\hat{q}\|} \tan \theta_1. \label{eq:pf_conv_thm4_3}
       \end{eqnarray}
       From Lemma~\ref{lem:conv_lem3}, Theorem~\ref{thm:conv_thm2} and
       \eqref{eq:pf_conv_thm4_3}, inequality \eqref{eq:pf_conv_thm4_2} satisfies
      \begin{align*}
            \sin \angle(x_1, \tilde{x}_1) &\leq \sin \theta_1 +
            \sin \angle(\hat{x}_1, \hat{q}_1) \\
            &\leq \sin \theta_1 + \|\hat{q}\| \sin \angle ( \hat{v}_1, \hat{q} ) \\
            &\leq \sin \theta_1 + \|\hat{q}\|  \frac{\| \hat{r} \| / \| \hat{q} \| }
            {\mbox{sep}(\lambda_1, ( \widehat{L}, \widehat{N} ))} \\
            &\leq \sin \theta_1 +  \frac{  | \lambda_1 |^2 m_0 + |\lambda_1| d_0 + k_0}
            {\mbox{sep}(\lambda_1, ( \widehat{L}, \widehat{N} ))} \tan \theta_1.
      \end{align*}
\end{proof}

From Theorem~\ref{thm:conv_thm3} we see that sep$(\lambda_1,
( \widehat{L}, \widehat{N} ))>0$ uniformly
is a sufficient condition for the convergence of
the  Ritz vector $\tilde{x}_1$. Furthermore, from
Lemma~\ref{thm:conv_thm1}, since the Ritz value $\mu_1$ approaches the
eigenvalue $\lambda_1$ as $\theta_1 \to 0$, by the
continuity argument we have sep$(\mu_1, ( \widehat{L}, \widehat{N} ))
\to$sep$(\lambda_1, ( \widehat{L}, \widehat{N} ))$.
However, as we have argued above, sep$(\mu_1, ( \widehat{L}, \widehat{N} ))$
can be arbitrarily small (and even be exactly zero) when $\mu_1$ is arbitrarily
near other eigenvalues (or is associated with a multiple eigenvalue)
of $( \widehat{L},\widehat{N} )$. Consequently, while the  Ritz value
converges unconditionally once $\theta_1 \to 0$, the corresponding  Ritz
vector may fail to converge or may converge very slowly or irregularly.

In the following, we give an example to illustrate that the  Ritz vector fails
to converge to the desired eigenvector.
\begin{example}\label{Example}
    Consider QEP \eqref{eq:QEP} with
    \begin{eqnarray*}
        M = \left[ \begin{array}{ccc}
           1 & 1 & 0 \\ 1 & 2 & 1 \\ 0 & 1 & 2
        \end{array} \right], \ D = \left[ \begin{array}{rrr}
           -5.5 & -5 & 0 \\ -5 & -11 & -3 \\ 0 & -3 & -4
        \end{array} \right], \ K = \left[ \begin{array}{rrr}
           6 & 6 & 0 \\ 6 & 9 & 2 \\ 0 & 2 & 2
        \end{array} \right].
    \end{eqnarray*}
    It is easy to see that $M$ and $K$ are symmetric positive definite and
    $(1, [0, 0, 1]^T)$ is an eigenpair of the QEP.
\end{example}
Suppose that we have come up with an orthonormal basis
\begin{eqnarray*}
    Q = \left[ \begin{array}{cc}
       0 & \frac{8}{\sqrt{73}} \\ 0 & -\frac{3}{\sqrt{73}} \\ 1 & 0
    \end{array} \right].
\end{eqnarray*}
Then we have $\sin\theta_1=0$ exactly, and the projected matrices are
\begin{align*}
    \widehat{M} &= Q^H M Q = \left[ \begin{array}{rr}
        2 & - \frac{3}{\sqrt{73}} \\ - \frac{3}{\sqrt{73}} & \frac{34}{73}
    \end{array} \right], \\
    \widehat{D} &= Q^H D Q = \left[ \begin{array}{rr}
        -4 & \frac{9}{\sqrt{73}} \\ \frac{9}{\sqrt{73}} & - \frac{211}{73}
    \end{array} \right], \\
    \widehat{K} &= Q^H K Q = \left[ \begin{array}{rr}
       2 & -\frac{6}{\sqrt{73}} \\ -\frac{6}{\sqrt{73}} & \frac{177}{73}
    \end{array} \right],
\end{align*}
from which it follows that
\begin{eqnarray*}
     \widehat{M} + \widehat{D} + \widehat{K} = 0.
\end{eqnarray*}
Since $\widehat{M} + \widehat{D} + \widehat{K}$ is zero, any nonzero vector
$\hat{x}_1$ with $\| \hat{x}_1 \| = 1$ is an eigenvector of $( \widehat{M},
\widehat{D},\widehat{K} )$ corresponding to the double eigenvalue one,
a  Ritz value equal to the desired eigenvalue exactly.
However, the  Rayleigh-Ritz method itself cannot tell us how
to pick up a suitable $\hat{x}_1$. In practice, we might well take $\hat{x}_1 =
[ 1/\sqrt{2}, 1/\sqrt{2}]^T$ and then the approximate eigenvector becomes
$[ 4\sqrt{2}/\sqrt{73}, \ -3/\sqrt{146},\ 1/\sqrt{2}]^T$,
which has no accuracy as an approximation of the desired eigenvector $[0,0,1]^T$ and
is completely wrong. Thus the method can fail even though the projection subspace
span$\{Q\}$ contains the desired eigenvector exactly.

In practice, we would not expect ${\rm span}\{Q\}$ to contain $x_1$ exactly.
Let us investigate the case that ${\rm span}\{Q\}$ contains an enough
accurate approximation to $x_1$, i.e., $\sin\theta_1$ is very small.
We perturb $Q$ by a matrix generated randomly
in a normal distribution by
$10^{-12}\times {\sf randn(3,2)}$ whose 2-norm is $2.2\times 10^{-12}$,
and the resulting
$$
\sin\theta_1=1.7\times 10^{-12}.
$$
The orthonormalized
$$
Q:=Q(Q^HQ)^{-1/2}=\left[\begin{array}{rr}
   -0.000000000001074  & 0.936329177568703\\
   -0.000000000001425  &-0.351123441589302\\
   1.000000000000000  &0.000000000000506\\
\end{array}
\right]
$$
and
$$
\widehat{M}=\left[\begin{array}{rr}
 1.999999999997149 &  -0.351123441589253\\
  -0.351123441589253&  0.465753424656353\\
\end{array}
\right],
$$
$$
\widehat{D}=\left[\begin{array}{rr}
 -3.999999999991449   & 1.053370324770698\\
  1.053370324770698& -2.890410958899234\\
\end{array}
\right],
$$
$$
\widehat{K}=\left[\begin{array}{rr}
1.999999999997149 &-0.351123441589253\\
-0.351123441589253 & 0.465753424656353\\
\end{array}
\right].
$$
We use the Matlab function {\sf polyeig.m} to solve the projected QEP, and the computed
$\mu_1=1.000000000009369$ and the associated eigenvector
$$
\hat{x}_1=[ 0.999982126253304,
-0.005978894038382]^T.
$$
So the  Ritz vector
$$
\tilde{x}_1=Q\hat{x}_1=[
 -0.005598212938803,
 0.002099329850230,
 0.999982126253300]^T
$$
and
$$
\sin\angle(x_1,\tilde{x}_1)\approx  0.005979,
$$
at least nine orders bigger than
$\sin\theta_1$! so $\tilde{x}_1$ is a very poor approximation to $x_1$ for the
given accurate subspace ${\rm span}\{Q\}$. It is also justified that the residual
norm of the  Ritz pair $(\mu_1,\tilde{x}_1)$ is
$$
\|(\mu_1^2M+\mu_1D+K)\tilde{x}_1\|
\approx  0.011958.
$$
The poor accuracy of $\tilde{x}_1$ is due to the fact that
there is another  Ritz value $\mu=1.000000000010143$ that is very
near to $\mu_1$, so that ${\rm sep}(\lambda_1, ( \widehat{L}, \widehat{N} ))$
in \eqref{boundqritzvector} is tiny.

\section{Convergence of refined  Ritz vectors} \label{sec:conv_refined_q_Ritz}

As we have seen in Section~\ref{sec:conv_q_Ritz}, the  Ritz vector may fail
to converge or converges very slowly. Since the  Ritz value is known to converge to
the simple eigenvalue $\lambda_1$ when $\sin \theta_1 \to 0$, this suggests us to deal
with non-converging  Ritz vector by retaining the  Ritz value but replacing
the  Ritz vector with a unit length vector $\tilde{z}_1 \in
\mbox{span}\{ Q \}$ with a suitably
small residual. Naturally, for a given  Ritz value $\mu_1$ we
construct $\tilde{z}_1 = Q \hat{z}_1$, where the unit length
$\hat{z}_1$ is required to be the optimal solution
\begin{eqnarray}
      \hat{z}_1 = \mbox{arg} \min_{\| z \| = 1} \left\| \left( \mu_1^2 M +
      \mu_1 D + K \right) Q z \right\|. \label{eq:optimal_RV}
\end{eqnarray}
The vector $\tilde{z}_1=Q \hat{z}_{1}$ is called a refined  Ritz
vector of $( M,D,K )$ corresponding to $\mu_{1}$ with respect to
${\rm span}\{Q\}$. Obviously, $\hat{z}_1$ is the right singular vector
of the $n\times m$ rectangular matrix
$\left( \mu_1^2 M +\mu_1 D + K \right) Q$ associated with its smallest
singular value. We can compute $\hat{z}_1$ reliably by a standard SVD algorithm or
generally cheaper but still numerically stable cross-product based SVD algorithms;
see \cite{Jia00,JiaSun11} and also \cite{Stewart_MaxAlg_Eig_01}.
For a detailed round-off error analysis on the latter ones,
we refer to \cite{Jia06}.

Before establishing the convergence of the refined  Ritz vector $\tilde{z}_1$,
we need two lemmas.

\begin{lemma} \label{lem:2}
For $W$ defined in \eqref{eq:mtx_W}, let $(\lambda_1, x_1)$ with
$\| x_1 \| = 1$ be the desired eigenpair of
      $( M,D,K )$ and $v_1 = \left[ \begin{array}{c} \lambda_1 x_1 \\
      x_1 \end{array} \right] / \sqrt{1 + | \lambda_1 |^2 }$.
Then it holds that
\begin{equation}
\sin\angle(v_1,{\rm span}\{W\})
=\sin\theta_1.
\end{equation}
\end{lemma}

\begin{proof}
By \eqref{eq:mtx_W} and the definition of $\sin\theta_1$, we have
\begin{align*}
& \hspace{4mm} \sin^2\angle\left(v_1,{\rm span}\left\{ W \right\}\right) \\
&= \frac{1}{1+\mid\lambda_1\mid^2}\min_{u, v\in{\rm span}\{Q\}}
\left\|\left[\begin{array}{c}
\lambda_1 x_1\\
x_1
\end{array}\right]-\left[\begin{array}{c}
u\\
v
\end{array}
\right]\right\|^2\\
&= \frac{1}{1+\mid\lambda_1\mid^2}\min_{u, v\in {\rm span\{Q\}}}(\|\lambda_1
x_1-u\|^2+\|x_1-v\|^2)\\
&= \frac{\mid\lambda_1\mid^2}{1+\mid\lambda_1\mid^2}\min_{u\in
{\rm span\{Q\}}}\|x_1-u\|^2
  +\frac{1}{1+\mid\lambda_1\mid^2}\min_{v\in {\rm span\{Q\}}}\|x_1-v\|^2\\
&= \frac{\mid\lambda_1\mid^2}{1+\mid\lambda_1\mid^2}\sin^2\theta_1+
\frac{1}{1+\mid\lambda_1\mid^2}\sin^2\theta_1\\
&= \sin^2\theta_1.
\end{align*}
\end{proof}

\begin{lemma}\label{lem:3}
Let $(A, B)$ be defined in \eqref{eq:mtx_A_B_1stGEP}. It holds that
\begin{equation}\label{qepgen}
\min_{\|z\|=1}\left\|(A-\mu_1 B)\left[\begin{array}{c}
\mu_1 Qz\\
Qz
\end{array}\right]\right\|=\sqrt{1+\mid\mu_1\mid^2}\min_{\| z \| = 1}
\left\| \left( \mu_1^2 M +
      \mu_1 D + K \right) Q z \right\|
\end{equation}
and the minimum is attained at $\hat{z}_1$.
\end{lemma}

\begin{proof}
Without the minimizations, for any $m$ dimensional vector $z$,
it is direct to verify that the two hand sides are equal. So
the assertion holds.
\end{proof}

\begin{theorem}
      Let $\mu_1$ be the  Ritz value of $( {M},  {D},  {K} )$
      approximating the desired simple eigenvalue $\lambda_1$. Suppose
      {\rm sep}$(\mu_1, ( L, N )) > 0$, where $L, N$ are defined
      in \eqref{eq:RR_AB}.
Then we have
      \begin{equation}\label{rconv}
      \sin \angle(x_1, \tilde{z}_1)<\frac{\sqrt{1+|\lambda_1|^2}
      \left(|\lambda_1-\mu_1|(\|B\|+\|A-\mu_1 B\|)+\|A-\mu_1 B\|\sin\theta_1\right)}
{\cos\theta_1{\rm sep}(\mu_1, ( L, N ))}.
      \end{equation}
\end{theorem}
\begin{proof}
Let $v_1 = \left[ \begin{array}{c} \lambda_1 x_1 \\
      x_1 \end{array} \right] / \sqrt{1 + | \lambda_1 |^2 }$.
     From Lemma~\ref{lem:conv_lem3}, we have
     \begin{align*}
           \sin \angle(x_1, \tilde{z}_1) &\leq \sqrt{1 + | \mu_1 |^2}
           \sin \angle(\left[ \begin{array}{c} \lambda_1 x_1 \\
           x_1 \end{array} \right], \left[ \begin{array}{c} \mu_1 Q \hat{z}_1 \\
           Q \hat{z}_1 \end{array} \right]) \nonumber \\
           &=  \sqrt{1 + | \mu_1 |^2} \sin \angle(v_1, \tilde{z}),
     \end{align*}
     where $ \tilde{z} = \left[ \begin{array}{c} \tilde{z}_2 \\ \tilde{z}_1
     \end{array} \right] \equiv \left[ \begin{array}{c}
     \mu_1 Q \hat{z}_1 \\   Q \hat{z}_1 \end{array} \right] /
     \sqrt{1+|\mu_1|^2 }$.
Let $P_W$ be the orthogonal projector onto the subspace
${\rm span}\{W\}$, where $W = \mbox{diag}(Q,Q)$. Then
$$
P_Wv_1=\left[\begin{array}{c}
\lambda_1 QQ^H x_1\\
QQ^H x_1
\end{array}
\right].
$$
Therefore, we get
$$
\|Q^Hx_1\|^{-1} \left( P_Wv_1-\left[\begin{array}{c}
(\lambda_1-\mu_1) QQ^H x_1\\
0
\end{array}
\right]\right)=\left[\begin{array}{c}
\mu_1 Q\frac{Q^H x_1}{\|Q^H x_1\|}\\
Q\frac{Q^H x_1}{\|Q^H x_1\|}
\end{array}
\right]:=\hat{v}_1,
$$
which is an approximate eigenvector of the desired form in
the left-hand side of \eqref{qepgen}
and $\frac{Q^Hx_1}{\|Q^H x_1\|}$ is a minimizer candidate for \eqref{qepgen}.
Define
$$
f=(I_n-P_W)v_1+f_2
$$
with
$$
f_2=\left[\begin{array}{c}
(\lambda_1-\mu_1) QQ^H x_1\\
0
\end{array}
\right].
$$
Then from $\cos\theta_1=\|Q^Hx_1\|$ we have
$$
\frac{\|f_2\|}{\cos\theta_1}\leq |\lambda_1-\mu_1|.
$$
From Lemma~\ref{lem:2} we get $\|(I_n-P_W)v_1\|=\sqrt{1+|\lambda_1|^2}\sin\theta_1$.
Therefore, we obtain
\begin{align*}
(A-\mu_1 B)\hat{v}_1&= \frac{(A-\mu_1 B)(P_W v_1-f_2)}{\cos\theta_1}\\
&= \frac{(A-\mu_1 B)(v_1-f)}{\cos\theta_1}\\
&= \frac{(\lambda_1-\mu_1)B v_1-(A-\mu_1 B)((I_n-P_W)v_1+f_2)}{\cos\theta_1}.
\end{align*}
Taking the norms gives
$$
\|(A-\mu_1 B)\hat{v}_1\|\leq\frac{\sqrt{1+|\lambda_1|^2}(|\lambda_1-
\mu_1|\|B\|+\|A-\mu_1 B\|\sin\theta_1)}
{\cos\theta_1}+|\lambda_1-\mu_1|\|A-\mu_1 B\|.
$$
From Lemma~\ref{lem:3}, by the optimality property of $\tilde{z}$ we have
{\small
\begin{align*}
 \frac{\|(A-\mu_1 B)\tilde{z}\|}{\sqrt{1+|\mu_1|^2}} &\leq
\frac{\|(A-\mu_1 B)\hat{v}_1\|}{\sqrt{1+|\mu_1|^2}}\\
&\leq
\frac{\sqrt{1+|\lambda_1|^2}(|\lambda_1-\mu_1|\|B\|+\|A-\mu_1 B\|\sin\theta_1)}
{\sqrt{1+|\mu_1|^2}\cos\theta_1}+\frac{|\lambda_1-\mu_1|\|A-\mu_1 B\|}
{\sqrt{1+|\mu_1^2|}}.
\end{align*}}
Since $\frac{\|(A-\mu_1 B)\tilde{z}\|}{\sqrt{1+|\mu_1|^2}}$ is a residual norm,
it is direct from
Theorem~\ref{thm:conv_thm2} that
$$
\sin\angle(v_1,\tilde{z})\leq \frac{\|(A-\mu_1 B)\tilde{z}\|}
{\sqrt{1+|\mu_1|^2}{\rm sep}(\mu_1, ( L, N ))}.
$$
Therefore, it holds from Lemma~\ref{lem:conv_lem3} that
{\small
\begin{align*}
\sin\angle(x_1,\tilde{z}_1)&\leq\sqrt{1+|\mu_1|^2}\sin\angle(v_1,\tilde{z})\\
&\leq\frac{\sqrt{1+|\lambda_1|^2}(|\lambda_1-\mu_1|\|B\|+\|A-\mu_1 B\|\sin\theta_1)}
{\cos\theta_1{\rm sep}(\mu_1, ( L, N ))}+
\frac{|\lambda_1-\mu_1|\|A-\mu_1 B\|}{{\rm sep}(\mu_1, ( L, N ))}\\
&<\frac{\sqrt{1+|\lambda_1|^2}
      (|\lambda_1-\mu_1|\left(\|B\|+\|A-\mu_1 B\|)+\|A-\mu_1 B\|\sin\theta_1\right)}
{\cos\theta_1{\rm sep}(\mu_1, ( L, N ))},
\end{align*}}
which proves (\ref{rconv}).
\end{proof}

Since $\mu_1$ is shown, as Corollary~\ref{cor:qritzvalue} indicates,
to converge to $\lambda_1$ as $\theta_1 \to 0$, we have sep$(\mu_1,
( L, N )) \to \mbox{sep}(\lambda_1, ( L, N ))$,
a positive constant independent of $\theta_1$,
provided that $\lambda_1$ is a simple eigenvalue of $( M,D,K )$.
So the refined  Ritz vector $\tilde{z}_1$ converges to $x_1$
once $\sin\theta_1 \to 0$.

We mention that Hochstenbach and Sleijpen \cite{hochsleij08} proposed a
refined Rayleigh--Ritz method for the polynomial eigenvalue problem
and derived an a priori bound for the residual norm of the refined
Ritz pair as the approximate eigenpair of the problem without invoking any
linearization; see Theorem 5.1 there.

We continue Example ~\ref{Example} to show considerable merits of refined
Ritz vectors. For the case that $x_1$ lies in ${\rm span}\{Q\}$ exactly,
recall that $\mu_1=\lambda_1$ exactly. It is easy to verify that the
smallest singular value of the matrix
$(\mu_1^2M+\mu_1D+K)Q$ is both exactly zero and simple, the optimal solution
$\hat{z}_1=[1,0]^T$ in \eqref{eq:optimal_RV} and
the refined  Ritz vector $\tilde{z}_1=Q\hat{z}_1=x_1$, exactly the desired
eigenvector! So in contrast to the  Ritz vector, the refined  Ritz vector can
pick up the desired eigenvector perfectly.

For the case that ${\rm span}\{Q\}$ is perturbed in the way described
in Example ~\ref{Example}, the optimal solution in \eqref{eq:optimal_RV} is
$$
\hat{z}_1=[1.000000000000000,0.000000000006175]^T
$$
and the refined  Ritz vector
$$
\tilde{z}_1=[
0.000000000004708,
  -0.000000000003593,
   1.000000000000000
]^T.
$$
So
$$
\sin \angle(x_1, \tilde{z}_1)=5.9\times 10^{-12},
$$
which is almost as small as $\sin\theta_1=1.7\times 10^{-12}$
and much more accurate than the
corresponding  Ritz vector $\tilde{x}_1$. Meanwhile, the computed
residual norm of the refined approximate eigenpair $(\mu_1,\tilde{z}_1)$
is
$$
\|(\mu_1^2M+\mu_1D+K)\tilde{z}_1\|=1.3\times 10^{-13},
$$
eleven orders smaller than that of the  Ritz pair $(\mu_1,\tilde{x}_1)$.

\section{Conclusions} \label{sec:conclusion}

Theoretically, we have proved that
there exists a  Ritz value of $( M,D,K )$ that unconditionally converges
to the desired eigenvalue when the angle between the subspace
span$\{ Q \}$ and the desired eigenvector tends to zero. However,
the associated  Ritz vector only converges conditionally. To this end, we
have proposed the refined  Ritz vector that is guaranteed to converge
unconditionally. We have presented some examples to demonstrate
our theory.

The purpose of this paper is not to present efficient and reliable
eigensolvers for QEPs, but rather to establish a general convergence
theory of the  Rayleigh-Ritz method and to show the unconditional convergence
of  Ritz values and refined  Ritz vectors and the conditional convergence
of  Ritz vectors. Refined  Ritz vectors may become a very valuable component
and make great improvement in flexible eigensolvers for QEPs. Numerical
experiments in \cite{JiaSun11} have shown that one can gain very much by
replacing  Ritz vectors by refined  Ritz vectors in second-order Arnoldi
type methods and their implicitly restarted algorithms.

\begin{acknowledgements}
We thank the editor Professor Michiel Hochstenbach and the
referee very much for their valuable suggestions and
comments that made us improve the presentation of the paper very substantially.
\end{acknowledgements}