Spectral Partitioning with Blends of Eigenvectors

Abstract: Many common methods for data analysis rely on linear algebra. We provide new results connecting data analysis error to numerical accuracy, which leads to the first meaningful stopping criterion for two way spectral partitioning. More generally, we provide pointwise convergence guarantees so that blends (linear combinations) of eigenvectors can be employed to solve data analysis problems with confidence in their accuracy. We demonstrate this theory on an accessible model problem, the Ring of Cliques, by deriving the relevant eigenpairs and comparing the predicted results to numerical solutions. These results bridge the gap between linear algebra based data analysis methods and the convergence theory of iterative approximation methods.

\subsection{Converging to a Single Eigenspace}\label{sec:convergesingle}
Let $M \in \mathbb{R}^{n \times n}$, $M = M^t$ be a general symmetric matrix.
Consider the solutions to the equation $M {\bf v} = \lambda {\bf v}$.
Because $M$ is symmetric, there are $n$ eigenvalues in $\mathbb{R}$ (counting multiplicities).
The set of all eigenvalues is the spectrum $\lambda(M)$, which we order decreasingly as $\lambda_1 \geq \lambda_2 \geq \cdots \geq \lambda_n$.
For $k=1,\dots,n$,  let ${\bf v}_k$ be an eigenvector associated with $\lambda_k$, $M {\bf v}_k = \lambda_k {\bf v}_k$, such that ${\bf v}_k^t {\bf v}_l = 0$ whenever $l \neq k$.
Define the eigenspace associated with $\lambda_k$ as the invariant subspace associated with $\lambda_k$, that is
\[
{\cal X}_k := \left\{ \, {\bf x} \in \mathbb{R}^n \,  : \,  M {\bf x} = \lambda_k {\bf x} \, \right\}.
\]
These definitions imply $\mbox{dim}({\cal X}_k) = \mbox{mult}(\lambda_k)$ and ${\cal X}_k = {\cal X}_l$ when $\lambda_k = \lambda_l$.

\begin{rem}
The results in this section are stated and proved in terms of generic symmetric matrix $M$ because they apply beyond spectral graph theory.
Spectral partitioning methods use the eigenvectors{} of $\hat{L} = I-{\hat{A}}$.
Counting eigenvalues from largest to smallest starting with 1, we see $\lambda_2\paren{{\hat{A}}} = \lambda_{n-1}\paren{\L}$ with the same eigenvectors.
Letting ${\bf k} = D^{1/2} {\bf 1}\norm{D^{1/2} {\bf 1}}^{-1}$, the normalized eigenvector of ${\hat{A}}$ associated with $\lambda_1\paren{{\hat{A}}}=1$, one can use $M = \ahat - {\bf k} {\bf k}^t$ in the results of this section.
Subtracting ${\bf k} {\bf k}^t$ moves the eigenvalue $1$ to $0$, or ${\bf k}$ is an eigenvector associated with $0 \in \lambda\paren{\ahat - {\bf k} {\bf k}^t}$.
Thus, for this $M$ and for $k$ where $\lambda_k\paren{M}>0$, we have
$$
  \lambda_k\paren{M} = \lambda_{k+1}\paren{{\hat{A}}}.
$$
In particular, for the Fiedler eigenvalue, $\lambda_1\paren{M} = \lambda_{2}\paren{{\hat{A}}}=1-\lambda_{n-1}\paren{\hat{L}}$,
 and the Fiedler vectors in the associated eigenspace correspond to extremal eigenvalue $\lambda_1\paren{M}$.
Computationally, implementations of iterative methods approximating the eigenvectors of $\lambda_2\paren{{\hat{A}}}$ perform
better with a routine applying the operator $\ahat - {\bf k} {\bf k}^t$.
\end{rem}

Let $({\bf x}, \mu)$, ${\bf x} \in \mathbb{R}^n$, $\mu \in \mathbb{R}$, be an approximate eigenpair of $M$ with $\norm{{\bf x}}
= 1$ and $\mu = \ray{{\bf x}}{M}$, the {\em Rayleigh quotient{}}, which minimizes the function $\norm{M{\bf x} - \theta {\bf x}}$ over all real values $\theta$.   Define the two-norm of the eigenresidual as $\epsilon = \norm{M {\bf x} - \mu {\bf x}}$.
As in~\cite{parlett1998symevp}, we have a simple eigenvalue bound.
By decomposing ${\bf x}$ in the eigenbasis ${\bf x} = \sum_{k=1}^n \alpha_k {\bf v}_k$, we see
$$
  \epsilon^2 = \| M {\bf x} - \mu {\bf x} \|^2 = \sum_{k=1}^n \alpha_k^2 (\lambda_k - \mu)^2 \geq
  \left( \sum_{k=1}^n \alpha_k^2 \right) \left(\min_{1 \leq k \leq n} (\lambda - \mu)^2 \right)
$$

meaning there exists an eigenvalue $\lambda_k$ within $\epsilon$ of $\mu$,

$$
  \min_{1\leq k \leq n} |\lambda_k - \mu| \leq \epsilon.
$$

Also in~\cite{parlett1998symevp}, we have bounds estimating convergence to an eigenspace in angle.
Define the eigengap for $\lambda_k$ as $\delta_k = \min_{\lambda \in \lambda\paren{M} \setminus \lambda_k} \abs{\lambda_k - \lambda}$.   Moreover, if $\epsilon$ is small compared to $\delta_k$ there exists a normalized eigenvector ${\bf v} \in {\cal X}_k$ with which ${\bf x}$ has a small angle,

$$
  \min_{{\bf v} \in {\cal X}_k} \sqrt{1 - \left< \, {\bf x}, \, {\bf v} \,\right> ^2} \leq \frac{\epsilon}{\delta_k}.
$$

Instead of presenting a proof of this well-known result, we derive a similar bound for $\ell_2$ and point-wise approximation to an eigenspace associated with an extremal eigenvector.

\newcommand{|\mu - \lambda_1| < \min_{\lambda \in \lambda(M) \setminus \lambda_1} |\mu - \lambda|}{|\mu - \lambda_1| < \min_{\lambda \in \lambda(M) \setminus \lambda_1} |\mu - \lambda|}
\begin{thm}\label{thm:singleevecerror}
Consider approximate eigenpair $({\bf x}, \mu)$ of symmetric $M \in \mathbb{R}^{n \times n}$ with $\norm{{\bf x}} = 1$ and $\mu = \ray{{\bf x}}{M}$.
Assume $$|\mu - \lambda_1| < \min_{\lambda \in \lambda(M) \setminus \lambda_1} |\mu - \lambda|.$$
Given eigenresidual $\epsilon$ and eigengap $\delta_1$, there exists an eigenvector ${\bf v} \in {\cal X}_1$ , with $\norm{{\bf v}}=1$, and error bound
$$\norm{{\bf x} - {\bf v}} \leq \frac{\sqrt{8}\epsilon}{\delta_1}.  $$
\end{thm}

\begin{proof}
Let $\alpha {\bf v}$ be the closest vector in ${\cal X}_1$ to ${\bf x}$.
Decompose ${\bf x}$ into its eigenvector components within ${\cal X}_1$ and perpendicular to ${\cal X}_1$, ${\bf x} = \alpha {\bf v} + \sum_{{\bf v}_k \perp {\cal X}_1} \alpha_k {\bf v}_k$.
Because $\mu$ is closer to $\lambda_1$ than any other eigenvalue, we have
\begin{eqnarray*}
\epsilon^2 & = & |\lambda_1- \mu|^2 \alpha^2 + \sum_{{\bf v}_k \perp {\cal X}_k}  |\lambda_k - \mu|^2 \alpha^2_k\
\geq \sum_{{\bf v}_k \perp {\cal X}_k}  |\lambda_k - \mu|^2 \alpha^2_k
\geq \frac{\delta_1^2}{4} \sum_{{\bf v}_k \perp {\cal X}_k}  \alpha^2_k. \\
\end{eqnarray*}
Rearranging gives
\begin{eqnarray*}
\frac{4 \epsilon^2}{\delta_1^2} & \geq & \sum_{{\bf v}_k \perp {\cal X}_k}  \alpha^2_k = 1- \alpha^2. \\
\end{eqnarray*}

\begin{eqnarray*}
\|{\bf x}-{\bf v}\|^2 & = & \|{\bf x} - \alpha{\bf v} \|^2 + \|{\bf v} - \alpha {\bf v}\|^2
\leq  \|{\bf x} - \alpha{\bf v} \|^2 + (1 - \alpha)^2
\leq  2(1-\alpha^2)
\leq  \frac{8 \epsilon^2}{\delta_1^2} ,
\end{eqnarray*}
\end{proof}

This result implies,
$$\frac{1}{n} \sum_{i\in{\cal V}} |x_i - v_i|^2 \leq \|{\bf x} - {\bf v}\|^2 \leq \frac{8 \epsilon^2}{n \delta_1^2}  $$
so if $\epsilon^2 / n$ is small compared to the $\delta_1^2$, then the average average error squared is also small.
Moreover we have a point-wise error bound,
$$\max_{i\in{\cal V}} |x_i - v_i| \leq \|{\bf x} - {\bf v}\| \leq \frac{\sqrt{8}\epsilon}{\delta_1}   $$

For a large graph, it is typical that Fiedler eigenvalue is so close to the next-to-largest eigenvalue, that the error bounds demand an extremely small eigenresidual for convergence.
Note that this error analysis is independent of the algorithm used to compute the solution.
Choosing ${\bf x}={\bf v}_3$ shows that the condition $|\mu - \lambda_1| < \min_{\lambda \in \lambda(M) \setminus \lambda_1} |\mu - \lambda|$ is necessary.
Thus for matrices $M$ with small $\delta_1$, we need to remove this condition.
When the $\delta_1$ is small, a reasonable number of iterations of an eigensolver may
produce a vector ${\bf x}$ with $\ray{{\bf x}}{M}$ close to $\lambda_1$ which may not be very close to the true extremal eigenspace.
In this case we examine convergence to a linear combination of eigenvectors associated with a range of eigenvalues.

\begin{comment}{\bf (Small Spectral Gap yet Easy to Partition)}
In some situations, a small spectral gap does not imply that the graph is difficult to partition.
\cref{sec:ROCstart} describes a class of graphs that are easy to partition with an approximate eigenvector,
but for which calculating an eigenvector with close point-wise approximation is extremely costly.
Let ${\cal G}_0$ be the ring of cliques{} in Section~\ref{sec:ROCdefinition} and form ${\cal G}$ by weighting
one internal edge $(i,j)$ with $1 - \eta$.
If the edge is chosen so that $i$ and $j$ are both interior vertices (no connections outside their blocks)
then the spectral gap for ${\cal G}$ gets arbitrarily small as $\eta \rightarrow 0$ by
We omit this algebra, but note it yields
$$
\delta \leq \frac{c \eta^2}{(b-1)^2},
$$
where $c < 1$ is a positive constant independent of $\eta$.
\end{comment}

\subsection{Converging to a Subspace Spanned by Multiple Eigenspaces}\label{sec:convergeblends}
\newcommand{\delta_{p,q}(\mu)}{\delta_{p,q}(\mu)}
\newcommand{\min_{\{ k < p\} \cup \{ k > q \}} |\lambda_k - \mu|}{\min_{\{ k < p\} \cup \{ k > q \}} |\lambda_k - \mu|}

This section generalizes the previous error bound to the distance between ${\bf x}$ and a subspace spanned by the eigenvectors associated with a range of eigenvalues.
Assume that linear combinations of eigenvectors associated with a range of eigenvalues $\clintervalcl{\lambda_q}{\lambda_p}$ are satisfactory for subsequent data analysis algorithms.
If the Rayleigh quotient{} is within $\clintervalcl{\lambda_q}{\lambda_p}$ and the eigenresidual is smaller than the distance between $\mu$ and any eigenvalue outside $\clintervalcl{\lambda_q}{\lambda_p}$, then the following theorem holds.

\begin{thm}\label{thm:blenderror}
Consider approximate eigenpair $({\bf x}, \mu)$ of symmetric $M \in \mathbb{R}^{n \times n}$  with $\|{\bf x}\| = 1$ and $\mu = {\bf x}^t M {\bf x} \in \clintervalcl{\lambda_q}{\lambda_p}$.
Define
\[
\delta_{p,q}(\mu) = \min_{\{ k < p\} \cup \{ k > q \}} |\lambda_k - \mu|
\qquad \mbox{and} \qquad
{\cal X}_p^q := \bigotimes_{k=p}^q {\cal X}_k.
\]
Given eigenresidual $\epsilon = \|M {\bf x} - \mu {\bf x} \| \le \pairmin{\mu-\lambda_{q-1}}{\lambda_{p+1}-\mu}$
there exists a vector ${\bf v} \in {\cal X}_p^q$, with $\|{\bf v}\|=1$,
$\ell_2$ error bound,
\[
\|{\bf x} - {\bf v}\| \leq \frac{\sqrt{2} \epsilon}{\delta_{p,q}(\mu)},
\]
\end{thm}

\begin{proof}
Let
${\bf x} = \sum_{k=1}^n \alpha_k {\bf v}_k$ and $\Pi {\bf x} = \sum_{k=p}^q \alpha_k {\bf v}_k$, the $\ell_2$-orthogonal projection onto ${\cal X}_p^q$.   Note $\sum_{k=1}^n \alpha_k^2 = 1$.    In the case where $\|\Pi {\bf x}\| = 1$, we can let ${\bf v} = {\bf x}$ and see the bound is clearly satisfied.   For $\| \Pi {\bf x} \| < 1$, we first demonstrate that $\|{\bf x} - \Pi {\bf x}\|$ is controlled by $\epsilon,$

\begin{eqnarray*}
\epsilon^2 & = & \|M {\bf x} - \mu {\bf x} \|^2  \\
\epsilon^2 & = & \sum_{k=1}^n \alpha_k^2 |\lambda_k - \mu|^2 \\
\epsilon^2 & = & \sum_{\{k< p\} \cup \{k > q\}}  \alpha_k^2 |\lambda_k - \mu|^2 + \sum_{k=p}^q \alpha_k^2 |\lambda_k - \mu|^2 \\
\epsilon^2 & \geq &  \delta_{p,q}(\mu)^2 \sum_{\{k< p\} \cup \{k > q\}}  \alpha_k^2  + \sum_{k=p}^q \alpha_k^2 |\lambda_k - \mu|^2 \\
\epsilon^2 & \geq &  \delta_{p,q}(\mu)^2 \sum_{\{k< p\} \cup \{k > q\}}  \alpha_k^2  \\
\epsilon^2 & \geq &  \delta_{p,q}(\mu)^2 \left( 1 - \sum_{k=p}^q \alpha_k^2 \right)  \\
\frac{\epsilon}{\delta_{p,q}(\mu)} & \geq & \|{\bf x} - \Pi {\bf x} \|.
\end{eqnarray*}

There is a unit vector in ${\cal X}_p^q$ that is also within some factor of $\epsilon$ to ${\bf x}$.   Let ${\bf v} = \Pi {\bf x} / \|\Pi {\bf x}\|$, then $ \|{\bf v} - \Pi {\bf x} \| = 1 - \|\Pi {\bf x} \| $.   We have

\begin{eqnarray*}
\| \Pi {\bf x} \|^2 + \|(I - \Pi) {\bf x} \|^2 & = & 1\\
\| \Pi {\bf x} \|^2  & = & 1 - \|(I - \Pi) {\bf x} \|^2 \\
\| \Pi {\bf x} \|^2 & \geq & {1 - \frac{\epsilon^2}{\delta_{p,q}(\mu)^2}} \\
\| \Pi {\bf x} \| & \geq & \sqrt{1 - \frac{\epsilon^2}{\delta_{p,q}(\mu)^2}}.
\end{eqnarray*}
Using the inequality $a \leq \sqrt{a}$ for $a \in (0,1)$, we see
\[
\|{\bf v} - \Pi {\bf x} \|
  = 1 - \|\Pi {\bf x} \|  \leq  1 - \sqrt{1 - \frac{\epsilon^2}{\delta_{p,q}(\mu)^2}}
  \leq \frac{\epsilon}{\delta_{p,q}(\mu)}.
\]
Then, because $({\bf x} - \Pi {\bf x})^t (\Pi {\bf x} - {\bf v}) = 0$, we have
\begin{eqnarray*}
\|{\bf x} - {\bf v}\|^2 =
\|{\bf x} - \Pi {\bf x}\|^2 +
\|\Pi {\bf x} - {\bf v}\|^2  \leq \frac{2 \epsilon^2}{\delta_{p,q}(\mu)^2}.
\end{eqnarray*}

\end{proof}

\begin{rem}
Note that the size of the {\em blend gap}, $\delta_{p,q}(\mu)$, is dependent on
\begin{inparaenum}[\itshape i\upshape)]
\item the size of $|\lambda_p - \lambda_{q}|$,
\item how internal Rayleigh quotient{} $\mu$ is within $\clintervalcl{\lambda_q}{\lambda_p}$,
\item and how far the exterior eigenvalues are, $|\lambda_q - \lambda_{q+1}|$ and $|\lambda_{p-1} - \lambda_p|$.
\end{inparaenum}
For a problem where the spectrum is not known a priori{}, it is difficult to state an acceptable interval $\clintervalcl{\lambda_q}{\lambda_p}$ for accomplishing a given data mining task.
\Cref{sec:ROCepairs} provides an example where one can choose $\clintervalcl{\lambda_q}{\lambda_p}$ a priori.
The Congress graph has $\lambda_{41}-\lambda_{42} \ge 0.4$ and the first $41$ eigenvectors indicate the natural clustering of the graph into sessions\cite{cucuringu2011localization}. This analysis thus applies to this real world network.
\end{rem}

For our application, $p=1$ and $\delta_{pq}=\mu - \lambda_{q+1}\paren*{\ahat - {\bf k} {\bf k}^t} \ge \lambda_q - \lambda_{q+1}$,
which can be much larger than the spectral gap $\lambda_1\paren{\ahat - {\bf k} {\bf k}^t} - \lambda_2\paren{\ahat - {\bf k} {\bf k}^t}$.
In Section~\ref{sec:convergesingle} the goal of computation is a single eigenvector and the output of the approximation is a blend of eigenvectors, the coefficients of the output in the eigenbasis of the matrix describes the error introduced by approximation.
In Section~\ref{sec:convergeblends} the goal of computation is a blend of eigenvectors, and
 we improve the error bound when the spectral gap is small.

\newcommand{\frac{\sqrt{2}\epsilon}{\delta{pq}(\mu)}}{\frac{\sqrt{2}\epsilon}{\delta{pq}(\mu)}}

\subsection{Definitions and Notation}\label{sec:definitions}
 Here we introduce the necessary concepts of data analysis quality and eigensolver accuracy.
For this work we focus on partitioning graphs to minimize conductance as defined below.
For any $S\subset V$, $S\cup \bar{S}=V$ represents a cut of the graph.
Define $\vol{S} = \sum_{i,j\in S} a_{i,j}$ as the total weight of the edges within $S$.
And define $\cut{S} = \sum_{i\in S, j\notin S} a_{i,j}$ as the total weight of edges with one
vertex in $S$ and one vertex in the complement of $S$.
The conductance of a cut $S$ is thus given by the formula~\cite{kannan2004goodbadspectral}:
\[
    \conductance{S} = \frac{\cut{S}}{\pairmin{\vol{S}}{\vol{\bar{S}}}}.
\]
For any vector ${\bf x}$, represent the sweep cut of ${\bf x}$ at $t$ as in \cref{eqn:sweepcut}.
\begin{equation}\label{eqn:sweepcut}
    S=\sweepcut{{\bf x}}{t} = \set{i\mid x_i > t}
\end{equation}
We denote by $\conductance{{\bf x}}$ the minimal conductance of a sweep cut of ${\bf x}$, that is $\min_t \conductance{S_x\paren{t}}$.
The conductance of the graph is defined as $\phi_G = \min_S \conductance{S}$.
If the graph has multiple partitions with conductance less than a value $\psi$, then the application might accept any of them.

The accuracy of a solution to the eigenvector problem can be measured in three quantities: Rayleigh quotient{}, error, and residual.
Spectral methods for finding low-conductance partitions rely on computing vectors ${\bf x}$ and corresponding scalars $\lambda$ that solve the equations $M {\bf x} = \lambda {\bf x}$ for some graph-associated matrix $M$.
The Rayleigh quotient{}, $\mu = {\bf x}^t M {\bf x}$ is an approximation to the eigenvalue $\lambda$.
The error $\norm{{\bf v}-{\bf x}}$ where ${\bf v}$ is the closest exact solution
is not accessible to a solver in general.
The solver can use the norm of the eigenresidual, $\norm{M{\bf x} - \mu{\bf x}}$, to determine when to stop iterations.
Throughout this paper $\norm{\cdot}$ will
be taken to mean the 2-norm with subscripts used to clarify when necessary.
In order to practically use an eigensolver, one must choose a residual tolerance $\norm{M{\bf x}-\mu{\bf x}} <
\epsilon$ sufficient to ensure that the computed eigenvector is accurate enough to solve the application problem.
This paper provides concrete residual tolerances for a specific model problem and provides tools for finding such
tolerances for more general graphs.

We briefly summarize notation for various graph matrices.
Let $\mathbf{1}$ be the all ones vector.
If
  $A$ is the adjacency matrix of a graph and
  $D$ is the diagonal matrix whose entries are $d_{i,i}=\paren{A\mathbf{1}}_i$,
  then $L=D-A$ is the combinatorial Laplacian,
  and $\hat{L} = I - D^{-\half} A D^{-\half}$ is the normalized Laplacian.
Solutions of the generalized eigenvector problem $L{\bf y}=D{\bf y}$ are scaled solutions to the eigenvector problem $D^{-\half} A D^{-\half} {\bf x} = {\bf x}$ where the scaling is ${\bf x} = D^{\half} {\bf y}$.
We refer to $D^{-\half} A D^{-\half}$ as $\hat{A}$, and use the identity
$\lambda_k(\hat{L}) = 1-\lambda_{n-k}(\hat{A})$ to replace computations involving small eigenvalues of the normalized Laplacian
matrix with computations involving large eigenvalues of the adjacency matrix.

Conductance is an appropriate measure of partition quality for spectral partitioning because of Cheeger's inequality
which bounds the conductance of the graph in terms of the eigenvalues of the Laplacian matrix.
\renewcommand{M}{\hat{L}}
\begin{thm}{General Cheeger Inequality\cite{mihail1989conductance}}\label{thm:cheeger}
    If ${\bf x}$ is a unit vector orthogonal to $D^{\half}\mathbf{1}$ such that $\ray{{\bf x}}{M} = \mu$ then $D^{-\half}{\bf x}$ has a sweep cut $S$ such that
  $\conductance{S} = \conductance{{\bf x}} \leq \sqrt{2\mu}$.
\end{thm}
When ${\bf x}$ satisfies $\hat{L}{\bf x}=\lambda_2{\bf x}$,
    $\phi_G \le \conductance{D^{-\half}{\bf x}} \le \sqrt{2\lambda_2\paren{L}}$.
This general form of Cheeger's inequality indicates that finding low-energy Laplacian eigenvectors is sufficient for
constructing low-conductance partitions of the graph.

In graph partitioning, the goal is to compute a partition of the graph that optimizes the chosen objective.
When applying spectral methods to graph partitioning, our goal is not to compute very accurate eigenpairs, but instead to partition the vertex set of a graph correctly.
Because spectral partitioning can be used recursively to find small clusters,
  we focus on splitting a graph into two parts.
Our results on the model problem indicate that approximate eigenvectors are sufficient to solve the data analysis
problem and are much faster to compute if the graph has the right structure.

\subsection{A Model Problem}\label{sec:rocjustification}

We use a simple model (the ring of cliques{}) to study the capabilities of spectral partitioning algorithms, form theory to
characterize performance, and potentially enhance these algorithms.   Such use of model problems is well-established in
the numerical analysis literature regarding iterative solutions to discretized partial differential equations (PDEs).
The Dirichlet Laplacian  on a unit square discretized on a Cartesian lattice is a simple problem with known eigenpairs
and is used to study the properties of various eigensolvers.    These simple model problems do not demonstrate the
algorithms perform well on real-world problems, but are incredibly important tools for algorithm development and theoretical analysis.   For spectral partitioning, the ring of cliques{} is one candidate model problem for which we can derive complete knowledge of the eigenpairs.
In PDEs, the order of discretization error (difference between continuous solution and discrete solution) provides the solver with a stopping criterion.   In spectral graph partitioning, we do not
have this luxury, and we must develop theory to understand how perturbations in a spectral embedding impact partitioning quality.
Another reason to develop a collection of model problems
is to enable careful study of this impact in well-understood situations.

In order to provide a striking example of our improved analysis \Cref{sec:ROCstart} studies our model problem in detail.
The goal is to understand when approximate eigenvectors have sweep cuts that correctly identify graph structures.
The ring of cliques{} has been studied  as
  ``the most modular network'' in order to demonstrate a resolution limit in the
  modularity maximization procedure for community detection~\cite{Fortunato2007resolutionlimit}.
For this family of highly structured graphs, the correct partition is unambiguous.
We use the ring of cliques{} to investigate how spectral embeddings for partitioning are affected by numerical error.
Because of the high degree of symmetry, the ring of cliques{} allows for a thorough closed form analysis producing formulas for the eigenvectors and eigenvalues.
A sweep cut using exact eigenvectors partitions the graph with small conductance and successful recovery of all the
clusters.
We quantify the effects of approximation error on sweep cut partitions of this graph.
Our findings demonstrate that despite a small spectral gap, which implies slow convergence of non-preconditioned eigensolvers, the ring of cliques{} is well
partitioned by low accuracy approximations to the eigenvectors.

Studying the ring of cliques{} provides guidance for practitioners on useful tolerances for eigensolvers.
We are able to construct the smallest perturbation that induces a mistake in the sweep cut partition.
This perturbation shows that when looking for clusters of size $b$ in a general graph the eigensolver tolerance must be
smaller than $\bigo{b^{-\half}}$.
Analysis of the ring of cliques{} provides an upper bound on the eigensolver accuracy that is
sufficient to recover community structure.

\subsection{Contributions}\label{sec:contributions}
This paper provides the following contributions.
\Cref{sec:blends} extends a known error bound on the computation of eigenvectors to the computation of linear combinations of eigenvectors.
By extending this classical error bound to linear combinations of eigenvectors,
  we find a condition on the spectrum of where numerically accurate blends are easy to achieve.
\Cref{thm:blendspaceconductance} provides a general condition under which approximate eigenvectors preserve sweep cuts.
\Cref{sec:ROCstart} analyzes a model problem and derives necessary and sufficient error tolerances for solving the model problem, which are essentially tight for some parameter regime.
We show for the model problem where the number of clusters is polynomial in the size of the clusters, the power method takes $\bigo{1}$ iterations to identify the clusters.

\section{Introduction}
\input{introduction.tex}

\subsection{Related Work}\label{sec:relatedwork}
\input{relatedwork.tex}
\renewcommand{M}{M}

\section{Blends of Eigenvectors}\label{sec:blends}
\input{convergencetheory.tex}

\newcommand{\bz}{{\bf z}}
\newcommand{\errs^\prime}{\bz^\prime}
\newcommand{\err}[1]{z_{#1}}
\renewcommand{\mathbf{1}}{\mathbf{1}}
\newcommand{\boldsymbol{\mu}}{\boldsymbol{\mu}}
\newcommand{\mean}[1]{\mu_{#1}}
\newcommand{\epsilon}{\epsilon}
\newcommand{Span\{\bv_1\dots \bv_q\}}{Span\{{\bf v}_1\dots {\bf v}_q\}}
\renewcommand{\half}{\frac{1}{2}}
\newcommand{\block}[1]{{\cal{B}}_{#1}}
\newcommand{\sqrt{2qn}}{\sqrt{2qn}}
\newcommand{{2qn}}{{2qn}}
\newcommand{\paren{1+2qn}^{-\half}}{\paren{1+2qn}^{-\half}}
\newcommand{{\cal{W}}}{{\cal{W}}}
\newcommand{{\mathbf{I}}}{{\mathbf{I}}}
\newcommand{\Proj}[1]{{\mathbf{P}_{#1}}}
\newcommand{\indicator}[2]{{\bf e}_{#1}^{#2}}
\newcommand{\minelt}[1]{\min_{i}{#1}_i}
\newcommand{\maxelt}[1]{\max_{i}{#1}_i}
\newcommand{\blockindicator}[2]{{\bf e}_{\block{#1}}}
\newcommand{b^{-\half}}{b^{-\half}}
\input{general.tex}

This section connects the eigenresidual to the error when computing blends of eigenvectors, and quantifies the impact of error on the sweep cut procedure.
 If the eigengap is small enough, then one cannot guarantee that the Rayleigh quotient{} is closest to the Fielder value,
 thus one cannot guarantee that the computed eigenvector is close to the desired eigenvector.
 In this small gap setting, a small eigenresidual{} indicates that the computed vector is close to the desired invariant
 subspace.
 \Cref{thm:blendspaceconductance} shows that vectors with small eigenresidual preserve low conductance sweep cuts for general
 graphs.
 \Cref{thm:blendspaceconductance} illustrates how the residual tolerance depends on both the blend gap $\delta_{p,q}(\mu)$ and the sensitivity $g$ of the
 eigenvectors.
The following section applies this theory to the ring of cliques{} in order to derive solver tolerances for
 graph partitioning.
\input{ringofcliques.tex}
\input{roc/minimalpertubation.tex}
\input{roc/experiment.tex}

\section{Conclusions}
\input{conclusions.tex}

\section{The Ring of Cliques}\label{sec:ROCstart}

To demonstrate the theory developed in \cref{sec:blends} and to make our arguments as clear as possible,
we employ a previously studied model problem, the ring of cliques{}~\cite{Fortunato2007resolutionlimit}.
Theorems~\ref{thm:Xnoise}~and~\ref{thm:Epairs} derive explicit
formulas for all eigenvalues and eigenvectors. These formulas determine the relevant residual tolerance.
Moreover, complete spectral knowledge gives a strong understanding the convergence properties of simple
iterative methods.

 The ring of cliques~has several attractive properties for analysis of spectral partitioning.
 The Community structure is as extreme as possible for a connected graph, so the solution is well-defined.
 Also, we can apply theorems about block circulant matrices~\cite{tee2005eigenvectors} to produce closed form solutions to the eigenvector problem.
 This graph serves as a canonical example of when solving an eigenproblem accurately is unnecessarily expensive to achieve data analysis success. This example shows that it is possible for the
 combinatorial structure of the data to be revealed faster than the algebraic structure of the associated matrices.
 The graph is simple to partition accurately as there are many cuts relatively close to the minimum.
 Any robust partitioning algorithm will correctly recover the cliques in this graph.
 However, a Fiedler eigenvector is difficult to calculate with guarantees of
  point-wise accuracy when using non-preconditioned iterative methods.
 An algorithm that computes a highly accurate eigenpair will be inefficient on large problem instances.
 \Cref{sec:rocbygeneral,sec:roc-perturbation} apply the tools from \Cref{sec:blends} in order to derive a residual tolerance sufficient for solving the ring of cliques{}.
 \Cref{sec:powermethod} bounds the number of power method iterations necessary to recover the ring of cliques{}, and
 \Cref{sec:roc_experiment} validates and illustrates these observations with an experiment.

\subsection{Definition}\label{sec:ROCdefinition}
A $q$-ring of $b$-cliques, $\Roc$, is parameterized by a block size $b$ and a number of blocks $q$.
Each block represents a clique of size $b$ and all possible internal connections exist within each individual set of $b$ vertices.
For each block, there is a single vertex called the {\it{corner}}\/ connected to the corners of the adjacent cliques.
These $q$ corners form a ring.
Each block also has  $\paren{b-1}$ {\it internal}\/ vertices that have no edges leaving the block.
The adjacency matrix associated with $\Roc$ is a sum of tensor products of simple matrices (identity, cycle, and rank-one matrices).
We have
$$
	A = I_q \otimes (\bfo_b \bfo_b^t - I_b) + C_q \otimes (\be_1 \be_1^t),
$$
where the $I_k$ are identity matrices of dimension $k$, ${\bf 1}_b$ is a constant vector with dimension $b$, ${\bf e}_1 \in
\mathbb{R}^b$ is the cardinal vector with a one in its first entry and zero elsewhere, and $C_q$ is the adjacency matrix
of a cycle graph on $q$ vertices.
The matrix $A$ and other matrices associated with this graph are {\em block-circulant}, which implies the eigenvectors are the Kronecker product of a periodic vector and the eigenvectors of a small eigenproblem defined by structure in the blocks.
\Cref{fig:rocvizgraph} shows the structure of the graph, and \Cref{fig:rocvizmat} shows the block structure of the
adjacency matrix.
\begin{figure}[htb]
	\centering
    \begin{minipage}[b]{0.45\textwidth}

        \includegraphics[width=\textwidth]{figures/spectral-layout.pdf}
	    \caption{A drawing of $\Roc$ laid out to show strucure.}
        \label{fig:rocvizgraph}
    \end{minipage}\quad
    \begin{minipage}[b]{0.45\linewidth
    \setlength{\arraycolsep}{3pt}

    \[
\begin{bmatrix}
J_b & \be_1 \be_1^t & 0 & \cdots & & \be_1 \be_1^t \\
\be_1 \be_1^t & J_b & \be_1 \be_1^t & 0 & & \\
0 & \be_1 \be_1^t & J_b & \be_1 \be_1^t & 0 & \\
\vdots && \ddots & \ddots & \ddots \\
&& 0 & \be_1 \be_1^t & J_b & \be_1 \be_1^t \\
\be_1 \be_1^t &&& 0 & \be_1 \be_1^t & J_b \\

\end{bmatrix}
    \] \\
$$
\mbox{where} \quad J_b = \bfo_b \bfo_b^t - I_b .
$$
    %
    \vspace{0.5cm}
    \caption{The adjacency matrix of $\Roc{}$ has block circulant structure.}
    \label{fig:rocvizmat}
    \end{minipage}
\end{figure}

Any partition that breaks a clique cuts at least $b-2$ edges while any partition that does not break any cliques cuts at most $q$ edges.
The best partition is break the ring into two contiguous halves by cutting exactly two edges.
There are $q/2$ partitions that achieve this minimal cut for even $q$.
We will consider any of these equivalent.
Any partition that breaks fewer than $b-2$ edges will be regarded as a good, but not optimal cut.
The fact that many partitions are close to optimal and then the vast majority of partitions are very far from optimal is a feature of this model problem.

\subsection{Eigenpairs of ROC normalized Adjacency Matrix}\label{sec:ROCepairs}
\begin{figure}[h]

	\label{fig:rocspect}
	\centering
	\includegraphics[width=1.0\textwidth]{figures/roc_spectrum.pdf}
	\caption{Distribution of eigenvalues of $\hat{A}$ for ${\cal R}_{b=8, q=32}$.
		The gray bars represent a histogram of the eigenvalues, including multiplcities.
		The red diamond represents a large multiplicity of $(n-2q)$ at $-(b-1)^{-1}$ corresponding to ${\cal X}_{noise}$ (see Theorem~\ref{thm:Xnoise}).
		There is a green interval near 1 containing the portion of the spectrum given by $\lambda^{(k)}_1$ and a blue interval near 0 containing $\lambda^{(k)}_1$ for $k=0, 1, ..., \lfloor q/2\rfloor$ (see Theorem~\ref{thm:Epairs}).
		The open left endpoint of each interval signifies that the eigenvalue corresponding to $\lambda^{(k)}_{\lceil q/2\rceil}$ is not present when $q$ is odd.}
\end{figure}

Due to the block-circulant structure of the ring of cliques{} we are able to compute the eigenvalues and  eigenvectors in a closed form.
Let $\cosvec{k}{q},\sinvec{k}{q}$ be the periodic vectors as defined in \cref{eqn:periodicdefs}.
\begin{equation}\label{eqn:periodicdefs}
 \cosvec{k}{q}_j = \cos\paren*{\frac{2\pi k}{q}j}\quad
 \sinvec{k}{q}_j = \sin\paren*{\frac{2\pi k}{q}j}\quad \text{for}\ j=\intrange{1}{q}
\end{equation}
These vectors are periodic functions with $k$ cycles through the interval $\clintervalcl{-1}{1}$ sampled at the $q$th roots of unity.
We employ results from \cite{Henson:2012:LSE} and \cite{tee2005eigenvectors} to derive the full spectrum of $\hat{A}$ and a full eigenbasis.
In summary, there is an eigenvalue $\lambdanoise{b}$ with a large multiplicity, $n - 2q = (b-2)q$.
Furthermore, the eigenspace associated with $\lambdanoise{b}$ can be represented in a basis that contains variation
internal to each clique, that is with eigenvectors of the form ${\bf h} \otimes {\bf e}_i$ where ${\bf e}_i$ is the $i$th standard
basis vector for each $i\in \set{1\dots q}$.
For this reason, we call $\lambda_{noise} = \lambdanoise{b}$ a noise eigenvalue.
The positive eigenvalues are called signal eigenvalues.
The signal eigenvectors have the form $\pervec{k}{q} \otimes (\xi {\bf e}_1 + {\bf g}_1)$, where $\pervec{k}{q}$ is either $\sinvec{k}{q}$ or $\cosvec{k}{q}$, ${\bf e}_1$ is one in its first entry, ${\bf g}_1$ is zero in its first entry and one elsewhere, and $\xi$ is a  scalar.
All of the internal members of the cliques take the same value in any eigenvector associated with $\lambda_k(\hat{A}) \neq (b-1)^{-1}$.

The slowly varying eigenvectors (associated with $\lambda_k(\hat{A}) \approx 1$) give nearly optimal
partitions of the graph.
Linear combinations of these slowly varying signal eigenvectors also give low conductance partitions.
There are $q-1$ non-localized{} eigenvectors with small positive or negative eigenvalues.
These eigenvectors have the internal clique members and their corner with different sign which causes them to
misclassify some of the corners.
The distribution of the eigenvalues of $\Roc$ is illustrated in \Cref{fig:roceigenvectorsplot}.
The rest of \cref{sec:ROCepairs} contains formulas for the eigenpairs and the details of their derivations.
\begin{thm}{\bf ($\Roc$ Noise Eigenpairs)}
\label{thm:Xnoise}
There is an eigenspace ${\cal X}_{noise}$ of multiplicity $(n - 2q)$ associated with eigenvalue
\[
\lambda_{noise} = \frac{-1}{b-1}.
\]
Any vector that is zero-valued on all corner vertices and sums to zero on each individual set of internal vertices is in ${\cal X}_{noise}$.
\end{thm}

\begin{proof}
    This is a specific case of locally supported eigenvectors~\cite{Henson:2012:LSE} (LSEVs{}) brought on by a high level of local symmetry in $\Roc$.
    For each clique $\clique{}$, let ${\bf x} = {\bf e}_i - {\bf e}_j$ for vertices $i$ and $j$ that are internal vertices of $\clique{}$.
    Both ${\bf x}$ and $\hat{A} {\bf x}$ are zero valued outside of $\clique{}$.
    Internally, due to $D^{-1/2} ({\bf e}_i - {\bf e}_j) = (b-1)^{-1/2}({\bf e}_i - {\bf e}_j)$ and the orthogonality ${\bf 1}_b^t ({\bf e}_i - {\bf e}_j) = 0$, we see
\[
\hat{A} ({\bf e}_i - {\bf e}_j) =
\frac{1}{b-1} ({\bf 1}_b {\bf 1}_b^t - I ) ({\bf e}_i - {\bf e}_j) = \frac{-1}{b-1} ({\bf e}_i - {\bf e}_j).
\]
Thus, ${\bf x}$ is an eigenvector of $\hat{A}$ associated with $-1/(b-1)$.
There are $(b-2)$ such linearly independent eigenvectors for $\clique{}$, and the same is true for all $q$ cliques.
Thus, we have a multiplicity of $q(b-2) = n - 2q$ for eigenvalue $\lambda_{noise} = (b-1)^{-1}$.

\end{proof}

These vectors are in the interior of the spectrum and thus are very well attenuated by the power-method\footnote{In a single iteration the shifted power method,  ${\bf x}_{k+1} = (\hat{A} + (b-1)^{-1} I) {\bf x}_{k}$,  perfectly eliminates all of the energy in the $(n-2q)$-dimensional eigenspace associated with $\lambda=-(b-1)^{-1}$. If the graph is perturbed with the addition and removal of a few edges, the eigenvectors become slightly less localized and the associated eigenvalues spread out to a short range of values and are not perfectly eliminated in a single iteration.   However, the power method or a Krylov method will rapidly attenuate the energy in the associated eigenspaces.}.
The remaining eigenvectors must be constant on the internal nodes of the blocks because of orthogonality to the LSEVs{}
which are spanned by ${\bf e}_i-{\bf e}_j$.
In any vector ${\bf v}$ the projection of ${\bf v}$ onto the global eigenvectors defines a mean value for the elements of the
blocks. Since all of the eigenvectors of interest are orthogonal to the constant vector, their entries must sum to zero.
So the LSEVs{} cannot change the mean of a block. The remaining eigenvectors are given in \cref{thm:Epairs}.

\begin{thm}{\bf (ROC Signal Eigenpairs)} For $k=0,..., \lceil \frac{q}{2} \rceil - 1$, define
\label{thm:Epairs}
\begin{eqnarray*}
\alpha_k & = & 2 \cos \left( \frac{2 \pi k}{q}\right) \\
\beta_k & = & \frac{1}{2} \left( \alpha_k \sqrt{\frac{b-1}{b+1}} - \sqrt{b^2-1} + \sqrt{\frac{b+1}{b-1}} \right) \\
\xi_1^{(k)} & = & \beta_k + \sqrt{\beta_k^2+(b-1)} \\
\xi_2^{(k)} & = & \beta_k - \sqrt{\beta_k^2+(b-1)} \\
\lambda_1^{(k)} & = & \frac{\xi^{(k)}_1}{\sqrt{b^2-1}} + 1 - \frac{1}{b-1} \\
\lambda_2^{(k)} & = & \frac{\xi^{(k)}_2}{\sqrt{b^2-1}} + 1 - \frac{1}{b-1} \\
\end{eqnarray*}
\begin{itemize}

Let ${\bf 1}_b$ and ${\bf 1}_q$ be the vectors of all ones in $\mathbb{R}^b$ and $\mathbb{R}^q$, respectively.  Also let ${\bf e}_1 \in \mathbb{R}^b$ have a one in its first entry, zero elsewhere and ${\bf g}_1 = {\bf 1}_b - {\bf e}_1$.   We have the following eigenpairs.

\item[(i)] For $k=0$, we have 2 eigenvalues of $\hat{A}$, $\lambda_1^{(0)}$ and $\lambda_2^{(0)}$, each with multiplicity 1.    The associated (unnormalized) eigenvectors are
\[
{\bf v}_1^{(0)} = \sqrt{b+1} ({\bf 1}_q  \otimes {\bf e}_1) + \sqrt{b-1} ({\bf 1}_q  \otimes {\bf g}_1)
\]
and
\[
{\bf v}_2^{(0)} = (b-1)^{3/2} ({\bf 1}_q  \otimes {\bf e}_1) - \sqrt{b+1} ({\bf 1}_q  \otimes {\bf g}_1)
\]
respectively.

\item[(ii)] For each $k=1, ..., \lceil \frac{q}{2} \rceil - 1$, we have 2 eigenvalues of $\hat{A}$, $\lambda_1^{(k)}$ and $\lambda_2^{(k)}$, each with multiplicity 2.
Two independent (unnormalized) eigenvectors associated with $\lambda_1^{(k)}$ are
\[
{\bf v}_{1,1}^{(k)} = \cosvec{k}{q} \otimes \left(\xi_1^{(k)}{\bf e}_1 + {\bf g}_1 \right)\label{eqn:kroneker-signalvectors}
\quad \mbox{and} \quad
{\bf v}_{1,2}^{(k)} = \sinvec{k}{q} \otimes \left(\xi_1^{(k)}{\bf e}_1 + {\bf g}_1 \right).
\]
Two independent (unnormalized) eigenvectors associated with $\lambda_2^{(k)}$ are
\[
{\bf v}_{2,1}^{(k)} = \cosvec{k}{q} \otimes \left(\xi_2^{(k)}{\bf e}_1 + {\bf g}_1 \right)
\quad \mbox{and} \quad
{\bf v}_{2,2}^{(k)} = \sinvec{k}{q} \otimes \left(\xi_2^{(k)}{\bf e}_1 + {\bf g}_1 \right).
\]
\item[(iii)] If $q$ is even, then for $k=\frac{q}{2}$, we have 2 eigenvalues of $\hat{A}$, $\lambda^{(q/2)}_1$ and $\lambda^{(q/2)}_2$, each with multiplicity 1.   The associated (unnormalized) eigenvectors are
\[
{\bf v}_1^{(q/2)} = \cosvec{q/2}{q} \otimes \left(\xi_1^{(q/2)}{\bf e}_1 + {\bf g}_1 \right)
\]
and
\[
{\bf v}_2^{(q/2)} = \sinvec{q/2}{q} \otimes \left(\xi_2^{(q/2)}{\bf e}_1 + {\bf g}_1 \right)
\]
respectively.

\end{itemize}

Note if values of $\lambda_p^{(k)}$ and $\lambda_q^{(l)}$ coincide for $(p,k)\neq(q,l)$ the eigenvalue multiplicities add up.

\end{thm}

\begin{proof}
    Let $D_b = \mbox{diag}((b+1){\bf 1} - 2 {\bf g}_1)$.   Matrix $\hat{A}$ can be organized into blocks that all
    co-commute with each other implying the eigenvectors are tensor products involving the eigenvectors of the blocks.
    We  decompose $\hat{A} = (I_q \otimes B_1) + (C_q \otimes B_2)$, where $I_q$ is the identity in $\mathbb{R}^{q}$,
    $C_q$ is the adjacency matrix of a $q$- cycle, $(C_q)_{ij} = 1 \Longleftrightarrow |i-j|=1$ mod $q$, $B_1 =
    D_b^{-1/2}({\bf 1}_b {\bf 1}_b^t-I)D_b^{-1/2}$, and $B_2 = \frac{1}{b+1}{\bf e}_1 {\bf e}_1^t$.  Because any eigenvector ${\bf y}$ of
    $C_q$ is also an eigenvector of $I_q$, eigenvectors of $\hat{A}$ have the form ${\bf y} \otimes {\bf z}$.      Vectors ${\bf z}$
    are derived by plugging various eigenvectors of $C_q$ into ${\bf y}$ and solving for a set of constraints that ${\bf z}$
    must satisfy for $({\bf y} \otimes {\bf z})$ to be an eigenvector associated with $\hat{A}$.

We describe the eigendecomposition of $C_q$.   For $k=0, \dots, \lceil \frac{q}{2}\rceil-1$, $\alpha_k = 2 \cos (2 \pi / k)$ is an eigenvalue of $C_q$.    For $k=0$, $\alpha_0$ is simple, and ${\bf 1}_b$ is the associated eigenvector.   For $k=1, ..., \lceil \frac{q}{2}\rceil$-1, $\alpha_k$ has multiplicity 2 and the associated $2$-dimensional eigenspace is span($\{\cosvec{k}{q},\, \sinvec{k}{q}$\}), as defined in (\ref{eqn:periodicdefs}).
If $q$ is even, then $\alpha_{q/2}$ is a simple eigenvalue as well and the associated eigenvector is $\cosvec{q/2}{q}$.
Letting ${\bf y}$ be an eigenvector associated with $\alpha_k$ and using the properties of Kronecker products, we see
\begin{eqnarray*}
\hat{A} ({\bf y} \otimes {\bf z}) & = &
\left[ (I_q \otimes B_1) + (C_q \otimes B_2) \right]({\bf y} \otimes {\bf z})
= \left[ (I_q {\bf y} \otimes B_1 {\bf z}) + (C_q {\bf y} \otimes B_2 {\bf z}) \right] \\
& =& \left[ ({\bf y} \otimes B_1 {\bf z}) + ( \alpha_k {\bf y} \otimes B_2 {\bf z}) \right] =
 \left[ ({\bf y} \otimes B_1 {\bf z}) + ( {\bf y} \otimes \alpha_k B_2 {\bf z}) \right]\\
& =& {\bf y} \otimes \left(B_1 {\bf z} +\alpha_k B_2 {\bf z} \right)  =
{\bf y} \otimes  \left[ (B_1 +\alpha_k B_2 ) {\bf z}\right] \\
\end{eqnarray*}
Here we see that if ${\bf z}$ is an eigenvector of $H_k := B_1 + \alpha_k B_2$, then $({\bf y} \otimes {\bf z})$ is an eigenvector associated with $\hat{A}$.   Observe that $H_k = D_b^{-1/2} ({\bf 1}_b {\bf 1}_b^t +\alpha_k {\bf e}_1 {\bf e}_1^t - I) D_b^{-1/2} $ is a scaling of a rank-2 shift from the identity matrix, where we would expect 3 eigenvalues: 2 simple and one of multiplicity $(b-2)$.

We can easily verify that there is a $(b-2)$-dimensional eigenspace of $H_k$ associated with $-1/(b-1)$.    The tensor products of these vectors are an alternative basis associated with the locally supported eigenvectors from Theorem~\ref{thm:Xnoise}.   The associated eigenspace of $H_k$ is orthogonal to span($\{{\bf e}_1, {\bf g}_1\}$).   Due to eigenvector orthogonality, the last two eigenvectors must be in the range of span($\{{\bf e}_1, {\bf g}_1\}$).  Note that $D_b^p {\bf e}_1=  (b+1)^p {\bf e}_1$ and $D_b^p {\bf g}_1=  (b-1)^p {\bf g}_1$.   We use this to solve for these eigenvectors and their associated eigenvalues in terms of $\alpha_k$ and $b$,
\begin{eqnarray*}
H_k ( \xi {\bf e}_1 + {\bf g}_1) & = & \lambda ( \xi {\bf e}_1 + {\bf g}_1) \\
({\bf 1}_b {\bf 1}_b^t + \alpha_k {\bf e}_1 {\bf e}_1^t - I_b )  D_b^{-1/2}( \xi {\bf e}_1 +  {\bf g}_1) & = & \lambda  D_b^{1/2}  (\xi {\bf e}_1 + {\bf g}_1)
\end{eqnarray*}
The right-hand side expands to
$$
\left( \lambda \xi \sqrt{b+1} \right) {\bf e}_1 + \left( \lambda \sqrt{b-1} \right) {\bf g}_1,
$$
The left-hand side expands and simplifies to
\begin{eqnarray*}
 \left( \frac{\xi}{\sqrt{b+1}} + \sqrt{b-1} \right) ({\bf e}_1 + {\bf g}_1) +  \left(  \frac{\xi \alpha_k }{\sqrt{b+1}} - \frac{\xi}{\sqrt{b+1}}  \right) {\bf e}_1+  \left( \frac{-1}{\sqrt{b-1}}  \right) {\bf g}_1  & = & \\
\left(  \frac{\xi \alpha_k}{\sqrt{b+1}} + \sqrt{b-1}  \right) {\bf e}_1+  \left(  \frac{\xi }{\sqrt{b+1}} + \sqrt{b-1} - \frac{1}{\sqrt{b-1}}   \right) {\bf g}_1  & = & \\
\end{eqnarray*}

The coefficients of ${\bf e}_1$ and ${\bf g}_1$ must be equal, individually, because they are not linearly dependent.   Equating the left-hand and right-hand sides and simplifying gives two nonlinear equations in $\xi$ and $\lambda$,
\begin{eqnarray}
 \frac{\xi \alpha_k}{b+1} + \sqrt{\frac{b-1}{b+1}} & = & \xi \lambda \label{eqn:nleqs1}\\
  \frac{\xi }{\sqrt{b^2-1}} + 1 - \frac{1}{b-1}  & = & \lambda. \label{eqn:nleqs2}
\end{eqnarray}
Multiplying the second equation by $\xi$, setting the left-hand sides of both equations equal to eliminate $\lambda$, then simplifying, yields the following quadratic equation in $\xi$,
$$
\xi^2 - \left( \alpha_k \sqrt{\frac{b-1}{b+1}} - \sqrt{b^2 - 1} + \sqrt{\frac{b+1}{b-1}}\right)\xi - \left( b-1\right) = 0,
$$
which is easily solved.   Define
$$
\beta_k = \frac{1}{2} \left(\alpha_k \sqrt{\frac{b-1}{b+1}} - \sqrt{b^2 - 1} + \sqrt{\frac{b+1}{b-1}} \right)\qq{and}
\gamma_k = b-1.
$$
Given $\xi$, $\lambda$ is determined by the second equation in (\ref{eqn:nleqs2}).   The solution set to nonlinear equations~(\ref{eqn:nleqs1})-(\ref{eqn:nleqs2}) is then
\begin{eqnarray*}
\xi^{(k)}_1 = \beta_k + \sqrt{\beta_k^2 + \gamma_k},
&&
\lambda^{(k)}_1 = \frac{\xi^{(k)}_1 }{\sqrt{b^2-1}} + 1 - \frac{1}{b-1} , \qq{and}\\
\xi^{(k)}_2 = \beta_k - \sqrt{\beta_k^2 + \gamma_k},
&&
\lambda^{(k)}_2 = \frac{\xi^{(k)}_2 }{\sqrt{b^2-1}} + 1 - \frac{1}{b-1}.
\end{eqnarray*}

Thus we have local eigenpairs of $H_k$, $((\xi^{(k)}_1 {\bf e}_1 + {\bf g}_1), \lambda_1^{(k)})$ and $( (\xi^{(k)}_2 {\bf e}_1 + {\bf g}_1), \lambda_2^{(k)})$.
The local eigenpairs  $((\xi^{(k)}_1 {\bf e}_1 + {\bf g}_1), \lambda_1^{(k)})$
yield global eigenpairs of $\hat{A}$ of the form
$( ({\bf c}_k \otimes (\xi^{(k)}_1 {\bf e}_1 + {\bf g}_1) ), \, \lambda_1^{(k)})$
and $( ({\bf s}_k \otimes (\xi^{(k)}_1 {\bf e}_1 + {\bf g}_1) ), \, \lambda_1^{(k)})$.
Similarly, $((\xi^{(k)}_1 {\bf e}_1 + {\bf g}_1), \, \lambda_2^{(k)})$ yield global eigenvectors of $\hat{A}$ associated with
 $\lambda_2^{(k)}$.
This accounts for the last $2q$ eigenpairs of $\hat{A}$.
\end{proof}

In order to illustrate these formulas, \Cref{fig:roceigenvectorsplot} shows
the computed eigenvectors for the graph ${\cal R}_{16,10}$ along with the eigenvalues.
Eigenvectors associated with eigenvalues close to $1$ have low conductance sweep cuts.
\begin{figure}[h]

    \includegraphics[width=1.0\textwidth]{figures/roc-eigenvectors.pdf}
    \caption{The eigenvectors of $\hat{A}$ are shown for ${\cal R}_{16,10}$. The eigenvectors with eigenvalues close to $1$
    (left) indicate the block structure with differing frequencies. The eigenvectors close to $-1$ (right) assign
    opposite signs to the internal vertices and corner vertices of each block.}
    \label{fig:roceigenvectorsplot}
\end{figure}

\begin{coro}
\label{cor:evals}
The asymptotic expansions of the eigenvalues are as follows.
\begin{itemize}
\item[(i)] For the signal eigenpairs, we see
$$
\lambda_1^{(k)} = 1- \frac{4-\alpha_k}{2 (b^2 - 1)} + \frac{1}{4(b-1)^2} + \frac{\alpha_k^2}{4(b+1)^2} + {\cal O}(b^{-3})
$$
\item[(ii)] For the non-signal (and non-noise) eigenpairs, we see
$$
\lambda_2^{(k)} = \frac{\alpha_k-1}{b+1} - \frac{\alpha_k^2}{4(b+1)^2} - \frac{\alpha_k}{2 (b^2 - 1)}  -  \frac{1}{4 (b - 1)^2} + {\cal O}(b^{-3})
$$
\item[(iii)] In particular, if $q$ is even we have
\begin{eqnarray*}
\lambda_2 = \lambda_1^{(1)} & \approx & 1 - \left[2 - \cos\left(\frac{2 \pi}{q}\right)\right] \frac{1}{b^2-1} +\frac{1}{4(b-1)^2} + \cos^2 \left(\frac{2 \pi}{q}\right) \frac{1}{(b+1)^2} ,
\\
\lambda_q = \lambda^{(q/2)}_{1} & \approx & 1 - \frac{3}{b^2-1} +\frac{1}{4(b-1)^2} + \frac{1}{(b+1)^2},
\\
\lambda_{q+1} = \lambda^{(0)}_{2} & \approx & \frac{1}{b+1}  -  \frac{1}{(b+1)^2} - \frac{1}{b^2 -1} - \frac{1}{(b-1)^2},
\\
\lambda_{n} = \lambda^{(q/2)}_{2} & \approx & \frac{-2}{b+1}  - \frac{1}{(b+1)^2}  + \frac{1}{b^2 -1} - \frac{1}{(b-1)^2}.
\end{eqnarray*}
In other words,
\begin{eqnarray*}
 1- \frac{C_2}{b^2} < \lambda_2 < 1, &&   1- \frac{C_q}{b^2} < \lambda_q < 1, \\ \\
0  < \lambda_{q+1} < \frac{C_{q+1}}{b}, &&   - \frac{C_n}{b} < \lambda_n < 0, \\
\end{eqnarray*}
where $C_2, C_q, C_{q+1}, $ and $C_n$ are positive quantities of order 1.
\end{itemize}
\end{coro}
\begin{proof}
These are seen through the formulas in Theorem~\ref{thm:Epairs} and first-order Taylor expansion of $\sqrt{\beta_k^2 + (b-1)}$ about leading asymptotic term $\frac{1}{4}((b^2-1)^2)$ and simplification.   Let $\theta = \sqrt{b-1}$ and $\eta = \sqrt{b+1}$.   Then
$$
\beta_k^2 = \frac{1}{4}\left(  \frac{\alpha_k \theta}{\eta} -\theta \eta + \frac{\eta}{\theta}   \right)^2 = \frac{\alpha_k^2 \theta^2}{4 \eta^2} + \frac{\theta^2 \eta^2}{4} + \frac{\eta^2}{4\theta^2} - \frac{\alpha_k \theta^2}{2} +\frac{\alpha_k}{2} - \frac{\eta^2}{2}.
$$
The first-order Taylor expansion we employ is $\sqrt{a^2+x} = a + \frac{1}{2} a^{-1} x + {\cal O}(a^{-3} x^2)$, which yields
\begin{eqnarray*}
\sqrt{\beta_k^2 + \theta^2} & = & \sqrt{\left[\frac{\theta \eta}{2}\right]^2 + \left[ \left(1 - \frac{\alpha_k}{2}\right)\theta^2 - \frac{\eta^2}{2} + \frac{\alpha_k^2 \theta^2 }{4 \eta^2} + \frac{\alpha_k}{2} +\frac{\eta^2}{4 \theta^2} \right]} \\
& = &  \frac{\theta \eta}{2} + \frac{1}{\theta \eta} \left[ \left(1 - \frac{\alpha_k}{2}\right)\theta^2 - \frac{\eta^2}{2} + \frac{\alpha_k^2 \theta^2 }{4 \eta^2} + \frac{\alpha_k}{2} +\frac{\eta^2}{4 \theta^2} \right] + {\cal O}(b^{-2}) \\
& = &  \frac{\theta \eta}{2} +  \left(1 - \frac{\alpha_k}{2}\right)\frac{\theta}{\eta} - \frac{\eta}{2 \theta} + \frac{\alpha_k^2 \theta }{4 \eta^3} + \frac{\alpha_k}{ 2 \theta \eta} +\frac{\eta}{4 \theta^3} + {\cal O}(b^{-2}). \\
\end{eqnarray*}
Now, we see
\begin{eqnarray*}
\xi^{(k)}_1 & = & \beta_k + \sqrt{\beta_k^2+\theta^2} = \frac{1}{2}\left(  \frac{\alpha_k \theta}{\eta} -\theta \eta + \frac{\eta}{\theta}   \right) + \sqrt{\beta_k+\theta^2} \\
& = & \frac{\theta}{\eta} + \frac{\alpha_k^2 \theta }{4 \eta^3} + \frac{\alpha_k}{ 2 \theta \eta} +\frac{\eta}{4 \theta^3} + {\cal O}(b^{-2}), \qquad \mbox{and } \\
\xi^{(k)}_2 & = & \beta_k - \sqrt{\beta_k^2+\theta^2} = \frac{1}{2}\left(  \frac{\alpha_k \theta}{\eta} -\theta \eta + \frac{\eta}{\theta}   \right) - \sqrt{\beta_k+\theta^2} \\
& = & -\theta\eta + \left( \alpha_k - 1\right) \frac{\theta}{\eta} + \frac{\eta}{\theta} - \frac{\alpha_k^2 \theta }{4 \eta^3} - \frac{\alpha_k}{ 2 \theta \eta} - \frac{\eta}{4 \theta^3} + {\cal O}(b^{-2}).
\end{eqnarray*}
Then, noting $\eta^{-1} - \theta^{-1} = -2 \eta^{-1}\theta^{-1}$, we see
\begin{eqnarray*}
\lambda^{(k)}_1 & = & 1 - \frac{1}{\theta^2} + \frac{\xi_1^{(k)}}{\theta \eta}\\
& = & 1 - \frac{1}{\theta^2} + \frac{1}{\eta^2} + \frac{\alpha_k}{2 \theta^2 \eta^2} + \frac{1}{4 \theta^4} + \frac{\alpha_k^2}{ 4 \eta^4 }\\
& = & 1 - \frac{4 - \alpha_k}{2 \theta^2 \eta^2} + \frac{1}{4 \theta^4} + \frac{\alpha_k^2}{4 \eta^4} + {\cal O}(b^{-3}).
\end{eqnarray*}
Substituting in for $\theta$ and $\eta$ gives the result presented in (i).   Similarly, we see (ii) via
$$
\lambda^{(k)}_2  =  1 - \frac{1}{\theta^2} + \frac{\xi_2^{(k)}}{\theta \eta} = \frac{\alpha_k-1}{\eta^2} - \frac{\alpha_k^2}{4 \eta^4} - \frac{\alpha_k}{2\theta^2 \eta^2} - \frac{1}{4 \theta^4} + {\cal O}(b^{-3}).
$$
Lastly, (iii) is seen by plugging in for specific values of $k$ and $\alpha_k = 2 \cos \left( \frac{2 \pi k }{q} \right) $.

\end{proof}

\begin{rem} We observe several facts:

\begin{itemize}
\item The vector $D^{-1/2 } \sigv{1}{0}$ is the constant vector.   It causes no errors, but does not help to separate any of the cliques.

\item The vectors $D^{-1/2} \sigv{1,1}{k}$ and $D^{-1/2} \sigv{1,2}{k}$ for $k=\intrange{1}{\ceil{q/2}-1}$ assign the same sign to the corners{} as the internal members of each block and are associated with positive eigenvalues.   Note that we can consider all these eigenvectors as signal eigenvectors,
$$
	{\cal X}_{signal} = \mbox{span}\left\{\sigv{1}{1}, \sigv{1}{2}, ... , \sigv{1}{\ceil{q}-1}\right\}.
$$
Because ${\cal X}_{signal}\perp {\cal X}_{noise}$, all sweep cuts of vectors in ${\cal X}_{signal}$ keep internal vertices of each clique together.

\item If $q$ is even, the vector $D^{-1/2} \sigv{2}{q/2}$ has values at the corners{} of opposite sign to the values of the internal vertices and the sign of each corner oscillates around the ring.
This is the most oscillatory global eigenvector.

\item The vectors $D^{-1/2} \sigv{2,1}{k}$ and $D^{-1/2} \sigv{2,2}{k}$ for $k=\intrange{1}{\ceil{q/2}-1}$ assign
    opposite signs to the corners{} and the internal members of each block. If vectors make large contributions to the blend we compute, then there is potential to misclassify several of the corner vertices.

\end{itemize}
\end{rem}

\begin{figure}
\includegraphics[width=3.0in]{figures/roc_asymptotic_delta.pdf}
\includegraphics[width=3.0in]{figures/roc_asymptotic_lambda.pdf}
\caption{Asymptotic estimates of spectral gaps (left) and Fiedler eigenvalues (right) for rings of cliques with parameters $b = 10,100,1000$ and $q=2,4, ..., 8096$.   Lines represent leading-order terms derived in Theorems~\ref{thm:roceigengap} and \ref{thm:fiedler} and '+' represent actual eigenvalues as given by the formulas in Theorem~\ref{thm:Epairs}. }
\label{fig:asymptotic}
\end{figure}

We use the previous result to derive asymptotic estimates of the eigengaps
associated with eigenvalues near 1 and the size of the Fiedler eigenvalue.
These asymptotic estimates are compact formulas in terms of $b$ and $q$.
Figure~\ref{fig:asymptotic} verifies these estimates empirically.

\begin{thm} {\bf (ROC Asymptotic Eigengap)}
\label{thm:roceigengap}
For the graph $\Roc$ the spectral gap relevant for computing an eigenvector associated with $\lambda_2(\hat{L})$ is asymptotically $\bigo{b^{-2} q^{-2}}$ for large $b$ and $q$.
\end{thm}

\begin{proof}
Because $\lambda_2(\hat{A}) = \lambda_3(\hat{A})$, the eigenvalues of interest are $\lambda_2(\hat{A}) =  \lambda^{(1)}_1$ and
$\lambda_4(\hat{A}) = \lambda^{(2)}_1$.    We will take Taylor expansions and collect the leading-order terms to understand the asymptotic behavior.   Define scalar function $f(x) = \sqrt{x+a}$, for constant $a$ that we define below. Using the formulas in the previous result and a bit of algebra, we see $ \lambda^{(1)}_1 -  \lambda^{(2)}_1 = $
\begin{equation}
\label{eqn:egap_roc}
\frac{\xi^{(1)}_1 - \xi^{(2)}_1}{\sqrt{b^2 -1}} =
 \frac{\beta_1 - \beta_2+\sqrt{\beta_1^2  + (b-1)} - \sqrt{\beta_2^2  + (b-1)} }{\sqrt{b^2 -1}} = \frac{\beta_1 - \beta_2 + f(x_1)-f(x_2)}{\sqrt{b^2 -1}}
\end{equation}
with
$$
x_k = \left( \frac{1}{4} - \frac{1}{2(b+1)}\right) \alpha_k^2 + \left(1 - \frac{b}{2}\right) \alpha_k
\qquad \mbox{and} \qquad
a = \frac{b^2}{4} + \frac{b}{2} - \frac{3}{2} + \frac{1}{2(b-1)}.
$$
We expand the two differences in the numerator of (\ref{eqn:egap_roc}) separately, concentrating on the $f(x_1) - f(x_2)$ first.   First-order Taylor expansion of $f(x)$ at $x_1$ yields
$$
f(x_2) = f(x_1) + f'(x_1)(x_2 - x_1) + \frac{f''(y)}{2} (x_2 - x_1)^2 .
$$
where $y \in (\min(x_1, x_2), \max(x_1, x_2))$.   Rearranging and plugging in, we see
$$
f(x_1) - f(x_2) = f'(x_1)(x_1 - x_2) - \frac{f''(y)}{2} (x_1 - x_2)^2 = \frac{x_1 - x_2}{2\sqrt{x_1 + a}} - \frac{(x_1 - x_2)^2}{4(y + a)^{3/2}}
$$

Further, assume $q >> 4 \pi k$ and use a  third-order Taylor expansion at 0 to see $\alpha_k = 2 - 4(\pi k/q)^2 + {\cal O}(q^{-4})$.   Similarly, $\alpha_k^2 = 4 \cos^2(2 \pi k/q) = 2 (1 + \cos(4 \pi k/q)) = 4 - 16(\pi k/q)^2 + {\cal O}(q^{-4})$.   Thus,
\begin{eqnarray*}
(x_1 - x_2) & =  & \left( \frac{1}{4} - \frac{1}{2(b+1)}\right) \left(\frac{48 \pi^2}{q^2} \right) +  \left(1 - \frac{b}{2}\right) \left(\frac{12 \pi^2}{q^2} \right) + {\cal O}\left( b q^{-4} \right) \\
& = &  \frac{-6 \pi^2 b}{q^2} + \frac{24 \pi^2}{q^2} + {\cal O}\left(b q^{-4} \right).
\end{eqnarray*}

We return to $\beta_1 - \beta_2$ and expand using the cosine Taylor expansions.

$$
\beta_1 - \beta_2 = \frac{\alpha_1 - \alpha_2}{2}\sqrt{\frac{b-1}{b+1}} = \frac{6\pi^2}{q^2}\sqrt{\frac{b-1}{b+1}} + {\cal O}(q^{-4})
$$

Lastly, $(y + a)^{3/2}$ is ${\cal O}(b^3)$ and $(x_1+ a)^{1/2}$ is ${\cal O}(b)$, so

$$
\lambda^{(1)}_1 -  \lambda^{(2)}_1 = \frac{6 \pi^2 }{q^2 (b + 1)} + \frac{-3 \pi^2 b}{q^2 \sqrt{x_1 + a} \sqrt{b^2 - 1}} +
\frac{12 \pi^2}{q^2 \sqrt{x_1 + a} \sqrt{b^2 - 1}}
+{\cal O}(b^{-1} q^{-4}).
$$
As $b \rightarrow \infty$ we see $2 b^{-1} \sqrt{x_1 + a} \rightarrow 1$, so the first two terms cancel asymptotically
and the third term is $\bigo{q^{-2}b^{-2}}$.
\end{proof}

\begin{thm} {\bf (ROC Asymptotic Fiedler Eigenvalue)}
\label{thm:fiedler} Let $b$ and $q$ be large and the same order.   For graph $\Roc$ the smallest nonzero eigenvalue of $\hat{L}$ is ${\cal O}(b^{-2}q^{-2})$.
\end{thm}
\begin{proof}
The eigenvalue of interest is $\lambda_2(\hat{L}) = 1- \lambda_2(\hat{A}) =  1-\lambda^{(1)}_1$.    Define scalar function $g(z) = \sqrt{z+1/4}$.   Using the formulas in Theorem~\ref{thm:Epairs} and a bit of algebra, we see
\begin{equation}
\label{eqn:eval_roc}
1 - \lambda^{(1)}_1=
\frac{1}{b-1} - \frac{\xi^{(1)}_1}{\sqrt{b^2 -1}} =
 \frac{1}{b-1}  - \frac{\beta_1 + \sqrt{\beta_1^2  + (b-1)}}{\sqrt{b^2 -1}} = \frac{1}{b-1} - \frac{\beta_1}{\sqrt{b^2 -1}} - g(z)
\end{equation}
with
$$
z = \frac{(1-\alpha_1)b}{2(b^2 -1)} + \frac{2\alpha_1-3}{2(b^2 -1)} +  \frac{1}{4(b-1)^2} + \frac{\alpha_1^2}{4(b+1)^2}.
$$
We derive the result by demonstrating that the larger terms in (\ref{eqn:eval_roc}) cancel.   Expanding the second term in in the right-hand-side yields
$$
\frac{\beta_1}{\sqrt{b^2 -1}} = \frac{\alpha_1}{2(b+1)} - \frac{1}{2} + \frac{1}{2(b-1)} = -\frac{1}{2}  + \frac{(1+\alpha_1)b}{2(b^2 - 1)} + \frac{1-\alpha_1}{2(b^2 - 1)} .
$$

A second-order Taylor expansion of $g(z)$ at zero shows for each $z$, we see
$g(z) = g(0) + g'(0) z + \frac{1}{2}g''(0) z^2 + \frac{1}{6}g'''(0) z^3+ \frac{1}{24}g^{(iv)}(y) z^4= \frac{1}{2} + z - z^2 +2 z^3 + {\cal O}(z^4) $, where
$$
g'(z) = \frac{1}{2\sqrt{z + 1/4}} ,
\qquad
 g''(z) = -\frac{1}{4 (z + 1/4)^{3/2}},
\qquad \mbox{and} \qquad
 g'''(z) = \frac{3}{8 (z + 1/4)^{5/2}}.
$$
Plugging in, we see the terms of $g(z)$ up to order $b^{-2}$ are
$$
\frac{1}{2} + \frac{(1-\alpha_1)b}{2(b^2 -1)}
 + \frac{2\alpha_1-3}{2(b^2 -1)} +  \frac{1}{4(b-1)^2} + \frac{\alpha_1^2}{4(b+1)^2}
- \frac{(1-\alpha_1)^2 b^2}{4(b^2 -1)^2}.
$$

The constant terms in $\beta_1/\sqrt{b^2 -1}$ and $g(z)$ cancel.   The order $b^{-1}$ terms in $1 - \lambda^{(1)}_1$ cancel to an order $b^{-2}$ term,
$$
\frac{1}{b-1}  -\frac{(1+\alpha_1)b}{2(b^2 - 1)} - \frac{(1-\alpha_1 )b}{2(b^2 - 1)}
= \frac{1}{b-1}  -\frac{b}{b^2 - 1}  = \frac{1}{b^2 - 1} .
$$
Combining fractions the order $b^{-2}$ terms in $1 - \lambda^{(1)}_1$, are reduced to
\begin{eqnarray*}
&&  \frac{1}{b^2 - 1} - \left[
 \frac{1-\alpha_1}{2(b^2 - 1)} +
 \frac{2\alpha_1-3}{2(b^2 -1)} +  \frac{1}{4(b-1)^2} + \frac{\alpha_1^2}{4(b+1)^2}
- \frac{(1-\alpha_1)^2 b^2}{4(b^2 -1)^2}
 \right] \\
 & = & \frac{(2 -  \alpha_1) b^2}{(b^2 - 1)^2} + \frac{(2 \alpha_1^2 -3) b}{2(b^2 - 1)^2} + \frac{-\alpha_1^2 + 2\alpha_1 - 9}{4(b^2-1)^2} \\
\end{eqnarray*}
Factoring in the other $b^{-3}$ terms and the cosine expansion $\alpha_1 = 2-4(\pi /q)^2+{\cal O}(q^{-4})$, we see
$$
\frac{4 \pi^2 b^2}{q^2 (b^2 - 1)^2} + {\cal O}\left(b^{-4} +  b^{-2} q^{-4} \right).
$$
\end{proof}

\newcommand{\conductance{\Dneghalf\bx}}{\conductance{D^{-\half}{\bf x}}}
\begin{thm}
    For any vector ${\bf x}\in\cX_{signal}$ of $\Roc$, $ \conductance{\Dneghalf\bx} \le q\conductance{\bv_2\paren{L}}$.
\end{thm}
\begin{proof}
    For any ${\bf x} \in {\cal X}_2$, let $S, \bar{S}$ be the partition given by the optimal sweep cut of $D^{-\half}{\bf x}$.
    Fiedler's nodal domain theorem implies at least one of $S, \bar{S}$ is a connected component.

    Because the eigenvectors are block constant, all vertices of each clique are assigned to the same side of the
    partition.
    These imply that $E\paren{S,\bar{S}}=2$.
    The optimal conductance partition is found when $\vol{S}=\vol{\bar{S}}= q {b\choose 2}$.
    Thus $\phi_G = \conductance{{\bf v}} = 2\paren{q {b\choose 2}}^{-1}$.

    For any ${\bf x}\in\cX_{signal}$, the optimal sweep cut of $D^{-\half}{\bf x}$ will partition the graph into two pieces one containing $k$ blocks
    and the other containing $n-k$ blocks for some $k\le \frac{n}{2}$.
    That is $\min\paren{{\vol{S}},{\vol{\bar{S}}}}=k{b\choose 2}$.
    Since only edges between cliques are cut, $E(S,\bar{S}) \le 2k$.
    Thus $\conductance{\Dneghalf\bx} \le 2{b\choose 2}^{-1}$. By assigning adjacent blocks to alternating sides of the partition,
    we see the bound is tight.

\end{proof}

Notice that the smallest eigenvalue of $\hat{L}$ scales as $\bigo{b^{-2}q^{-2}}$,
  but the optimal conductance scales as $\bigo{b^{-2}q^{-1}}$,
  and that the worst case sweep cut partition of a blend has conductance $2{b\choose 2}^{-1}$ independent of $q$.
The remainder of this section shows that by accepting this factor of $q$ in conductance, one gains tremendously in
computational efficiency.

\subsection{A residual tolerance for ring of cliques{}}\label{sec:rocbygeneral}
In order to derive a residual tolerance for the ring of cliques{}, we show that for any vector in $Span\{\bv_1\dots \bv_q\}$
at least one block is sufficiently far from the other blocks in spectral coordinates.
\begin{lemma}\label{thm:mus-apart}
    Define $\block{i} = \set{ib+2\dots ib+q}$ as the vertex blocks of $\Roc$.
    For any vector ${\bf x}$, let $\mean{j} = \abs{\block{i}}^{-1}\sum_{i\in\block{j}}x_i$.
    Let ${\cal{W}}$ be the span of
    $\set{{\bf e}_{ib+1}\mid i \in 0\dots q-1} \cup \set{\blockindicator{i}{q}\mid  i \in 0\dots q-1}$.
    For any vector ${\bf x}\in {\cal{W}}$, $\norm{\boldsymbol{\mu}}_\infty > n^{-1/2}\norm{{\bf x}}_2$.
\end{lemma}
\begin{proof}
    By construction of ${\bf x}$, $\mean{i} = x_j$ for all $j\in\block{i}$.
    Thus $\norm{{\bf x}}_\infty = \norm{\boldsymbol{\mu}}_\infty$.
    Equivalence of norms implies $\norm{{\bf x}}_2 < \sqrt{n}\mean{q}$.
\end{proof}

We are able to apply \cref{thm:blendspaceconductance} and derive a residual tolerance for recovering the ring of cliques{}.
\begin{coro}\label{thm:roc-residbound-general}
    If ${\bf x}$ is an approximate eigenvector of $\Roc$ with eigenresidual less than $\frac{C}{q\sqrt{n}}$ for some constant $C$ and ${\bf x}^t\mathbf{1}=0$,
    then $\conductance{D^{-\half}{\bf x}}\le 2 {b\choose 2}^{-1}$
\end{coro}
\begin{proof}
In the setting of \cref{thm:blendspaceconductance},
  choose $G=\Roc$, $\psi=2 {b\choose 2}^{-1}$.
In the notation of \cref{thm:mus-apart} applied to sorted ${\bf v}$,
for all  ${\bf v} \in V$, $g_v = \max_i \paren{\mean{i+1}-\mean{i}} \ge q^{-1}\norm{\boldsymbol{\mu}}_\infty \ge \paren{q\sqrt{n}}^{-1}$.
For some $C\in\bigo{1}$, $\delta_{p,q}(\mu) g > \frac{C}{q\sqrt{n}}$.
So \Cref{thm:blendspaceconductance} implies computing ${\bf x}$ to a residual tolerance of $\frac{C}{q\sqrt{n}}$ is sufficient to guarantee $\conductance{{\bf x}} \le 2{b\choose 2}^{-1}$.
\end{proof}
\Cref{thm:roc-residbound-general} gives a sufficient condition on approximate eigenvectors of $\Roc$ such that ${\bf x}$ partitions the graph at least as well as any partition that recovers the cliques.
\Cref{thm:minimal-perturbation-lb} and \Cref{thm:minimal-perturbation-ub} using analysis specialized for $\Roc$ in
\cref{sec:roc-perturbation} to construct the minimal perturbation that causes the sweep cut proceedure to fail.

\subsection{Minimal Perturbation}\label{sec:roc-perturbation}
We want to find the minimal error at which a vector can make a mistake.
The effects of the corner vertices only enter into the constants of the following results,
  and for clarity of exposition we omit handling of the corner vertices.
\Cref{thm:minimal-perturbation-lb} shows that no perturbation with norm less than $\paren{1+2qn}^{-\half}$ can induce a mistake in the sweep cut.
\Cref{thm:minimal-perturbation-ub} constructs a perturbation that induces a mistake in the sweep cut and has norm less than $b^{-\half}$.
For the parameter regime $q\in\bigo{1}$, the bounds in \Cref{thm:roc-residbound-general}, \Cref{thm:minimal-perturbation-lb}, and \Cref{thm:minimal-perturbation-ub} are all equivalent up to constant factors.

Using the same notation as \cref{thm:mus-apart},
we say that a vector ${\bf y}$ recovers all the cliques of $\Roc{}$
    if there is a threshold $t\in\opintervalop{\minelt{y}}{\maxelt{y}}$

    such that for all $\block{i}$ with $j,k\in\block{i}$,
    $y_{j} < t$ if and only if  $y_{k} < t$.

\begin{thm}\label{thm:minimal-perturbation-lb}
    Let ${\cal{W}}$ be defined as in \cref{thm:mus-apart},
    and let $\Proj{{\cal{W}}}$ be the orthogonal projector onto ${\cal{W}}$.
For any vector ${\bf y}$ orthogonal to $\mathbf{1}$, define $\bz=$ $\paren{{\bf I}-\Proj{\blockconstspace}}{\bf y}$.
If $\norm{\bz}_2\le\paren{1+2qn}^{-\half}\norm{{\bf y}}_2$,
    then ${\bf y}$ recovers all the cliques of $\Roc{}$.
\end{thm}
\begin{proof}
    Define ${\bf x} = \Proj{{\cal{W}}}{\bf y}$.

    Without loss of generality, relabel the vertices and blocks such that $\mean{i} \le \mean{i+1}$.
    Let $\alpha_i = \maxelt{\err{}}$ and $\beta_i = \minelt{\err{}}$ for each $\block{i}$.
    Note that $\alpha_i > 0$ and $\beta_i < 0$ since $\bz \perp \blockindicator{i}{q}$.

    The vector ${\bf y}$ recovers all the cliques if and only if there is a $\block{i}$ where
    $\alpha_{i} - \beta_{i+1} \le \mean{i+1} - \mean{i} $.
    In this case, a threshold can be chosen in
        $\opintervalop{
            \mean{i}+\alpha_{j}}{
            \mean{i+1}+\beta_{k}}
        $.
    Suppose that ${\bf y}$ does not recover all the cliques,
    then for all $\block{i}$ $\alpha_{i} - \beta_{i+1} > \mean{i+1}-\mean{i}$.
    This implies $\sum_{i=0}^{q-1} \alpha_{i}-\beta_{i+1} > \sum_{i=0}^{q-1} \mean{i+1}-\mean{i}$.

    Thus  we can bound the 1-norm error as follows:
    \[
        \norm{\bz}_1
        \ge \sum_i\paren{\alpha_{i} - \beta_{i+1}}

        \ge \mean{q} - \mean{1}

        \ge n^{-\half}\norm{{\bf x}}_2.
    \]
    Since $\bz$ must have at least $q$ nonzero entries $\norm{\bz}_2 > \paren{2qn}^{-\half}\norm{{\bf x}}_2$.

    Applying $\norm{{\bf y}}^2 = \norm{{\bf x}}^2 + \norm{\bz}^2$, we see that

    ${\paren{1+2qn}^{-\half}}\norm{{\bf y}} < \norm{\bz}_2$.

\end{proof}

The proof of Theorem~\ref{thm:minimal-perturbation-lb} yields a construction
for the minimal perturbation of ${\bf x}$ that does not recover all the cliques.

\begin{thm}\label{thm:minimal-perturbation-ub}
    For any unit vector ${\bf x}\in{\cal{W}}$ orthogonal to $\mathbf{1}$,
    there exists a perturbation $\bz$ where $\norm{\bz} < b^{-\half}, \Proj{{\cal{W}}}\bz = 0$
        such that ${\bf y} = {\bf x}+\bz$ does not recover all the cliques.
\end{thm}
\begin{proof}
For any ${\bf x}\in{\cal{W}}$,
set $\alpha_0=0, \beta_{q-1}=0, \alpha_i=-\beta_{i+1} = \frac{\mean{i+1}-\mean{i}}{2}$.
\[
    \norm{\bz}_2^2
    = \sum_{i=0}^{q-1} \alpha_i^2 + \beta_{i+1}^2

    = \half\sum_{i=0}^{q-1}\paren{\mean{i+1}-\mean{i}}^2

    = b^{-1}\norm{{\bf x}}_2^2 -2\sum_{i=0}^{q-1}\mean{i+1}\mean{i}

    < b^{-1}\norm{{\bf x}}_2^2
\]
\end{proof}
Theorem~\ref{thm:minimal-perturbation-lb} implies that $\paren{1+2qn}^{-\half}$ is a sufficient accuracy to ensure recovery of all the cliques, and Theorem~\ref{thm:minimal-perturbation-ub} implies that for some elements of the top invariant subspace
accuracy less than $b^{-\half}$ is necessary to ensure recovery of all the cliques from that vector.

\Cref{fig:minimal-perturbation-lowerbounds} lends validation to the formulas in \cref{thm:minimal-perturbation-lb}.
The experiment shown is to take a random (gaussian unit norm) linear combination of ${\cal X}_{signal}$, and then construct the minimal perturbation that makes a mistake.
\Cref{fig:minimal-perturbation-lowerbounds} shows the minimum over all samples as a function of $n$.
This experiment is conducted for three different parameter regimes, $q=25$, $b=25$, and $b=q=\sqrt{n}$.
One can see that the lower bound from \cref{thm:minimal-perturbation-lb} is below the empirical observation,
and that this lower bound is within a constant factor of the observed size of the minimal perturbation.
\begin{figure}
    \centering
    \includegraphics[width=0.9\textwidth]{figures/roc-minimal-perturbation-lowerbounds.pdf}
    \caption{Empirical measurements of minimal error perturbations on a log-log scale. Lower bounds are shown in the same color with dashed lines.}\label{fig:minimal-perturbation-lowerbounds}
\end{figure}

We can now apply Theorem~\ref{thm:singleevecerror} and Theorem~\ref{thm:blenderror}
to determine the residual tolerance for an eigensolver for graph paritioning.
The residual tolerance can be no larger than that for the the ring of cliques,
but some graphs may require smaller tolerances.

\subsection{The Power Method}\label{sec:powermethod}

\newcommand{\mbox{erf}}{\mbox{erf}}
\newcommand{k^*}{k^*}
\newcommand{1-\zeta}{1-\zeta}
\newcommand{\lambda^*}{\lambda^*}
From the eigenvalues and error tolerances above, one can determine an upper bound on the number of iterations required by the power method to recover all the cliques in $\Roc{}$.

\begin{thm}\label{thm:rocpowermethod}
    Let ${\bf x}_0$ be sampled from ${\cal N}_n\paren{0, 1}$.
    Let ${\bf x}_k$ be the $k$-th iteration of the power method, ${\bf x}_{k+1} \longleftarrow \hat{A} {\bf x}_k / \|{\bf x}_k\|$ for $\Roc{}$.
    Let $\zeta = \left(\frac{e^{7/8}}{8}\right)^{q/2} + \left(\frac{2}{e}\right)^{(n-q)/2}$.
    There is a $k^*$ of $\bigo{\log_b q}$ such that
    for $k \geq k^*$ a sweep cut based on ${\bf x}_k$ makes no errors
    with probability at least $1-\zeta$.
\end{thm}
\begin{proof}
First, we bound $\|\paren{{\bf I}-\Proj{\blockconstspace}} {\bf x}_0\|^2$ and $\|\Proj{\blockconstspace} {\bf x}_0\|^2$ probabilistically.   Each entry in ${\bf x}_0$ is independently sampled from ${\cal N}\paren{0, 1}$.   For any orthonormal basis of $\mathbb{R}^n$, $\{{\bf v}_k\}_{k=1}^n$, the distribution of each ${\bf v}_k^t {\bf x}_0$ is also ${\cal N}\paren{0, 1}$.   Therefore, the distribution of $\|\Proj{\blockconstspace} {\bf x}_0\|^2 $ is a $\chi^2$-distribution of order $q$, which has expected value $q$ and cumulative distribution function $\gamma(q/2, z/2)/\Gamma(q/2)$, where $\Gamma(\cdot)$ is the gamma function and $\gamma(\cdot, \cdot)$ is the lower incomplete gamma function.    Let $c_0 \in (0,1)$, using Chernoff bounds we have
$$
p_0 := \mbox{Prob} \left[ \|\Proj{\blockconstspace} {\bf x}_0\|^2 > c_0 q \right] = 1- \frac{\gamma\left(\frac{q}{2}, \frac{c_0 q }{2}\right)}{\Gamma\left(\frac{q}{2}\right)} \geq 1-\left(  c_0 e^{1 - c_0} \right) ^{q/2}
$$

Similarly, $\|\paren{{\bf I}-\Proj{\blockconstspace}} {\bf x}_0\|^2 $ is from a $\chi^2$-distribution  of order order $n-q$, with expected value $n-q$ and known cdf.    Let $c_1 \in (1,\infty)$, we have
$$
p_1 := \mbox{Prob} \left[ \|\Proj{\blockconstspace} {\bf x}_0\|^2 < c_1 (n-q) \right] = \frac{\gamma\left(\frac{n-q}{2}, \frac{c_1 (n-q) }{2}\right)}{\Gamma\left(\frac{n-q}{2}\right)} \geq 1-\left(  c_1 e^{1 - c_1} \right) ^{(n-q)/2}
$$

The union of events $\left[ \|\Proj{\blockconstspace} {\bf x}_0\|^2 > c_0 q \right]$ and $\left[ \|\Proj{\blockconstspace} {\bf x}_0\|^2 < c_1 (n-q) \right]$ is a subset of all possibilities for which $\left[ \frac{ \|\paren{{\bf I}-\Proj{\blockconstspace}} {\bf x}_0\|^2 }{\|\Proj{\blockconstspace} {\bf x}_0\|^2} < \frac{c_1 (n-q)}{ c_0 q} \ \right]$ holds.   Therefore, setting $c_0 = 1/8$ and $c_1 = 2$, we see

$$
\mbox{Prob} \left[ \frac{ \|\paren{{\bf I}-\Proj{\blockconstspace}} {\bf x}_0\|^2 }{\|\Proj{\blockconstspace} {\bf x}_0\|^2} < \frac{c_1 (n-q)}{ c_0 q} \ \right] > p_0 p_1 >1 - \zeta
$$
where $\zeta := \left(\frac{e^{7/8}}{8}\right)^{q/2} + \left(\frac{2}{e}\right)^{(n-q)/2}$ is a small positive constant
when $q,b > 4$.
Because a sweep cut does not depend on the norm of a vector, we consider the iteration, ${\bf x}_{k} \longleftarrow
\frac{1}{\lambda_q}\hat{A} {\bf x}_{k-1}$ which is equivalent to the power method.
Letting $\lambda^* = \pairmax{\abs{\lambda_{q+1}}}{\abs{\lambda_n}}$,
this iteration accentuates vector
components in the range of $\Proj{\blockconstspace}$ by a factor greater than 1 and attenuates those orthogonal to this space by
factors less than $\lambda^*/ \lambda_q$.   If $\|\Proj{\blockconstspace} {\bf x}_0\|^2 > c_0 q$, then

$$\|{\bf x}_k\|^2 \geq \|\Proj{\blockconstspace} {\bf x}_k\|^2 \geq \|\Proj{\blockconstspace} {\bf x}_0\|^2 \geq c_0 q.$$

Also, if $\|\paren{{\bf I}-\Proj{\blockconstspace}} {\bf x}_0\|^2 \leq c_1 (n-q)$, then

$$  \|\paren{{\bf I}-\Proj{\blockconstspace}} {\bf x}_k\|^2 \leq \left( \frac{\lambda^*}{\lambda_q}\right)^{2k}  \|\paren{{\bf I}-\Proj{\blockconstspace}} {\bf x}_0\|^2 \leq \left(
\frac{\lambda^*}{\lambda_q}\right)^{2k} c_1 (n-q).$$

Therefore, under the assumptions on ${\bf x}_0$, the $k$-th iteration satisfies

$$
\frac{\norm{\paren{{\bf I}-\Proj{\blockconstspace}} {\bf x}_k}}{\norm{{\bf x}_k}} \leq \left( \frac{\lambda^*}{\lambda_q}\right)^{k} \sqrt{\frac{c_1}{c_0} (b-1)}
= 4 \left( \frac{\lambda^*}{\lambda_q}\right)^{k} \sqrt{b-1}.
$$

By \cref{thm:minimal-perturbation-lb}, if this ratio is less than $\paren{1+2qn}^{-1/2}$, then ${\bf x}_k$ makes no errors. We see this is ensured by

    $$
    k \geq k^* := \left\lceil \frac{ \log 4 + \log (b-1) + \log(1+2qn)}{2\log\left( \lambda_q / \lambda^*\right)} \right\rceil.
    $$
Revisiting Corollary~\ref{cor:evals} we that $\lambda_q > 1 - C_q/b^2$ and $\lambda^* < \max(C_{q+1}, C_n)/b$ so
$\lambda_{q} / \lambda^* = C^* b$, where $C^*$ is an order 1 constant.   Plugging this in we see that $k^*$ is in $\bigo{\log_b q}$.

\end{proof}

\subsection{Experiment}\label{sec:roc_experiment}
Here we show the results of a numerical experiment in order to lend intuition and validation to the theorems.
Take $\Roc{}$ and a random seed vector $\bx^{(0)}$.
Then apply power iteration ${\bf x}^{(i)} = (\ahat - {\bf k} {\bf k}^t) {\bf x}^{(i-1)}$.
Far $b=20$ and $q=30$ the relevant measures of convergence are shown in \cref{tab:pmeth}.
\Cref{fig:pmeth} illustrates the convergence behavior in terms of the conductance of all sweep cuts,
	and the reordered adjacency matrix represented in sparsity plots.
\Cref{tab:pmeth} shows that the convergence to the Fiedler vector stalls after iteration 3,
	but convergence to the space orthogonal to ${\cal X}_{noise}$ continues unabated.   Letting $\Pi$ be the projection onto ${\cal X}_{noise}^\perp$, we measure $\|\Pi {\bf x}^{(i)}\|$ for each iteration.
Applying \cref{thm:rocpowermethod}, we calculate that $k^* =5$ iterations will perfectly resolve the clique structure with probability at least $1-\zeta = 0.99999998575$.  After one iteration the sweep cut did not split any cliques, but only a single clique is shaved off.   After 3 iterations a nearly optimal partition is found.

\begin{minipage}[h]{\textwidth}
\centering
\begin{tabular}{ccc}
First Iteration & Second Iteration & Third Iteration \\
\includegraphics[width=132pt]{figures/roc_q30_b20_pmeth_i1_cnf_final.pdf}&
\includegraphics[width=132pt]{figures/roc_q30_b20_pmeth_i2_cnf_final.pdf}&
\includegraphics[width=132pt]{figures/roc_q30_b20_pmeth_i3_cnf_final.pdf}\\
\includegraphics[width=132pt]{figures/roc_q30_b20_pmeth_i1_spy_final.pdf}&
\includegraphics[width=132pt]{figures/roc_q30_b20_pmeth_i2_spy_final.pdf}&
\includegraphics[width=132pt]{figures/roc_q30_b20_pmeth_i3_spy_final.pdf}
\end{tabular}
\captionof{figure}{First several iterations of the power method applied to ${\cal R}_{b=20,q=30}$.    Above: sweep conductance of $A$ reordered by sorting the 1$^{st}$ (top-left), 2$^{nd}$ (top middle), and 3$^{rd}$ iterations (top-right).   Horizontal axis represents which vertex to split at under the induced ordering; vertical axis is the conductance for each split on a log scale.   Below: matrix sparsity plots of $A$ reordered by sorting the 1$^{st}$ (bottom-left), 2$^{nd}$ (bottom-middle), and 3$^{rd}$ iterations (bottom-right).  Red lines demonstrate which edges are cut for the optimal cut in each ordering.   }
\label{fig:pmeth}

\vspace{.5cm}
\begin{tabular}{|c|c|c|c|c|c|}
\hline
      $i$ & $\epsilon$  & $\| \Pi {\bf x}^{(i)} \|$ &    $\mu$    & $\phi({\bf x})$ & $\sqrt{2 \mu}$ \\ \hline
        0 & 1.24806e+01 & 2.25433e+01 & 1.52966e+00 & 2.03094e+00 & 1.74909e+00   \\
        1 & 9.20534e-01 & 5.02679e-02 & 8.37276e-01 & 1.05263e-02 & 1.29404e+00   \\
        2 & 1.02629e-01 & 1.45977e-02 & 5.67751e-02 & 6.68577e-03 & 3.36972e-01   \\
        3 & 1.08242e-02 & 8.10095e-04 & 1.01284e-02 & 4.89853e-03 & 1.42326e-01   \\
        4 & 1.01568e-02 & 4.28591e-05 & 9.87761e-03 & 4.89853e-03 & 1.40553e-01   \\
        5 & 1.01262e-02 & 2.26698e-06 & 9.85062e-03 & 4.89853e-03 & 1.40361e-01   \\
        6 & 1.01022e-02 & 1.19907e-07 & 9.82549e-03 & 4.89853e-03 & 1.40182e-01   \\
        7 & 1.00782e-02 & 6.34217e-09 & 9.80038e-03 & 4.89853e-03 & 1.40003e-01   \\
        8 & 1.00543e-02 & 3.35448e-10 & 9.77526e-03 & 4.89853e-03 & 1.39823e-01   \\
        9 & 1.00303e-02 & 1.77422e-11 & 9.75013e-03 & 4.89853e-03 & 1.39643e-01   \\
       10 & 1.00063e-02 & 9.38388e-13 & 9.72498e-03 & 4.89853e-03 & 1.39463e-01   \\
 $\cdots$ & $\cdots$ & $\cdots$ & $\cdots$  & $\cdots$ & $\cdots$ \\
 $\infty$ & 0.00000e+00 & 0.00000e+00 & 1.14176e-04 & 3.50018e-04 & 1.51113e-02   \\
&  && $\lambda_2(\hat{L})$ & $\phi({\bf v}_2) = \phi_{{\cal G}}$ & $\sqrt{2 \lambda_2(\hat{L})}$   \\  \hline
\end{tabular}
\captionof{table}{Table corresponding to Figure~\ref{fig:pmeth} with 10 iterations.  Convergence to the Fiedler eigenpair is slow, yet convergence to the orthogonal complement of ${\cal X}_{noise}$ is rapid (column 2). }
\vspace{.5cm}

\label{tab:pmeth}
\end{minipage}