A matrix representation of graphs and its spectrum as a graph invariant

Abstract: We use the line digraph construction to associate an orthogonal matrix with each graph. From this orthogonal matrix, we derive two further matrices. The spectrum of each of these three matrices is considered as a graph invariant. For the first two cases, we compute the spectrum explicitly and show that it is determined by the spectrum of the adjacency matrix of the original graph. We then show by computation that the isomorphism classes of many known families of strongly regular graphs (up to 64 vertices) are characterized by the spectrum of this matrix. We conjecture that this is always the case for strongly regular graphs and we show that the conjecture is not valid for general graphs. We verify that the smallest regular graphs which are not distinguished with our method are on 14 vertices.

\section{Introduction}

Graphs are often conveniently represented using matrices, for
example, the adjacency matrix, the Laplacian matrix, \emph{etc.}
\cite{g}. Many important properties of a graph are encoded in the
eigenvalues of the matrix representation. However, eigenvalues
generally fail to separate isomorphism classes. In this paper, we
consider some matrix representations inspired by the notion of
coined quantum walks \cite{a}. Since strongly regular graphs give
rise to relatively large sets of non-isomorphic graphs which are
cospectral with respect to the commonly used matrix representations,
we use these graphs as a testing ground for the representations that
we define. Let $G$ be a graph and let $U$ be the orthogonal matrix
inducing a coined
quantum walk on $G$, where the coins are Grover matrices. For every given $k$%
, we define a digraph, $D$, with an arc $(i,j)$ if and only if $%
U_{i,j}^{k}>0 $. For $k=1$, we express the adjacency eigenvalues of
$D$ in terms of the ones of $G$. When considering strongly regular
graphs, we describe how to construct directly $D$ from $G$. For
strongly regular graphs, we conjecture that the eigenvalues of the
adjacency matrix of $D$ distinguish $G$ from its cospectral mates
for $k=3$. We verify this conjecture numerically for all strongly
regular graphs up to $64$ vertices contained in the tables of Spence
\cite{s} (the graphs srg$(16,9,4,6)$ were obtained from the tables
of McKay \cite{mc}). By providing counterexamples, we show that our
method fails to distinguish general graphs. Hopefully these
counterexamples will help understanding what graphs are cospectral
with respect to the representations described here.

\bigskip

The remainder of the paper is organized as follows. In Section 2, we
recall the definition of the orthogonal matrix $U$ and give a
formula for its eigenvalues. We define digraphs derived from powers
of $U$, and propose using these digraphs to distinguish the original
graph from its cospectral mates. In Section 3, we focus on strongly
regular graphs. We describe some structural properties of the
digraph $D$ obtained from $U^{3}$. In Section
4, we list the sets of strongly regular graphs for which the eigenvalues of $%
D$ successfully distinguish a graph from its cospectral mates. We
conclude with counterexamples involving general graphs.

\bigskip

Recently, dynamical processes based on quantum evolution have been
considered for attacking the graph isomorphism problem \cite{gu,
sjc}. In this context, Shiau \emph{et al.} \cite{sjc} gave evidence
that quantum walks alone are not successful. The setting they
describe is as follows. Let
$\{|j\rangle :1\leq j\leq n\}$ be an orthonormal basis of a Hilbert space ($%
|j\rangle $ is a ray and $\langle i|$ is the linear functional which
maps each $\left\vert j\right\rangle $ to the usual inner product,
which is
denoted by $\langle i%
{\vert}%
j\rangle $). Given a graph $G$, let us define the Hamiltonian
$H=-\sum M(G)_{a,b}c_{a}^{\dagger }c_{b}$, where the operator
$c_{a}^{\dagger }c_{b}$ is given by $\langle i|c_{a}^{\dagger
}c_{b}|j\rangle =\delta _{i,a}\delta
_{b,j}$ and $M(G)$ is the adjacency matrix of $G$. The evolution for a time $%
t$ of the initial states $\{|\psi _{j}(0)\rangle =|j\rangle :1\leq
j\leq n\}$
is governed by the Schr\"{o}dinger equation $i\frac{d|\psi _{j}(0)\rangle }{%
dt}=H|\psi _{j}(0)\rangle $. In \cite{sjc}, it was shown that the
matrix whose entries are $O_{i,j}=\langle \psi _{i}(0)|\psi
_{j}(t)\rangle $ do not help in distinguishing pairs of strongly
regular graphs with the same set of
parameters. However, by making use of some techniques described in \cite{ru}%
, it was also given an alternative method which distinguished
numerically all known strongly regular graphs up to $29$ vertices.
It would be interesting to investigate potential connections between
\cite{sjc} and the present paper.

\section{Representing graphs with orthogonal matrices}

Let $G=(V,E)$ be a simple undirected graph (that is $G$ is loopless
and without multiple edges). Let $D_{G}=(V,A)$ be the directed graph
obtained
from $G$ by replacing every edge $\{i,j\}\in E(G)$ with the pair of arcs $%
(i,j)$ and $(j,i)$. The degree of a vertex $i$ is denoted by $d(i)$.
A graph is said to be $k$-regular if every of its vertices has
degree $k$. Unless otherwise stated, in this paper we will consider
only graphs with minimum degree $3$.

\begin{definition}
Given a graph $G$, we denote by $U(G)$\ the matrix defined as follows:%
\[
\text{for all }(i,j),(k,l)\in A(D_{G}),
\]%
\[
U(G)_{(i,j),(k,l)}:=\left\{
\begin{tabular}{ll}
$\frac{2}{d(j)}-\delta_{i,l},$ & if $j=k;$ \\
$0,$ & otherwise$.$%
\end{tabular}
\right.
\]
\emph{\ }
\end{definition}

The matrix $U(G)$ induces a coined quantum walk on $G$, where the
coins are Grover matrices \cite{a}. Here are some basic observations
about $U(G)$:

\begin{itemize}
\item If $|E(G)|=m$ then $U(G)$ is $2m\times2m$ and the vertices of degree
zero do not contribute to the dimension of $U_{G}$.

\item Given $i\in V(G)$, if $d(i)=1$ and $\{i,j\}\in E(G)$, then $%
U(G)_{(j,i),(i,j)}=1$, and this is the unique non-zero entry in the
row indexed by $(j,i)$ and in the column indexed by $(i,j)$.

\item Given $j\in V(G)$, if $d(j)=2$ and $\{i,j\},\{j,k\}\in E(G)$, then $%
U(G)_{(i,j),(j,i)}=U(G)_{(k,j),(j,k)}=0$ and $%
U(G)_{(i,j),(j,k)}=U(G)_{(k,j),(j,i)}=1$. So, if $G$ is $2$-regular,
the matrix $U(G)$ has exactly a one in every row and every column
and it is then a permutation matrix.

\item Since $G$ is undirected, $U(G)_{(i,j),(j,i)}\neq0$ and $%
U(G)_{(j,i),(i,j)}\neq0$ for every $\{i,j\}\in E(G)$. However, we
have assumed that the minimum degree of $G$ is $3$, and then $U(G)$
is not
symmetric. In fact, for three distinct vertices $i,j,l\in V(G)$ such that $%
\{i,j\},\{j,l\}\in E(G)$, we have $U(G)_{(i,j),(j,l)}\neq0$, but $%
U(G)_{(j,l),(i,j)}=0$ since $i\neq l$.
\end{itemize}

The \emph{support} of an $n\times n$ matrix $M$, denoted by
$\underline{M}$, is the $n\times n$ $\left( 0,1\right) $-matrix with
elements defined as
follows:%
\[
\underline{M}_{i,j}:=\left\{
\begin{tabular}{ll}
$1$ & if $M_{i,j}\neq0;$ \\
$0$ & otherwise.%
\end{tabular}
\right.
\]

Let $M(G)$ be the adjacency matrix of a graph (digraph) $G$. The
\emph{line digraph} of a digraph $D$, denoted by
$\overrightarrow{L}D$, is the digraph
defined as follows: $V(\overrightarrow{L}D)=A(D)$ and $((i,j),(k,l))\in A(%
\overrightarrow{L}D)$ if and only if $j=k$. If $d(i)>2$ for every
$i\in V(G)$
then one can verify that $\underline{U(G)}=M(\overrightarrow{L}D_{G})$ \cite%
{si}.

\bigskip

Graphs (or digraphs) $G$ and $H$ are \emph{isomorphic} (written
$G\cong H$)
if there is a permutation matrix $P$ such that $M(G)=PM(H)P^{-1}$, where $%
M(G)$ and $M(H)$ are the adjacency matrices of the graphs $G$ and
$H$, respectively. Harary and Norman \cite{hn} proved that if $D$
and $F$ are digraphs without sources or sinks then
$\overrightarrow{L}D\cong
\overrightarrow{L}F$ if and only if $D\cong F$ ($i\in V(D)$ is a \emph{source%
} if there is no $j$ such that $(j,i)\in A(D)$; a \emph{sink }if
there is no $j$ such that $(i,j)\notin A(D)$). Since $D_{G}$ is
without sources or
sinks, it follows that $G\cong H$ if and only if $\underline{U(G)}=Q%
\underline{U(H)}Q^{-1}$, for some permutation matrix $Q$. According
to Definition 1, this fact is sufficient to ensure that $G\cong H$
if and only if $U(G)=QU(H)Q^{-1}$, for some permutation matrix $Q$.
In words, $U(G)$ \textquotedblleft faithfully\textquotedblright
\emph{\ }represents $G$.

\bigskip

The \emph{spectrum} of a matrix $M$ is the multiset of the eigenvalues of $M$%
, and is denoted by Sp$(M)=\{[\lambda_{i}]^{m_{i}}:1\leq i\leq m\}$, where $%
\lambda_{1},\lambda_{2},...,\lambda_{k}$ are the eigenvalues of $M$ and $%
m_{1},m_{2},...,m_{k}$ their respective multiplicities. Let $T(G)$
be the
matrix with $ij$-th entry%
\[
T(G)_{i,j}:=\left\{
\begin{tabular}{ll}
$\frac{1}{d(j)},$ & if $M(G)_{i,j}=1;$ \\
$0,$ & otherwise.%
\end{tabular}
\ \ \right.
\]
We can obtain Sp$(U(G))$ from Sp$(T(G))$, as shown in the next
result.

\begin{proposition}
\label{ft}Let $G$ be a graph on $n$ vertices and $m$ edges. The
matrix $U(G)$ has $2n$ eigenvalues of the form
\[
\lambda=\lambda_{T}\pm i\sqrt{1-\lambda_{T}^{2}},
\]
where $\lambda_{T}$ is an eigenvalue of the matrix $T(G)$. The remaining $%
2(m-n)$ eigenvalues of $U(G)$ are $\pm1$ with equal multiplicities.
\end{proposition}

\begin{proof}
Let $M=M(G)$, $U=(G)$ and $T=T(G)$. Since $G$ is undirected, we have $%
M_{i,j}^{2}=M_{i,j}M_{j,i}=M_{i,j}$. By Definition 1,%
\[
U_{(i,j),(k,l)}=M_{i,j}M_{k,l}\delta_{j,k}\left(
\frac{2}{d(k)}-\delta _{i,l}\right) .
\]
Let $\mathbf{u}$ be an eigenvector of $U$ with eigenvalue $\lambda$
(notice that the entries of $\mathbf{u}$ have two indices as it is
for the rows and the columns of $U$). By making use of this
equation, we can write
\begin{align*}
\lambda u_{\left( i,j\right) } & =\sum_{k,l}U_{(i,j),(k,l)}u_{\left(
k,l\right) } \\
& =2\frac{M_{i,j}}{d(j)}\sum_{l}M_{j,l}u_{\left( j,l\right)
}-M_{i,j}u_{\left( j,i\right) }.
\end{align*}
Let $\mathbf{t}$ be an eigenvector of $T$ with eigenvalue
$\lambda_{T}$. Define
\[
u_{\left( i,j\right) }=\frac{M_{i,j}t_{j}}{d(j)}-\lambda^{\ast}\frac {%
M_{i,j}t_{i}}{d(i)}.
\]
Then
\begin{align*}
& 2\frac{M_{i,j}}{d(j)}\sum_{l}M_{j,l}u_{\left( j,l\right)
}-M_{i,j}u_{\left( j,i\right) } \\
& =\frac{M_{i,j}t_{j}}{d(j)}(2\lambda_{T}-\lambda^{\ast})-M_{i,j}\frac{t_{i}%
}{d(i)}.
\end{align*}
Since $U$ is real-orthogonal, its eigenvalues have magnitude $1$. It
follows that
\[
\lambda u_{\left( i,j\right) }=\lambda\left( \frac{M_{i,j}t_{j}}{d_{j}}%
(2\lambda^{\ast}\lambda_{T}-(\lambda^{\ast})^{2})-\lambda^{\ast}M_{i,j}\frac{%
t_{i}}{d_{i}}\right) .
\]
Therefore, $\mathbf{u}$ is an eigenvector of $U$ if $2\lambda^{\ast}%
\lambda_{T}-(\lambda^{\ast})^{2}=1$ or
$\lambda+\lambda^{\ast}=2\lambda_{T}$ ($\lambda^{\ast}$ is the
complex conjugate of $\lambda$). It is easy to verify that the
remaining eigenvalues are $\pm1$.
\end{proof}

\bigskip

Two graphs $G$ and $H$ are said to be \emph{cospectral} (or \emph{isospectral%
}), with respect to a given matrix representation, if the spectra of
the matrices representing the graphs are identical (see,
\emph{e.g.}, \cite{w}).
For example, let $G$ and $H$ such that%
\[
M(G)=\left(
\begin{array}{ccccc}
0 & 1 & 1 & 0 & 0 \\
1 & 0 & 0 & 1 & 0 \\
1 & 0 & 0 & 1 & 0 \\
0 & 1 & 1 & 0 & 0 \\
0 & 0 & 0 & 0 & 0%
\end{array}
\right)
\]
and%
\[
M(H)=\left(
\begin{array}{ccccc}
0 & 0 & 0 & 0 & 1 \\
0 & 0 & 0 & 0 & 1 \\
0 & 0 & 0 & 0 & 1 \\
0 & 0 & 0 & 0 & 1 \\
1 & 1 & 1 & 1 & 0%
\end{array}
\right) .
\]
Then Sp$(M(G))=$ Sp$(M(H))$. The following corollary is a
straightforward consequence of Proposition \ref{ft}.

\begin{corollary}
Given graphs $G$ and $H$, \emph{Sp}$(U(G))=$ \emph{Sp}$(U(H))$ iff \emph{Sp}$%
(T(G))=$ \emph{Sp}$(T(H))$.
\end{corollary}

If $G\cong H$ then Sp$(M(G))=$ Sp$(M(H))$, but the converse is not
necessarily true. This is the case for the graphs $G$ and $H$
considered in the above example, since Sp$(M(G))=$
Sp$(M(H))=\{[-2]^{1},[0]^{3},[2]^{2}\}$ but, of course, $G\ncong H$.
Also, Sp$(T(G))=$ Sp$(T(H))$ does not imply
that $G\cong H$. For instance, if $G$ is a $k$-regular graph then Sp$%
(T(G))=\{[\frac{1}{k}\lambda _{i}]^{m_{i}}:1\leq i\leq k\}$, where Sp$%
(M(G))=\{[\lambda _{i}]^{m_{i}}:1\leq i\leq k\}$. So, for two
$k$-regular
graphs $G$ and $H$, Sp$(M(G))=$ Sp$(M(H))$ if and only if Sp$(T(G))=$ Sp$%
(T(H))$, or, equivalently, Sp$(U(G))=$ Sp$(U(H))$. Two non-isomorphic $k$%
-regular cospectral graphs $G$ and $H$ are then not distinguished by
the spectra of $U(G)$ and $U(H)$.

\bigskip

Now, let $M$ and $N$ be matrices such that Sp$(M)=$
Sp$(N)=\{[\lambda
_{i}]^{m_{i}}:1\leq i\leq n\}$. For any natural number $p$, Sp$(M^{p})=$ Sp$%
(N^{p})=\{[\lambda _{i}^{p}]^{m_{i}}:1\leq i\leq n\}$, but it is not
necessarily the case that Sp$(\underline{M^{p}})=$
Sp$(\underline{N^{p}})$. This is again the case of the graphs $G$
and $H$ in the above example:
\[
\underline{M(G)^{2}}=\left(
\begin{array}{ccccc}
1 & 0 & 0 & 1 & 0 \\
0 & 1 & 1 & 0 & 0 \\
0 & 1 & 1 & 0 & 0 \\
1 & 0 & 0 & 1 & 0 \\
0 & 0 & 0 & 0 & 0%
\end{array}%
\right) ;
\]%
\[
\underline{M(H)^{2}}=\left(
\begin{array}{ccccc}
1 & 1 & 1 & 1 & 0 \\
1 & 1 & 1 & 1 & 0 \\
1 & 1 & 1 & 1 & 0 \\
1 & 1 & 1 & 1 & 0 \\
0 & 0 & 0 & 0 & 1%
\end{array}%
\right) .
\]%
Sp$(\underline{M(G)^{2}})=\{[0]^{3},[2]^{2}\}$ and Sp$(\underline{M(H)^{2}}%
)=\{[0]^{3},[1],[4]\}$. The entry $M(G)_{i,j}^{p}$ equals the number
of
paths of length $p$ between vertices $i$ and $j$. Note that $%
M(G)_{1,1}^{2}=2 $ and $\underline{M(G)^{2}}_{1,1}=1$. This implies
that matrix $\underline{M(G)^{p}}$ contains generally less
information about the structure of a graph $G$, since
$\underline{M(G)^{p}}_{i,j}=1$ if and only if the number of paths of
length $p$ between vertices $i$ and $j$ is non-zero. Despite this
fact, it is legitimate to ask the following question: given graphs
$G$ and $H $ such that Sp$(M(G))=$ Sp$(M(H))$ what conditions imply
that Sp$(\underline{M(G)^{p}})\neq $ Sp$(\underline{M(H)^{p}})$, for
some $p$? If a graph $G$ with diameter $p$ is connected and
non-bipartite then $\underline{M(G)^{p}}=J $, where $J$ denotes the
all-ones matrix. As a
consequence, if two graph $G$ and $H$ have diameter $p$ then Sp$(\underline{%
M(G)^{q}})=$ Sp$(\underline{M(H)^{q}})$ for every $q\geq p$, even if Sp$%
(M(G))\neq $ Sp$(M(H))$. The spectra Sp$(\underline{M(G)^{q}})$ and Sp$(%
\underline{M(H)^{q}})$ are indeed of no use in distinguishing $G$
and $H$ if these graphs have diameter $2$.

\bigskip

Given a graph $G$, since the matrix $U(G)$ is real-orthogonal, the
support
of $U(G)^{p}$, for some $p$, is presumably different from the support of $M(%
\overrightarrow{L}D_{G})^{p}$. This is because of the contribution
of the negative entries in $U(G)$. On the basis of this observation,
we define the following matrix.

\begin{definition}
Given a graph $G$, we denote by $S^{+}(U(G)^{p})$\ the matrix
defined as
follows:\emph{\ }%
\[
\begin{tabular}{l}
$S^{+}(U(G)_{i,j}^{p}):=\left\{
\begin{tabular}{ll}
$1$ & if $U(G)_{i,j}^{p}>0;$ \\
$0$ & otherwise$.$%
\end{tabular}
\ \right. $%
\end{tabular}
\]
\end{definition}

Suppose that Sp$(M(G))=$ Sp$(M(H))$ and $G\ncong H$. What conditions
need $G$
and $H$ to satisfy in order that Sp$(S^{+}(U(G)^{p}))\neq $ Sp$%
(S^{+}(U(H)^{p}))$ for some $p\geq 2$? Strongly regular graphs seem
to provide a good testing ground to approach this question. Strongly
regular graphs have been investigated in many different contexts,
including group theory, algebraic graph theory, design of
experiments, finite geometries, error-correcting codes, \emph{etc.}
(see, \emph{e.g.}, \cite{br, c}). The fastest known algorithm for
testing the isomorphism of strongly regular
graphs on $n$ vertices was designed by Spielman \cite{sp} and runs in $%
n^{O(n^{1/3}\log n)}$ number of steps. A \emph{strongly regular
graph} with parameters $(n,d,r,s)$ (for short, a srg$(n,d,r,s)$) is
a $d$-regular graph on $n$ vertices such that any two adjacent
vertices have exactly $r$ common neighbours and any two nonadjacent
vertices have exactly $s$ common neighbours \cite{c}. Strongly
regular graphs are interesting in our context principally for the
following two reasons:

\begin{itemize}
\item For the strongly regular graphs $G$ and $H$ with identical sets of
parameters, we have Sp$(M(G))=$ Sp$(M(H))$ even if $G\ncong H$. In
particular, the adjacency matrix of a srg$(n,d,r,s)$ has exactly
three eigenvalues, $[d]^{1}>[e^{+}]^{m^{+}}\geq \lbrack
e^{-}]^{m^{-}}$, such that
$e^{+}=\frac{1}{2}(r-s+\sqrt{\Delta })$ and $e^{-}=\frac{1}{2}(r-s-\sqrt{%
\Delta })$, with $\{m^{+},m^{-}\}=\frac{1}{2}(n-1\pm \frac{2k+(n-1)(s-r)}{%
\sqrt{\Delta }})$ and $\Delta =\left( s-r\right) ^{2}+4\left(
d-s\right) $.

\item The diameter of a connected strongly regular graph is $2$ (see, \emph{%
e.g.}, \cite{c}).
\end{itemize}

\section{On the spectra of $S^{+}(U(G))$, $S^{+}(U(G)^{2})$ and $%
S^{+}(U(G)^{3})$}

In this section, we give formulas to compute the eigenvalues of
$S^{+}(U(G))$ and $S^{+}(U(G)^{2})$ from the eigenvalues of $M(G)$,
when $G$ is strongly regular. It follows that neither $S^{+}(U(G))$
nor $S^{+}(U(G)^{2})$ are of any help in distinguishing $G$. The
eigenvalues of $S^{+}(U(G)^{3})$ do not depend entirely on the
eigenvalues of $M(G)$ and it seems difficult to give a formula for
Sp$(S^{+}(U(G)^{3}))$ in terms of properties of $G$. On the other
hand we show how to construct $S^{+}(U(G)^{3})$ from $G$. We
conjecture that Sp$(S^{+}(U(G)^{3}))$ distinguishes $G$ from its
cospectral mates.

\bigskip

Given a digraph $D$, let $P(D,x)=\det (xI-M\left( D\right) )$ be the
characteristic polynomial of $M\left( D\right) $. It is known that
\cite{lz,
r}%
\[
P(\overrightarrow{L}D,x)=x^{%
{\vert}%
A\left( D\right)
{\vert}%
-%
{\vert}%
V\left( D\right)
{\vert}%
}P\left( D,x\right) .
\]%
Then
\[
\text{Sp}(\underline{U(G)})=\text{ Sp}(M(G))\cup
\{[0]^{nk-(1+m^{+}+m^{-})}\}.
\]

The following two propositions show that the spectra of $S^{+}(U(G))$ and $%
S^{+}(U(G)^{2})$ are determined by the parameters of $G$. Therefore
the spectra of these matrices do not characterize $G$ itself.

\begin{proposition}
Let $G$ be a $k$-regular graph on $n$ vertices. Let $\mathbf{e}$ be
an eigenvector of $M(G)$ with eigenvalue $\lambda_{M}$ and let
$\mathbf{u}$ be
an eigenvector of $S^{+}(U(G))$ with eigenvalue $\lambda$. Then $S^{+}(U(G))$ has $%
2n$ eigenvalues of the form
\[
\lambda=\frac{\lambda_{M}}{2}\pm i\sqrt{k-1-\lambda_{M}^{2}/4},
\]
with eigenvectors having entries
\[
u_{i,j}=M(G)_{i,j}\mathbf{e}_{j}-\frac{1}{\lambda}M(G)_{i,j}e_{i}.
\]
The remaining $n(k-2)$ eigenvalues of $S^{+}(U(G))$ take the values
$\pm1$.
\end{proposition}

\begin{proof}
Observe that
\[
S^{+}(U(G)_{(i,j),(k,l)})=(1-\delta_{i,l})M_{i,j}M_{k,l}\delta_{j,k}.
\]
We then have%
\begin{align*}
& \sum_{(x,y)}S^{+}(U(G)_{(a,b),(x,y)})u_{x,y} \\
& =M_{a,b}\sum_{y}M_{b,y}u_{b,y}-M_{a,b}u_{b,a} \\
& =M_{a,b}\sum_{y}M_{b,y}e_{y}-\frac{1}{\lambda}M_{a,b}e_{b} \\
& \sum_{y}M_{b,y}-M_{a,b}e_{a}+\frac{1}{\lambda}M_{a,b}e_{b} \\
& =M_{a,b}e_{b}(\lambda-(k-1)/\lambda)-M_{a,b}e_{a} \\
& =\lambda\left( M_{a,b}e_{b}-\frac{1}{\lambda}M_{a,b}e_{a}\right)
\end{align*}
Then these eigenvalues and eigenvectors account for $2n$ of the
eigenvalues
and eigenvectors of $S^{+}(U(G))$. It is easy to verify that the remaining $%
n(k-2)$ eigenvalues take the values $\pm1$.
\end{proof}

\begin{proposition}
Let $G$ be a $k$-regular graph on $n$ vertices. Let $\mathbf{e}$ be
an eigenvector of $M(G)$ with eigenvalue $\lambda_{M}$ and let
$\mathbf{u}$ be
an eigenvector of $S^{+}(U(G)^{2})$ with eigenvalue $\lambda$. Then $%
S^{+}(U(G)^{2})$ has $2n$ eigenvalues of the form%
\[
\lambda=\frac{\lambda_{M}^{2}}{2}+2-k\pm i\lambda_{M}\sqrt{%
k-1-\lambda_{M}^{2}/4}
\]
with eigenvectors having entries
\[
u_{i,j}=M(G)_{i,j}e_{j}-\frac{\lambda^{\ast}-2+k}{\lambda_{M}(k-1)}%
A_{i,j}e_{i}.
\]
The remaining $n(k-2)$ eigenvalues of $S^{+}(U(G)^{2})$ take the
value $2$.
\end{proposition}

\begin{proof}
Observe that
\[
S^{+}(U(G)^{2})=M_{i,j}M_{j,k}M_{k,l}(2\delta_{i,k}\delta_{j,l}+1-\delta
_{i,k}-\delta_{j,l}).
\]
We then have{\small
\begin{align*}
\lefteqn{\sum_{(x,y)}S^{+}(U(G)_{(a,b),(x,y)})u_{x,y}} \\
&
=\sum_{x,y}M_{a,b}M_{b,x}M_{x,y}(2\delta_{a,x}\delta_{b,y}+1-\delta
_{a,x}-\delta_{b,y}) \\
& \left( M_{x,y}e_{y}-\frac{\lambda_{M}(\lambda^{\ast}-2+k)}{k-1}%
M_{x,y}e_{x}\right) \\
& =M_{a,b}e_{b}(\lambda_{M}^{2}-\lambda^{\ast})-M_{a,b}e_{a} \\
& \left(
\frac{(2-k)(\lambda^{\ast}-2+k)+\lambda_{M}^{2}(k-1)}{\lambda
_{M}(k-1)}\right) \\
& =M_{a,b}e_{b}\lambda-M_{a,b}\mathbf{e}_{a} \\
& \left( \frac{(\lambda^{\ast}-2+k)(2-k+\lambda-2+k)}{\lambda_{M}(k-1)}%
\right) \\
& =\lambda u_{a,b}.
\end{align*}
}We can also write
\begin{align*}
& \sum_{(x,y)}S^{+}(U(G)_{(a,b),(x,y)})u_{(x,y)} \\
& =2M_{a,b}u_{a,b}+M_{a,b}\sum_{x,y}M_{b,x}M_{x,y}u_{x,y} \\
&
-M_{a,b}\sum_{y}M_{b,a}M_{a,y}u_{a,y}-M_{a,b}\sum_{x}M_{b,x}M_{x,b}u_{x,b}.
\end{align*}
If
\[
\begin{tabular}{lll}
$\sum_{y}M_{a,y}u_{a,y}=0$ & and & $\sum_{y}M_{x,b}u_{x,b}=0$%
\end{tabular}
\]
then
\[
\sum_{(x,y)}S^{+}(U(G)_{(a,b),(x,y)})u_{x,y}=2M_{a,b}u_{a,b}
\]
and $\mathbf{u}$ is an eigenvector with eigenvalue $2$, provided that $%
M_{a,b}=0$ and then $u_{a,b}=0$.
\end{proof}

\bigskip

The following observation allows us to construct $S^{+}(U(G)^{3})$
directly
from a strongly regular graph $G$, without the need of first constructing $%
U(G)$.

\begin{proposition}
Let $G$ be a srg$(n,k,r,s)$. Then $S^{+}(U(G)_{(i,j),(l,m)}^{3})=1$
if and only if one of the following conditions holds:

\begin{enumerate}
\item $i=m$, $j\neq l$ and
\[
s+\left( r-s\right) M(G)_{j,l}-k+\frac{k^{2}}{4}>0
\]
(which always holds if $i=m$, $j\neq l$ and $k>4$);

\item $i=l$, $m\neq j$ and $M(G)_{j,m}<\frac{2r}{k}$;

\item $i=l$ and $m=j$;

\item $i\neq l$, $m=j$ and $M(G)_{i,l}<\frac{2r}{k}$;

\item $i\neq l$, $i\neq m$, $j\neq l$, $j\neq m$ and%
\[
\frac{2}{k}\left[ s+(r-s)M(G)_{j,l}\right] >M(G)_{i,l}+M(G)_{j,m}.
\]
\end{enumerate}
\end{proposition}

\begin{proof}
Let $G$ be a srg$(n,k,r,s)$. Let $A=M(G)$ and $U=U(G)$. We define the \emph{%
amplitude} of a given path to be the product of the entries of $U$
corresponding to the transitions along the path. The entry $%
S^{+}(U(G)_{(i,j),(l,m)}^{3})$ is given by the sum of the amplitudes
of all possible paths of length three from $(i,j)$ to $(l,m)$. In
what follows we count all these paths to give conditions for
$S^{+}(U(G)_{(i,j),(l,m)}^{3})=1$. We begin by observing that $i\neq
j$ and $l\neq m$ since $G$ is a simple graph. The cases in the
following table completely describe the possible ways in which the
vertices $i,j,l,\text{ and }m$ can be identified with one
another:%
\[
\begin{tabular}{||l|l|l||}
\hline $i=m(\neq l)$ & $i=l(\neq m)$ & $i\neq m$ and $i\neq l$
\\ \hline
Case A: $j=l$ & Case C: $j=m$ & Case E: $j=m(\neq l)$ \\
Case B: $j\neq l$ & Case D: $j\neq m$ & Case F: $j=l(\neq m)$ \\
&  & Case G: $j\neq m$ and $j\neq l$ \\ \hline
\end{tabular}
\]

For each case we count the possible paths below and thus determine $%
U_{(i,j),(l,m)}^{3}$. Note that all paths of length three must be of
the form
\[
(i,j)\rightarrow(j,x)\rightarrow(x,l)\rightarrow(l,m),
\]
where $x\in\{i,m\}$ or $x$ is some vertex other than $i,j,l$ or $m$.

\textbf{Case A ($i=m$ and $j\neq l$)} There is the unique path that
takes place only on the vertices $\{i,j,l,m\}$:
\[
(i,j)\rightarrow(j,i)=(j,m)\rightarrow(m,l)\rightarrow(l,m),
\]
and has amplitude $\frac{2}{k}(\frac{2}{k}-1)^{2}$. The other paths
are of
the form%
\[
(i,j)\rightarrow(j,x)\rightarrow(x,l)\rightarrow(l,m),\;x\notin\{i,j,l,m\},
\]
which have amplitude $\frac{8}{k^{3}}$. If $\{j,l\}\in E(G)$ there
are $r-1$ such paths, if $\{j,l\}\notin E(G)$ there are $s-1$. Thus,
\[
U_{(i,j),(l,m)}^{3}=\frac{2}{k}\left\{ \left( \frac{2}{k}-1\right) ^{2}+%
\frac{4}{k^{2}}[s+(r-s)A_{j,l}-1]\right\} .
\]

\textbf{Case B ($i=m$ and $j=l$)} There is the unique path that
takes place only on the vertices $\{i,j,l,m\}$:
\[
(i,j)\rightarrow(j,i)\rightarrow(i,j)\rightarrow(j,i)=(l,m),
\]
which has amplitude $(\frac{2}{k}-1)^{3}$. There are $k$ other paths
of the form
\[
(i,j)\rightarrow(j,x)\rightarrow(x,j)\rightarrow(j,i)=(l,m),\;x\notin
\{i,j,l,m\},
\]
which have amplitude $\frac{8}{k^{3}}$. Thus,
\[
U_{(i,j),(l,m)}^{3}=\frac{2}{k}-1.
\]

\textbf{Case C ($i=l$ and $j\neq m$) }There are no paths which take
place only on the vertices $\{i,j,l,m\}$. There are paths
\[
(i,j)\rightarrow(j,x)\rightarrow(x,l)\rightarrow(l,m),\;x\notin\{i,j,l,m\}
\]
of amplitude $\frac{8}{k^{3}}$. If $\{j,m\}\in E(G)$ there are $r-1$
such paths, if $\{j,m\}\notin E(G)$ there are $r$. Thus,
\[
U_{(i,j),(l,m)}^{3}=\frac{4}{k^{2}}\left(
\frac{2r}{k}-A_{j,m}\right) .
\]

\textbf{Case D ($i=l$ and $j=m$) }There are no paths which take
place only on the vertices $\{i,j,l,m\}$. There are $r$ paths
\[
(i,j)\rightarrow(j,x)\rightarrow(x,l)\rightarrow(l,m),\;x\notin\{i,j,l,m\}
\]
with amplitude $\frac{8}{k^{3}}$. Thus,
\[
U_{(i,j),(l,m)}^{3}=\frac{8}{k^{3}}r.
\]

\textbf{Case E ($i\neq m,i\neq l$ and $j=m$) }If $\{i,l\}\notin
E(G)$ then there are no paths which take place only on the vertices
$\{i,j,l,m\}$. However, there are $r$ paths
\[
(i,j)\rightarrow(j,x)\rightarrow(x,l)\rightarrow(l,m),\;x\notin\{i,j,l,m\}
\]
with amplitude $\frac{8}{k^{3}}$. If $\{i,l\}\in E(G)$ there is the
unique path
\[
(i,j)\rightarrow(j,i)\rightarrow(i,l)\rightarrow(l,m)
\]
with amplitude $\frac{4}{k^{2}}(\frac{2}{k}-1)$. When $\{i,l\}\in
E(G)$ there are also $r-1$ paths
\[
(i,j)\rightarrow(j,x)\rightarrow(x,l)\rightarrow(l,m),\;x\notin\{i,j,l,m\}
\]
with amplitude $\frac{8}{k^{3}}$. Thus,%
\[
U_{(i,j),(l,m)}^{3}=\frac{4}{k^{2}}\left(
\frac{2r}{k}-A_{i,l}\right) .
\]

\textbf{Case F ($i\neq m,i\neq l$ and $j=l$) }There are two paths%
\[
(i,j)\rightarrow(j,i)\rightarrow(i,j)\rightarrow(j,m)=(l,m)
\]
and
\[
(i,j)\rightarrow(j,l)\rightarrow(l,j)\rightarrow(j,m)=(l,m)
\]
each of amplitude $\frac{2}{k}(\frac{2}{k}-1)^{2}$. There are also
$k-2$ paths
\[
(i,j)\rightarrow(j,x)\rightarrow(x,l)\rightarrow(l,m),\;x\notin\{i,j,l,m\}
\]
with amplitude $\frac{4}{k^{2}}(\frac{2}{k}-1)$. Thus,%
\[
U_{(i,j),(l,m)}^{3}=0.
\]

\textbf{Case G ($i\neq m,i\neq l,j\neq l$ and $j\neq l$)} We
consider the cases $\{j,l\}\in E(G)$ and $\{j,l\}\notin E(G)$
separately. Firstly, we consider the case $\{j,l\}\in E(G)$. There
are $r-A_{i,l}-A_{j,m}$ paths
\[
(i,j)\rightarrow(j,x)\rightarrow(x,l)\rightarrow(l,m),\;x\notin\{i,j,l,m\}
\]
of amplitude $\frac{8}{k^{3}}$. There are also $A_{i,l}+A_{j,m}$
paths of
amplitude $\frac{4}{k^{2}}(\frac{2}{k}-1)$. If they exist, these are%
\[
(i,j)\rightarrow(j,m)\rightarrow(m,l)\rightarrow(l,m)
\]
and%
\[
(i,j)\rightarrow(j,i)\rightarrow(i,l)\rightarrow(l,m).
\]
Now, if $\{j,l\}\notin E(G)$ the analysis is identical but with $s$
replacing $r$. Thus,
\[
U_{(i,j),(l,m)}^{3}=\frac{4}{k^{2}}\left\{ \frac{2}{k}\left[ s+(r-s)A_{j,l}%
\right] -(A_{i,l}+A_{j,m})\right\} .
\]

Notice that the conditions in the statement of the proposition are
mutually exclusive. Each case corresponds to $U_{(i,j),(l,m)}^{3}>0$
for one of the cases A-F above. The condition $1$ corresponds to the
case A, $2$ to C, $3$ to D, $4$ to E and $5$ to F. We have
$U_{(i,j),(l,m)}^{3}\leq0$ for all the other cases.
\end{proof}

\bigskip

We conclude with the following conjecture.

\begin{conjecture}
Let $G$ and $H$ be strongly regular graphs with the same set of
parameters. Then $G\cong H$ if and only if Sp$(S^{+}(U(G)^{3}))=$
Sp$(S^{+}(U(H)^{3}))$.
\end{conjecture}

If the conjecture is true then isomorphism of strongly regular
graphs can be tested in polynomial time.

\section{Computations}

The table below contains the parameters of the strongly regular
graphs for which we have tested the validity of the conjecture. We
have tested also the complements of these graphs. In all cases the
conjecture resulted to be
valid. The number of non-isomorphic srg$(n,k,r,s)$ is denoted by $N(n,k,r,s)$%
. The graphs srg$(16,9,4,6)$ where obtained from \cite{mc}; all
other graphs where obtained from \cite{s}:

\bigskip

\[
\begin{tabular}{||l|l||l|l||}
\hline $(n,k,r,s)$ & $N(n,k,r,s)$ & $(n,k,r,s)$ & $N(n,k,r,s)$
\\ \hline\hline
(16,6,2,2) & 2 & (35,18,9,9) & 227 \\
(16,9,4,6) & 2 & (36,14,4,6) & 180 \\
(25,12,5,6) & 15 & (36,15,6,6) & 32,548 \\
(26,10,3,4) & 10 & (40,12,2,4) & 28 \\
(28,12,6,4) & 4 & (45,12,3,3) & 78 \\
(29,14,6,7) & 41 & (64,18,2,6) & 167 \\ \hline
\end{tabular}%
\]

\bigskip

The method that we have described appears then to be successful for
strongly regular graphs. In general, there are cases in which the
method fails. We have verified that Sp$(S^{+}(U(G)^{3}))$
distinguishes all regular graphs up to $13$ vertices. However, we
have found $4$-regular graphs on $14$ vertices which are not
distinguished by our method. Two graphs for which the method fails
are drawn in the figure below:

\begin{figure}[ht]
\begin{center}
\includegraphics[scale=.35]{regU.eps}
\end{center}
\end{figure}

The computer programmes used for our calculations can be downloaded
from
\begin{center}
\texttt{www-users.cs.york.ac.uk/\symbol{126}wilson/qwalks.html}
\end{center}
or asked directly to the authors. These are MATLAB and C codes.

\bigskip

\noindent \textbf{Acknowledgments} We would like to thank Lalit
Jain, Chris Godsil, Toufik Mansour, Peter Cameron \ and Dennis Shasha for their
interest in this work. We would like to thank the anonymous referees
for their help in improving the content and the presentation of the
paper. One of the referees noted that there are $3$-regular graphs
on $16$ vertices that are not distinguished by the technique
described in the paper.


Kernel interpolation with continuous volume sampling

Abstract: A fundamental task in kernel methods is to pick nodes and weights, so as to approximate a given function from an RKHS by the weighted sum of kernel translates located at the nodes. This is the crux of kernel density estimation, kernel quadrature, or interpolation from discrete samples. Furthermore, RKHSs offer a convenient mathematical and computational framework. We introduce and analyse continuous volume sampling (VS), the continuous counterpart -- for choosing node locations -- of a discrete distribution introduced in (Deshpande & Vempala, 2006). Our contribution is theoretical: we prove almost optimal bounds for interpolation and quadrature under VS. While similar bounds already exist for some specific RKHSs using ad-hoc node constructions, VS offers bounds that apply to any Mercer kernel and depend on the spectrum of the associated integration operator. We emphasize that, unlike previous randomized approaches that rely on regularized leverage scores or determinantal point processes, evaluating the pdf of VS only requires pointwise evaluations of the kernel. VS is thus naturally amenable to MCMC samplers.

\section{Introduction}
\label{sec:introduction}
Kernel approximation is a recurrent task in machine learning  \citep*{HaTiFr09}[Chapter 5], signal processing \citep{Uns00} or numerical quadrature \citep{Lar72}. Expressed in its general form, we are given a reproducing kernel Hilbert space $\mathcal{F}$ (RKHS; \citealp{BeTh11}) of functions over $\mathcal{X}$, with a symmetric kernel $k: \X \times \X \rightarrow \mathbb{R}_{+}$, and an element $\mu : \X \rightarrow \mathbb{R}$ of $\mathcal{F}$. We ask for conditions on a \emph{design} $\bm{x} = (x_{1},\dots,x_N)\in \mathcal{X}^N$, and on the corresponding weights $w_1,\dots,w_N$, such that the RKHS norm
\begin{equation}
\left\Vert \mu - \sum_{i=1}^N w_{i}k(x_{i},.)\right\Vert_{\mathcal{F}}
\label{e:residual}
\end{equation}
is small. In other words, $\mu$ should be well reconstructed in $\mathcal{F}$ by the weighted design.

Measuring the error in RKHS norm has a computational advantage. Indeed, minimizing \eqref{e:residual} boils down to minimizing a quadratic form; and given a design $\bm{x}$ such that $\Det \bm{K}(\bm{x}) = \Det (k(x_i,x_j)) >0$, Equation~\eqref{e:residual} has a unique set of minimizing weights. The minimizer corresponds to the weights $\hat{\bm{w}} = \bm{K}(\bm{x})^{-1} \mu(\bm{x})$, where $\mu(\bm{x})\in\mathbb{R}^N$ contains the evaluation of $\mu$ at the $N$ design nodes $x_{i}$. Whenever the weights are chosen to be $\hat{\bm{w}}$, the sum in \eqref{e:residual} takes the same values as $\mu$ at the nodes $x_{i}$, and the optimal value of \eqref{e:residual} is thus called \emph{interpolation error}; otherwise we speak of \emph{approximation error}.
Note that when the kernel $k$ is bounded, guarantees in RKHS norm translate to guarantees in the supremum norm.

In this work, we propose and analyze the interpolation based on a random design drawn from a distribution called \emph{continuous volume sampling}, which favours designs $\bm{x}$ with a large value of $\Det \bm{K}(\bm{x})$. After introducing this new distribution, we prove non-asymptotic guarantees on the interpolation error which depend on the spectrum of the kernel $k$. Previous kernel-based randomized designs, both i.i.d. \citep{Bac17} and repulsive \citep{BeBaCh19}, can be hard to compute in practice since they require access to the Mercer decompostion of $k$. We show here that continuous volume sampling enjoys similar error bounds as well as some additional interpretable geometric properties, while having a joint density that can be evaluated as soon as one can evaluate the RKHS kernel $k$. In particular, this opens the possibility of Markov chain Monte Carlo samplers \citep{ReGh19}.

The rest of the article is organized as follows. Section~\ref{s:relatedWork} reviews kernel-based interpolation. In Section~\ref{sec:VS_DPP}, we define continuous volume sampling and relate it to projection determinantal point processes, as used by \cite{BeBaCh19}. Section~\ref{sec:main_results} contains our main results while Section~\ref{sec:steps_proof} contains sketches of all proofs with pointers to the appendices for missing details. Finally, in Section~\ref{sec:discussion}, we conclude and discuss some consequences of our results beyond kernel interpolation.

\paragraph{Notation and assumptions.}\label{paragraph:notation}
We assume that $\mathcal{X}$ is equipped with a Borel measure $\mathrm{d}\omega$, and that the support of $\mathrm{d}\omega$ is $\mathcal{X}$.
 Let $\mathbb{L}_{2}(\mathrm{d}\omega)$ be the Hilbert space of square integrable, real-valued functions on $\mathcal{X}$, with inner product $\langle \cdot, \cdot \rangle_{\mathrm{d}\omega}$, and associated norm $\|.\|_{\mathrm{d}\omega}$.

\begin{assumption}\label{hyp:integrable_diagonal}
$\displaystyle \int_{\X} k(x,x) \mathrm{d}\omega(x) < +\infty$.
\end{assumption}
Under Assumption~\ref{hyp:integrable_diagonal}, define the integral operator
\begin{equation}
\bm{\Sigma} f (\cdot) = \int_{\mathcal{X}} k(\cdot,y)f(y) \mathrm{d}\omega(y), \quad f \in \mathbb{L}_{2}(\mathrm{d}\omega).
\end{equation}
By construction, $\bm{\Sigma}$ is self-adjoint, positive semi-definite, and trace-class \citep{Sim05}.
For $m \in \Ns$, denote by $e_{m}$ the $m$-th eigenfunction of $\bm{\Sigma}$, normalized so that $\|e_{m}\|_{\mathrm{d}\omega} = 1$, and $\sigma_{m}$ the corresponding eigenvalue. Assumption~\ref{hyp:integrable_diagonal} implies that the embedding operator $I_{\mathcal{F}}: \mathcal{F} \longrightarrow \mathbb{L}_{2}(\mathrm{d}\omega)$ is compact; moreover, since $\mathrm{d}\omega$ is of full support in $\mathcal{X}$, $I_{\mathcal{F}}$ is injective \citep{StCh08}. This implies a Mercer-type decomposition of $k$,
\begin{equation}
k(x,y)= \sum\limits_{m \in \mathbb{N}^{*}}\sigma_{m}e_{m}(x)e_{m}(y),
\end{equation}
where $\mathbb{N}^{*} = \mathbb{N} \smallsetminus \{0\}$ and the convergence is pointwise \citep{StSc12}. The eigenvalues $(\sigma_m)$ are assumed to be non-increasing. Moreover, for $m \in \mathbb{N}^{*}$, we write $e_{m}^{\mathcal{F}} = \sqrt{\sigma_{m}}e_{m}$. Since $I_{\mathcal{F}}$ is injective, $(e_{m}^{\mathcal{F}})_{m \in \mathbb{N}^{*}}$ is an orthonormal basis of $\mathcal{F}$ \citep{StSc12}. Unless explicitly stated, we assume that $\mathcal{F}$ is dense in $\mathbb{L}_{2}(\mathrm{d}\omega)$, so that $(e_{m})_{m \in \mathbb{N}^{*}}$ is an orthonormal basis of $\mathbb{L}_{2}(\mathrm{d}\omega)$. For more intuition, under these assumptions, $f \in \mathcal{F}$ if and only if $\sum_m \sigma_{m}^{-1} \langle f,e_{m} \rangle_{\mathbb{L}_{2}(\mathrm{d} \omega)}^{2}$ converges.
For $\bm{x} \in \X^{N}$, we define $\bm{K}(\bm{x}) := k(x_{i},x_{j})_{i,j \in [N]}$. If $\Det \bm{K}(\bm{x})>0$,  the subspace $\mathcal{T}(\bm{x}) = \Span k(x_{i},.)_{i \in [N]}$ is of dimension $N$; we denote by $\Pi_{\mathcal{T}(\bm{x})}$ the $\langle.,. \rangle_{\F}$-orthogonal projection on $\mathcal{T}(\bm{x})$.
Finally, for $N \in \Ns$, we will often sum over the sets
\begin{align}
	\:\mathcal{U}_N^m &=  \{ U \subset \Ns, |U| = N,\: m \notin U \},\\
  \:\mathcal{U}_N &=  \{ U \subset \Ns, |U| = N \}.
\end{align}
Finally, define the approximation error
\begin{equation}\label{def:E}
	\mathcal{E}(\mu;\bm{x},\bm{w}) = \|\mu - \sum\limits_{i \in [N]} w_{i} k(x_{i},.)\|_{\F},
\end{equation}
where $[N]=\{1,\dots,N\}$. If $\Det \bm{K}(\bm{x}) >0$, let $\hat{\bm{w}} = \bm{K}(\bm{x})^{-1} \mu(\bm{x})$ and define the interpolation error
\begin{align}\label{def:Ei}
	\mathcal{E}(\mu;\bm{x}) &= \|\mu - \sum\limits_{i \in [N]} \hat{w} _{i} k(x_{i},.)\|_{\F}\\
  & = \|\mu - \Pi_{\mathcal{T}(\bm{x})} \mu\|_{\F}.
\end{align}

\section{Related work}\label{s:relatedWork}
This section reviews some results on kernel interpolation to better situate our contributions. The literature on this topic is prolific and cannot be covered in details here. In particular, we start by reviewing results on optimal kernel quadrature, a particular case of kernel interpolation.

\subsection{Interpolation for optimal kernel quadrature}\label{sec:review_optimal_kernel_quadrature}

Given $g \in \Ltwo$, kernel quadrature deals with approximating the integrals
\begin{equation}
\:\int_{\X} fg \,\mathrm{d}\omega \approx \sum\limits_{i \in [N]} w_{i}f(x_{i}),\quad f\in\F,
\end{equation}
where the weights $w_{i}$ do not depend on $f$.
In principle, it is easy to control the error uniformly in $f\in\mathcal{F}$. Indeed,
\begin{equation}\label{eq:upper_bound_integration_error}
\left| \int_{\X} fg \,\mathrm{d}\omega - \sum_{i \in [N]} w_{i}f(x_{i})\right| \leq \|f\|_{\F} \, \mathcal{E}(\mu_{g};\bm{x},\bm{w}),
\end{equation}
where
$ \displaystyle
\mu_{g} = \int_{\X}g(x)k(x,.) \mathrm{d}\omega(x){=\bm{\Sigma} g}
$
is the so-called \emph{embedding}\footnote{In case $g =1$, $\mu_{g}$ is classically called the mean-element of the measure $\mathrm{d} \omega$ \citep{SmGrSoSc07}.} of $g$ in the RKHS $\F$.

An upper bound on the approximation error of $\mu_{g}$ implies an upper bound on the integration error that is uniform over any bounded subset of $\F$. This observation sparked intense research on the kernel approximation of embeddings $\mu_{g}$. Among kernel approximation results, we pay a particular attention to interpolation, i.e., approximation with optimal weights. In the sequel, we call \emph{optimal kernel quadrature} the quadrature based on optimal weights $\hat{\bm{w}}$ minimizing \eqref{e:residual} for a given set of nodes.

\cite{Boj81} proved that, for $g=1$, the interpolation of $\mu_{g}$ using the uniform grid over $\X = [0,1]$ has an error in $\mathcal{O}(N^{-2s})$ if $\F$ is the periodic Sobolev space of order $s$, and that any set of nodes leads to that rate at least. A similar rate was proved for $g$ not constant \citep{NoUlWo15} even though it is only asymptotically optimal in that case.

In the quasi-Monte Carlo (QMC) literature, several designs were investigated for $\X = [0,1]^{d}$, $g = 1$ and $\F$ that may not even be a Hilbert space; see \citep{DiPi10}. In this context, the term \emph{QMC quadrature rule} means a low discrepancy sequence, loosely speaking a ``well-spread" set of nodes, along with uniform weights $w_{i} \equiv 1/N$. If $\F$ is a Korobov space of order $s \geq 1$, the Halton sequence of nodes \citep{Hal64} leads to $\mathcal{E}(\mu_1; \bm{x}, (1/N))^2$ in $\mathcal{O}(\log(N)^{2d} N^{-2})$ and higher-order digital nets converge faster as $\mathcal{O}(\log(N)^{2sd} N^{-2s})$ \citep{DiPi14}[Theorem 5].

These rates are naturally inherited if the uniform weights are replaced by the respective optimal weights $\hat{\bm{w}}$, as observed by \cite{BOGOS2019}. In particular, \cite{BOGOS2019} emphasize that the bound for higher-order digital nets attains the optimal rate in this RKHS.
For optimal kernel quadrature based on Halton sequences, this inheritance argument does not explain the fast $\mathcal{O}(\log(N)^{2sd} N^{-2s})$ rates observed empirically by \cite{Oett17}.

Beside the hypercube, optimal kernel quadrature has been considered on the hypersphere equipped with the uniform measure \citep{EhGrCh19}, or on $\mathbb{R}^{d}$ equipped with the Gaussian measure \citep{KaSa19}. In these works, the design construction is adhoc for the space $\X$ and
$g$ is usually assumed to be constant. Another approach is offered by optimisation algorithms that we review in Section~\ref{sec:sequential_algos}. Before that, we clarify the subtle difference between optimal kernel quadrature and kernel interpolation.

\subsection{Kernel interpolation beyond embeddings}\label{sec:review_kernel_interpolation_beyond_mu}

Besides the approximation of the embeddings $\mu_{g}$ discussed in Section~\ref{sec:review_optimal_kernel_quadrature}, theoretical guarantees for the kernel interpolation of a general $\mu\in\F$ are sought \emph{per se}. The Shannon reconstruction formula for bandlimited signals \citep{Sha48} is implicitly an interpolation by the sinc kernel.
The RKHS approach for sampling in signal processing was introduced in \citep{Yao67} for the Hilbert space of bandlimited signals;
see also \citep{NaWa91} for generalizations.
Remarkably, in those RKHSs, every $\mu \in \F$ is an embedding $\mu_{g}$ for some $g \in \Ltwo$: $k$ is a projection kernel of infinite rank. In general, for a trace-class kernel, the subspace spanned by the embeddings $\mu_{g}$ is strictly included in $\F$. More precisely, every $\mu_{g}$ satisfies
\begin{equation}
\|\bm{\Sigma}^{-1/2} \mu_{g}\|_{\F} = \|\bm{\Sigma}^{1/2} g\|_{\F} = \|g\|_{\Ltwo} < +\infty.
\end{equation}
This condition is more restrictive than what is required for a generic $\mu$ to belong to $\F$, i.e., $\|\mu\|_{\F}< +\infty$, so that kernel interpolation is more general than optimal kernel quadrature. The proposed approach will permit to deal with any $\mu\in\F$.

Scattered data approximation \citep{Wend04} is another field where quantitative error bounds for kernel interpolation on $\X\subset\mathbb{R}^d$ are investigated; see \citep{ScWe06} for a modern review. In a few words, these bounds typically depend on quantities such as the \emph{fill-in distance} $\varphi(\bm{x}) = \sup_{y \in \X} \min_{i \in [N]}\|y-x_{i}\|_{2}$,
 so that the interpolation error converges to zero  as $N\rightarrow \infty$ if $\varphi(\bm{x})$ goes to zero.
 Any node set can be considered, as long as $\varphi(\bm{x})$ is small.
Using these techniques, \cite{OaGi16} proposed another application of kernel interpolation: the construction of functional control variates in Monte Carlo integration.
Finally, note that the application of these techniques is restricted to compact domains: the fill-in distance is infinite if $\X$ is not compact, even for ``well-spread" sets of nodes.

\subsection{Optimization algorithms}\label{sec:sequential_algos}

Optimization approaches offer a variety of algorithms for the design of the interpolation nodes.
\cite{DeM03} and \cite{DeScWe05} proposed greedily maximizing the so-called   \emph{power function}
\begin{equation}
p(x;\bm{x}) = \left[k(x,x) - k_{\bm{x}}(x)^{\Tran} \bm{K}(\bm{x})^{-1} k_{\bm{x}}(x)\right]^{1/2},
\label{e:power}
\end{equation}
where $k_{\bm{x}}(x) = (k(x,x_{i}))_{i \in [N]}$.
This algorithm leads to an interpolation error that goes to zero with $N$ for a kernel of class $\mathcal{C}^{2}$ \citep{DeScWe05}. Later, \cite{SaHa17} proved better convergence rates for smoother kernels. Again, these results assume that the domain $\X$ is compact.
Other greedy algorithms were proposed in the context of Bayesian quadrature (BQ) such as Sequential BQ \citep{HuDu12}, or Frank-Wolfe BQ \citep{BrOaGiOs15}.
These algorithms sequentially minimize $\mathcal{E}(\mu_{g}; \bm{x})$, for a fixed $g \in \Ltwo$. The nodes are thus adapted to one particular $\mu_{g}$ by construction.
In general, each step of these greedy algorithms requires to solve a non-convex problem with many local minima \citep{Oett17}[Chapter 5]. In practice, costly approximations must be employed such as local search in a random grid \citep{LaLiBa15}

An alternative approach, that is very related to our contribution and has raised a lot of recent interest, is to observe that the squared power function \eqref{e:power} can be upper bounded by the inverse of $\Det \bm{K}(\bm{x})$ \citep{Sch05,Tan19}.
 Designs that maximize $\Det \bm{K}(\bm{x})$ are called \emph{Fekete points}; see e.g. \citep{BoMa02,BoDe11}.
\cite{Tan19} proposed to approximate $\Det \bm{K}(\bm{x})$ using the Mercer decomposition of $k$, followed by a rounding of the solution of a $D$-experimental design problem, yet without a theoretical analysis of the interpolation error.
\cite{KaSaTa19} proved that for the uni-dimensional Gaussian kernel, the approximate objective function of \citep{Tan19} is actually convex. Moreover, \cite{KaSaTa19} analyze their interpolation error; see also Section~\ref{sec:main_theorems_2}.
Finally, we emphasize that these algorithms require the knowledge of a Mercer-type decomposition of $k$ so that they cannot be implemented for any kernel; moreover, the approximate objective function may be non-convex in general.

\subsection{Random designs}

In this section, we survey random node designs with uniform-in-$g$ approximation guarantees for the embeddings $\mu_{g}$ in the RKHS norm.
\cite{Bac17} studied the quadrature resulting from sampling i.i.d. nodes $(x_j)$ from some proposal distribution $q$. He proved that when the proposal is chosen to be
\begin{equation}
	q_{\lambda}^*(x) \propto \sum\limits_{m \in \Ns} \frac{\sigma_{m}}{\sigma_{m}+\lambda} e_{m}(x)^{2},
\label{e:proposalBach}
\end{equation}
with $\lambda >0$,
and the number of points $N$ satisfies $N \geq 5 d_\lambda \log(16 d_\lambda / \delta)$ with $d_\lambda = \Tr \bm{\Sigma}(\bm{\Sigma} + \lambda \bm{I})^{-1}$, then with probability larger than $1-\delta$,
\begin{equation}\label{eq:Bach_bound}
\sup\limits_{\|g\|_{\mathrm{d}\omega} \leq 1} \inf\limits_{ \|\bm{w}\|^{2}\leq \frac{4}{N}} \Big\| \mu_{g} - \sum\limits_{j \in [N]} \frac{w_{j}}{q_\lambda(x_{j})^{1/2}} k(x_{j},.)\Big\|_{\mathcal{F}}^{2} \leq 4\lambda .
\end{equation}

The bound in \eqref{eq:Bach_bound} gives a control on the approximation error of $\mu_{g}$ by the subspace spanned by the $k(x_{j},.)$, and this control is uniform over $g$ in the unit ball of $\mathbb{L}_{2}(\mathrm{d}\omega)$.
Note that for a fixed value of $\lambda$, the upper bound in \eqref{eq:Bach_bound} guarantees that the approximation error is smaller than $4\lambda$. It does not however guarantee that the error goes to zero as $N$ increases since it appears that $\lambda$ should decrease as $N$ increases.
This coupling of $N$ and $\lambda$ combined with the condition $N \geq d_{\lambda} \log d_{\lambda}$ makes it intricate to derive a convergence rate from \eqref{eq:Bach_bound}.
 Moreover, the optimal density $q_{\lambda}^*$ is only implicitly available in general through the limit in \eqref{e:proposalBach}, which makes sampling and pointwise evaluation difficult in practice.

\cite{BeBaCh19} proposed a related kernel-based quadrature, but using nodes sampled from a repulsive joint distribution called a \emph{projection determinantal point process} (DPP); see \citep{HoKrPeVi06} and our Section~\ref{sec:VS_DPP}. In particular, the repulsion is characterized by the first eigenfunctions $(e_{n})_{n \in [N]}$ of the integration operator $\bm{\Sigma}$. The weights $\hat{\bm{w}}$ are chosen again by minimizing the residual error \eqref{e:residual},
which gives the uniform bound
\begin{equation}\label{eq:proj_DPP_bound}
\EX \sup\limits_{\|g\|_{\mathrm{d}\omega} \leq 1} \mathcal{E}(\mu_{g}; \bm{x})^{2} \leq 2(N^{2} r_{N} + o(N^{2} r_{N})) ,
\end{equation}
where $r_{N} = \sum_{m \geq N+1} \sigma_{m}$. This result can be improved by further restricting $g$ to be an eigenfunction of $\bm{\Sigma}$, leading to
\begin{equation}\label{eq:proj_DPP_bound_eigen}
\EX \sup\limits_{g \in \{e_{n};~ n\geq 1\}} \mathcal{E}(\mu_{g}; \bm{x})^{2} \leq 2(N r_{N} + o(N r_{N})).
\end{equation}

Now for smooth kernels, such as the Gaussian kernel or the Sobolev kernel with a large regularity parameter, the upper bounds in \eqref{eq:proj_DPP_bound} and \eqref{eq:proj_DPP_bound_eigen} do converge to $0$ as $N$ goes to $+\infty$. Furthermore, sampling from the recommended projection DPP can be implemented easily, although it still requires the knowledge of the Mercer decomposition of $k$, unlike the method that we introduce here in Section~\ref{sec:VS_DPP}.

Since the bounds in \eqref{eq:proj_DPP_bound} and \eqref{eq:proj_DPP_bound_eigen} are uniform-in-$g$, they also concern interpolation. One downside of the analysis in \citep{BeBaCh19} is that these upper bounds are rather pessimistic: experimental results suggest faster rates in $\mathcal{O}(\sigma_{N})$. If one could prove these rates, then kernel quadrature or interpolation using DPPs would reach known lower bounds, which we now quickly survey.

\subsection{Lower bounds}\label{sec:lower_bounds}
When investigating upper bounds for kernel interpolation errors, it is useful to remember existing lower bounds, so as to evaluate the tightness of one's results. In particular, $N$-widths theory \citep{Pin12} implies lower bounds for kernel interpolation errors, which once again show the importance of the spectrum of $\bm{\Sigma}$.

 The $N$-width of $\mathcal{S} = \{ \mu_{g} = \bm{\bm{\Sigma}}g, \: \|g\|_{\Ltwo} \leq 1\}$ with respect to the couple $(\Ltwo, \F)$ \citep[Chapter 1.7]{Pin12} is defined as the square root of
\begin{equation}
	d_{N}(\mathcal{S})^{2}  = \inf\limits_{\substack{Y \subset \F\\ \mathrm{dim}Y = N}}\, \sup\limits_{\|g\|_{\mathrm{d}\omega} \leq 1}\, \inf\limits_{y \in Y}\, \|\bm{\Sigma} g -y \|_{\F}^{2}.  \\
\end{equation}
In interpolation, we do use a subspace $Y \subset \F$ spanned by $N$ independent functions $k(x_{i},.)$, so that
\begin{equation}
\sup\limits_{\|g\|_{\mathrm{d}\omega} \leq 1} \mathcal{E}(\bm{\Sigma} g;\bm{x})^{2} \geq d_{N}(\mathcal{S})^{2}.
\end{equation}
Applying \citep[Theorem 2.2, Chapter 4]{Pin12} to the adjoint of the embedding operator $I_{\F}$ \cite{StSc12}[Lemma 2.2], it comes
$d_{N}(\mathcal{S})^{2}~=~\sigma_{N+1}$.
One may object that some QMC sequences seem to breach this lower bound. For example, in the Korobov space $(d = 2, s \geq 1)$, $\sigma_{N+1} = \mathcal{O}(\log(N)^{2s} N^{-2s})$ \citep{Bac17}, while the interpolation of $\mu_{g}$ with $g = 1$ using a Fibonacci lattice leads to an error in $\mathcal{O}(\log (N) N^{-2s}) = o(\sigma_{N+1})$ \citep{BiTeYu12}[Theorem~4]. But this is the rate for one particular $\mu_{g}$, and it cannot be achieved uniformly in $g$.

\section{Volume sampling and DPPs}\label{sec:VS_DPP}
In this section, we introduce a repulsive distribution that we call \emph{continuous volume sampling} (VS) and compare it to projection determinantal point processes (DPPs; \citep{HoKrPeVi06}). Both continuous VS and projection DPPs are parametrized using a reference measure $\mathrm{d} \omega$ and a repulsion kernel $\KDPP : \X \times \X \rightarrow \mathbb{R}_{+}$.

\subsection{Continuous volume sampling}
\begin{definition}[Continuous volume sampling]\label{def:VS}
Let $N \in \mathbb{N}^{*}$ and $\bm{x} = \{x_{1}, \dots, x_{N}\}\subset \X$. We say that $\bm{x}$ follows the volume sampling distribution
 if $(x_{1}, \dots ,x_{N})$ is a random variable of $\X ^{N}$, the law of which is absolutely continuous with respect to $\otimes_{i \in [N]} \mathrm{d}\omega$, and the density writes
\begin{equation}
  \label{e:CVS_density}
f_{\VS}(x_{1}, \dots ,x_{N}) \propto \Det \bm{K}(\bm{x}).
\end{equation}
\end{definition}

Two remarks are in order. First, under Assumption~\ref{hyp:integrable_diagonal}, the density $f_{\VS}$ in  \eqref{e:CVS_density} indeed integrates to 1. Indeed, Hadamard's inequality yields

\begin{align}
\int_{\X ^{N}} \Det \bm{K}(\bm{x}) \MuTen & \leq \int_{\X ^{N}} \prod\limits_{i \in [N]} k(x_{i},x_{i}) \otimes\mathrm{d}\omega(x_{i}) \nonumber \\
& = \left(\int_{\X}k(x,x) \mathrm{d}\omega(x) \right)^{N}  < +\infty.
\end{align}

Second, the determinant in \eqref{e:CVS_density} is invariant to permutations, so that continuous volume sampling can indeed be seen as defining a random set $\bm{x} = \{x_1,\dots,x_N\}$.

In the following, we denote, for any symmetric and continuous kernel $\tilde{k}$ satisfying Assumption~\ref{hyp:integrable_diagonal},
\begin{equation}
Z_{N}(\tilde{k}) := \int_{\X^{N}} \Det \bm{\tilde{K}}(\bm{x}) \otimes\mathrm{d}\omega(x_{i}).
\end{equation}

\subsection{Continuous volume sampling as a mixture of DPPs}

Definition~\ref{def:VS} could be mistaken with the definition of a determinantal point process (DPP; \citealp{Mac75}). However, the cardinal of a DPP sample is a sum of Bernoulli random variables \citep{HoKrPeVi06}, while volume sampling is supported on subsets of $\X$ with cardinality exactly equal to $N$. This property is convenient for approximation tasks where the number of nodes $N$ is fixed. While it is not a DPP, volume sampling is actually a mixture of DPPs.

\begin{proposition} \label{prop:VS_decomposition}
For $U \subset \mathbb{N}^{*}$ define the projection kernel
\begin{equation}
\mathfrak{K}_{U}(x,y) = \sum\limits_{u \in U} e_{u}(x)e_{u}(y).
\end{equation}
For $N \in \mathbb{N}^{*}$, we have
\begin{equation}
  \label{e:mixture}
f_{\VS}(x_{1}, \dots, x_{N}) \propto \sum\limits_{ U \in \:\mathcal{U}_N} \prod\limits_{u \in U} \sigma_{u} \Det (\mathfrak{K}_{U}(x_{i},x_{j}))_{(i,j)},
\end{equation}
and the normalization constant is equal to
\begin{equation}\label{eq:normalization_constant_VS}
Z_{N}(k) = \mathrm N! \sum\limits_{ U \in \mathcal{U}_{N}} \prod\limits_{u \in U} \sigma_{u}.
\end{equation}
\end{proposition}
The proof of this proposition is given in Appendix~\ref{sec:proof_VS_decomposition}. Observe that
for every $U \subset \:\mathcal{U}_N$,
\begin{equation}
(x_{1}, \dots , x_{N}) \mapsto \frac{1}{N!} \Det (\mathfrak{K}_{U}(x_{i},x_{j}))_{(i,j) \in [N]\times [N] },
\end{equation}
defines a well-normalized probability distribution on $\X^{N}$, called the {\em projection DPP} associated to the marginal kernel $\mathfrak{K}_{U}$  \citep{HoKrPeVi06}. Among all DPPs, only projection DPPs have a deterministic cardinality, equal to the rank of $\mathfrak{K}_{U}$ \citep{HoKrPeVi06}. Interestingly, the largest weight in the mixture \eqref{e:mixture} corresponds to the projection DPP of marginal kernel $\mathfrak{K}_{[N]}$ proposed in \citep{BeBaCh19} for kernel quadrature. The following lemma gives an upper bound on this weight $\delta_N$ using the eigenvalues of $\bm{\Sigma}$.
\begin{lemma}\label{lemma:projection_DPP_weight}
For $N \in \Ns$, define
\begin{equation}
\delta_{N} = \prod\limits_{n \in [N]} \sigma_{n} \bigg/\sum\limits_{ U \in \: \:\mathcal{U}_N} \prod\limits_{u \in U} \sigma_{u}.
\end{equation}
Then for all $N \in \Ns$, $\displaystyle \delta_{N} \leq \sigma_{N} / r_{N}$.
\end{lemma}
In particular, if the spectrum of $k$ decreases polynomially, then $\delta_{N} = \mathcal{O}(1/N)$, so that as $N$ grows, volume sampling becomes more different from the projection DPP of \cite{BeBaCh19}. In contrast, if the spectrum decays exponentially, then $\delta_{N} = \mathcal{O}(1)$.

\subsection{Sampling algorithms}

A projection DPP can be sampled exactly as long as one can evaluate the corresponding projection kernel $\KDPP$ \citep{HoKrPeVi06}. For kernel quadrature \citep{BeBaCh19}, evaluating $\KDPP$ requires the knowledge of the Mercer decomposition of the RKHS kernel $k$. The algorithm of \cite{HoKrPeVi06} implements the chain rule for projection DPPs, and each conditional is sampled using rejection sampling; see \cite{GaBaVa19} for recommendations on proposals. This suggests using the mixture in Proposition~\ref{prop:VS_decomposition} to sample from the volume sampling distribution.
Again, such an algorithm requires explicit knowledge of the Mercer decomposition of the kernel or at least a decomposition onto an orthonormal basis of $\F$ as in \citep{KaSaTa19}. This is a strong requirement that is undesirable in practice.

The fact that the joint pdf \eqref{e:CVS_density} only requires evaluating $k$ pointwise suggests that volume sampling is \emph{fully kernelized}, in the sense that a sampling algorithm should be able to bypass the need for a kernel decomposition, thus making the method very widely applicable.
One could proceed by rejection sampling. Yet the acceptance ratio would likely scale poorly with $N$.  A workaround would be to use an MCMC sampler similar to what was proposed in \citep{ReGh19}. We leave investigating the efficiency of such an MCMC approach to volume sampling to future work.

\section{Main results}\label{sec:main_results}
In this section, we give a theoretical analysis of kernel interpolation on nodes that follow the continuous volume sampling distribution. We state our main result in Section~\ref{sec:main_theorems}, an uniform-in-$g$ upper bound of $\EX_{\VS} \|\mu_{g} - \Pi_{\mathcal{T}(\bm{x})} \mu_{g}\|_{\F}^{2}$. We give an upper bound for a general $\mu \in \F$ in Section~\ref{sec:main_theorems_2}.

\subsection{The interpolation error for embeddings $\mu_{g}$}\label{sec:main_theorems}

The main theorem of this article decomposes the expected error for an embedding $\mu_g$ in terms of the expected errors $\epsilon_m$ for eigenfunctions of the kernel.
\begin{theorem}\label{thm:main_result_1}
Let $\displaystyle g = \sum\limits_{m\in \Ns} g_m e_m$ satisfy $\| g\|_{\mathrm{d}\omega} \leq 1$.  Then under Assumption~\ref{hyp:integrable_diagonal},
\begin{equation}\label{eq:main_result_EX_VS_err_mu}
\EX_{\VS} \|\mu_{g} - \Pi_{\mathcal{T}(\bm{x})} \mu_{g}\|_{\F}^{2} = \sum\limits_{m \in \mathbb{N}^{*}} g_{m}^{2} \epsilon_{m},
\end{equation}
where $\epsilon_{m} = \sigma_{m} \left(\sum\limits_{ U \in \: \:\mathcal{U}_N} \prod\limits_{u \in U} \sigma_{u} \right)^{-1}  \sum\limits_{  U \in \: \:\mathcal{U}_N^m} \prod\limits_{u \in U} \sigma_{u}$.
In particular, the sequence $(\epsilon_m)_{m \in \Ns}$ is non-increasing and
\begin{equation}\label{eq:upper_bound_sup_epsilon}
\sup_{\| g\|_{\mathrm{d}\omega} \leq 1} \EX_{\VS} \|\mu_{g} - \Pi_{\mathcal{T}(\bm{x})} \mu_{g}\|_{\F}^{2} \leq \sup\limits_{m \in \mathbb{N}^{*}} \epsilon_{m} = \epsilon_1.
\end{equation}
Moreover,
\begin{equation}\label{eq:ineq_r_N}
\epsilon_{1} \leq \sigma_{N} \left(1+ \beta_{N}\right),
\end{equation}
where $\displaystyle \beta_{N} = \min_{M \in [2:N]} \left[(N-M+1)\sigma_N\right]^{-1} \sum_{m \geq M} \sigma_m$.
\end{theorem}

In other words, under continuous volume sampling, $\epsilon_{1}$ is a uniform upper bound on the expected squared interpolation error of \emph{any} embedding $\mu_{g}$ such that $\|g\|_{\mathrm{d}\omega} \leq 1$. We shall see in Section~\ref{sec:decomposition_error} that $\epsilon_m = \EX_{\VS} \|\mu_{e_{m}} - \Pi_{\mathcal{T}(\bm{x})} \mu_{e_{m}}\|_{\F}^{2}$.

Now, for $N_{0} \in \Ns$, a simple counting argument yields, for $m \geq N_{0}$, $\epsilon_{m} \leq \sigma_{N_{0}}$. Actually, for $m \geq N_{0}$, $\|\mu_{e_{m}}\|_{\F}^{2} \leq \sigma_{N_{0}}$, independently of the nodes.

Inequality~\eqref{eq:ineq_r_N} is less trivial and makes continuous volume sampling distribution worth of interest: the upper bound goes to $0$ as $N \rightarrow +\infty$, below the initial error $\sigma_{N_0}$.
 Moreover, the convergence rate is $\mathcal{O}(\sigma_{N})$, matching the lower bound of Section~\ref{sec:lower_bounds} if the sequence $(\beta_{N})_{N \in \Ns}$ is bounded. In the following proposition, we prove that it is the case as soon as the spectrum decreases polynomially (e.g., Sobolev spaces of finite smoothness) or exponentially (e.g., the Gaussian kernel).

\begin{proposition}\label{prop:constant_bound}
If $\sigma_{m} = m^{-2s}$ with $s >1/2$ then
\begin{equation}
\forall N \in \Ns, \: \beta_{N} \leq \left(1+\frac{1}{2s-1}\right)\left(1+\frac{1}{2s-1}\right)^{2s-1}.
\end{equation}
If $\sigma_{m} = \alpha^{m}$, with $\alpha \in [0,1[$, then
\begin{equation}
\forall N \in \Ns, \: \beta_{N} \leq \frac{\alpha}{1-\alpha}.
\end{equation}
\end{proposition}
In both cases, the proof uses the fact that
\begin{equation}
\beta_{N} \leq [(N-M_{N}+1)\sigma_N]^{-1} \sum_{m \geq M_{N}} \sigma_m,
\end{equation}
for a well designed sequence $M_{N}$. For example, if $\sigma_{m}~=~m^{-2s}$, we take $M_{N} = \lceil{N/c \rceil}$ with $c >1$; if $\sigma_{m} = \alpha^{m}$ we take $M_N = N$. We give a detailed proof in the appendices.

 For a general kernel, if an asymptotic equivalent of $\sigma_{N}$ is known \citep{Wid63,Wid64}, it should be possible to give an explicit construction of $M_N$. Indeed,
\begin{equation}
 \beta_{N} \leq \frac{\sigma_{M_{N}}}{\sigma_{N}} + [{(N-M_{N}+1)\sigma_{N}}]^{-1} \sum\limits_{m \geq N+1}\sigma_{m},
\end{equation}
and $M_{N}$ should be chosen to control both terms in the RHS.
Figure~\ref{fig:the_only_one_for_the_moment} illustrates the upper bound of Theorem~\ref{thm:main_result_1} and the constant of Proposition~\ref{prop:constant_bound} in case of the periodic Sobolev space of order $s=3$. We observe that $\EX_{\VS} \mathcal{E}(\mu_{e_m};\bm{x})^{2}$ respects the upper bound: it starts from the initial error level $\sigma_m$ and decreases according to the upper bound for $N \geq m$.

\subsection{The interpolation error of any element of $\F$}\label{sec:main_theorems_2}
Theorem~\ref{thm:main_result_1} dealt with the interpolation of an embedding $\mu_{g}$ of some function $g\in\Ltwo$. We now give a bound on the interpolation error for any $\mu\in\F$. We need the following assumption, which is relatively weak; see Proposition~\ref{prop:constant_bound} and the discussion that follows.

\begin{assumption}\label{hyp:beta_N_bounded}
	There exists $B >0$ such that $\beta_{N} \leq B$.
\end{assumption}

\begin{theorem}\label{thm:slow_rates}
Let $\mu \in \F$.
Assume that $\|\bm{\Sigma}^{-r}\mu \|_{\F} < +\infty$ for some $r \in [0,1/2]$. Then, under Assumption~\ref{hyp:beta_N_bounded},
\begin{equation}
\EX_{\VS} \mathcal{E}(\mu;\bm{x})^{2} \leq (2+B) \sigma_{N}^{2r} \|\bm{\Sigma}^{-r} \mu\|_{\F}^{2} = \mathcal{O}(\sigma_{N}^{2r}).
\end{equation}
\end{theorem}
In other words, the expected interpolation error depends on the smoothness parameter $r$. For $r =1/2$, we exactly recover the rate of Theorem~\ref{thm:main_result_1}. In contrast, for $r<1/2$, the rate $\mathcal{O}(\sigma_{N}^{2r})$ is slower. For $r=0$, our bound is constant with $N$. Note that assuming more smoothness $(r>1/2)$ does not seem to improve the rate $\mathcal{O}(\sigma_{N})$.

Let us comment on this bound in two classical cases. First, consider the uni-dimensional Sobolev space of order $s$. Assumption~\ref{hyp:beta_N_bounded} is satisfied by Proposition~\ref{prop:constant_bound} and the squared error scales as $\mathcal{O}(N^{-4sr})$. Moreover, for this family of RKHSs, $\|\bm{\Sigma}^{-r}.\|_{\F}$ can be seen as the norm in the Sobolev space of order $(2r+1)s$, and we recover a result in \citep{ScWe06}[Theorem 7.8] for quasi-uniform designs. By using the norm in the RKHS $\F$ of rougher functions, we upper bound the interpolation error of $\mu$ belonging to the smoother RKHS $\bm{\Sigma}^{r} \F$. Second, we emphasize again that our result is agnostic to the choice of the kernel, as long as Assumption~\ref{hyp:beta_N_bounded} holds. In particular, Theorem~\ref{thm:slow_rates} applies to the Gaussian kernel: the rate is slower $\mathcal{O}(\sigma_{N}^{2r})$ yet still exponential. Finally, recall that for $f \in \F$
\begin{equation}
|f(x)|^{2} = |\langle f, k(x,.)\rangle_{\F}|^{2} \leq\|f\|_{\F}^{2} k(x,x),
\end{equation}
so that, bounds on the RKHS norm imply bounds on the uniform norm if the kernel $k$ is bounded. Therefore, for $r \in [0,1/2]$, our result improves on the rate $\mathcal{O}(N^{2}\sigma_{N}^{2r})$ of approximate Fekete points \citep{KaSaTa19}.

\subsection{Asymptotic unbiasedness of kernel quadrature}\label{sec:unbiased_property}
As explained in Section~\ref{sec:review_optimal_kernel_quadrature}, kernel interpolation is widely used for the design of quadratures.
In that setting, one more advantage of continuous volume sampling is the consistency of its estimator. This is the purpose of the following result.
 \begin{theorem}\label{thm:EX_VS_integration_error}
Let $f \in \F$, and $g \in \Ltwo$. Then
\begin{align}
\mathcal{B}_{N}(f,g) & \triangleq \EX_{\VS} \left( \int_{\X}fg\,\mathrm{d}\omega -  \sum\limits_{i \in N} \hat{w}_{i}f(x_{i}) \right). \nonumber\\
 & = \sum\limits_{n \in \Ns} \langle f,e_{n} \rangle_{\mathrm{d}\omega} \langle g,e_{n} \rangle_{\mathrm{d}\omega}\left(1- \EX_{\VS}\tau_{n}^{\F}(\bm{x}) \right) .
\end{align}
Moreover,
$\mathcal{B}_{N}(f,g) \rightarrow 0$ as $N \rightarrow +\infty$.
\end{theorem}
Compared to the upper bound on the integration error given by \eqref{eq:upper_bound_integration_error}, the bias term in Theorem~\ref{thm:EX_VS_integration_error} takes into account the interaction between $f$ and $g$. For example, if for all $n \in \Ns$, $\langle f,e_{n} \rangle_{\mathrm{d}\omega} \langle g,e_{n} \rangle_{\mathrm{d}\omega} = 0$, the quadrature is unbiased for every $N$.
Theorem~\ref{thm:EX_VS_integration_error} is a generalization of a known property of regression estimators based on volume sampling in the discrete setting \citep{BeTe90,DeWa17,DeWaHs18,DeWaHs19}.

\begin{figure}[]
    \centering
\includegraphics[width= 0.8\textwidth]{figures/EX_VS_err_n_s_3.pdf}\\
\caption{\label{fig:the_only_one_for_the_moment} The value of $\EX_{\VS} \mathcal{E}(\mu_{e_{m}};\bm{x})^{2}$ for $m \in \{1,2,3,4,5\}$ for the periodic Sobolev space ($s =3,d=1$) compared to the theoretical upper bound (UB) of Theorem~\ref{thm:main_result_1}.}
\end{figure}

\section{Sketch of the proofs}\label{sec:steps_proof}
 The proof of Theorem~\ref{thm:main_result_1} decomposes into three steps. First, in Section~\ref{sec:decomposition_error}, we write $\mathcal{E}(\mu_{g};\bm{x})^{2}$ as a function of the square of the interpolation errors $\mathcal{E}(\mu_{e_{m}};\bm{x})^{2}$ of the embeddings
$\mu_{e_{m}}$. Then, in Section~\ref{sec:closed_formulas}, we give closed formulas for $\EX_{\VS} \mathcal{E}(\mu_{e_{m}};\bm{x})^{2}$ in terms of the eigenvalues of $\bm{\Sigma}$. Finally, the inequality \eqref{eq:ineq_r_N} is proved using an upper bound on the ratio of symmetric polynomials \citep{GuSi12}. The details are given in Appendix~\ref{app:proof_ineq_r_N}. Finally, The proofs of Theorem~\ref{thm:slow_rates} and Theorem~\ref{thm:EX_VS_integration_error} are straightforward consequences of Theorem~\ref{thm:main_result_1}. The details are given in Appendix~\ref{app:proof_slow_rates} and Appendix~\ref{app:proof_bias}.

\subsection{Decomposing the interpolation error}
\label{sec:decomposition_error}
Let $\bm{x} \in \mathcal{X}^{N}$ such that $\Det \bm{K}(\bm{x}) > 0$. For $m_{1}, m_{2} \in \mathbb{N}^{*}$, let the \emph{cross-leverage score} between $m_{1}$ and $m_{2}$ associated to $\bm{x}$ be
\begin{equation}\label{eq:cross_lvs_def}
\tau_{m_{1},m_{2}}^{\F}(\bm{x}) = e_{m_{1}}^{\F}(\bm{x})^{\Tran} \bm{K}(\bm{x})^{-1} e_{m_{2}}^{\F}(\bm{x}).
\end{equation}
When $m_1=m_2=m$, we speak of the $m$-th leverage score\footnote{Our definition is consistent with the leverage scores used in matrix subsampling  \citep{DrMaMu06}. Loosely speaking, $\tau_{m}^{\F}(\bm{x})$ is the leverage score of the $m$-th column of the semi-infinite matrix  $(e_{n}^{\F}(x_{i}))_{(i,n) \in [N] \times \Ns}$.} associated to $\bm{x}$, and simply write $\tau_{m}^{\F}(\bm{x})$.

By Lemma~\ref{lemma:lvs_identities},
 the $m$-th leverage score is related to the interpolation error of the $m$-th eigenfunction $e_{m}^{\F}$. Indeed,
\begin{equation}
\|e_{m}^{\F} - \Pi_{\mathcal{T}(\bm{x})} e_{m}^{\F}\|_{\F}^{2} = 1- \tau_{m}^{\F}(\bm{x}) \in [0,1].
\end{equation}
Similarly, for the cross-leverage score,
\begin{equation}
\langle \Pi_{\mathcal{T}(\bm{x})} e_{m_{1}}^{\F}, \Pi_{\mathcal{T}(\bm{x})} e_{m_{2}}^{\F} \rangle_{\F} = \tau_{m_{1},m_{2}}^{\F}(\bm{x}) \in [-1,1].
\end{equation}

For $g \in \Ltwo$, the interpolation error of the embedding $\mu_{g}$ can be expressed using the (cross-)leverage scores.
\begin{lemma}\label{lemma:error_decomposition}
If $\Det \bm{K}(\bm{x}) > 0$, then,
\begin{equation}
\label{e:error_decomposition_in_lemma}
\mathcal{E}(\mu_{g};\bm{x})^{2}  = \sum\limits_{m \in \mathbb{N}^{*}}  g_{m}^{2} \sigma_{n}\bigg(1- \tau_{m}^{\F}(\bm{x})\bigg) - \sum\limits_{m_{1}\neq m_{2} \in \mathbb{N}^{*}}  g_{m_{1}}g_{m_{2}} \sqrt{\sigma_{m_{1}}} \sqrt{\sigma_{m_{2}}} \tau_{m_{1},m_{2}}^{\F}(\bm{x}).
\end{equation}
\end{lemma}
In particular, with probability one, a design sampled from the continuous volume sampling distribution in Definition~\ref{def:VS} satisfies \eqref{e:error_decomposition_in_lemma}. Furthermore, we shall see that the expected value of the (cross-) leverage scores has a simple expression.

\subsection{Explicit formulas for expected leverage scores}
\label{sec:closed_formulas}
Proposition~\ref{prop:EX_VS_lvs} expresses expected leverage scores in terms of the spectrum of the integration operator.
\begin{proposition}\label{prop:EX_VS_lvs}
For $m~\in~\mathbb{N}^{*}$,
\begin{equation}\label{eq:lvs_formula}
\EX_{\VS} \tau_{m}^{\F}(\bm{x})  = \frac{1}{\sum\limits_{U \in \: \:\mathcal{U}_N} \prod\limits_{u \in U}\sigma_{u}}  \sum\limits_{\substack{U \in \: \:\mathcal{U}_N \\ m \in U}} \prod\limits_{u \in U}\sigma_{u}.
\end{equation}
Moreover, for $m_{1},m_{2} \in \mathbb{N}^{*}$ such that $m_{1} \neq m_{2}$, we have
\begin{equation}\label{eq:cross_lvs_zero}
\EX_{\VS} \tau_{m_{1},m_{2}}^{\F}(\bm{x}) = 0.
\end{equation}
\end{proposition}

In Appendix~\ref{app:proof_main_result_1}, we combine Lemma~\ref{lemma:error_decomposition} with Proposition~\ref{prop:EX_VS_lvs}. This concludes the proof of Theorem~\ref{thm:main_result_1} by Beppo Levi's monotone convergence theorem.

It remains to prove Proposition~\ref{prop:EX_VS_lvs}. Again, we proceed in two steps. First, our Proposition~\ref{thm:EX_lvs_phi} yields a characterization of $ \EX_{\VS}\tau_{m}^{\F}(\bm{x})$ and $\EX_{\VS}\tau_{m_{1},m_{2}}^{\F}(\bm{x})$ in terms of the spectrum of three perturbed versions of the integration operator $\bm{\Sigma}$. Second, we give explicit forms of these spectra in Proposition~\ref{prop:perturbed_kernels_spectrum} below. The idea is to express $\EX_{\VS} \tau_{m}(\bm{x})^{\F}$ as the normalization constant \eqref{eq:normalization_constant_VS} of a perturbation of the kernel $k$.
The same goes for $\EX_{\VS} \tau_{m_{1},m_{2}}^{\F}(\bm{x})$.

Let $t\in\mathbb{R}_{+}$ and $\bm{\Sigma}_{t}$, $\bm{\Sigma}_{t}^{+}$ and $\bm{\Sigma}_{t}^{-}$ be the integration operators\footnote{We drop from the notation the dependencies on $m,m_{1}$ and $m_{2}$ for simplicity.} on $\Ltwo$, respectively associated with the kernels
\begin{equation}\label{eq:k_t}
k_{t}(x,y) = k(x,y) + t e_{m}^{\F}(x) e_{m}^{\F}(y),
\end{equation}
\begin{equation}\label{eq:k_t_plus}
k_{t}^{+}(x,y) = k(x,y)
+ t \left( e_{m_{1}}^{\F}(x) + e_{m_{2}}^{\F}(x) \right) \left( e_{m_{1}}^{\F}(y) + e_{m_{2}}^{\F}(y) \right),
\end{equation}
\begin{equation}\label{eq:k_t_minus}
k_{t}^{-}(x,y) = k(x,y) + t \left( e_{m_{1}}^{\F}(x) - e_{m_{2}}^{\F}(x) \right) \left( e_{m_{1}}^{\F}(y) - e_{m_{2}}^{\F}(y) \right).
\end{equation}
By Assumption~\ref{hyp:integrable_diagonal}, and by the fact that $(e_{m})_{m \in \Ns}$ is an orthonormal basis of $\Ltwo$, all three kernels also have integrable diagonals (see Assumption~\ref{hyp:integrable_diagonal}).
In particular, they define RKHSs that can be embedded in $\Ltwo$. Moreover, recalling the definition \eqref{eq:normalization_constant_VS} of the normalization constant $Z_{N}$ of volume sampling, the following quantities are finite
\begin{align}
\phi_{m}(t) = & Z_{N}(k_{t}) , \quad\phi_{m_{1},m_{2}}^{+}(t) = Z_{N}(k_{t}^{+}),&\text{ and }\quad \phi_{m_{1},m_{2}}^{-}(t) = Z_{N}(k_{t}^{-}).
\end{align}
Remember that by Proposition~\ref{prop:VS_decomposition},
\begin{equation}\label{eq:Z_kt}
\phi_{m}(t) = N! \sum\limits_{U \in \: \:\mathcal{U}_N} \prod\limits_{u \in \: U} \tilde{\sigma}_{u}(t),
\end{equation}
where $\displaystyle \{\tilde{\sigma}_{u}(t), \: u \in \Ns\}$ is the set of eigenvalues\footnote{For a given value of $t$, the eigenvalues $\tilde{\sigma}_{u}(t)$ are not necessarily decreasing in $u$. We give explicit formulas for these eigenvalues in Proposition~\ref{prop:perturbed_kernels_spectrum}, and the order satisfied for $t=0$ is not necessarily preserved for $t>0$. This does not change anything to the argument since these eigenvalues only appear in quantities such as $\phi_{m}(t)$ which are invariant under permutation of the eigenvalues.} of $\bm{\Sigma}_{t}$. Similar identities are valid for $\phi_{m_{1},m_{2}}^{+}(t)$ and $\phi_{m_{1},m_{2}}^{-}(t)$ with the eigenvalues of $\bm{\Sigma}_{t}^{+}$ and $\bm{\Sigma}_{t}^{-}$ respectively.

\begin{proposition}\label{thm:EX_lvs_phi}
The functions $\phi_m$, $\phi_{m_1,m_2}^{+}$ and $\phi_{m_1,m_2}^{-}$ are right differentiable in zero. Furthermore,
\begin{equation}\label{eq:EX_lvs_n_delta_phi_n}
\EX_{\VS} \tau_{m}^{\F}(\bm{x})  = \frac{1}{Z_{N}(k)}  \frac{\partial \phi_{m}}{ \partial t }\bigg|_{t = 0^{+}},
\end{equation}
and
\begin{equation}\label{eq:EX_VS_cross_lvs_formula}
\EX_{\VS} \tau_{m_{1},m_{2}}^{\F}(\bm{x}) = \frac{1}{4Z_{N}(k)} \bigg(\frac{\partial \phi_{m_{1},m_{2}}^{+}}{ \partial t } - \frac{\partial \phi_{m_{1},m_{2}}^{-}}{ \partial t }\bigg)\bigg|_{t = 0^{+}}.
\end{equation}
\end{proposition}
 The details of the proof are postponed to Appendix~\ref{app:proof_EX_lvs_phi}. We complete this proposition with a description of the spectrum of the operators $\bm{\Sigma}_{t}$, $\bm{\Sigma}_{t}^{+}$ and $\bm{\Sigma}_{t}^{-}$ using the spectrum of $\bm{\Sigma}$.
\begin{proposition}\label{prop:perturbed_kernels_spectrum}
The eigenvalues of $\bm{\Sigma}_{t}$ write
\begin{equation}
\tilde{\sigma}_{u}(t) =  \left\{
    \begin{array}{ll}
        \sigma_{u} & \mbox{if} \: u \neq m, \\
        (1 + t)\sigma_{u} & \mbox{if}\: u = m.
    \end{array}
\right.
\end{equation}
Moreover, the eigenvalues of $\bm{\Sigma}_{t}^{+}$ and $\bm{\Sigma}_{t}^{-}$ satisfy
\begin{equation}\label{eq:equality_positive_negative_spectrum}
\{\tilde{\sigma}_{u}^{+}(t), \: u \in \Ns \} = \{\tilde{\sigma}_{u}^{-}(t), \: u \in \Ns \}.
\end{equation}
\end{proposition}
The proof is based on the observation that the perturbations in \eqref{eq:k_t}, \eqref{eq:k_t_plus}, and \eqref{eq:k_t_minus} only affect a principal subspace of dimension $1$ or $2$; see Appendix~\ref{app:proof_perturbed_kernels_spectrum}.

Combining the characterization of $\EX_{\VS} \tau_{m}^{\F}(\bm{x})$ and $\EX_{\VS} \tau_{m_{1},m_{2}}^{\F}(\bm{x})$ given in Proposition~\ref{thm:EX_lvs_phi}, and Proposition~\ref{prop:perturbed_kernels_spectrum}, we prove Proposition~\ref{prop:EX_VS_lvs}; see details in Appendix~\ref{sec:proof_EX_VS_lvs}.

\section{Conclusion}\label{sec:discussion}

We deal with interpolation in RKHSs using random nodes and optimal weights. This problem is intimately related to kernel quadrature, though interpolation is more general.
We introduced continuous volume sampling (VS), a repulsive point process that is a mixture of DPPs, although not a DPP itself.
VS comes with a set of advantages. First, interpretable bounds on the interpolation error can be derived under minimalistic assumptions. Our bounds are close to optimal since they share the same decay rate as known lower bounds. Moreover, we provide explicit evaluations of the constants appearing in our bounds for some particular RKHS (Sobolev, Gaussian). Second,
VS provides a fully kernelized approach, which may permit sampling without knowing the Mercer decomposition of the kernel \citep{ReGh19} unlike previous work on random designs. Investigating efficient samplers is deferred to future work.

Volume sampling was originally introduced on a finite domain \citep{DRVW06}, and has been used for matrix subsampling for linear regression and low-rank approximations \citep{DeWa17,BeBaCh18}. Similarly to \citep{BeBaCh19}, our work is another direct connection between subsampling columns of a matrix and the continuous problem of interpolating an element of an RKHS.

Compared to \citep{KaSaTa19}, our analysis yields a sharper upper bound while circumventing the analysis of the Lebesgue constant.
Yet, it would be interesting to analyze this constant under continuous volume sampling to get an idea of the stability of the algorithm. Another potential extension of this work would be the analysis of interpolation under regularization as in \citep{Bac17}.

\subsection*{Acknowledgments}
AB acknowledges support from ANR grant \textsc{BoB} (ANR-16-CE23-0003), Région Hauts-de-France, and Centrale Lille. RB acknowledges support from ERC grant \textsc{Blackjack} (ERC-2019-STG-851866).

\newpage

Machine Learning (stat.ML); Machine Learning (cs.LG); Numerical Analysis (math.NA); Probability (math.PR)