Avoiding unwanted results in locally linear embedding: A new understanding of regularization

Abstract: We demonstrate that locally linear embedding (LLE) inherently admits some unwanted results when no regularization is used, even for cases in which regularization is not supposed to be needed in the original algorithm. The existence of one special type of result, which we call ``projection pattern'', is mathematically proved in the situation that an exact local linear relation is achieved in each neighborhood of the data. These special patterns as well as some other bizarre results that may occur in more general situations are shown by numerical examples on the Swiss roll with a hole embedded in a high dimensional space. It is observed that all these bad results can be effectively prevented by using regularization.

\section{Introduction}

Let $\mathcal{X}=\{x_i\}_{i=1}^N$ be a collection of points
in some high dimensional space $\mathbb{R}^D$. The general goal of
manifold learning (or nonlinear dimensionality reduction) is to
find for $\mathcal{X}$ a representation $\mathcal{Y}=\{y_i\}_{i=1}^N$ in
some lower dimensional $\mathbb{R}^d$, under the assumption
that $\mathcal{X}$ lies on some unknown submanifold of
$\mathbb{R}^D$.

Locally linear embedding (LLE) \cite{lle}, due to its
simplicity in idea as well as efficiency in implementation, is a
popular manifold learning method, which has been
studied a lot and has many variant and modified
versions \cite{hlle2003,de2003supervised,kouro2005,
chang2006robust, mlle2007,chenliu2011,wu2018,wang2015bearing}.
The goal of this paper is to point out a fundamental problem,
which seems to have not been addressed yet,
and then propose a simple solution.
Precisely, we will demonstrate
that LLE inherently admits some unwanted results when no
regularization is used, even for cases in which regularization
is not supposed to be needed in the original algorithm.
And the solution is just to use regularization in any case.

The existence of one special type of unwanted results,
which we call ``projection patterns'', will be mathematically
proved in the situation that an exact local linear relation
is achieved in each neighborhood of the dataset $\mathcal X$.
Such projection patterns, as indicated by their name,
are basically direct projections of $\mathcal{X}$ from $\mathbb{R}^D$
onto a $d$-dimensional hyperplane, ignoring the
geometry of the data totally.
Due to the using of regularization, they do not appear
for most artificial datasets, which reside in some low
dimensional $\mathbb{R}^D$ (mostly $\mathbb{R}^3$).
The problem is that for general data in a high
dimensional $\mathbb{R}^D$, the
regularization is supposed to be unnecessary and not employed.
We will show by numerical examples that this practice is risky.
The idea is simple:
performing some embeddings of
the Swiss roll with a hole into a high dimensional $\mathbb{R}^D$
such that
regularization is not used in the original LLE algorithm, and
then comparing the results to those with regularization
being used. It turns out that, if regularization is not used,
the projection phenomenon always occurs as predicted when an isometric embedding is applied, while if some further perturbation is added
to the embedding, more bizarre results may appear.
By contrast, with regularization, all the bad results are
effectively prevented.

\section{Projection patterns in LLE}\label{sec:lle}

Let us begin with a quick review of the LLE procedure. As in the introduction,
the dataset $\mathcal X=\{x_{i}\}_{i=1}^{N}$ is a subset of
$\mathbb R^{D}$ and we want to find for it a representation
$\mathcal Y=\{y_{i}\}_{i=1}^{N}$ in some lower dimensional
$\mathbb R^{d}$.
\begin{enumerate}[align=left]
\item[Step 1.] For each $i=1,\ldots,N$, let
$\mathcal U_{i}=\{x_{i_{1}},\ldots,x_{i_{k}}\}
\subset\mathcal X\setminus\{x_{i}\}$ be a $k$-nearest
neighborhood of $x_{i}$. (For simplicity we consider
$k$ to be fixed for all $i$.)
\item[Step 2.] Set
$w^{(i)}=(w_1^{(i)},\ldots,w_k^{(i)})\in\mathbb{R}^k$ to be
a solution of the problem
\begin{equation}
\underset{(w_{1},\ldots,w_{k})\in\mathbb R^{k}}{\mbox{argmin}}
\|x_{i}-\sum_{j=1}^{k}w_{j}x_{i_{j}}\|^{2}\quad\mbox{s.t.}
\quad\sum_{j=1}^{k}w_{j}=1.\tag{P1}\label{eq:p1}
\end{equation}
\item[Step 3.] With $w^{(i)}$ given from Step 2, set $\mathcal Y$ to be
a solution of the problem
\begin{equation}
\underset{\{y_{i}\}_{i=1}^{N}\subset\mathbb R^{d}}{\mbox{argmin}}
\sum_{i=1}^{N}\|y_{i}-\sum_{j=1}^{k}w_{j}^{(i)}y_{i_{j}}\|^{2}
\quad\mbox{s.t.}\quad YY^{T}=I,\tag{P2}\label{eq:p2}
\end{equation}
where $Y$ denotes the matrix
$\left[y_{1}\ \cdots\ y_{N}\right]\in\mathbb R^{d\times N}$,
and $I$ is the identity matrix.
\end{enumerate}
For convenience, in the following we reserve the index $i$
to be the one which runs from $1$ to $N$, and $i_{1},\ldots,i_{k}$
are the indices for the corresponding $k$-nearest points being
chosen in Step 1.

The following remark suggests a possible problem of the LLE
algorithm, which turns out to be what happens to the
``unwanted results'' that will be discussed later.

\begin{rem}\label{rem:tei}
Step 3 is stated in accordance with the original
idea. It is not what is exactly executed in the algorithm.
To be precise, write
\begin{align*}
\sum_{i=1}^{N}\|y_{i}-\sum_{j=1}^{k}w_{j}^{(i)}y_{i_{j}}\|^{2}
=\|(I-W)Y^T\|^2,
\end{align*}
where $W=(W_{ij})$ is the $N\times N$ matrix defined by
\begin{align*}
W_{is} = \bigg\{
\begin{array}{cl}
w_j^{(i)} &  \mbox{if }\ s= i_j\ (j=1,\ldots,k)\\[3pt]
0 &  \mbox{if }\ s\notin \{i_{1},\ldots,i_{k}\},
\end{array}
\end{align*}
and $\|\cdot\|$ for matrices denotes the Frobenius norm.
Then, it is known that solutions of \eqref{eq:p2} are given by
\begin{equation}
Y^{T}=\left[g_{1}\ \cdots\ g_{d}\right],\label{eq:lley}
\end{equation}
where $g_{1},\ldots,g_{d}$ are any selection of
orthonormal eigenvectors of
$(I-W)^T(I-W)$ with corresponding eigenvalues
$0=\lambda_1\le \lambda_2\le \ldots\le\lambda_d$.
However, in this way one of the $g_{j}$'s
might be $\frac{1}{\sqrt{N}}\boldsymbol 1_{N}$, corresponding to the
eigenvalue $0$. Here $\boldsymbol{1}_N\in \mathbb R^N$ denotes the vector
whose components all equal $1$.
This eigenvector is redundant for our
purpose, since the dimension reduced data $\mathcal Y$ given by
\eqref{eq:lley} is trivially isometric to that given by
$[g_2\ g_3\ \cdots\ g_d]$. Therefore, to find
an effective $d$-dimensional representation for $\mathcal X$, we
would like to exclude $\frac{1}{\sqrt{N}}\boldsymbol 1_N$ from our
selection of eigenvectors. In the LLE algorithm, what is
executed is to find out the first $d+1$ eigenvectors,
and set $g_1,\ldots,g_d$ to be the second to the $(d+1)$-th ones
respectively, in the anticipation that the first eigenvector
should be $\frac{1}{\sqrt{N}}\boldsymbol{1}_N$. However, note that this
approach may break down when the multiplicity of the
eigenvalue $0$ is greater than one, hence there
are more than one eigenvectors corresponding to it.
In this situation, it is
not guaranteed that the first eigenvector being found by computer
is $\frac{1}{\sqrt{N}}\boldsymbol 1_{N}$.
Indeed, it is not necessary that
any one of the eigenvectors been found must be
$\frac{1}{\sqrt{N}}\boldsymbol 1_{N}$.
\end{rem}

Now we turn to the resolution of \eqref{eq:p1}. It can be
rewritten as
\[
\underset{w\in\mathbb R^{k}}{\mbox{argmin}}\|Z_{i}w\|^{2}\quad
\mbox{s.t.}\quad\boldsymbol 1_{k}^{T}w=1,
\]
where
\[
Z_{i}=\begin{bmatrix}x_{i_{1}}-x_{i} & \cdots & x_{i_{k}}-x_{i}\end{bmatrix}\in\mathbb R^{D\times k}.
\]
Let $C_{i}=Z_{i}^{T}Z_{i}$, then
$\|Z_i w\|^2 = w^T Z_i^T Z_i w = w^T C_i w$.
By the method of Lagrange multiplier, any minimizer $w$ of \eqref{eq:p1}
satisfies
\begin{equation}\label{eq:lm}
\left\{ \begin{aligned} & C_i w=\lambda\boldsymbol 1_{k}\\
 & \boldsymbol 1_{k}^{T}w=1
\end{aligned}
\right.
\end{equation}
for some $\lambda\in\mathbb R$. If $C_{i}$
is invertible, \eqref{eq:lm} has a unique solution
\[
w^{(i)}=\frac{C_{i}^{-1}\boldsymbol 1_{k}}{\boldsymbol 1_{k}^{T}C_{i}^{-1}\boldsymbol 1_{k}}.
\]
On the other hand, if $C_{i}$ is singular, \eqref{eq:lm} may have
multiple solutions. In this situation it is suggested that we
add a small regularization term $\epsilon I$ to $C_{i}$, and set
\begin{align}\label{eq:rew}
w^{(i)}=w^{(i)}(\epsilon)=\frac{(C_{i}+\epsilon I)^{-1}\boldsymbol 1_{k}}
{\boldsymbol 1_{k}^{T}(C_{i}+\epsilon I)^{-1}\boldsymbol 1_{k}}.
\end{align}

For efficiency, in the LLE algorithm whether the regularization
term is used or not is not precisely determined by the
invertibility or not of $C_i$, but by the following rule:
used if $k>D$, and not used if $D\ge k$. Note that this rule is only
a convenient one but not in exact accordance with the original
idea. Precisely, if $k>D$, then it's true that $C_i$ must be singular,
but $D\ge k$ does not guarantee that $C_i$ is non-singular.
Nevertheless, as we will advocate using regularization
(namely the formula \eqref{eq:rew})
no matter $C_i$ is singular or not, we do not care about
this problem.

A common perception of \eqref{eq:rew} is that it provides
a stable way to solve \eqref{eq:lm}, at the cost of a small amount of error.
In fact,
\[
w^{(i)}(0^{+}):=\lim_{\epsilon\to0^{+}}w^{(i)}(\epsilon)
\]
exists and is an exact solution of \eqref{eq:lm}.
In this thread, it seems natural to
expect better performance of LLE if $w^{(i)}(\epsilon)$ is
replaced by $w^{(i)}(0^{+})$. However, that is not the case.
In Figure \ref{fig:1},
we give some numerical examples
on the Swiss roll with a hole for
different values of $\epsilon$, and we see that
when $\epsilon$ is too small, the resulting $\mathcal Y$ becomes
far from what it ought to be. Actually, when $\epsilon$ approaches zero,
we see that $\mathcal Y$ converges to a ``projection pattern''.
That is, it looks like $\mathcal Y$ is simply some
projection of the Swiss roll onto the plane, ignoring the
rolled-up nature of the original data.
We are going to explain that this phenomenon is not a
coincidence, and is not due to any instability
problem in solving \eqref{eq:lm}. In fact, such projection
mappings are inherently allowed in the LLE procedure
when no regularization is used.
\begin{figure}
\includegraphics[width=1\textwidth]{Fig1.eps}
\caption{Results of applying LLE to the Swiss roll with a hole
with different sizes of $\epsilon$. Notice that the rightmost
image is a 2D one.
\label{fig:1}}
\end{figure}

To explain the phenomenon, we formulate precisely two assumptions:
\begin{enumerate}[label=(A\arabic{enumi}), ref = (A\arabic{enumi})]
\item $x_{i}-\sum_{j=1}^{k}w_{j}^{(i)}x_{i_{j}}=0$ for all $i=1,\ldots,N$. \label{a1}
\item $\mbox{rank}(X)\ge d$, where
$X:=\begin{bmatrix}x_{1} & \cdots & x_{N}\end{bmatrix}\in\mathbb R^{D\times N}$. \label{a2}
\end{enumerate}
Assumption \ref{a1} is the same as saying that zero is achieved as the minimum in
\eqref{eq:p1} for all $i$. Note that this is the normal situation for cases with
$k>D$ -- the same cases for which regularization is employed
in the LLE algorithm. As for \ref{a2}, it
is a natural assumption which holds in all cases of interest.
To see it, suppose $\mbox{rank}(X)=r<d$, then some $r$ columns from $X$
span all the others, and hence $\mathcal X$ lies entirely on some $r$
dimensional subspace. In this case there is no point to pursue an embedding
of $\mathcal X$ in $\mathbb R^{d}$.

Recall that
$\mbox{rank}(XX^{T})=\mbox{rank}(X)$,
and hence \ref{a2} implies $XX^{T}$ has at least $d$ nonzero
eigenvalues (counting multiplicity). Moreover, since $XX^{T}$ is
positive semi-definite, all of its nonzero eigenvalues are positive.
With these in mind, we can now state our key observation.
\begin{thm}
\label{thm:m1} Assume \ref{a1} and \ref{a2}.
Let $u_{1},\ldots,u_{d}\in\mathbb R^{D}$ be any collection of
orthonormal eigenvectors of $XX^{T}$ whose corresponding
eigenvalues, denoted $\lambda_{1},\ldots,\lambda_{d}$, are all positive.
Then $\mathcal Y:=\{Ax_{i}\}_{i=1}^{N}$
is a solution of \eqref{eq:p2}, where $A\in\mathbb{R}^{d\times D}$
is given by
\begin{equation}
A=\begin{bmatrix}\lambda_{1}^{-1/2}\\
 & \ddots\\
 &  & \lambda_{d}^{-1/2}
\end{bmatrix}\begin{bmatrix}u_{1}^{T}\\
\vdots\\
u_{d}^{T}
\end{bmatrix}.\label{eq:ad}
\end{equation}
\end{thm}

\begin{proof}
The local linear relations in \ref{a1}
are preserved by applying $A$, that is
\[
Ax_{i}-\sum_{j=1}^{k}w_{j}^{(i)}Ax_{i_{j}}=0\quad\forall\,i.
\]
As a consequence, by setting $\mathcal Y=\{y_{i}\}_{i=1}^{N}=\{Ax_{i}\}_{i=1}^{N}$,
we have
\[
\sum_{i=1}^{N}\|y_{i}-\sum_{j=1}^{k}w_{j}^{(i)}y_{i_{j}}\|^{2}=0,
\]
and hence $\{y_{i}\}_{i=1}^{N}$ is a minimizer of the cost function in
Problem \eqref{eq:p2}. It remains to show that the constraint
$YY^{T}=I$ is also satisfied. Note that $Y=AX$,
and hence the constraint is
\[
AXX^{T}A^{T}=I.
\]
The validity of this equality is easy to check from the definition
\eqref{eq:ad} of $A$, and the fact that
$XX^{T}u_{\ell}=\lambda_{\ell}u_{\ell}$
for $\ell=1,\ldots,d$.
\end{proof}
Now, what does $\mathcal Y = \{Ax_{i}\}_{i=1}^{N}$ in Theorem
\ref{thm:m1} look like? Note that
\begin{equation}
x\mapsto\sum_{\ell=1}^{d}(x\cdot u_{\ell})u_{\ell}\label{eq:op}
\end{equation}
is the orthogonal projection of $x\in\mathbb R^{D}$ onto the
$d$-dimensional subspace $\mbox{span}(u_{1},\ldots,u_{d})$.
By endowing this subspace with its own coordinate system with respect to
$u_{1},\ldots,u_{d}$, we can regard the projection as a mapping from
$\mathbb R^{D}$ into $\mathbb R^{d}$, which is expressed by the coefficients
in \eqref{eq:op}:
\[
x\mapsto\begin{bmatrix}x\cdot u_{1}\\
\vdots\\
x\cdot u_{d}
\end{bmatrix}=\begin{bmatrix}u_{1}^{T}\\
\vdots\\
u_{d}^{T}
\end{bmatrix}x.
\]
This is the first half of the action of $A$. The second half
is a further multiplication of the diagonal matrix
$\mbox{diag}(\lambda_{1}^{-1/2},\ldots,\lambda_{d}^{-1/2})$,
which is nothing but performing some rescalings of the
coordinates. In summary, $x\mapsto Ax$ is an
orthogonal projection from $\mathbb R^{D}$ to $\mathbb R^{d}$
followed by some rescalings of coordinates in $\mathbb R^{d}$.
This explains the projection phenomenon observed.

\begin{rem}\label{rem:cc}
What Theorem \ref{thm:m1} tells us is that, when
\ref{a1} and \ref{a2} hold, some projection patterns are
solutions of \eqref{eq:p1}, and hence are candidates for
being the result of LLE. Logically speaking, it does not preclude
the possiblity that there are other kinds of results (which we do
not know).
\end{rem}

\section{Numerical examples for high dimensional data}\label{sec:num}

The projection phenomenon does not
pertain exclusively to cases with $k>D$.
For one thing, even if $D\ge k$, if $\mathcal{X}$ lies on some
$D'$-dimensional hyperplane with $D'<k$,
then \ref{a1} still holds in general (we take
\ref{a2} for granted and will no longer mention it), and
Theorem \ref{thm:m1} applies.
Of course, such a degenerate case is of little interest.
But for another, it is conceivable that
the projection effect may still have its influence
when \ref{a1} is only approximately true while to a high degree.
This statement can partly be supported by the third and fourth images
in Figure \ref{fig:1}, where $\epsilon$
is very small and \ref{a1} is almost true.
However, those images correspond to the situation $k>D=3$.

To acquire some direct evidences about what could
happen to high dimensional data,
we have performed several experiments in which
the Swiss roll with a hole was first embedded in a high dimensional
$\mathbb{R}^D$, and then applied LLE to it with $k$ chosen
to be smaller than $D$.
For our purpose, we only considered isometric embeddings
and perturbed isometric embeddings
so that the embedded datasets are basically the same
Swiss roll with a hole.
It turns out that for isometrically embedded data
(which lie entirely on some three dimensional
hyperplane in $\mathbb R^D$), the projection phenomenon is obvious
(see however Remark \ref{c1} at the end of this section).
Somewhat surprising to us is that when rather small
perturbations are added to the isometric embeddings, the outcomes
become quite unpredictable but not merely perturbations of
projection patterns.
It is observed that all these unwanted results -- projection
patterns and others -- are associated with the problem mentioned
in Remark \ref{rem:tei}. That is, the matrix $(I-W)^T(I-W)$
corresponding to them is very singular.
And in any case, by using regularization we see this problem can be
effectively avoided, and considerably improved results can be obtained.
See Figure \ref{fig:2} for some examples, the details of which
are given in the next paragraph.
\begin{figure}
\includegraphics[width=\textwidth]{Fig2.eps}
\caption{Examples of applying LLE to the Swiss roll with a hole
embedded in dimensional spaces. From left to right,
embeddings $E_1,E_2$ and $E_3$ are applied respectively (see descriptions in the text).
Top row: results of the original LLE; Bottom row: corresponding results with regularization.
\label{fig:2}}
\end{figure}

Let $\mathcal X =\{x_i\}$ represents a Swiss roll with a hole dataset
in $\mathbb{R}^3$.
Figure \ref{fig:2} shows, from left to right, the results of
applying LLE to
three different datasets $E_1\mathcal X,E_2\mathcal X$ and $E_3\mathcal X$
with $k=12$.
The top row are results of the original LLE (no regularization),
and the bottom
row are corresponding results for which the regularized weight
vector \eqref{eq:rew}  is used for each $i$, with
$\epsilon = 10^{-3}\mbox{trace}(C_i)$.
$E_i$, $i=1,2,3$, are given as follows:
\begin{itemize}
\item $E_1$ is some randomly generated $18\times 3$ matrix whose
columns form an orthonormal set of vectors.
This gives rise to a linear isometric mapping from $\mathbb R^3$ into
$\mathbb R^{18}$.
\item $E_2$ is the mapping from $\mathbb R^3$ to $\mathbb R^{19}$ obtained
by perturbing $E_1$ in an extra dimension:
\begin{align*}
E_2 x = (E_1 x, 0.1\sin (\sum_{j=1}^{18}(E_1 x)_j)),
\end{align*}
where $(E_1 x)_j$ denotes the $j$-th component of $E_1 x$.
\item  $E_3:\mathbb R^3\to\mathbb R^{18}$ is another perturbation of $E_1$ given by
\begin{align*}
E_3 x = E_1 x + 0.1(\sin((E_1 x)_1),\sin((E_1x)_2),\ldots,\sin((E_1x)_{18})).
\end{align*}
\end{itemize}
Here are some remarks:
\begin{enumerate}[label=(\alph{enumi}), ref = (\alph{enumi})]
\item For isometrically embedded data, the results without
regularization are not always typical projection patterns
as the left top image in Figure \ref{fig:2}. Sometimes additional
deformation or distortion is also present (see Figure \ref{fig:3}).
We do not know if it reflects the fact that there are
other possible results than
projection patterns (cf. Remark \ref{rem:cc}), or is simply
the cause of errors in numerical simulations.\label{c1}
\begin{figure}
\includegraphics[width=\textwidth]{Fig3.eps}
\caption{Some ``non-typical'' projection patterns
obtained by applying LLE to the Swiss roll with a hole
isometrically embedded in a high dimensional space.
\label{fig:3}}
\end{figure}
\item The using of sine function in $E_2$ and $E_3$ is a random choice
and bears no significance. We can still see the projection effect
for $E_2\mathcal X$ without regularization (top middle image of Figure \ref{fig:2}),
while the corresponding result for $E_3\mathcal X$ (top right) is totally inexplicable.
One difference between $E_2\mathcal X$ and $E_3\mathcal X$ that might
be important is that the former lies on
a 4-dimensional hyperplane in $\mathbb R^{19}$, while the latter does not
lie on any lower dimensional hyperplane in $\mathbb R^{18}$.
\item The above examples show clearly that the condition $k<D$ doesn't imply
regularization is unnecessary.
However, as is mentioned, the criterion based on the size relation between
$k$ and $D$
is only something for convenience. How about using the original idea:
performing regularization or not according to whether $C_i$ is singular
or not? Experiments (not shown here) show that
it doesn't help either.
In fact, for $E_3\mathcal X$, the majority of the $C_i$'s
are already non-singular, and even if all the singular
$C_i$'s are regularized (with non-singular $C_i$'s unchanged),
the result still looks as bizarre as the original one.
\end{enumerate}

\section{Conclusion}\label{sec:con}

We have demonstrated that LLE inherently admits some unwanted results
if no regularization is used, even
for cases in which regularization is supposed to be
unnecessary in the original algorithm.
The true merit of regularization is hence not (or at least not merely)
for solving \eqref{eq:lm} stably, at the cost of a small
amount of error.
On the contrary, by deliberately distorting the local linear relations,
it protects LLE from some bad results. As a consequence,
we suggest that one uses regularization in any case when applying
LLE.
Of course, our investigation is far from comprehensive.
More examples, especially high dimensional real world data,
should be examined. Moreover, using regularization
alone is in no way a promise of good results.

\section*{Acknowledgment}
This work is supported by Ministry of Science
and Technology of Taiwan under grant number MOST110-2636-M-110-005-.
The author would also like to thank Chih-Wei Chen
for valuable discussions.

\bibliographystyle{plain}

Numerical Analysis (math.NA); Machine Learning (cs.LG); Machine Learning (stat.ML)


Exact and Approximation Algorithms for Many-To-Many Point Matching in the Plane

Abstract: Given two sets $S$ and $T$ of points in the plane, of total size $n$, a {many-to-many} matching between $S$ and $T$ is a set of pairs $(p,q)$ such that $p\in S$, $q\in T$ and for each $r\in S\cup T$, $r$ appears in at least one such pair. The {cost of a pair} $(p,q)$ is the (Euclidean) distance between $p$ and $q$. In the {minimum-cost many-to-many matching} problem, the goal is to compute a many-to-many matching such that the sum of the costs of the pairs is minimized. This problem is a restricted version of minimum-weight edge cover in a bipartite graph, and hence can be solved in $O(n^3)$ time. In a more restricted setting where all the points are on a line, the problem can be solved in $O(n\log n)$ time [Colannino, Damian, Hurtado, Langerman, Meijer, Ramaswami, Souvaine, Toussaint; Graphs Comb., 2007]. However, no progress has been made in the general planar case in improving the cubic time bound. In this paper, we obtain an $O(n^2\cdot poly(\log n))$ time exact algorithm and an $O( n^{3/2}\cdot poly(\log n))$ time $(1+\epsilon)$-approximation in the planar case. Our results affirmatively address an open problem posed in [Colannino et al., Graphs Comb., 2007].

\section{Introduction}
Let $G=(V=S\cup T, E)$ be a simple bipartite graph where each edge has a non-negative real weight, and no vertex is isolated. The \emph{many-to-many} matching problem on $G$ is to find a subset of edges  $E'\subseteq E$ of minimum total weight such that for each vertex $v\in V$ there is an edge in $E'$ incident on $v$. This is often referred to as the  \emph{minimum-weight edge cover} problem. A standard method to compute a minimum-weight edge cover of $G$ in polynomial-time is to reduce the problem to the minimum-weight perfect matching problem on an equivalent graph, see \cite{schrijver-book,ColanninoDHLMRST07,eiter1997distance,FerdousPK18}. Since the reduction takes $O(|V|+|E|)$ time, the running time is the same as the fastest known algorithm for computing a minimum-weight perfect matching. A faster $\frac{3}{2}$-approximation algorithm is proposed in \cite{FerdousPK18}.

Motivated by the computational problems in musical rhythm theory, Colannino et al. \cite{ColanninoDHLMRST07} studied the many-to-many matching problem in a geometric setting. Suppose we are given two sets $S$ and $T$ of points in the plane, of total size $n$. A \emph{many-to-many} matching between $S$ and $T$ is a set of pairs $(p,q)$ such that $p\in S$, $q\in T$ and for each $r\in S\cup T$, $r$ appears in at least one such pair. The \emph{cost of a pair} $(p,q)$ is the (Euclidean) distance $d(p,q)$ between $p$ and $q$. The \emph{cost of a set of pairs} is the sum of the costs of the pairs in the set. In the geometric \emph{minimum-cost many-to-many matching} problem, the goal is to compute a many-to-many matching such that the cost of the corresponding set of pairs is minimized. For points on a line,  an $O(n\log n)$ time dynamic-programming algorithm is proposed in \cite{ColanninoDHLMRST07}. In the same paper, the importance of the many-to-many matching problem for points in the plane is stated in the context of melody matching. Furthermore, the authors state the task of generalizing their work to this version in the plane as an important open problem, especially since geometry has helped in designing efficient algorithms for the computation of the minimum-weight perfect matching for points in the plane (see, e.g., \cite{vaidya1989geometry,varadarajan1999approximation}). Several  variants of the $1$-dimensional many-to-many matching problem are considered in \cite{Rajabi-AlniB16,abs-1904-05184,abs-1904-03015}.

\subsection{Our Results and Techniques}
In this work, we design several exact and approximation algorithms for minimum-cost many-to-many matching in the plane, thus affirmatively addressing the open problem posed in \cite{ColanninoDHLMRST07}.  First, we obtain an $O(n^{2}\cdot poly(\log n))$ time\footnote{We use the notation $poly()$ to denote a polynomial function} exact algorithm for this problem using a connection to minimum-weight perfect matching. We note that our time-bound matches the time bound for solving minimum-weight bipartite matching for points in the plane. Next, for any $\epsilon > 0$, we obtain a $(1+\epsilon)$-approximation algorithm for this problem with improved $O((1/\epsilon^c)\cdot n^{3/2}\cdot poly(\log n))$ running time for some small constant $c$. We also obtain a simple $2$-approximation in $O(n\log n)$ time.

Next, we give an overview of our techniques. A major reason behind the scarcity of results for geometric many-to-many-matching is the lack of techniques to directly approach this problem. The $O(n^3)$ algorithm known for general graphs reduces the problem to (minimum-weight) bipartite perfect matching and uses a graph matching  algorithm to solve the problem on the new instance. However, this standard reduction \cite{keijsper1998efficient} from many-to-many matching to regular matching changes the weights of the edges in a convoluted manner. Hence, even if one starts with the planar Euclidean distances, the new interpoint distances in the constructed instance cannot be embedded in the plane or even in any metric space. As the algorithms for planar bipartite (perfect) matching heavily exploit the properties of the plane, they cannot be employed for solving the new instance of bipartite perfect matching. Thus, even though there is a wealth of literature for planar bipartite matching, no progress has been made in understanding the structure of many-to-many matching in the plane.

In our approach, we use a rather unconventional connection to bipartite perfect matching. First, we use a different reduction to convert our instance of many-to-many matching in the plane to an equivalent instance of bipartite perfect matching. The new instance cannot be embedded in the plane, however it does not modify the original distances between the points. Rather the new bipartite graph constructed is the union of (i) the original geometric bipartite graph, (ii) a new bipartite clique all of whose edges have the  same weight and (iii) a linear number of additional edges. Thus, even though we could not use a planar matching algorithm directly, we could successfully use ideas from the literature of planar matching exploiting the structure of the constructed graph. A similar reduction was used in \cite{eiter1997distance} in the context of computation of link distance of graphs. However, we cannot use this reduction directly, as we cannot afford to explicitly store the constructed graph which contains $\Omega(n^2)$ edges. Recall that we are aiming for an $O((1/\epsilon^c)\cdot n^{3/2}\cdot poly(\log n))$ time bound. Nevertheless, we exploit the structure of this graph to implicitly store it in  $O(n)$ space.

In Section \ref{sec:exact}, we show that the Hungarian algorithm can be implemented on our constructed graph in $O(n^{2}\cdot poly(\log n))$ time using ideas from planar matching. To obtain a subquadratic time bound, we implement the algorithm due to Gabow and Tarjan \cite{gabow1989faster}. A straightforward implementation of this algorithm might need $\Omega(n^2)$ time. We note that this algorithm has been used in several works on planar matching for obtaining efficient algorithms \cite{RaghvendraA20,varadarajan1999approximation,AsathullaKLR20}. Our algorithm is closest to the one in \cite{varadarajan1999approximation} among these algorithms from a moral point of view. The difference is that their algorithm is for planar points. However, we deal with a graph which is partly embeddable. Nevertheless, in Section \ref{thm:ptas-perfect}, we show that using additional ideas and data structures, the Gabow-Tarjan algorithm can be implemented in $O(n^{3/2}\cdot poly(\log n))$ time, albeit with a $(1+\epsilon)$-factor loss on the quality of the solution.
In Section \ref{sec:2approx}, we describe our 2-approximation which the $(1+\epsilon)$-approximation algorithm uses as a subroutine for preprocessing.

\section{Preliminaries}

In the \emph{bipartite perfect matching} problem, we are given an edge-weighted bipartite graph $G=(R,B,E)$ containing a perfect matching, and the goal is to find a perfect matching having the minimum cost or sum of the edge-weights. For our convenience, sometimes we would assume that the edges of $G$ are not given explicitly. For example, $R$ and $B$ might be two sets of points in the plane, and $G$ is the complete bipartite graph induced by the bipartition $(R,B)$. In this case, the points in $R\cup B$ can be used to implicitly represent the graph $G$. In our case, we will use similar implicit representation of input graphs for designing subquadratic algorithm. Now, we have the following lemma, which reduces our problem to an equivalent instance of bipartite perfect matching.

\begin{lemma}\label{lem:equiv-many-perfect}
Given an instance $I'$ of minimum-cost many-to-many matching, one can compute in $O(n\log n)$ time an instance of bipartite perfect matching $I$ such that (i) if there is a many-to-many matching for $I'$ of cost $C$, there is a perfect matching for $I$ of cost at most $C$, and (ii) if there is a perfect matching for $I$ of cost $C$, there is a many-to-many matching for $I'$ of cost $C$.
\end{lemma}
\vspace{-6mm}

\begin{figure}[!ht]
		\begin{center}
			\includegraphics[width=.7\textwidth]{graph-construction.pdf}
			\vspace{-2mm}
			\caption{The figure on the left shows an instance $I'$ of minimum-cost many-to-many matching along with the interpoint distances, where $S=\{u_1,u_2\}$ and $T=\{v_1,v_2,v_3\}$. The right figure depicts the graph constructed from $I'$ along with the edge weights, where $R_0=\{u_1,u_2\}, R_1=\{\widehat{v_1},\widehat{v_2},\widehat{v_3}\}, B_0=\{v_1,v_2,v_3\}$ and $B_1=\{\widehat{u_1},\widehat{u_2}\}$. Solution pairs (or edges) are shown in bold.}
			\label{fig:graph}
		\end{center}
\end{figure}
\vspace{-10mm}
\begin{proof}
Given the instance $I'$ consisting of the two sets of points $S$ and $T$, we construct a bipartite graph $G=(R=R_0\cup R_1,B=B_0\cup B_1,E)$, where $R_0=S$, $B_0=T$, $R_1$ contains copies of the points in $T$, $B_1$ contains copies of the points in $S$. $E$ contains all the edges of $E_0=R_0\times B_0$ and $E_1=R_1\times B_1$, and also the ones in $E_2=\{(u,\hat{u})\mid u\in R_0,\hat{u}\text{ is the copy of } u \text{ in } B_1\}$ and $E_3=\{(\hat{v},v)\mid v\in B_0,\hat{v}\text{ is the copy of } v \text{ in } R_1\}$. The weight of each edge $(u,v)\in E_0$ is the distance between the points $u$ and $v$. The weight of each edge in $E_1$ is 0. The weight of any edge $(u,\hat{u})\in E_2$ is the distance between $u\in S$ and its closest neighbor in $T$. The weight of any edge $(\hat{v},v)\in E_3$ is the distance between $v\in T$ and its closest neighbor in $S$. See Figure \ref{fig:graph} for an example. Note that $G$ can be represented implicitly in $O(n)$ space, where $|S\cup T|=n$. Let $I$ be the constructed instance of bipartite perfect matching that consists of $G$. As the closest neighbors of $n$ points in the plane can be found in total $O(n\log n)$ time using Voronoi diagram (see Section  \ref{sec:2approx}),
construction of $G$ takes $O(n\log n)$ time.

Now, suppose $I'$ has a many-to-many matching $M'$. Consider the pairs in $M'$ as the edges of a graph $G'$ with vertices being the points in $S\cup T$. We will use the term pairs and edges in $G'$ interchangeably. First, note that wlog we can assume that $G'$ does not contain any path of length 3. Otherwise, we can remove the middle edge of such a path from $M'$, and $M'$ still remains a many-to-many matching. Thus, each component in $G'$ is a star. We compute a perfect matching in $G$ from $M'$ as follows. For each star in $G'$ having only one edge $(u,v)$ with $u\in S, v\in T$, add $(u,v)\in E_0$ to $M$. Also, add the edge $(\hat{v},\hat{u})\in E_1$ to $M$. Now, consider any star $H=\{(u,v_1),(u,v_2),\ldots,(u,v_t)\}$ in $G'$; wlog assume that $u\in S$. Add $(u,v_1)\in E_0$ to $M$. Also, add the edge $(\hat{v_1},\hat{u})\in E_1$ to $M$. For each $2\le i\le t$, add the edge $(\hat{v_i},v_i)\in E_3$ to $M$. It is not hard to verify that all the vertices of $G$ are matched in $M$. Also, as the weight of a star edge $(u,v_i)$ above is at least the weight of $(\hat{v_i},v_i)$ in $G$ by definition, the cost of $M$ is at most the cost of $M'$.

Next, suppose $I$ has a perfect matching $M$; we construct a many-to-many matching $M'$ for $I'$. Consider any $u\in S=R_0$. If $(u,\hat{u})\in M$, add the pair $(u,u')$ to $M'$, where $u'$ is the closest neighbor of $u$ in $T$. Otherwise, $u$ is matched (in $M$) to some $v_1\in B_0$. In this case, simply add the pair $(u,v_1)$ to $M'$. Similarly, consider any $v\in T=B_0$. If $(\hat{v},v)\in M$, add the pair $(v',v)$ to $M'$, where $v'$ is the closest neighbor of $v$ in $S$. Otherwise, $v$ is matched (in $M$) to some $u_1\in R_0$. In this case, simply add the pair $(u_1,v)$ to $M'$. It is not hard to verify that $M'$ is a many-to-many matching, and the cost of $M'$ is same as the cost of $M$.
\end{proof}

Now, consider any matching in a graph. An \emph{alternating} path is a path whose edges alternate between matched and unmatched edges. Similarly, one can define \emph{alternating} cycles and trees. A vertex is called \emph{free} if it is not matched. An \emph{augmenting} path is an alternating path which starts and ends at free vertices. Given an augmenting path $P$ w.r.t. a matching $M$, we can augment $M$ by one edge if we remove the edges of $P\cap M$ from $M$ and add the edges in $P\setminus M$ to $M$. The new matching is denoted by $M \oplus P$. Throughout the paper, $m$ and $n$ denote the number of edges and vertices, respectively, unless otherwise specified. We denote the weight or cost of an edge $(u,v)$ by $c(u,v)$. In our discussions, a path can be treated as an ordered set of vertices or edges depending on the context.

\section{An Exact Algorithm}
\label{sec:exact}

Consider the instance $I$ obtained by the reduction in Lemma \ref{lem:equiv-many-perfect}. In this section, we prove the following theorem.

\begin{theorem}\label{thm:exact-perfect}
Bipartite perfect matching can be solved exactly on $I$ in time $O(n^{2}\cdot poly(\log n))$, and hence there is an $O(n^{2}\cdot poly(\log n))$ time exact algorithm for minimum-cost many-to-many matching.
\end{theorem}

In the rest of this section, we prove Theorem \ref{thm:exact-perfect}. To prove this theorem we use the Hungarian algorithm \cite{kuhn1956variants} for computing a (minimum-cost) bipartite perfect matching.

In the Hungarian algorithm, there is a dual variable $y(v)$ corresponding to each vertex $v$. Every feasible matching $M$ must satisfy the following two conditions.
\begin{align}
    y(u)+y(v) & \le c(u,v) \text{ for every edge } (u,v)\label{eq:1}\\
    y(u)+y(v) &= c(u,v) \text{ for every edge } (u,v) \in M\label{eq:2}
\end{align}
Given any matching $M$ and an augmenting path $P$, the net-cost or augmentation cost is defined as,
\begin{align*}
    \phi(P) = \sum_{(u,v) \in P\setminus M} c(u,v)-\sum_{(u,v) \in P\cap M} c(u,v)
\end{align*}
$\phi(P)$ is basically the cost increment for augmenting $M$ along $P$. The net-cost of any alternating cycle can be defined in the same way. The Hungarian algorithm starts with an empty matching $M$. In every iteration, it computes an augmenting path of the minimum net-cost and augments the current matching. The algorithm halts once a perfect matching is found. If there is a perfect matching in the graph, this algorithm returns one after at most $n$ iterations. Moreover, an augmenting path can be found in $O(m)$ time leading to $O(mn)$ running time in total.

It is possible to show that any perfect matching is a minimum-cost matching if and only if there is no negative net-cost alternating cycle with respect to it. Moreover, a feasible matching with dual values $\{y(v)\}$ satisfies this property. Thus, it is sufficient to find a perfect matching that is feasible.

For finding an augmenting path, a Hungarian search procedure is employed. Hungarian search uses Dijkstra's shortest path algorithm to find a minimum net-cost path in the graph where the value $y(u)+y(v)$ is subtracted from the weight of each edge $(u,v)$. This along with the first feasibility condition ensure that each edge has a non-negative weight, and hence there is no negative cycle in the graph. So, one can correctly employ Dijkstra's algorithm to find such a shortest path. Finally, one can show that the minimum net-cost augmenting path with original weights corresponds to a shortest path with the modified weights, and vice versa. After augmenting the current matching with the newly found path, the dual values are adjusted appropriately to ensure feasibility of the new matching.

Now, we describe in detail how the Hungarian search procedure is implemented in each iteration. An edge $(u,v)$ is called \emph{admissible} if $y(u)+y(v)=c(u,v)$. It can be shown that it suffices to find an augmenting path consisting of only admissible  edges. Let $M$ be the current matching and $F$ be the free vertices of $R$. To obtain the desired augmenting path, a forest $\mathcal{F}$ is grown whose roots are in $F$. Each tree in $\mathcal{F}$ is an augmenting tree rooted at a vertex in $F$. Once $\mathcal{F}$ contains an augmenting path the search is completed. At any moment, let $R'$ be the vertices of $R$ in $\mathcal{F}$ and $B'$ be the vertices of $B$ not in $\mathcal{F}$. Initially, $R'=F$ and $B'=B$. Also, let \[\delta=\min_{u\in R',v\in B'} \{c(u,v)-y(u)-y(v)\}.\] Note that $\delta=0$ means there is an admissible edge. In each step, if $\delta=0$, an admissible edge $(u,v)$ is selected where $u\in R'$ and $v\in B'$. If $v$ is free, $(u,v)$ is added to $\mathcal{F}$ and the desired augmenting path is found. Otherwise, let $v$ be matched to $u'$ (which is not in $\mathcal{F}$ by an invariant). In this case, the edges $(u,v)$ and $(v,u')$ are added to $\mathcal{F}$; $u'$ is added to $R'$ and $v$ is removed from $B'$.

If at some moment, $\delta$ becomes more than 0 (no admissible edge), we perform dual adjustments. In particular, for each $u\in R'$, $y(u)$ is updated to $y(u)+\delta$ and for each $v\in \mathcal{F}\cap B$, $y(v)$ is updated to $y(v)-\delta$. This ensures that at least one edge becomes admissible, e.g., the edge corresponding to which the $\delta$ value is achieved. Thus, eventually the search halts with an augmenting path in $\mathcal{F}$.

It can be shown  that if $\delta$ can be computed efficiently, then the desired augmenting path can also be found efficiently \cite{gabow1989faster,AgarwalES99,vaidya1989geometry}. For that purpose, another variable $\Delta$ is maintained. Also, for each vertex $v$, a weight $\sigma_v$ is stored. In the beginning of each step, $\Delta=0$, $\sigma_v=y(v)$. When the edges $(u,v)$ and $(v,u')$ are added to $\mathcal{F}$, $\sigma_{u'}$ is updated to $y(u')-\Delta$ and $\sigma_{v}$ is updated to $y(v)+\Delta$. Once $\delta$ becomes more than 0, $\Delta$ is updated to $\Delta+\delta$.

Note that the weight of a vertex is updated only once when it is added to $\mathcal{F}$. We have the following observation.

\begin{observation}
\cite{gabow1989faster,AgarwalES99,vaidya1989geometry} For each $u\in R'$, the current dual value of $u$, $y(u)$, is equal to $\sigma_u+\Delta$. For each $v\in \mathcal{F}\cap B$, the current dual value of $v$, $y(v)$, is equal to $\sigma_v-\Delta$.
\end{observation}

It follows from the above observation that $\delta$ can be equivalently expressed as follows. \[\delta=\min_{u\in R',v\in B'} \{c(u,v)-\sigma_u-\sigma_v\}-\Delta. \]

Hence, Hungarian search boils down to the following task ignoring the trivial details. We need to maintain two sets $R'\subseteq R$ and $B'\subseteq B$. Initially, $B'=B$. In each step, a vertex $r$ is added to $R'$ and a vertex $b$ is removed from $B'$. Additionally, each vertex $v$ has a weight $\sigma_v$. In every step, the goal is to maintain the bichromatic closest pair, which is the pair $(r,b)\in R'\times B'$ with the minimum $c(r,b)-\sigma_r-\sigma_b$ value. In the following, we construct a data structure that can be used to perform the above task (a Hungarian search) in $O(n\cdot poly(\log n))$ time. As we need at most $n$ such searches, Theorem \ref{thm:exact-perfect} follows.

Recall that in our instance $I$, we are given the graph $G=(R=R_0\cup R_1,B=B_0\cup B_1,E)$, where $R_1$ contains copies of the vertices in $B_0$, $B_1$ contains copies of the vertices in $R_0$. $E=E_0\cup E_1\cup E_2\cup E_3$, where $E_0=R_0\times B_0$, $E_1=R_1\times B_1$, $E_2=\{(u,\hat{u})\mid u\in R_0,\hat{u}\in B_1\}$, and $E_3=\{(\hat{v},v)\mid v\in B_0,\hat{v}\in R_1\}$. Let $n=|R|=|B|$.

Our data structure $\mathcal{D}$ is a collection of three data structures. First, we construct the dynamic bichromatic closest pair data structure from \cite{kaplan2020dynamic} for the two point sets $R_0$ and $B_0$ with the distance function $c(r,b)-\sigma_r-\sigma_b$ for each pair $(r,b)$. We refer to this data structure as $\mathcal{D}_1$. Initially, it contains only the points in $R'\cup B'$. Next, we construct two max-heaps $H_1^r$ and $H_1^b$ for the vertices in $R_1\cap R'$ and $B_1\cap B'$, respectively. Initially, $H_1^r$ contains vertices in $R'$ and $H_1^b$ contains all the vertices in $B_1$. The key value of each vertex $v$ is its weight $\sigma_v$. Using $H_1^r$ and $H_1^b$, the pair $(u,v)\in (R_1\cap R')\times (B_1\cap B')$ with the maximum $\sigma_u+\sigma_v$ value can be found in $O(1)$ time.
We also construct another min-heap $H_{23}$ to store the edges in $E_2\cup E_3$ with key value $c(r,b)-\sigma_r-\sigma_b$ for each pair $(r,b)$. Initially, it is empty. In every iteration, it contains only those edges $(u,v)$ such that $u\in R'$ and $v\in B'$. We also maintain a global closest pair $(r,b)$ over the three data structures $\mathcal{D}_1$, $H_1^r\cup H_1^b$ and $H_{23}$ with the minimum key value $c(r,b)-\sigma_r-\sigma_b$.

Using $\mathcal{D}$, we implement each step as follows. If $v\in B_0$, remove $v$ from $\mathcal{D}_1$. Also remove $(\hat{v},v)$ from $H_{23}$ if it is in $H_{23}$. If $v\in B_1$, remove $v$ from $H_1^b$. Also remove the edge $(u',v)$ with $u'\in R_0$ from $H_{23}$ if its in $H_{23}$. If $u\in R_0$, add $u$ to $\mathcal{D}_1$. Also add the edge $(u,\hat{u})$ to $H_{23}$ if $\hat{u}\in B'$. If $u\in R_1$, add $u$ to $H_1^r$. Also add the edge $(u,v')$ with $v'\in B_0$ to $H_{23}$ if $v'\in B'$.

It is not hard to verify that at each step the correct global closest pair is stored in $\mathcal{D}$. The correctness for  $\mathcal{D}_1$ and $H_{23}$ follow trivially. Also, as the weight of the edges in $E_1$ are same, it is sufficient to find a pair $(u,v)\in E_1$ for which $\sigma_u+\sigma_v$ value is maximized. As $E_1$ contains all the edges in $R_1\times B_1$, equivalently it suffices to find a $u\in R_1$ with the maximum $\sigma_u$ value and a $v\in B_1$ with the maximum $\sigma_v$ value.

We note that the total number of steps in a Hungarian search is at most $n$. Also, $O(n)$ operations (insertions, deletions and searching) on $\mathcal{D}_1$ can be performed in amortized $O(n\cdot poly(\log n))$ time \cite{kaplan2020dynamic}. The construction time and space of $\mathcal{D}_1$ are also $O(n\cdot poly(\log n))$. Moreover, the $O(n)$ operations on all max-heaps can be performed in total $O(n\log n)$ time. It follows that Hungarian search can be implemented in $O(n\cdot poly(\log n))$ time leading to the desired running time of our algorithm.

\section{A 2-approximation in $O(n\log n)$ Time}
\label{sec:2approx}

In this section, we prove the following theorem.

\begin{theorem}\label{thm:2approx}
There is an $O(n\log n)$ time 2-approximation algorithm for minimum-cost many-to-many matching.
\end{theorem}

Our algorithm is as follows. For each point $u\in S$, we add the pair $(u,q)$ to the solution matching $M$ which minimizes $d(u,q)$ over all $q\in T$. Similarly, for each point $v\in T$, we add the pair $(p,v)$ to $M$ which minimizes $d(p,v)$ over all $p\in S$. Thus, for each point in a set, we basically add the pair corresponding to its nearest neighbor in the other set. By definition, the computed solution $M$ is a many-to-many matching. Next, we argue how to implement this algorithm in $O(n\log n)$ time. First, we compute the Voronoi diagram of the points in $T$, compute a triangulation of the Voronoi cells and then construct Kirkpatrick's planar point location data structure \cite{Kirkpatrick83} using the triangulation. Using this data structure, for any $u\in S$, its nearest neighbor in $T$ can be computed in $O(\log n)$ time. Also, the data structure uses $O(n)$ space and $O(n\log n)$ construction time including the preprocessing time. Using a similar data structure for points in $S$, we can compute the nearest neighbors of the points in $T$ in $O(n\log n)$ time. The following lemma completes the proof of Theorem \ref{thm:2approx}.

\begin{lemma}
The cost of the matching $M$ computed as in above is at most twice the optimal cost.
\end{lemma}

\begin{proof}
For each point $p$ in $S$ (resp. in $T$), let $\eta(p)$ be its nearest neighbor in $T$ (resp. in $S$). Note that the cost of $M$, cost$(M)$ $=\sum_{p\in S\cup T} d(p,\eta(p))$. Now, consider any optimal matching $M^*$. For each $p \in S\cup T$, let $c^*(p)$ be the cost of any pair $(p,q^*)$ in $M^*$, i.e., a pair containing $p$. Then, $c^*(p)$ must be at least the distance between $p$ and its nearest neighbor in the other set, i.e.,  $c^*(p)\ge d(p,\eta(p))$. It follows that, the cost of $M$,  \[\sum_{p\in S\cup T} d(p,\eta(p))\le \sum_{p\in S\cup T} c^*(p)=\sum_{p_1\in S} c^*(p_1)+\sum_{p_2\in T} c^*(p_2)\le \text{cost}(M^*)+\text{cost}(M^*)=2\cdot \text{cost}(M^*).\]
\end{proof}

\section{An Improved $(1+\epsilon)$-approximation}
Consider the instance $I$ obtained by the reduction in Lemma \ref{lem:equiv-many-perfect}. In this section, we prove the following theorem.

\begin{theorem}\label{thm:ptas-perfect}
A $(1+\epsilon)$-approximate bipartite perfect matching for $I$ can be computed in time $O((1/\epsilon^c)\cdot n^{3/2}\cdot poly(\log n))$ for some constant $c$, and hence there is an $O((1/\epsilon^c)\cdot n^{3/2}\cdot poly(\log n))$ time $(1+\epsilon)$-approximation algorithm for minimum-cost many-to-many matching.
\end{theorem}

In the rest of this section, we prove Theorem \ref{thm:ptas-perfect}. In particular, we show that the bipartite perfect matching algorithm by Gabow and Tarjan \cite{gabow1989faster} can be implemented on the instance $I$ in the mentioned time.

The Gabow-Tarjan algorithm is based on a popular scheme called the bit-scaling paradigm. The algorithm is motivated by two classic matching algorithms: Hopcroft-Karp \cite{hopcroft1973n} for maximum cardinality bipartite matching with $O(m\sqrt{n})$ running time and Hungarian algorithm \cite{kuhn1956variants} for minimum-cost bipartite matching with $O(mn)$ running time. The Hopcroft-Karp algorithm chooses an augmenting path of the shortest length. The Hungarian algorithm, as mentioned before, chooses an augmenting path whose augmentation cost is the minimum. When the weights on the edges are small an augmenting path of the shortest length approximates the latter path. The Gabow-Tarjan algorithm scales the weights in a manner so that all the effective weights are small. This helps to combine the ideas of the two algorithms, which leads towards an $O(m\sqrt{n}\log (n N))$ time algorithm for (minimum-cost) bipartite perfect matching, where $N$ is the largest edge weight.

Next, we describe the Gabow-Tarjan algorithm. This algorithm is based on the ideas of the Hungarian algorithm. However, here instead of a feasible matching we compute a \emph{1-feasible} matching. A matching $M$ is called 1-feasible if it satisfies the following two conditions.
\begin{align}
    y(u)+y(v) & \le c(u,v)+1 \text{ for every edge } (u,v)\label{eq:3}\\
    y(u)+y(v) &= c(u,v) \text{ for every edge } (u,v) \in M\label{eq:4}
\end{align}
Note that the only difference is that now the sum of dual variables $y(u)+y(v)$ can be 1 plus the cost of the edge. This additive error of 1 on every unmatched edge ensures that longer augmenting paths have larger cost. As an effect, the algorithm picks short augmenting paths as in the Hopcroft-Karp algorithm.

A \emph{1-optimal} matching is a perfect 1-feasible matching. Note that a 1-optimal matching costs more than the original optimal matching. However, as the error is at most +1 for every edge, one can show the following.

\begin{lemma}
\cite{gabow1989faster} Let $M$ be a 1-optimal matching and $M'$ be any perfect matching. Then $c(M')\ge c(M)-n$.
\end{lemma}

It follows from the above lemma that, to annihilate the error introduced, one can scale the weight of each edge by a factor of $(n+1)$, and then with the scaled weights the cost of a 1-optimal matching is same as the cost of any optimal matching. Let $\overline{c}(u,v)$ be the scaled cost of $(u,v)$, i.e., $\overline{c}(u,v)=(n+1)\cdot {c}(u,v)$. Let $k=\lfloor\log ((n+1) N)\rfloor+1$ be the maximum number of bits needed to represent any new weight.

The Gabow-Tarjan algorithm runs in $k$ different scales. In each scale $i$ ($1\le i\le k$), the most significant $i$ bits of $\overline{c}(u,v)$ are used for defining the current cost of each edge $(u,v)$. The dual values are also modified to maintain 1-feasibility of an already computed perfect matching in the following way: $y(v)\leftarrow 2y(v)-1$ for every vertex $v$. Then with the current edge costs and dual values, the algorithm computes a 1-optimal matching.

By the above claim, that any 1-optimal matching is also optimal with scaling factor $n+1$, the 1-optimal matching computed by the algorithm at $k$-th scale must be optimal.

To find a 1-optimal matching on a particular scale a procedure called \emph{match} is employed which we describe below. Before that we need a definition. Consider any 1-feasible matching $M$. An edge $(u,v)$ is called \emph{eligible} if it is in $M$ or $y(u)+y(v)=c(u,v)+1$. It can be shown that for the purpose of computing a 1-optimal matching it suffices to consider the augmenting paths which consist of eligible edges only.

\vspace{2mm}
\noindent\fbox{
\parbox{\textwidth}{
\noindent \textbf{The} \emph{match} \textbf{procedure}

\noindent Initialize all the dual variables $y(v)$ to 0 and $M$ to $\emptyset$. Repeat the following two steps until a perfect matching is obtained in step 1.
\begin{enumerate}
    \item Find a maximal set $\mathcal{A}$ of augmenting paths of eligible edges. For each path $P\in \mathcal{A}$, augment the current matching $M$ along $P$ to obtain a new matching which is also denoted by $M$. For each vertex $v\in P\cap B$, decrease $y(v)$ by 1. (This is to ensure that the new matching $M$ also is 1-feasible.) If $M$ is perfect, terminate.
    \item Employ a Hungarian search to adjust the values of the dual variables (by keeping $M$ 1-feasible), and find an augmenting path of eligible edges.
\end{enumerate}
}}
\vspace{2mm}

Note that the number of free vertices in every iteration of \emph{match} is at least 1 less than that in the previous iteration, as step 2 always ends with finding an augmenting path. By also showing that the dual value of a variable is increased by at least 1 in every call of Hungarian search, they proved that $O(\sqrt{n})$ iterations are sufficient to obtain a 1-optimal matching. It can be shown that each iteration of \emph{match} can be executed in general bipartite graphs in $O(m)$ time leading to the complexity of $O(m\sqrt{n})$ in each scale. As we have $O(\log (nN))$ scales, in total the running time is $O(m\sqrt{n}\log (nN))$.

Next, we use the Gabow-Tarjan algorithm to compute a perfect matching for our instance of bipartite perfect matching. In particular, we show that by exploiting the structure of our instance, it is possible to implement every iteration of \emph{match} in $O(n \cdot poly(\log n))$ time.
First, we show that one can consider a modified instance with bounded aspect ratio of the weights, for the purpose of computing a $(1+\epsilon)$-approximation.

Let OPT be the optimal cost of perfect matching on the instance $I$. Recall that the instance $I$ consists of the graph $G=(R=R_0\cup R_1,B=B_0\cup B_1,E)$. Given the implicit representation of $G$, we compute a 2-approximate solution for minimum-cost many-to-many matching on the two sets of points $R_0$ and $B_0$ using the algorithm in Theorem \ref{thm:2approx}. Let $C$ be the cost of this solution. By Lemma \ref{lem:equiv-many-perfect}, $\text{OPT}\le C\le 2\text{OPT}$. We construct a new instance $I_1$ which consists of the implicit representation of the same graph $G$. Additionally, we assume that any edge with weight more than $C$ in $I_1$ will not be part of the solution, and each edge has cost at least  $\epsilon C/(2n)$. Note that it is not possible to explicitly set the weight of each and every edge if we are allowed to spend $o(n^2)$ time. Thus, for the time being, we make the above assumptions implicitly. Later, we will make them explicit in our algorithm. Note that the construction time of $I_1$ is dominated by the time of the 2-approximation algorithm, which is $O(n\log n)$. We obtain the following lemma.

\begin{restatable}{lemma}{boundedaspectratio}\label{lem:bounded-aspect-ratio}
Given $I$, one can compute in $O(n\log n)$ time another instance $I_1$ of bipartite perfect matching, such that (i) the weight of every edge is in $[\epsilon C/(2n),C]$ where $\text{\emph{OPT}}\le C\le 2\text{\emph{OPT}}$, (ii) $I_1$ has a perfect matching of weight at most $(1+\epsilon) \text{\emph{OPT}}$, and (iii) any perfect matching in $I_1$ of weight $C'$ is also a perfect matching in $I$ of weight at most $C'$.
\end{restatable}

\begin{proof}
(i) follows by the above construction of $I_1$. (ii) follows from the facts that OPT does not contain any edge of weight more than $C$ and the weight of an edge in OPT is increased by at most $\epsilon C/(2n)\le \epsilon \text{OPT}/n$ in $I_1$. (iii) follows, as each edge in $I_1$ is also present in $I$ with possibly equal or lesser cost.
\end{proof}

Henceforth, we solve the problem on $I_1$. Note that the minimum edge weight in $I_1$ is $\epsilon C/(2n)$ and the maximum is $C$. By scaling the weights by $2n/(\epsilon C)$, we can assume wlog that the edge weights in $I_1$ are in $[1,n^2]$. Moreover, we can assume that each edge weight $w$ is rounded up to the nearest integer at least $w$. We can afford to remove the fractions, as each fraction costs less than 1, which is at most $\epsilon \text{OPT}/n$ w.r.t. the original weights. For our convenience, we also divide the weights into $O(\log_{1+\epsilon} n)$ classes as follows. For each weight $a$ (an integer) with $(1+\epsilon)^i \le a < (1+\epsilon)^{i+1}$, $a$ is rounded to the largest integer in the range $[(1+\epsilon)^i,(1+\epsilon)^{i+1})$, where $0\le i\le \lceil 2\log_{1+\epsilon} n\rceil$. We denote this largest integer corresponding to the $i$-th weight class by $w_i$. We note that the above weight scalings are performed implicitly. It is not hard to verify that these still preserve a $(1+O(\epsilon))$-approximate solution, which is sufficient for our purpose. Henceforth, we treat $\epsilon$ as a constant and hide function of $\epsilon$ in time complexity as a constant in $O()$ notation. It will not be hard to verify that the dependency on $\epsilon$ that we hide is $(1/\epsilon)^c$ for some true small constant $c$.

To implement the Gabow-Tarjan algorithm, we show how the \emph{match} procedure can be implemented efficiently. To implement step 1 of \emph{match}, we store the information about the input graph $G$ in a data structure that we refer to as  MATCH. We allow the following operation on MATCH. In the following, we denote the current matching by $M$.

\vspace{-4mm}
\subparagraph*{FIND$\_$MAXIMAL$\_$APS.} Find a maximal set of vertex disjoint augmenting paths of eligible edges with respect to $M$.

Given the MATCH data structure, we implement the step 1 of \emph{match} as follows. The dual values are stored in an array indexed by the vertices. Note that the dual values remain fixed in step 1 while the augmenting paths are found. Afterwards, the dual values are updated.  We first make a call to FIND$\_$MAXIMAL$\_$APS to obtain a maximal set $\mathcal{A}$ of paths. For each $P\in \mathcal{A}$, we augment $M$ along $P$ to obtain a new matching $M$. Also, for each $v\in P\cap B$, we decrease $y(v)$ by 1.

In Section \ref{sec:DS}, we show how to construct and maintain {MATCH} so that the above subroutine can be performed in time $O(n \cdot poly(\log n))$. The building time of {MATCH} is $O(n \cdot poly(\log n))$, and it takes $O(n \cdot poly(\log n))$ space. Thus, by noting that $\mathcal{A}$ contains disjoint paths, step 1 can be implemented in $O(n \cdot poly(\log n))$ time and space.

We also need another data structure which will help us implement step 2 of \emph{match}, which we refer to as the Hungarian search data structure. As described before,
Hungarian search boils down to
maintaining a bichromatic closest pair $(r,b)$ of two sets
with the minimum $c(r,b)-\sigma_r-\sigma_b$ value.
However, here we have to be more careful, as for an unmatched eligible edge $(u,v)$, $y(u)+y(v)=c(u,v)+1$. In contrast, in the Hungarian algorithm, we had $y(u)+y(v)=c(u,v)$ in that case. Hence, we have to consider the $c(u,v)+1-y(u)-y(v)$ value as the distance of such an unmatched pair $(u,v)$. This apparently makes our life harder, as now we have to deal with two types of distance functions: one for matched pairs and one for unmatched pairs. However, we use the following observation to consider only one type of distances, which follows from our description of Hungarian search in Section \ref{sec:exact}.

\begin{observation}
Consider any matched edge $(u,v)$ with $u\in R$ and $v\in B$. In the Hungarian search, if $u\in R'$, i.e., $u$ is already in the forest, then $v$ must also be in the forest, i.e., $v\notin B'$.
\end{observation}

Recall that for computing $\delta$, we look into the pairs $(u,v)$ where $u\in R'$ and $v\in B'$. By the above observation, it suffices to probe only unmatched edges. Hence, we can again work with only one distance function $c(r,b)+1-\sigma_r-\sigma_b$ for the purpose of computing $\delta$. As the $+1$ term is common in all distances, we also drop that, and work with our old distance function.

In Section \ref{sec:DS}, we show how to construct and maintain Hungarian search data structure so that the task of maintaining closest pair can be performed  in total $O(n \cdot poly(\log n))$ time. Moreover, the data structure uses $O(n \cdot poly(\log n))$ construction time and space.

\begin{lemma}
Using the MATCH and Hungarian search data structures, one can implement the Gabow-Tarjan algorithm on the instance $I_1$ in time $O(n^{3/2}\cdot poly(\log n))$, i.e., a minimum cost bipartite perfect matching in $I_1$ can be computed in time $O(n^{3/2}\cdot poly(\log n))$.
\end{lemma}

\section{Data Structures}
\label{sec:DS}
\subsection{The MATCH Data Structure}
We would like to construct a data structure where given a matching at a fixed scale, a maximal set of augmenting paths can be computed efficiently. In particular, we are given the graph $G=(R=R_0\cup R_1,B=B_0\cup B_1,E)$, where $R_1$ contains copies of the vertices in $B_0$, $B_1$ contains copies of the vertices in $R_0$. $E=E_0\cup E_1\cup E_2\cup E_3$, where $E_0=R_0\times B_0$, $E_1=R_1\times B_1$, $E_2=\{(u,\hat{u})\mid u\in R_0,\hat{u}\in B_1\}$, and $E_3=\{(\hat{v},v)\mid v\in B_0,\hat{v}\in R_1\}$. Let $n=|R|=|B|$.
Also, note that each edge weight is an integer $w_i$ for $0\le i\le \lceil (c'\log n)/\epsilon\rceil$, where $c'$ is a constant. Let $E_i^0$ be the set of edges in $R_0\times B_0$ with weights $w_i$. We define a bi-clique cover for each $E_i^0$ as a collection $\mathcal{C}_i=\{(P_{i1},Q_{i1}),(P_{i2},Q_{i2}),\ldots,(P_{it(i)},Q_{it(i)})\}$ where $P_{ij}\subseteq R_0$, $Q_{ij}\subseteq B_0$, all the edges in $P_{ij}\times Q_{ij}$ are in $E_i^0$ and $\cup_{j=1}^{t(i)} P_{ij}\times Q_{ij} = E_i^0$. The size of $E_i^0$ is  $\sum_{j=1}^{t(i)} |P_{ij}|+|Q_{ij}|$. Given the points in $R_0$ and $B_0$, using standard range searching data structures, one can compute such a bi-clique cover of size $O((n/{\epsilon})\log^2 n)$ in $O((n/{\epsilon})\log^2 n)$ time. We note that bi-clique covers are also used for the algorithm in \cite{varadarajan1999approximation}. Let $\mathcal{C}=\mathcal{C}_i$. Thus $\mathcal{C}$ can be computed in  $O((n/{\epsilon}^2)\log^3 n)$ time and space.

In MATCH we store the bi-clique covers in $\mathcal{C}$ corresponding to the edges in $E_0$. Also, we store the edges in $E_2$ along with their weights in an array $A_2$ indexed by the vertices of $R_0$ and the edges in $E_3$ and their weights in an array $A_3$ indexed by the vertices of $B_0$. For finding an augmenting path efficiently, we need to store additional information. Before describing that, we describe in more detail how the augmenting paths are found.

The maximal set of augmenting paths are found by a careful implementation of depth first search. In this implementation, vertices can be labeled as marked. Initially all vertices are unmarked. We select any free unmarked vertex of $R$ and initialize a path $P$ at that vertex. $P$ is extended from the last vertex $u$ (in $R$ as an invariant) as follows. We probe an eligible edge $(u,v)$. If $v$ is already marked, the next eligible edge is considered. If no such edge exists, the last two edges (one unmatched and one matched) are deleted from $P$. If $P$ becomes empty, a new path is initialized. For the remaining cases, the following subroutine is called.
\vspace{-2mm}

\subparagraph*{AUGMENTING$\_$PATH$(v)$.} If $v$ is unmarked and free, we have found an augmenting path; $v$ is marked, $P$ is added to $\mathcal{A}$ and a new path is initialized. In this case, return DONE. If $v$ is unmarked, but matched with another vertex $w$, $(u,v)$ and $(v,w)$ are added to $P$; $v,w$ are marked and the extension continues from $w$ (in $R$).

\begin{restatable}{observation}{unmatchedonly}\label{obs:unmatched_only}
In the above procedure, we always maintain the invariant that when we extend a path it suffices to probe only unmatched edges.
\end{restatable}
\begin{proof}
\begin{claim}
In AUGMENTING$\_$PATH$(v)$, we always maintain the invariants that when we extend a path $P$ from the last vertex $u$, (i) $u\in R$ and (ii) if $u$ is not the first vertex of $P$, the last edge $(v,u)\in P$ is a matched edge.
\end{claim}

The above claim helps us restrict the search from a vertex of $R$. Now, whenever we extend the path $P$ from the last vertex $u$, there are two cases. Either $P$ contains a single vertex $u$, and in this case $u$ is free; we probe an unmatched eligible edge $(u,v)$. Or, $P$ contains at least two edges, and in this case the last edge of $P$ is a matched edge by the above observation; we need to probe an unmatched eligible edge $(u,v)$.  Thus, it suffices to probe only unmatched edges for the purpose of extension of $P$.
\end{proof}

Note that in the above procedure we cannot afford to probe all the unmatched  edges. So, we have to implement the above step carefully. First, note that an edge $(u,v)$ ($u \in R$) is never scanned twice. When $(u,v)$ is probed the first time, if $v$ is unmarked, it becomes marked in all the cases. Also, once $v$ is marked, $(u,v)$ is never used to extend $P$. Thus, we can eliminate $(u,v)$ from further probing.

From the above discussion, as $E_2$ and $E_3$ contain $O(n)$ edges in total they can be probed in $O(n)$ time. However, $E_0$ and $E_1$ contain $\Omega(n^2)$ edges and thus for probing them we need specialized data structures. Next, we describe those.

Let $c(u,v)$ be the weight of $(u,v)$ at the current scale. $y(u)$ is the dual value of the vertex $u$ which remains fixed throughout the augmenting paths finding process. Again consider the bi-clique covers in $\mathcal{C}$. For each $0\le i\le \lceil (c'\log n)/\epsilon\rceil$, $w_i$ denotes the weight of the edges in $i$-th class. Also, let $\ell$ be the weight class to which the edges in $E_1$ belong, i.e., all of their weights are $w_{\ell}$. For each such $i$ and $1\le j\le t(i)$, we store in \emph{match} the vertices of $Q_{ij}$ in a Red-Black tree $T_{ij}$ with $w_i+1-y(v)$ as the key of each such vertex $v$. Moreover, for each $u\in R_0$, we keep an ordered set of indexes $I(u)=\{(i,j)\mid u\in P_{ij}\}$. Similarly, define the index set $I(v)$ for $v\in B_0$. We also store the vertices of $B_1$ in a Red-Black tree $T_1$ with $w_{\ell}+1-y(v)$ as the key of $v \in B_1$.

\begin{observation}
MATCH uses $O(n \cdot poly(\log n))$ construction time and space.
\end{observation}

The above space bound follows from the fact that the space complexity of MATCH is dominated by the space needed for the Red-Black trees, which is $O(|B_1|)+\sum_{(i,j)} O(|Q_{ij}|)=O((n/{\epsilon}^2)\log^3 n)$, as a Red-Black tree uses linear space. The time bound follows trivially.
\vspace{-4mm}

\subparagraph*{FIND$\_$MAXIMAL$\_$APS.} Let $F$ be the set of free vertices and $\Pi$ be the set of vertices that are already marked. Initially $\Pi=\emptyset$. Let $\mathcal{A}$ be the set of augmenting paths found so far, which is initialized to $\emptyset$. For each vertex $u\in R_1$, set its $E_1$-failed flag to 0. While there is a vertex $r_1\in (R\cap F)\setminus \Pi$, do the following.
\begin{itemize}
    \item Initialize a path $P$ at $r_1$.
    \item While $P$ is not empty, do the following.
    \begin{itemize}
        \item Let $P=\{r_1,b_1,\ldots,r_{\tau-1},b_{\tau-1},r_{\tau}\}$ be the current augmenting path that we need to extend.
        \item (Case 1. $r_{\tau}\in R_0$) Access the array $A_2$ to find whether the copy of $r_{\tau}$ in $B_1$, i.e., $\hat{r}_{\tau}$, is marked and $(r_{\tau},\hat{r}_{\tau})$ is eligible.
        \begin{itemize}
            \item If $\hat{r}_{\tau}$ is unmarked and $(r_{\tau},\hat{r}_{\tau})$ is eligible, call the subroutine AUGMENTING$\_$PATH ($\hat{r}_{\tau}$). If this subroutine returns DONE, terminate this while loop. Otherwise, jump to the next iteration.
            \item Otherwise, search the Red-Black tree $T_{ij}$ where $(i,j)$ is the first index in $I(r_{\tau})$, to find a vertex $b_{\tau}$ with key value $y(r_{\tau})$. If such a vertex $b_{\tau}$ is found, call the subroutine AUGMENTING$\_$PATH($b_{\tau}$). Remove $b_{\tau}$ from all the Red-Black trees with indexes in $I(b_{\tau})$, as it is marked in the subroutine. If this subroutine returns DONE, terminate this while loop. Otherwise, jump to the next iteration. If no such vertex $b_{\tau}$ is found in $T_{ij}$, remove the index $(i,j)$ from $I(r_{\tau})$ and repeat the above step (performed for $(i,j)$) for the next index in  $I(r_{\tau})$. If $I(r_{\tau})$ becomes empty, remove $r_{\tau}$ and $b_{\tau-1}$ from $P$, and continue to the next iteration.
        \end{itemize}

        \item (Case 2. $r_{\tau}\in R_1$) Access the array $A_3$ to find whether the original copy of $r_{\tau}$ in $B_0$, say $v$, is marked and $(r_{\tau},v)$ is eligible. If $v$ is unmarked and $(r_{\tau},v)$ is eligible, call the subroutine AUGMENTING$\_$PATH($v$). If this subroutine returns DONE, terminate this while loop. Otherwise, jump to the next iteration. If $E_1$-failed is not set, i.e., it is 0 for $r_{\tau}$, search the Red-Black tree $T_1$ to find a vertex $b_{\tau}$ with key value $y(r_{\tau})$. If such a vertex $b_{\tau}$ is found, call the subroutine AUGMENTING$\_$PATH($b_{\tau}$). Remove $b_{\tau}$ from $T_1$, as it is marked in the subroutine. If this subroutine returns DONE, terminate this while loop. Otherwise, jump to the next iteration. If no such vertex $b_{\tau}$ is found in $T_1$, set $E_1$-failed flag for $r_{\tau}$ to 1, remove $r_{\tau}$ and $b_{\tau-1}$ from $P$, and continue to the next iteration.
    \end{itemize}
\end{itemize}

The above procedure is self-explanatory. Next, we prove its correctness and bound the implementation time.

\begin{restatable}{lemma}{matchcorrectness}\label{lem:match-correctness}
FIND$\_$MAXIMAL$\_$APS correctly computes a maximal set of disjoint augmenting paths.
\end{restatable}

\begin{proof}
The proof of the lemma follows from the proof of correctness of the Gabow-Tarjan algorithm, assuming all eligible edges are considered if needed, while extending the path $P$ from $r_{\tau}$. Consider any eligible edge $e=(r_{\tau},v)$. Here we show that $e$ is considered for extension of $P$. If $v$ is marked, $v$ was already included at some augmenting path. As we are looking for disjoint paths, we do not need to consider the edge $(r_{\tau},v)$, and hence the correctness follows in this case. Thus, wlog, we can assume that $v$ is unmarked. First, assume $r_{\tau}\in R_0$. Then if $v\in B_1$, $e$ is explicitly considered by the subroutine. Otherwise, $v\in B_0$. In this case, suppose $(i,j)$ be an index such that $e\in (P_{ij},Q_{ij})$. Note that initially $(i,j)$ is in $I(r_{\tau})$. Thus, if needed, $T_{ij}$ can be searched, and as $v$ is unmarked, $v\in T_{ij}$. Now, as $e$ is eligible and unmatched (by Observation \ref{obs:unmatched_only}), $y(r_{\tau})+y(v)=w_{i}+1$. Thus, the key of $v$, $w_i+1-y(v)$ must be equal to $y(r_{\tau})$ and $v$ can be found eventually while searching $T_{ij}$. Now, assume $r_{\tau}\in R_1$. Then if $v\in B_0$, again $e$ is explicitly considered by the subroutine. Otherwise, $v\in B_1$. In this case, $v$ is in the tree $T_1$, as $v$ is unmarked. Now, as $e$ is eligible and unmatched (by Observation \ref{obs:unmatched_only}), $y(r_{\tau})+y(v)=w_{\ell}+1$. Thus, the key of $v$, $w_{\ell}+1-y(v)$ must be equal to $y(r_{\tau})$ and $v$ can be found eventually while searching $T_1$.
\end{proof}

\begin{restatable}{lemma}{matchtime}\label{lem:match-time}
FIND$\_$MAXIMAL$\_$APS can be implemented in $O(n\cdot poly(\log n))$ time.
\end{restatable}

\begin{proof}
For any vertex $u\in R$, let $L_u$ be the total time spent over the course of the procedure to extend paths that end at $u$, in particular, to find eligible edges of the form $(u,v')$ with $v'\in B$. Also, for $v\in B$, let $K_v$ be the total time spent to remove $v$ from the Red-Black tree(s) once it is marked. Note that the time of FIND$\_$MAXIMAL$\_$APS is dominated by the time $\sum_{u\in R} L_u+\sum_{v\in B} K_v$. Let us analyze $\sum_{u\in R} L_u$ at first. In the first case, $u\in R_0$. As mentioned before, probing of the edges in $E_2\cup E_3$ can be done in $O(n)$ time, using the arrays $A_2$ and $A_3$. So, let us focus on the remaining edges. Note that when we try to find an eligible edge for $u$, we search in $T_{ij}$, where $(i,j)\in I(u)$. Either no desired vertex $v$ is found, in which case, $(i,j)$ is removed from $I(u)$. Otherwise, a vertex $v\in Q_{ij}$ is found, but it is deleted from all Red-Black trees, and so it cannot appear in future search. Thus, removal of indexes from $I(u)$ and failed searches cost $O(| I(u)|\log n)$ time in total. The total number of vertices in $B_0$ that successfully appear in the result of the searches corresponding to all free vertices in $R_0$ is $\sum_{(i,j)} |Q_{ij}|$. Hence, \[\sum_{u\in R_0} L_u = \sum_{u\in R_0} O(| I(u)|\log n)+\sum_{(i,j)} O(|Q_{ij}|\log n)=\sum_{(i,j)} O((|P_{ij}|+|Q_{ij}|)\log n).\]
The last inequality follows, as the sum of the sizes of the index sets in $\{I(u)\mid u\in R_0\}$ is bounded by the sum of the sizes of the sets in $\{P_{ij}\}$.

Now, consider the case when $u\in R_1$. In this case, when we try to find an eligible edge $(u,v)$ for $u$ with $v\in B_1$, we search in $T_1$. Either no such $v$ is found, in which case, $E_1$-failed flag for $u$ is set to 1, and $T_1$ is never searched w.r.t. $u$ again. Or, such a vertex $v$ is found, in which case, it is deleted from $T_1$, and so it cannot appear in future search. Hence, \[\sum_{u\in R_1} L_u = O(|R_1|\log n)+O(|B_1|\log n)=O(n\log n).\]

For $v\in B_1$, $K_v$ is bounded by $O(\log n)$. For $v\in B_0$, $K_v$ is bounded by the time needed to delete $v$ from all $T_{ij}$ in all of which it appears. Thus, \[\sum_{v\in B} K_v=O(|B_1|\log n)+\sum_{(i,j)} O(|Q_{ij}|\log n).\]
As $\sum_{(i,j)} |P_{ij}|+|Q_{ij}|=O((n/{\epsilon}^2)\log^3 n)$, it follows that, \[\sum_{u\in R} L_u+\sum_{v\in B} K_v=O((n/{\epsilon}^2)\log^4  n).\] Hence, the lemma follows.
\end{proof}

\subsection{Hungarian Search Data Structure}
Recall that in Hungarian search, we need to maintain two sets $R'\subseteq R$ and $B'\subseteq B$. Initially, $R'=\emptyset$ and $B'=B$. In each iteration, a vertex is added to $R'$ and removed from $B'$. Additionally, each vertex $v$ has a weight $\sigma_v$. In every iteration, the goal is to maintain the bichromatic closest pair, which is the pair $(u,v)\in R'\times B'$ with the minimum $c(u,v)-\sigma_u-\sigma_v$ value.

Consider the bi-clique cover $\mathcal{C}$ and in particular a pair $(P_{ij},Q_{ij})$ in $\mathcal{C}$. Then any edge in $P_{ij}\times Q_{ij}$ has the same weight $w_i$. Thus, the pair $(u,v)$ with the maximum $\sigma_u+\sigma_v$ value has the minimum $c(u,v)-\sigma_u-\sigma_v$ value in $P_{ij}\times Q_{ij}$. In other words, it is sufficient to keep track of a vertex $u\in P_{ij}$ with the maximum $\sigma_u$ value and a vertex $v\in Q_{ij}$ with the maximum $\sigma_v$ value.

Now, we describe the Hungarian Search data structure. For each index $(i,j)$ in $\mathcal{C}$, we construct two max-heaps $H_{ij}^r$ and $H_{ij}^b$, for $P_{ij}\cap R'$ and $Q_{ij}\cap B'$, respectively. Initially, $H_{ij}^r$ is empty and $H_{ij}^b$ contains all the vertices in $Q_{ij}$. The key value of each vertex $v$ is its weight $\sigma_v$. Note that using $H_{ij}^r$ and $H_{ij}^b$, the pair $(u,v)\in (P_{ij}\cap R')\times (Q_{ij}\cap B')$ with the maximum $\sigma_u+\sigma_v$ value can be found in $O(1)$ time. Similarly, we construct two heaps $H_1^r$ and $H_1^b$ for the vertices in $R_1\cap R'$ and $B_1\cap B'$, respectively. Initially, $H_1^r$ is empty and $H_1^b$ contains all the vertices in $B_1$. The key value of each vertex $v$ is its weight $\sigma_v$. Again using $H_1^r$ and $H_1^b$, the pair $(u,v)\in (R_1\cap R')\times (B_1\cap B')$ with the maximum $\sigma_u+\sigma_v$ value can be found in $O(1)$ time. We also construct another max-heap $H_{23}$ to store the edges in $E_2\cup E_3$. Initially, it is empty. In every iteration, it contains only those edges $(u,v)$ such that $u\in R'$ and $v\in B'$. Finally, to keep track of the global maximum pair, we create another max-heap $H$. $H$ stores a maximum pair of $(P_{ij}\cap R')\times (Q_{ij}\cap B')$ for each index $(i,j)$ and maximum pairs of $(R_1\cap R')\times (B_1\cap B')$ and $H_{23}$.

Note that the space complexity of the  data structure is $\sum_{(i,j)} O(|P_{ij}|+|Q_{ij}|)+O(n)=O((n/{\epsilon}^2)\log^3 n)$. This follows, as a max-heap uses linear space.

\begin{observation}
The Hungarian Search  data structure uses $O(n \cdot poly(\log n))$ construction time and space.
\end{observation}

Next, we describe a procedure which will help us implement Hungarian search. This procedure takes a pair $(u,v)$ as input, where $u\in R\setminus R'$ and $v\in B'$. We need to add $u$ to $R'$ and remove $v$ from $B'$ while maintaining the correct maximum pair.

\subparagraph*{UPDATE$\_$CLOSEST$\_$PAIR$(u,v)$.} If $v\in B_0$, remove $v$ from all $H_{ij}^b$ that contains $v$. Also remove $(\hat{v},v)$ from $H_{23}$ if its in $H_{23}$. If $v\in B_1$, remove $v$ from $H_1^b$. Also remove the edge $(u',v)$ with $u'\in R_0$ from $H_{23}$ if its in $H_{23}$. If $u\in R_0$, add $u$ to all $H_{ij}^r$ such that $P_{ij}$ contains $u$. Also add the edge $(u,\hat{u})$ to $H_{23}$ if $\hat{u}\in B'$. If $u\in R_1$, add $u$ to $H_1^r$. Also add the edge $(u,v')$ with $v'\in B_0$ to $H_{23}$ if $v'\in B'$. Update the maximum pairs in $H$ accordingly by selecting the updated maximum pairs from the other max-heaps.

\begin{restatable}{lemma}{hstime}\label{lem:hstime}
Hungarian search can be performed in $O(n \cdot poly(\log n))$ time.
\end{restatable}

\begin{proof}
We note that there are at most $n$ iterations where in each iteration a vertex $u\in R\setminus R'$ is added to $R'$ and a vertex $v\in B'$ is removed from $B'$. Moreover, each vertex of $R$ is added to the data structure only once and each vertex of $B$ is removed from the data structure only once. To implement each iteration, we make a call to UPDATE$\_$CLOSEST$\_$PAIR($u,v$). The correctness follows trivially by construction of the data structure. Next, we analyze the time complexity. First, note that all the insertions and deletions in $H_1^b$, $H_1^r$ and $H_{23}$ can be performed in total time $O(n\log n)$. Also, the insertions of the vertices in $R$ to all $H_{ij}^r$ takes in total $\sum_{(i,j)} O(|P_{ij}|\log n)$ time. Similarly, the deletions of the vertices in $B$ from all $H_{ij}^b$ takes in total $\sum_{(i,j)} O(|Q_{ij}|\log n)$ time.  Note that an update to an element (maximum pair) in $H$ is triggered by an update in another max-heap. As we can have $O(n)+\sum_{(i,j)} O(|P_{ij}|+|Q_{ij}|)$ updates in total in other max-heaps, the updates in $H$ can be performed in total time $O(n\log n)+\sum_{(i,j)} O(|P_{ij}|+|Q_{ij}|)\log n$. Thus, the time taken over all iterations is $O(n\log n)+\sum_{(i,j)} O(|P_{ij}|+|Q_{ij}|)\log n=O((n/{\epsilon}^2)\log^4  n)$, which completes the proof of the lemma.
\end{proof}

Computational Geometry (cs.CG); Data Structures and Algorithms (cs.DS)


Conjectured lower bound for the clique number of a graph

Abstract: It is well known that $n/(n - \mu)$, where $\mu$ is the spectral radius of a graph with $n$ vertices, is a lower bound for the clique number. We conjecture that $\mu$ can be replaced in this bound with $\sqrt{s^+}$, where $s^+$ is the sum of the squares of the positive eigenvalues. We prove this conjecture for various classes of graphs, including triangle-free graphs, and for almost all graphs.

\section{Introduction}

Let $G$ be  a graph, with no isolated vertices, with $n$ vertices, edge set $E$ with $|E| = m$, average degree $d$, chromatic number $\chi(G)$ and clique number $\omega(G)$.
We also let $A$ denote the adjacency matrix of $G$ and let $\mu = \mu_1 \ge \ldots \ge \mu_n$ denote the eigenvalues of $A$. The inertia of $A$ is the ordered triple $(\pi, \nu, \gamma)$ where $\pi$, $\nu$ and $\gamma$ are the numbers counting muliplicities of positive, negative and zero eigenvalues of $A$ respectively. Let

\[
s^+ = \sum_{i=1}^\pi \mu_i^2  \quad\mbox{and}\quad s^- = \sum_{i=n-\nu+1}^n \mu_i^2.
\]

Note that:

\[
\sum_{i=1}^n \mu_i^2 = \mathrm{tr}(A^2) = 2m = s^+ + s^-.
\]

\section{Replacing $\mu^2$ with $s^+$}

Edwards and Elphick \cite{edwards83} proved that

\[
\frac{2m}{2m - \mu^2} \le \chi(G)
\]

and Ando and Lin \cite{ando15} proved a conjecture due to Wocjan and Elphick \cite{wocjan13} that

\[
\frac{2m}{2m - s^+} = 1 + \frac{s^+}{s^-} \le \chi(G).
\]

As another example of replacing $\mu^2$ with $s^+$, Hong \cite{hong93} proved for graphs with no isolated vertices that $\mu^2 \le 2m - n + 1$, and Elphick \emph{et al} \cite{elphick16} proved that for almost all connected graphs $s^+ \le 2m - n + 1$. Similarly Favaron \emph{et al} \cite{favaron93} proved that $\omega(G) \le 2m/\mu$ and Wu and Elphick \cite{wu17} proved the doubly stronger result that $\chi(G) \le 2m/\sqrt{s^+}.$ Finally Stanley \cite{stanley87} proved that

\[
\mu \le \frac{\sqrt{8m + 1} - 1}{2},
\]

and Wu and Elphick \cite{wu17} proved that

\[
\sqrt{s^+} \le \frac{\sqrt{8m + 1} - 1}{2}.
\]

So in all of these cases we can strengthen known bounds by replacing $\mu^2$ with $s^+$. The next section considers the same replacement for a well known lower bound for the clique number.

\section{Conjectured bound for the clique number}

The concise version of Tur\'an's theorem states that:

\begin{equation}\label{turan}
\frac{n}{n - d} \le  \omega(G).
\end{equation}

This bound was improved by Caro \cite{caro79} and Wei \cite{wei81} using degrees as follows:

\[
\sum_{i=1}^n \frac{1}{n - d_i} \le \omega(G);
\]

and by Wilf \cite{wilf86} using the spectral radius as follows:

\begin{equation}\label{wilf}
 \frac{n}{n - \mu} \le \omega(G).
\end{equation}

Bound (2) was strengthened by Nikiforov \cite{nikiforov02} who proved the following conjecture of Edwards and Elphick \cite{edwards83}.

\begin{equation}\label{vlado}
\frac{2m}{2m - \mu^2} \le \omega(G).
\end{equation}

Note that for regular graphs, all of these bounds equal $n/(n - d)$. Wocjan and Elphick \cite{wocjan13} noted that

\[
\frac{2m}{2m - s^+} \not\le \omega(G).
\]

An alternative strengthening of Wilf's bound is provided by the following conjecture, which we have tested against the thousands of named graphs with up to 40 vertices in the Wolfram Mathematica database, and found no counter-example. Aouchiche \cite{aouchiche16} has tested this conjecture using his powerful AGX software, and also found no counter-example. Conjecture 1 exceeds $n/(n - d)$ for all regular graphs with more than one positive eigenvalue.
\begin{con}
For any graph $G$

\[
\frac{n}{n - \sqrt{s^+}} \le \omega(G).
\]
This conjecture is exact, for example,  for complete regular multipartite graphs.

\end{con}

We can prove this conjecture for the following classes of graphs.

\subsection{Proof for triangle-free graphs}
\begin{proof}

Let $t$ denote the number of triangles in a graph. It is well known that:

\[
\sum_{i=1}^n \mu_i^3 = \mathrm{tr}(A^3) = 6t,
\]

so for triangle-free graphs

\[
\sum_{i=1}^\pi \mu_i^3 = -\sum_{i=n-\nu+1}^n \mu_i^3.
\]

Therefore, using that $\mu \ge |\mu_n|$

\[
s^- \ge \frac{\sum_{i=n-\nu+1}^n \mu_i^3}{\mu_n} = \frac{\sum_{i=1}^\pi \mu_i^3}{|\mu_n|} \ge \frac{\mu^3}{|\mu_n|} \ge \mu^2.
\]

Therefore, using the lower bound on the largest eigenvalue $\mu \ge 2m/n$, the equality $\frac{1}{2}(s^+ + s^-) = m$ combined with the arithmetic-geometric-mean inequality, and the above lower bound on $s^-$, we obtain
\[
\sqrt{s^+} \le \mu \frac{n}{2m} \sqrt{s^+} \le \frac{n}{2m} \sqrt{s^-} \sqrt{s^+} \le \frac{n}{2m} \frac{2m}{2} = \frac{n}{2}.
\]

\end{proof}

\subsection{Proof for weakly perfect graphs}

\begin{proof}
Weakly perfect graphs have $\omega(G) = \chi(G)$. Therefore using the result due to Ando and Lin \cite{ando15} discussed above and that $\mu \ge 2m/n$:

\[
\frac{n}{n - \sqrt{s^+}} \le \frac{2m}{2m - s^+} \le \chi(G) = \omega(G).
\]

\end{proof}

\subsection{Proof for some strongly regular graphs}

We do not know how to prove this conjecture for all strongly regular graphs. However we can prove the conjecture for the subset of strongly regular graphs which are Kneser graphs. The Kneser graph $KG_{p,k}$ is the graph whose vertices correspond to the $k-$element subset of a set of $p$ elements, and where two vertices are joined if and only if the corresponding sets are disjoint. The Kneser graphs with $k = 2$ are strongly regular, with only three distinct eigenvalues. For these graphs

\[
n = {p \choose 2},    \omega = \left\lfloor \frac{p}{2}\right \rfloor,  2m = {p \choose 2}{p - 2 \choose 2} \mbox{  and  } p \ge 2k = 4.
\]

The eigenvalues (see Godsil and Royle \cite{godsil01}) are:

\[
(-1)^i{p - 2 - i \choose 2 - i} \mbox{  with multiplicity  } {p \choose i} - {p \choose i - 1}, \mbox{  for  } i = 0,1,2.
\]

We are seeking to prove that

\[
\frac{n}{n - \sqrt{s^+}} \le \frac{p-1}{2} \le \left\lfloor \frac{p}{2}\right \rfloor = \omega(KG_{p,2}),
\]

which re-arranges to

\[
s^+ = 2m - s^- \le \frac{n^2(p - 3)^2}{(p - 1)^2} = \frac{p^2(p - 3)^2}{4}.
\]

Inserting the negative eigenvalues this becomes

\[
{p \choose 2}{p - 2 \choose 2} - (p - 1){p - 3 \choose 1}^2 \le \frac{p^2(p - 3)^2}{4}.
\]

Simple algebra reduces this to

\[
2p^2 - 9p + 6 \ge 0
\]

which is true for all $p \ge 4$.

\subsection{Proof for almost all graphs}
\begin{proof}

We use the Erdos-Renyi random graph model $G_n(p)$, which consists of all graphs with $n$ vertices in which edges are chosen independently with probability $p$. Bollob\'as and Erdos \cite{bollobas76} proved that the clique number is almost always $x$ or $x+1$ where

\[
x = \frac{2\log n}{\log (1/p)} + O(\log \log n).
\]

Since almost all graphs have all degrees very close to $n/2$ we let $p = 0.5$. Therefore
\[
s^+ \le s^+ + s^- = 2m \approx n \, \frac{n}{2}
\]

So for almost all graphs

\[
\frac{n}{n - \sqrt{s^+}} \le \frac{n}{n - n/\sqrt{2}} \approx 3.4 <\frac{2\log n}{\log 2} \approx \omega(G).
\]

\end{proof}

\section{Conclusion}

Lower bounds for the clique number are often proved using the Motzkin-Straus \cite{motzkin65} inequality, which can be expressed as follows. For any adjacent vertices $i$ and $j$ such that $i < j$ we write $i \sim j$. Then for any vector $(p_1, \ldots, p_n)$ with $p_i \ge 0$ for all $i$ and $\sum_{i=1}^n p_i = 1$:

\[
\sum_{i \sim j} p_i p_j \le \frac{\omega - 1}{2\omega}.
\]

It is however not evident how to use this approach in the context of Conjecture 1, where the number of positive eigenvalues varies greatly between graphs with $n$ vertices.

\section*{Acknowledgements}

This research was supported in part by the National Science Foundation Award 1525943 .