Tverberg theorems over discrete sets of points

Abstract: This paper discusses Tverberg-type theorems with coordinate constraints (i.e., versions of these theorems where all points lie within a subset $S \subset \mathbb{R}^d$ and the intersection of convex hulls is required to have a non-empty intersection with $S$). We determine the $m$-Tverberg number, when $m \geq 3$, of any discrete subset $S$ of $\mathbb{R}^2$ (a generalization of an unpublished result of J.-P. Doignon). We also present improvements on the upper bounds for the Tverberg numbers of $\mathbb{Z}^3$ and $\mathbb{Z}^j \times \mathbb{R}^k$ and an integer version of the well-known positive-fraction selection lemma of J. Pach.

\section*{Introduction}

Consider $n$ points in $\ensuremath{\mathcal{R}}^d$ and a positive integer $m\geq 2$. If $n\geq (m-1)(d+1)+1$, the points can always be partitioned into
$m$ subsets whose convex hulls contain a common point. This is the celebrated theorem of Tverberg \cite{Tverberg:1966tb}, which
has been the topic of many generalizations and variations since it was first proved in 1966 \cite{barany+soberonsurvey,jesus+xavier+frederic+nabil}.
In this paper we focus on new versions of Tverberg-type theorems where some of the coordinates of the points are restricted to discrete subsets of a Euclidean space.  The associated  discrete Tverberg numbers are much harder to compute than their classical real-version counterparts
(see for instance the complexity discussion of \cite{Onn:1991wz}).

We begin our work remembering the following unpublished Tverberg-type result of Doignon.
Consider $n$ points with coordinates in $\mathbb{Z}^2$ and a positive integer $m\geq 3$. If $n\geq 4m-3$, then
the points can be partitioned into $m$ subsets whose convex hulls contain a common point in $\mathbb{Z}^2$.
According to Eckhoff~\cite{Eckhoff:2000jw} this result was stated by Doignon in a conference.

A partition of points where the intersection of the convex hulls contains at least one lattice point is called an {\em integer $m$-Tverberg partition} and
such a common point is an {\em integer Tverberg point} for that partition.
Regarding the case $m=2$,  the integer $2$-Tverberg partitions are  called {\em integer Radon partitions}. Any configuration of at least six
points in $\mathbb{Z}^2$ admits an integer Radon partition. This was proved by Doignon in his PhD thesis~\cite{doignonthesis} and later discovered independently by Onn~\cite{Onn:1991wz}. All these values for $\mathbb{Z}^2$ are optimal as shown by following examples. The $5$-point configuration $\left\{(0,0),(0,1),(2,0)(1,2),(3,2)\right\}$, exhibited by Onn in the cited paper, has no Radon partition. To address the optimality when $m\geq 3$, consider the set $\{(i,i),(i,-i+1)\colon i = -m+2, -m+3 , \ldots, m-2, m-1\}.$ (According to Eckhoff~\cite{Eckhoff:2000jw}, this set was proposed by Doignon during the aforementioned conference.)
This set has $4m-4$ points and a moment of reflection might convince the reader that it has no integer $m$-Tverberg partition.

More generally, one can define the {\em Tverberg number} $\Tv(S,m)$ for any subset $S$ of $\ensuremath{\mathcal{R}}^d$ and an integer $m\geq 2$ as
the smallest positive integer $n$ with the following property: Any multiset of $n$ points in $S$ admits a partition into $m$ subsets $A_1,A_2,\ldots, A_m$ with
\[\left(\bigcap_{i=1}^m\conv(A_i)\right)\cap S\neq\varnothing.\]
  (Here, by ``partition of a multiset'', we mean
that each element of a multiset $A$ is contained in a number of submultisets $A_1, \dots, A_m$ so that the sum of its multiplicities in the $A_i$ is equal to its multiplicity in $A$.)  If no such number exists, we say that $\Tv(S,m) = \infty$. Note that Doignon's theorem, together with the discussion that follows, allows us to say
\[\Tv(\mathbb{Z}^2,m)=\left\{\begin{array}{ll} 6 & \mbox{if $m=2$}, \\ 4m-3 & \mbox{otherwise.}\end{array}\right.\]

\subsection*{Our contributions}

Our first main result generalizes Doignon's theorem. We determine the exact $m$-Tverberg number (when $m$ is at least three) for any discrete subset $S$ of $\ensuremath{\mathcal{R}}^2$, as considered in \cite{integertverberg2017}. Before stating this result we recall the \emph{Helly number $\He(S)$} of a discrete subset $S$ of $\ensuremath{\mathcal{R}}^d$ as the smallest positive integer with the following property: Suppose $\mathcal{F}$ is a finite family of convex sets in $\mathbb{R}^d$, and that $\cap \mathcal{G}$ intersects $S$ in at least one point for every subfamily $\mathcal{G}$ of $\mathcal{F}$ having at most $\He(S)$ members. Then $\cap \mathcal{F}$ intersects $S$ in at least one point. If no such integer exists,
we say that $\He(S) = \infty$.

Then we have the following theorem. (The theorem is stated for $S$ with finite Helly number, as any $S \subset \ensuremath{\mathcal{R}}^d$ with $\He(S) = \infty$ has $\Tv(S,m) = \infty$ for all $m \geq 2$~\cite{hellylevi}.)

\begin{theorem}~\label{Stverberg}

Suppose $S$ is a discrete subset of $\ensuremath{\mathcal{R}}^2$ with $\He(S) < \infty$.
If $m \geq 3$, then $$\Tv(S,m) = \He(S)(m-1) + 1.$$
Regarding the case $m = 2$, if $\He(S) \leq 3$, then $$\Tv(S,2) = \He(S) + 1,$$ and if $\He(S) \geq 4$, then $$\He(S) + 1 \leq \Tv(S,2) \leq \He(S) + 2,$$

and both values are possible.

\end{theorem}

In particular we present a proof of Doignon's theorem, the special case of Theorem~\ref{Stverberg} where $S = \mathbb{Z}^2$.

\remark{Theorem~\ref{Stverberg} shows that $S$-Tverberg numbers of planar sets are very closely related to $S$-Helly numbers (see \cite{Ave2013, Averkovetal-tightbounds} and the references there). However, for the case $\He(S) = 4$, the bounds on $\Tv(S,2)$ given above cannot be improved. For example, $S' = \{(0,0),(0,1),(1,0),(1,1)\}$ and $\mathbb{Z}^2$ both have Helly number  four, but $\Tv(\mathbb{Z}^2,2) = 6$, while the pigeonhole principle implies that $\Tv(S',2) = 5$.}
\medskip

Our second main result improves the upper bound on the integer Tverberg numbers for the three-dimensional case $S = \mathbb{Z}^3$.
\begin{theorem} \label{3dZ} The following inequality holds for all $m\geq2$:
\[\Tv(\mathbb{Z}^3, m) \leq 24m- 31.\]
\end{theorem}

Our third main result is an inequality that will be used to derive improved bounds on $S$-Tverberg numbers when $S$ is a product of a Euclidean space with some subset $S'$ of a Euclidean space.

\begin{theorem} \label{mixedint}
Let $S' \subset \ensuremath{\mathcal{R}}^j$ be a subset of a Euclidean space. Then for all positive integers $k$ and all $m\geq 2$, we have
 \[\Tv(S' \times \ensuremath{\mathcal{R}}^k, m) \leq \Tv(S' , \Tv(\ensuremath{\mathcal{R}}^k,m)).\]
\end{theorem}

For example, choosing $S$ of the form $\mathbb{Z}^j \times \ensuremath{\mathcal{R}}^k$ leads to the ``mixed integer'' case. Then Theorem~\ref{mixedint} implies that for all positive integers $j, k$ and all $m\geq 2$, we have
\[\Tv(\mathbb{Z}^j \times \ensuremath{\mathcal{R}}^k, m) \leq \Tv(\mathbb{Z}^j , \Tv(\ensuremath{\mathcal{R}}^k,m)).\]
Moreover, we will use Theorem~\ref{mixedint} to obtain the following bound:
\begin{equation}~\label{mixedbound}
2^j(m-1)(k+1) + 1 \leq \Tv(\mathbb{Z}^j \times \ensuremath{\mathcal{R}}^k,m) \leq j2^j(m-1)(k+1) + 1.\end{equation}

Our fourth main result is a generalization of \emph{Pach's positive-fraction selection lemma}~\cite{P98} (see \cite{karasevetal} for related bounds).
Here is Pach's result: Given an integer $d$, there exists a constant $c_d$ such that for any set $P$ of $n$ points in $\Re^d$, there exists
a point $\boldsymbol{q} \in \Re^d$, and $(d+1)$ disjoint subsets of $P$, say $P_1, \ldots, P_{d+1}$, such that $|P_i| \geq c_{d} \cdot n$ for all $i$ and
the simplex defined by every transversal of $P_1, \ldots, P_{d+1}$ contains $\boldsymbol{q}$. (By ``transversal", we mean a set containing exactly one element from
each $P_i$.)

Unfortunately the point $\boldsymbol{q}$ need not be an integer point; furthermore, the proof uses the so-called ``second selection lemma" that currently does not exist for integer points (see Pach~\cite{P98} and Matou\v{s}ek~\cite[Chapter 9]{M02}). In Section~\ref{IPFSL}, we strengthen the above theorem, such that,
as a consequence, the theorem now extends to the integer case---indeed, to any scenario where one has points of high half-space depth in the following sense:

Given a finite set $P$ of points in $\Re^d$ and a point $\boldsymbol{q} \in \Re^d$, we say that $\boldsymbol{q}$
is \emph{of half-space depth $t$ with respect to $P$} if any half-space containing $\boldsymbol{q}$
contains at least $t$ points of $P$ (when the context is clear,
we will simply say that $\boldsymbol{q}$ is of \emph{depth $t$}). Then here is our theorem.

\begin{theorem}
For any integer $d \geq 1$ and real number $\alpha \in (0, 1]$, there exists a constant
$c_{d, \alpha}$ such that the following holds.
For any set $P$ of $n$ points in $\mathbb{R}^d$ and any point $\boldsymbol{q} \in \Re^d$ of half-space depth at least $\alpha \cdot n$,
there exist $(d+1)$ disjoint subsets of $P$, say $P_1, \ldots, P_{d+1}$, such
that
  \begin{itemize}
  \item $|P_i| \geq c_{d, \alpha} \cdot n$ for   $i = 1, \ldots, (d+1)$, and
  \item every   simplex defined by a transversal of $P_1, \ldots, P_{d+1}$ contains $\boldsymbol{q}$.
  \end{itemize}
\label{thm:depthcovering}
\end{theorem}

\remark{Our proof yields a constant $c_{d, \alpha}$ whose value is exponential in the dimension $d$.}

Note that the existence of integer points of high half-space depth (Lemma~\ref{centerpt}) together with Theorem~\ref{thm:depthcovering} implies the following integer version of the positive-fraction selection lemma.

\begin{corollary}
Let $P$ be a set of $n \geq (d+1)$ points in $\mathbb{Z}^d$. Then there exists a
point $\boldsymbol{q} \in \mathbb{Z}^d$, and $(d+1)$ disjoint subsets of $P$, say $P_1, \ldots, P_{d+1}$,
such that $|P_i| \geq c_{d, 2^{-d}} \cdot n$ for all $i = 1, \ldots, (d+1)$, and
the simplex defined by every  transversal of $P_1, \ldots, P_{d+1}$ contains $\boldsymbol{q}$.
\end{corollary}

\remark{In particular, this implies that $\boldsymbol{q}$ belongs to many \emph{distinct} Tverberg partitions---at
least $\left(\lceil c_{d, 2^{-d}} \cdot n \rceil !\right)^d$  \emph{distinct} Tverberg partitions,
with each such Tverberg partition containing $\left\lceil c_{d, 2^{-d}} \cdot n \right\rceil$ sets.}

\medskip

\subsection*{Related  Results and Organization of the paper}
The problem of computing the Tverberg number for $\mathbb{Z}^d$ with $d\geq 3$ seems to be challenging.
It has been identified as an interesting problem since the 1970's~\cite{problemsinconvexity} and yet the following inequalities
are almost all that is known about this problem: for the general case, De Loera et al.~\cite{integertverberg2017} proved
\begin{equation}\label{DL}
2^d(m-1)+1\leq\Tv(\mathbb{Z}^d,m)\leq d2^d(m-1)+1, \quad \mbox{for $d\geq 1$ and $m\geq 2$.}
\end{equation} Two special cases get better bounds:
\begin{equation}\label{special}
\Tv(\mathbb{Z}^3,2)\leq 17 \qquad\mbox{and}\qquad 5\cdot 2^{d-2}+1\leq\Tv(\mathbb{Z}^d,2)\quad \mbox{for $d\geq 1$.}
\end{equation}
The left-hand side inequality is due to Bezdek and Blokhuis~\cite{Bezdek2003} and the right-hand side was
proved by Doignon in his PhD thesis (and rediscovered by Onn).

Previously established bounds for the ``mixed integer'' case include  the bounds for the Radon number ($2$-Tverberg number) found by Averkov and Weismantel~\cite{AW2012}. \[ 2^j(k+1) + 1 \leq \Tv(\mathbb{Z}^j \times \ensuremath{\mathcal{R}}^k, 2) \leq (j+k)2^j(k+1)- j -k + 2.\]
Later, De Loera et al.~\cite{integertverberg2017} gave the following general bound for all mixed integer Tverberg numbers:
\[\label{known-mixed} \Tv(\mathbb{Z}^j \times \ensuremath{\mathcal{R}}^k, m) \leq (j+k)2^j(m-1)(k+1)+ 1.\]
Note that \eqref{mixedbound} above is a simultaneous improvement of both of these.

Previous bounds and related work on more general $S$-Tverberg numbers can also be found in \cite{integertverberg2017}, including the following bound for any discrete $S\subset \ensuremath{\mathcal{R}}^d$:
$$\Tv(S,m) \leq \He(S)(m-1)d + 1.$$

The following lemma about integer points of high half-space depth is used throughout the paper. See \cite{basu+oertel} for a proof and related results.

\begin{lemma} \label{centerpt} Consider a multiset $A$ of points in $\mathbb{Z}^d$. If $|A|\geq 2^d (m-1)+1$ (counting multiplicities), then there is a point $\boldsymbol{q} \in \mathbb{Z}^d$ of half-space depth $m$ in $A$.
\end{lemma}

The paper is organized as follows.
In Section~\ref{TS}, we prove Theorem ~\ref{Stverberg} using a somewhat similar strategy to Birch's proof of the planar case of the original Tverberg theorem~\cite{Birch:1959ii}. In Section~\ref{TZ3}, we prove Theorem~\ref{3dZ} using techniques reminiscent of those in \cite{integertverberg2017}. In Section~\ref{CONS}, we  prove Theorem \ref{mixedint} (adapting an approach by Mulzer and Werner~\cite[Lemma 2.3]{MW13}) and collect some consequences of the main theorems presented above, including (\ref{mixedbound}). Finally, in Section \ref{IPFSL}, we prove Theorem~\ref{thm:depthcovering} by proving a new lemma and adapting the methods of Pach in \cite{P98}.

\section{Tverberg Numbers over Discrete Subsets of $\mathbb{R}^2$: Proof of Theorem~\ref{Stverberg}}~\label{TS}
We start with the proof of the special case $S = \mathbb{Z}^2$ because it nicely illustrates the techniques of the more general proof of Theorem~\ref{Stverberg}.
\subsection{Proof of the special case $S = \mathbb{Z}^2$} \label{TZ2}
The theorem will follow easily from the following two lemmas, the first covering the case $m\geq 3$ and the second the case $m=2$.

\begin{lemma}\label{Tplane1}
Consider a multiset $A$ of points in $\mathbb{Z}^2$ with $|A| \geq 4m-3$ and $m \geq 3$. If $\boldsymbol{p} \notin A$ is a point of depth $m$, then there is an $m$-Tverberg partition of $A$ with $\boldsymbol{p}$ as Tverberg point.
\end{lemma}

\begin{lemma}\label{Tplane2}
Consider a multiset $A$ of points in $\mathbb{Z}^2$ with $|A| \geq 6$. If $\boldsymbol{p} \notin A$ is a point of depth two, then there is a Radon partition of $A$ with $\boldsymbol{p}$ as Tverberg point.
\end{lemma}

\begin{proof}[Proof of special case $S = \mathbb{Z}^2$.]
Consider a multiset $A$ of at least $4m-3$ points in $\mathbb{Z}^2$. By Lemma~\ref{centerpt}, $A$ has an integer point $\boldsymbol{p}$ of depth $m$. If $\boldsymbol{p}$ is an element of $A$ with multiplicity $\mu$, then take the singletons $\{\boldsymbol{p}\}$ as $\mu$ of the sets in the Tverberg partition. Then $\boldsymbol{p}$ is a point of depth $m-\mu$ of the remaining $4m-\mu-3$ points. If $\mu \geq m$, we are done, and if $\mu = m-1$, the point $\boldsymbol{p}$ is in the convex hull of the remaining points and we take them to be the last set in the desired partition. If $\mu \leq m-3$, according to Lemma~\ref{Tplane1}, there is an $(m-\mu)$-Tverberg partition of the remaining points with $\boldsymbol{p}$ as Tverberg point. There is thus an $m$-Tverberg partition of $A$ with $\boldsymbol{p}$ as Tverberg point. The case $\mu = m - 2$ is treated similarly with the help of Lemma~\ref{Tplane2} in place of Lemma~\ref{Tplane1}.
\end{proof}

\begin{proof}[Proof of Lemma~\ref{Tplane1}]
Since $\boldsymbol{p}$ is not in $A$, up to a radial projection, we can assume that the points of $A$ are arranged in a circle around $\boldsymbol{p}$. Define $q$ and $r$ to be respectively the quotient and the remainder of the Euclidean division of $|A|$ by $m$. Define moreover $e$ to be $\lceil \frac{r}{q} \rceil$.

Suppose first that $\boldsymbol{p}$ is a point of depth $m + e$. In such a case, we arbitrarily select  a first point in $A$, and label clockwise the points with elements in $[m]$ according to the following pattern:
\[1, 2, \ldots, m, 1, 2, \ldots, e , 1 , 2 , \ldots, m , 1, 2, \ldots, e , 1, 2 , \ldots m, 1, 2 , \ldots k,\]
where $k=|A|-qm-(q-1)e$. Note that we have $k\leq e$.
Each half-plane delimited by a line passing through $\boldsymbol{p}$ contains at least $m+e$ consecutive points in this pattern and thus has at least one point with each of the $m$ different labels. Partitioning the points so that each subset consists of all points with a fixed label, we therefore obtain an $m$-Tverberg partition with $\boldsymbol{p}$ as Tverberg point.

Suppose now that $\boldsymbol{p}$ is not a point of depth $m + e$. There is thus a closed half-plane $H_+$, delimited by a line passing through $\boldsymbol{p}$, with $|H_+ \cap A| < m + e$. The complementary closed half-plane to $H_+$, which we denote by $H_-$, is such that $|H_- \cap A| >  4m-3 - (m+e)$. Define $\ell$ to be $|H_- \cap A|$. Since $e\leq \frac m 3$, we have $\ell\geq 2m$. Denote the points in $H_- \cap A$ by $\boldsymbol{x}_1,\ldots,\boldsymbol{x}_{\ell}$, where the indices are increasing when we move clockwise. We label $\boldsymbol{x}_i$ with $r + i$ from $\boldsymbol{x}_1$ to $\boldsymbol{x}_{m-r}$, and then label $\boldsymbol{x}_{m-r+j}$ with $j$ from $\boldsymbol{x}_{m-r +1}$ to $\boldsymbol{x}_m$. We then continue labeling the points of $A$, still moving clockwise, using labels $1, 2, \ldots , m , \ldots, 1, 2 , \ldots m, 1, 2, \ldots r.$ See Figure~\ref{fig:labeling} for an illustration of the labeling scheme.

\begin{figure}
  \centering
  \includegraphics[width=10cm ]{labeling.png}
  \caption{Labeling of the points in the half-plane $H_-$.}
  \label{fig:labeling}
\end{figure}
The labeling pattern is such that any sequence of $m$ consecutive points either has all $m$ labels, or contains the two consecutive points $\boldsymbol{x}_m$ and $\boldsymbol{x}_{m+1}$. Let us prove that any closed half-plane $H$ delimited by a line passing through $\boldsymbol{p}$ contains at least one point with each label. Once this is proved, the conclusion will be immediate by taking as subsets of points those with same labels, as above.

If such an $H$ does not simultaneously contain $\boldsymbol{x}_m$ and $\boldsymbol{x}_{m+1}$, then $H$ contains at least one point with each label. Consider thus a closed half-plane $H$ delimited by a line passing through $\boldsymbol{p}$ and containing $\boldsymbol{x}_m$ and $\boldsymbol{x}_{m+1}$. Note that according to Farkas' lemma (\cite{Sch03} Theorem 5.3) , $\boldsymbol{x}_{m+1}$ cannot be separated from $\boldsymbol{x}_1$ and $\boldsymbol{x}_{\ell}$ by a line passing through $\boldsymbol{p}$, since they are all in $H_-$. This means that either $H$ contains $\boldsymbol{x}_1, \boldsymbol{x}_2, \ldots, \boldsymbol{x}_{m + 1}$, or $H$ contains $\boldsymbol{x}_{m+1}, \boldsymbol{x}_{m+2}, \ldots, \boldsymbol{x}_{\ell}$. In any case, $H$ contains a point with each label.
\end{proof}

\begin{proof}[Proof of Lemma \ref{Tplane2}]
As before, we assume that the points in $A$ are arranged on a circle centered at $\boldsymbol{p}$. If $|A|$ is even, it clearly suffices to label the points in order, alternating between 1 and 2. We may therefore assume that $|A|$ is odd, and thus $|A| \geq 7$. If $\boldsymbol{p}$ is a point of depth three, it suffices to label the points alternating labels between 1 and 2, except with two consecutive points labeled 1.
If $|A|$ is odd but $\boldsymbol{p}$ is not a point of depth three, then $|A| \geq 7$ and there is a half-plane $H_+$ containing $\boldsymbol{p}$ with $|H_+ \cap A| = 2$. The complementary half-plane $H_-$ has $|H_- \cap A| \geq 5$ and we follow a similar strategy as in the second half of Lemma~\ref{Tplane1}. Namely, we denote the points in $H_- \cap A$ by $\boldsymbol{x}_1,\ldots,\boldsymbol{x}_{\ell}$, where the indices are increasing when we move clockwise. Then we label $\boldsymbol{x}_1$ with 2, $\boldsymbol{x}_2$ with 1, $\boldsymbol{x}_3$ with 1, and $\boldsymbol{x}_4$ with 2. We continue this pattern for $\alpha \geq 5$, labeling $\boldsymbol{x}_\alpha$ with 1 if $\alpha$ is odd, and $\boldsymbol{x}_\alpha$ with 2 if $\alpha$ is even. For the remaining points in $A$ we continue labeling clockwise, alternating between the labels 1 and 2.

The labeling pattern is such that any sequence of $2$ consecutive points either has both labels, or contains the two consecutive points $\boldsymbol{x}_2$ and $\boldsymbol{x}_3$. As in Lemma~\ref{Tplane1} it suffices to show that any closed half-plane $H$ delimited by a line passing through $\boldsymbol{p}$ contains at least one point with each label.

If such an $H$ does not simultaneously contain $\boldsymbol{x}_2$ and $\boldsymbol{x}_3$, then $H$ contains at least one point with each label. Consider thus a closed half-plane $H$ delimited by a line passing through $\boldsymbol{p}$ and containing $\boldsymbol{x}_2$ and $\boldsymbol{x}_3$. Note that according to Farkas' lemma, $\boldsymbol{x}_{3}$ cannot be separated from $\boldsymbol{x}_1$ and $\boldsymbol{x}_4$ by a line passing through $\boldsymbol{p}$, since they are all in $H_-$. This means that either $H$ contains $\boldsymbol{x}_1, \boldsymbol{x}_2, \boldsymbol{x}_3$, or $H$ contains $\boldsymbol{x}_{3}$ and $\boldsymbol{x}_4$. In any case, $H$ contains a point with each label.
\end{proof}

\subsection{Proof of the general case}~\label{TSA}

The proof of the general case is split into three lemmas addressing the lower bound, the upper bound for $\He(S) \geq 4$,
and the upper bound for $\He(S) \leq 3$, respectively.

\begin{lemma}~\label{Stverberglowerbound} For any discrete set $S\subset \ensuremath{\mathcal{R}}^2$ with finite Helly number $\He(S)$, we have $\Tv(S,m) > \He(S)(m-1)$.
\end{lemma}

\begin{proof} It suffices to exhibit a subset $R \subseteq S$, of cardinality $|R| = \He(S)(m-1)$, with the property that no point in $S$ is of half-space depth $m$ with respect to $R$. By Lemma 2.6 in \cite{amentaetal2017helly}, there exists a set $R'$ of $\He(S)$ points in $S$ in convex position with the property that $\conv(R')\cap S = R'$. Let $R$ be the multiset given by taking each point in $R'$ with multiplicity $(m-1)$, so $|R| = \He(S)(m-1)$. No points of $S-R$ are in $\conv(R)$. Since $R'$ was taken to be in convex position, for any point in $R$, there exists a line such that one side of that line has at most $m-1$ points in $R$. Thus $S$ cannot contain a point of half-space depth $m$ with respect to $R$.
\end{proof}

\begin{lemma}~\label{helly4} For any discrete set $S\subset \ensuremath{\mathcal{R}}^2$ with finite Helly number $\He(S) \geq 4$, we have $\Tv(S,m) \leq \He(S)(m-1) + 1$ whenever $m \geq 3$. For the case $m = 2$, we have $\Tv(S,2) \leq \He(S) + 2$.
\end{lemma}

\begin{proof}
The proof of Lemma~\ref{helly4} is the same as the proof of Theorem 1. In particular, we can use Lemmas 2 and 3 as they are stated, except that we use the following result (Theorem 2 in \cite{basu+oertel} with $\mu$ being the uniform probability measure on $A$) in place of Lemma~\ref{centerpt}. For any discrete discrete subset $S$ of a Euclidean space with finite Helly number $\He(S)$, and any set $A \subseteq S$ with $|A| \geq \He(S) (m-1) + 1$, there  exists a point $\boldsymbol{p} \in S$ that is of half-space depth $m$ with respect to $A$.
\end{proof}

\begin{lemma} For a discrete set $S\subset \ensuremath{\mathcal{R}}^2$ with finite Helly number $\He(S) \leq 3$, we have $\Tv(S,m) \leq \He(S)(m-1) + 1$.
\end{lemma}
\begin{proof}
The case $\He(S) = 1$ implies that $S$ consists of a single point, so the result trivially follows. If $\He(S) = 2$, it must be that all points in $S$ are collinear (as any set containing a non-degenerate triangle has Helly number at least 3), and thus we can take median of any set with at least $2(m-1) + 1$ points in $S$ as the desired $m$-Tverberg point. Thus for the remainder of the proof we assume that $\He(S) = 3$.

Given any set $A$ of $\He(S)(m-1) = 3m-2$ points in $S$, there exists an $m$-Tverberg partition, say $\mathcal{P}$ by the classical Tverberg theorem. We denote by $K_1, \dots, K_m$ the $m$ convex hulls of the subsets in $\mathcal{P}$. As $\bigcap_{1 \leq i \leq m} K_i$ is a nonempty polygon, say $Q$, (possibly just a point or line segment) we pick an arbitrary vertex $\boldsymbol{q}$ of $Q$.

It suffices to show that $\boldsymbol{q} \in S$. We can assume that $\boldsymbol{q}$ is not a vertex of any $K_i$, since otherwise $\boldsymbol{q} \in A \subseteq S$.

Since $\boldsymbol{q}$ is a vertex of $Q$, it must be contained in a one dimensional face $F_1$ of at least one $K_i$. Since $\boldsymbol{q}$ is not a vertex of any $K_i$, in fact $\boldsymbol{q}$ is in the relative interior of $F_1$. For $\boldsymbol{q}$ to be a vertex of $Q$, it must also be in another one dimensional face, say $F_2$, of some other $K_i$, such that $F_1$ is not parallel to $F_2$. Moreover, $\boldsymbol{q}$ must be in the relative interior of $F_2$, and we also have $F_1 \cap F_2 = \{\boldsymbol{q}\}$.

Denote by $\{\boldsymbol{a},\boldsymbol{b}\}$ and $\{\boldsymbol{c},\boldsymbol{d}\}$ the vertices of $F_1$ and $F_2$ respectively. We have that $\boldsymbol{a},\boldsymbol{b},\boldsymbol{c},\boldsymbol{d} \in S$ are the vertices of a convex quadrilateral with diagonals intersecting at $\boldsymbol{q}$, by the assumption that $F_1$ and $F_2$ are non parallel. Out of the four triangles $\conv(\{\boldsymbol{a},\boldsymbol{b},\boldsymbol{c}\}), \conv(\{\boldsymbol{a},\boldsymbol{b},\boldsymbol{d}\}), \conv(\{\boldsymbol{a},\boldsymbol{c},\boldsymbol{d}\}), \conv(\{\boldsymbol{b},\boldsymbol{c},\boldsymbol{d}\})$, any three have at least one vertex in common, and therefore intersect in $S$. Since $\He(S) = 3$, the four triangles therefore all intersect in $S$. This intersection point is $\boldsymbol{q}$, the point where the diagonals of the quadrilateral intersect.
\end{proof}

\section{Tverberg Numbers over $\mathbb{Z}^3$: Proof of Theorem~\ref{3dZ}}\label{TZ3}

We state the following lemma without proof; it is a consequence, upon close inspection of the argument, of the proof of the main theorem in the already mentioned paper by Bezdek and Blokhuis~\cite{Bezdek2003}.

\begin{lemma}\label{BB}
Consider a multiset $A$ of at least $17$ points in $\mathbb{R}^3$ and a point $\boldsymbol{p}$ of depth $3$ in $A$. There is a bipartition of $A$ into two subsets whose convex hulls contain $\boldsymbol{p}$.
\end{lemma}

Next, we prove the following.

\begin{lemma}\label{BBplus}
Consider a multiset $A$ of points in $\ensuremath{\mathcal{R}}^3$ with $|A| \geq 24m-31$ and $m\geq 2$. If $\boldsymbol{p} \notin A$ is a point of depth $3m-3$, then there is an $m$-Tverberg partition of $A$ with $\boldsymbol{p}$ as Tverberg point.
\end{lemma}
\begin{proof}
Since $\boldsymbol{p}$ is not an element of $A$, we assume without loss of generality that the points of $A$ are located on a sphere centered at $\boldsymbol{p}$, as in the proof of Theorem~\ref{Stverberg}.

We claim that there exist pairwise disjoint subsets $X_1, X_2, \ldots, X_{m-2}$ of $A$, each having $\boldsymbol{p}$ in its convex hull and each being of cardinality at most $4$. (Here ``pairwise disjoint'' means that each element of $A$ is present in a number of $X_i$'s that does not exceed its multiplicity in $A$.) We proceed by contradiction. Suppose that we can find at most $s < m-2$ such subsets $X_i$'s. Then, by Carath\'eodory's theorem, $\boldsymbol{p}$ is not in the convex hull of the remaining points in $A$. Therefore there is a half-space $H_+$ delimited by a plane containing $\boldsymbol{p}$ such that
$H_+ \cap A \subseteq \bigcup_{i=1}^s X_i$. On the other hand, since each $X_i$ contains $\boldsymbol{p}$ in its convex hull (and we can assume the $X_i$ are minimal with respect to containing $\boldsymbol{p}$), we have $|H_+ \cap X_i |\leq 3$ for all $i\in[s]$. Therefore $|H_+ \cap A| \leq \left|H_+ \cap  \left(\bigcup_{i=1}^s X_i\right)\right| \leq 3s < 3(m-2)$, which is a contradiction since $\boldsymbol{p}$ is a point of depth $3m-3$ in $A$. There are thus $m-2$ disjoint subsets $X_1, X_2, \ldots, X_{m-2}$ as claimed.

Let $X$ denote $\bigcup_{i=1}^{m-2}X_i$. Consider an arbitrary half-space $H_+$ delimited by a plane containing $\boldsymbol{p}$. Since $|H_+ \cap X_i| \leq 3$ for all $i$, we have $|H_+ \cap X| \leq 3(m-2)$. Furthermore $|H_+ \cap A| \geq 3m-3$, so $|H_+ \cap (A \setminus X) | \geq 3$. Since $H_+$ is arbitrary, $\boldsymbol{p}$ is a point of depth $3$ of $A\setminus X$. Also, $|A\setminus X| \geq |A| - 4(m-2) \geq 20m- 23 \geq 17$, so Lemma~\ref{BB} implies that $A\setminus X$ can be partitioned into two sets whose convex hulls contain $\boldsymbol{p}$. With the subsets $X_i$, we have therefore an $m$-Tverberg partition of $A$, with $\boldsymbol{p}$ as Tverberg point.
\end{proof}
From these two lemmas we can now finish the proof of Theorem~\ref{3dZ}.
\begin{proof}[Proof of Theorem~\ref{3dZ}]
Consider a multiset $A$ of $24m-31$ points in $\mathbb{Z}^3$. The case $m=2$ is the already mentioned result by Bezdek and Blokhuis. Assume that $m \geq 3$. Applying Lemma~\ref{centerpt}, $A$ has an integer point $\boldsymbol{p}$ of depth $3m-3$. If $\boldsymbol{p}$ is an element of $A$ with multiplicity $\mu$, then take the singletons $\{\boldsymbol{p}\}$ as $\mu$ of the sets in the Tverberg partition.

If $\mu\geq m$, we are done. If $\mu=m-1$, the point $\boldsymbol{p}$ is still in the convex hull of points in $A$, and thus we are done. And if $\mu \leq m-2$, the point $\boldsymbol{p}$ is still a point of depth $3m-\mu-3\geq 3(m-\mu)-3$ of the remaining $24m-\mu-31 \geq 24(m-\mu)-31$ points. Thus, we may apply Lemma~\ref{BBplus} to get an $(m-\mu)$-Tverberg partition of the remaining points, with $\boldsymbol{p}$ as Tverberg point, and conclude the result.
\end{proof}
\section{Tverberg Numbers over $Q \times \ensuremath{\mathcal{R}}^k$: Proof of Theorem \ref{mixedint}}\label{CONS}
In this section, we prove Theorem~\ref{mixedint}. We adapt an approach by Mulzer and Werner~\cite[Lemma 2.3]{MW13} and show how the results of our paper
can be combined to improve known bounds and to determine new exact values for the Tverberg number in the mixed integer case, as well as better bounds for certain $S$-Tverberg numbers.
\begin{proof}[Proof of Theorem \ref{mixedint}] Let $t = \Tv(\ensuremath{\mathcal{R}}^k,m) = (m-1)(k+1) + 1$. Choose a multiset $A$ in $S' \times \ensuremath{\mathcal{R}}^k$ with $|A| \geq\Tv(S',t)$. It suffices to prove that $A$ can be partitioned into $m$ subsets whose convex hulls contain a common point in $S' \times \ensuremath{\mathcal{R}}^k$.

Let $A'$ be the projection of $A$ onto $S'$. Since $|A'| \geq\Tv(S' , t)$, there is a partition of $A'$ into $t$ submultisets $Q'_1,\ldots,Q'_t$ whose convex hulls contain a common point $\boldsymbol{q}$ in $S'$. The $Q'_i$ are the projections onto $S'$ of $t$ disjoint subsets $Q_i$ forming a partition of $A$.  For each $i\in[t]$, we can find a point $\boldsymbol{q}_i\in\conv(Q_i)$ projecting onto $\boldsymbol{q}$.

The $t$ points $\boldsymbol{q}_1,\ldots,\boldsymbol{q}_t$ belong to $\{\boldsymbol{q}\}\times\ensuremath{\mathcal{R}}^k$. As $t=\Tv(\ensuremath{\mathcal{R}}^k,m)$, there exists a partition of $[t]$ into $I_1,\ldots,I_m$ and a point $\boldsymbol{p}\in\{\boldsymbol{q}\}\times\ensuremath{\mathcal{R}}^k$ such that $\boldsymbol{p}\in\conv\left(\bigcup_{i\in I_{\ell}}\boldsymbol{q}_i\right)$ for all $\ell\in[m]$. For each $\ell\in [m]$, define $A_{\ell}$ to be $\bigcup_{i\in I_{\ell}}Q_i$. We have, for each $\ell\in[m]$
\[\boldsymbol{p}\in\conv\left(\bigcup_{i\in I_{\ell}}\boldsymbol{q}_i\right)\subseteq\conv\left(\bigcup_{i\in I_{\ell}}\conv(Q_i)\right)=\conv(A_{\ell})\]
 and the $A_{\ell}$ form the  desired partition.
\end{proof}

Here are the new bounds and exact values we get:

\begin{enumerate}
\item $\Tv(\mathbb{Z} \times \mathbb{R}^k,m) = 2(m-1)(k+1) +1$.
\item $\Tv(\mathbb{Z}^2 \times \mathbb{R}^k,m) = 4(m-1)(k+1)+1$.
\item $\Tv(\mathbb{Z}^3 \times \mathbb{R}^k,m) \leq 24(m-1)(k+1) -7$.
\item\label{last} $2^j(m-1)(k+1) + 1 \leq \Tv(\mathbb{Z}^j \times \ensuremath{\mathcal{R}}^k,m) \leq j2^j(m-1)(k+1) + 1$.
\item If $S' \subset \ensuremath{\mathcal{R}}^2$ with finite Helly number $\He(S')$, then $$\Tv(S' \times \mathbb{R}^k,m) \leq \He(S')(m-1)(k+1)+1.$$
\end{enumerate}

The lower bound in \eqref{last} is obtained by repeated applications of Proposition~\ref{stack} below. The upper bounds follow from Theorem~\ref{mixedint}, combined with the fact that $\Tv(\mathbb{Z},m) = 2m-1$, Theorem~\ref{Stverberg} for $S = \mathbb{Z}^2$, Theorem~\ref{3dZ}, the upper bound in Equation~\eqref{DL}, and Theorem~\ref{Stverberg} respectively.
\begin{proposition}~\label{stack}
Let $j$ and $k$ be two non-negative integers. Then we have
 \[\Tv(\mathbb{Z}^{j+1} \times \ensuremath{\mathcal{R}}^k,m) > 2\Tv(\mathbb{Z}^j \times \ensuremath{\mathcal{R}}^k, m)-2.\]
\end{proposition}
We prove Proposition~\ref{stack} by following the idea of the proof of Proposition 2.1 in \cite{Onn:1991wz}.
\begin{proof}[Proof of Proposition~\ref{stack}]
Assume toward a contradiction that
$$\Tv(\mathbb{Z}^{j+1} \times \ensuremath{\mathcal{R}}^k,m) \leq 2\Tv(\mathbb{Z}^j \times \ensuremath{\mathcal{R}}^k, m)-2.$$ Choose
$A$ to be a set of $\Tv(\mathbb{Z}^j \times \ensuremath{\mathcal{R}}^k,m) -1$ points in $\mathbb{Z}^j \times \ensuremath{\mathcal{R}}^k$ with no $m$-Tverberg partition.
Let $A_i = \{(\boldsymbol{a},i) \colon \boldsymbol{a} \in A,i = \{0,1\}\}.$ Since $A_0 \cup A_1 \subset \mathbb{Z}^{j+1} \times \ensuremath{\mathcal{R}}^k$ has cardinality $2\Tv(\mathbb{Z}^j \times \ensuremath{\mathcal{R}}^k, m)-2$, there exists an $m$-Tverberg partition $Y_1, Y_2, \dots, Y_m$ of $A_0 \cup A_1$ with $\boldsymbol{p} \in \bigcap_{i \in [m]} \conv(Y_i)$. Furthermore $\boldsymbol{p}$ is in $\mathbb{Z}^{j+1} \times \ensuremath{\mathcal{R}}^k$. That implies either $\boldsymbol{p} \in \conv(A_0)$ or $\boldsymbol{p} \in \conv(A_1)$. In either case $A_0$ or $A_1$ has an $m$-Tverberg partition, a contradiction with our choice of $A$.
\end{proof}

\section{A Generalized Fraction Selection Lemma: Proof of Theorem~\ref{thm:depthcovering}}\label{IPFSL}

Our proof relies on
the   simplicial partition theorem of Matou\v{s}ek, used in a  similar
manner as in~\cite{MR17}, which states  the following.
\begin{theorem}[\cite{M92}; see also~\cite{Cha00}]~\label{simplicialpartition}
 Given an integer $d \geq 1$ and a parameter $r$, there exists a constant $c_d \geq 1$ such that
for any set $P$ of $n$ points in $\Re^d$,
there exists an integer $s$
and a partition $\left\{P_1, \ldots, P_{s}\right\}$ of $P$ such that
\begin{itemize}
\item for each $i = 1, \ldots, s$, $\frac{n}{r} \leq |P_i|  \leq   \frac{2n}{r} $, and
\item any hyperplane intersects the convex hull of less than $c_d \cdot r^{1-\frac{1}{ d }}$ sets of the partition.
\end{itemize}
The constant $c_d$ is independent of $P$ and depends only on $d$.
\end{theorem}

We now prove the following key lemma.

\begin{lemma}
For any integer $d \geq 1$, there exists a constant
$c_{d}$ such that the following holds.
For any set $P$ of $n$ points in $\mathbb{R}^d$ and a real number $\alpha \in (0, 1]$,
there exists a partition $\mathcal{P} = \left\{ P_1, \ldots, P_{r} \right\}$,
$r = \left\lceil \left(\frac{4c_d}{\alpha}\right)^d \right\rceil$, of
$P$ such that
\begin{itemize}
\item $\frac{n}{2 r} \leq |P_i|  \leq \frac{2n}{r} $ for each $i = 1, \ldots, r$, and
\item the convex hull of any transversal $Q$ of $\mathcal{P}$ contains all
points in $\mathbb{R}^d$ of half-space depth at least $\alpha \cdot n$.
\end{itemize}
\label{lemma:depthpartitioning}
\end{lemma}
\begin{proof}
Apply the simplicial partition theorem (Theorem~\ref{simplicialpartition}) to $P$ with
$r = \left\lceil \left(\frac{4c_d}{\alpha}\right)^d \right\rceil$, and
let the resulting partition be $\left\{P'_1, \ldots, P'_s\right\}$.
Note that as $\frac{n}{r} \leq |P'_i|  \leq   \frac{2n}{r} $ for each $i = 1, \ldots, s$,
we have $\frac{r}{2} \leq s \leq r$. Now partition arbitrarily each of
$r-s$ biggest sets in $\left\{P'_1, \ldots, P'_s\right\}$ into two equal parts,
and let the resulting partition be $\left\{P_1, \ldots, P_r\right\}$.
Clearly each set of this partition has size in the interval $\left[ \frac{n}{2r}, \frac{2n}{r}\right]$.
This proves the first part.
Note also that each hyperplane intersects the convex hull of at most twice as many sets, i.e.,
less than $2 c_d \cdot r^{1-\frac{1}{ d }}$ sets of the partition $\left\{P_1, \ldots, P_r\right\}$.

To see the second part, let
$\boldsymbol{c}$ be any  point of half-space depth at least $\alpha \cdot n$, and
$Q$ any transversal of $\ensuremath{\mathcal{P}}$. For contradiction, assume
that $\boldsymbol{c} \notin \mathrm{conv}\left(Q\right)$. Then there exists
a hyperplane $H$ containing $\boldsymbol{c}$ in one of its two open half-spaces, say $H^-$,
and containing $\mathrm{conv}\left(Q\right)$ in the half-space $H^+$.
We will show that then there exists an index $i \in \left\{1, \ldots, r\right\}$
such that $P_i \subseteq H^-$. But then $P_i \cap Q = \varnothing$, a contradiction to
the fact that $Q$ is a transversal of $\ensuremath{\mathcal{P}}$.

It remains to show the existence of a set $P_i \in \ensuremath{\mathcal{P}}$ such that $P_i \subseteq H^-$.
Towards this, we bound $\left| P \cap H^- \right|$. Each point  of $P$
lying in $H^-$  belongs to a set $P' \in \ensuremath{\mathcal{P}}$ such that either
\begin{itemize}
\item $P'\subseteq H^-$, in which case we are done, or
\item $P' \nsubseteq H^-$. As $H^-$ contains at least one point of $P'$, we must have $\mathrm{conv}\left(P' \right) \cap H \neq \varnothing$. As argued earlier, there are less than
$2 c_d \cdot r^{1-\frac{1}{d}}$ such sets.
\end{itemize}
Thus we have
\begin{align}
\left|P \cap H^- \right| <  2 c_d  \cdot r^{1-\frac{1}{d}} \cdot   \frac{2n}{r}
= \frac{4 c_d\cdot n  }{\left\lceil  \left( \frac{4c_d }{\alpha}\right)^d\right\rceil^{\frac{1}{d}}}  \leq \alpha \cdot n.
\label{eq:upperbound}
\end{align}
On the other hand, as $\boldsymbol{c}$ has half-space depth at least $\alpha \cdot n$ and $\boldsymbol{c} \in H^-$,
we have $\left| P \cap H^- \right| \geq \alpha n$,
a contradiction to inequality~(\ref{eq:upperbound}).
\end{proof}

\remark{In particular, for $r= \left\lceil \left(\frac{4c_d}{\alpha}\right)^d \right\rceil$, there exist at least $\left( \frac{n}{2r} \right)^{r}$ $r$-sized subsets, each of whose convex hull contains all integer
points of depth at least $\alpha \cdot n$.}

\medskip

\begin{proof}[Proof of Theorem~\ref{thm:depthcovering}.]
Given the point set $P$ in $\Re^d$ and a point $\boldsymbol{q} \in \Re^d$ of half-space depth
$\alpha \cdot n$, apply
Lemma~\ref{lemma:depthpartitioning} with $P$ and $\alpha$ to get a partition consisting
of $r \geq d + 1$ sets,
where $r = \left\lceil \left(\frac{4c_d}{\alpha}\right)^d \right\rceil$.
By discarding at most $\frac{n}{2}$ points of $P$, we can derive a partition on the remaining points of $P$, say
$\mathcal{P} = \left\{P_1, \ldots, P_r \right\}$, such that the $P_i$'s are equal-sized disjoint subsets of $P$,
i.e., $|P_i| = \frac{n}{2r}$ for all $i = 1, \ldots, r$. Furthermore,
every transversal of $\mathcal{P}$ contains all points in $\Re^d$ of half-space depth at least $\alpha n$,
and thus $\boldsymbol{q}$.

For each transversal $Q$ of $\mathcal{P}$,
the point $\boldsymbol{q}$ lies in the convex hull of $Q$, and by Carath\'eodory's theorem, there exists
a $(d+1)$-sized subset of $Q$ whose convex hull also contains $\boldsymbol{q}$.
By the pigeonhole principle, there must exist $(d+1)$ sets of $\mathcal{P}$, say
the sets $P_1, \ldots, P_{d+1}$, such that at least
\begin{equation}
\frac{ \left(\frac{n}{2r}\right)^r }{\binom{r}{d+1} \left(\frac{n}{2r}\right)^{r-(d+1)}}
\geq \frac{ \left(\frac{n}{2r}\right)^{(d+1)}}{ \left(\frac{er}{d+1}\right)^{d+1}}
= \frac{1}{\left(\frac{er}{d+1}\right)^{d+1}} \cdot \prod_{i=1}^{d+1} |P_i|.
\label{eq:densityarg}
\end{equation}
distinct transversals of $\left\{P_1, \ldots, P_{d+1} \right\}$ contain $\boldsymbol{q}$.

\begin{sloppypar}

The rest of the proof  follows the one of Pach~\cite{P98}. In brief, we view the $P_i$'s as parts of a $(d+1)$-partite hypergraph with vertices corresponding to points in  $P$ and a hyperedge corresponding to each transversal of $\mathcal{P}$ containing $\boldsymbol{q}$. As there are $\Omega\left(n^{d+1}\right)$ such transversals by inequality~(\ref{eq:densityarg}), we apply a weak form of the hypergraph version of Szemer\'edi's regularity
lemma (see \cite{M92} Theorem 9.4.1) to derive
the existence of constant-fraction sized subsets  $P'_1 \subseteq P_1, \ldots, P'_{d+1} \subseteq P_{d+1}$ such that the following is true, for some constant $c'_d$:

\mystate{for \emph{any}   $P''_1 \subseteq P'_1, \ldots, P''_{d+1} \subseteq P'_{d+1}$,  with $|P''_i| \geq c'_d \cdot |P'_i|$ for $i = 1, \ldots, d+1$, we have the property
that there exists at least one transversal of $\left\{ P''_1, \ldots, P''_{d+1} \right\}$ whose convex hull contains $\boldsymbol{q}$.}

Then the same-type lemma (\cite{Barany:1998gi} Theorem 2) applied
to $\left\{P'_1, \ldots, P'_{d+1}, \left\{\boldsymbol{q}\right\} \right\}$
gives constant-fraction sized subsets $X_1 \subseteq P'_1, \ldots, X_{d+1} \subseteq P'_{d+1}$ such that each transversal of
$\left\{X_1, \ldots, X_{d+1} \right\}$ has the same order type with respect to $\boldsymbol{q}$.
\end{sloppypar}

We can set up the parameters for the same-type lemma and the weak regularity lemma such
that $|X_i| \geq c'_d \cdot |P'_i|$, for all $i = 1, \ldots, d+1$.
Then the weak regularity lemma implies that there exists
at least one transversal of $\left\{X_1, \ldots, X_{d+1}\right\}$
that contains $\boldsymbol{q}$. However, as each transversal of $\left\{X_1, \ldots, X_{d+1}\right\}$ has the same order type,
it must be that \emph{each} transversal
of $\left\{X_1, \ldots, X_{d+1} \right\}$   contains $\boldsymbol{q}$. These are the required subsets.

The size of each $X_i$ is a constant-fraction of $n$, say $|X_i| \geq c_{d, \alpha} \cdot n$,
where the constant $c_{d, \alpha}$ depends on the constants in inequality~(\ref{eq:densityarg}),
in the weak regularity lemma and in the same-type lemma. All of these depend only on $\alpha$ and $d$.
\end{proof}

\section{Acknowledgments}
This work was partially supported by NSF grant DMS-1440140, while the first and third authors were in residence at the
Mathematical Sciences Research Institute in Berkeley,  California, during the Fall 2017 semester. The first and second
authors were also partially supported by NSF grants DMS-1522158 and DMS-1818169. The fourth author was supported by
ANR grant SAGA (ANR-14-CE25-0016). Last, but not least, we are grateful for the information and comments we received
from  N. Amenta, G. Averkov, A. Basu, J.P. Doignon, P. Sober\'on, D. Oliveros, and S. Onn. In particular we thank Pablo Sober\'on
for noticing that the same methods used in an earlier version could be extended to yield Theorem~\ref{Stverberg}.
\bibliographystyle{amsalpha}

Metric Geometry (math.MG); Computational Geometry (cs.CG); Combinatorics (math.CO)


Abstract: Multiresolution Matrix Factorization (MMF) was recently introduced as an alternative to the dominant low-rank paradigm in order to capture structure in matrices at multiple different scales. Using ideas from multiresolution analysis (MRA), MMF teased out hierarchical structure in symmetric matrices by constructing a sequence of wavelet bases. While effective for such matrices, there is plenty of data that is more naturally represented as nonsymmetric matrices (e.g. directed graphs), but nevertheless has similar hierarchical structure. In this paper, we explore techniques for extending MMF to any square matrix. We validate our approach on numerous matrix compression tasks, demonstrating its efficacy compared to low-rank methods. Moreover, we also show that a combined low-rank and MMF approach, which amounts to removing a small global-scale component of the matrix and then extracting hierarchical structure from the residual, is even more effective than each of the two complementary methods for matrix compression.

\section{MMF on nonsymmetric matrices}

Based on the MMF approach described in~\Cref{section:mmf}, in the sequel we provide two approaches that extend MMF to general square matrices.

\subsection{MMF via an additive decomposition}

We begin by observing that any square matrix $A\in\R^{n\times n}$ can be written as the sum of a symmetric matrix and a skew-symmetric matrix as follows
\begin{align}
\label{eq:square_decomp}
A = \underbrace{\frac{A + A^T}{2}}_\text{symmetric} + \underbrace{\frac{A - A^T}{2}}_\text{skew-symmetric}
\end{align}

We propose performing MRA on each term separately. Since the first term is symmetric, it directly yields to a multiresolution matrix factorization. In particular, the factorization of $\frac{A + A^T}{2}$ can be obtained via a parallel variant of MMF such as pMMF~\citep{kondor2015parallel}. Thus, the problem of computing a multiresolution factorization of an asymmetric square matrix is now reduced to computing a multiresolution factorization for a skew-symmetric matrix.

\subsubsection{MMF for skew-symmetric matrices}
For skew-symmetric matrices, the MRA axioms, especially MRA1, are no longer applicable. The reason being that under the definition of smoothness in MRA1 $\sup_{v\in V\subseteq\R^n} \frac{\inner{v,Av}}{\inner{v,v}}$, for a skew-symmetric matrix $A$, the smoothness is always zero.

In symmetric MMF, the choice of the sparse orthogonal factors and the diagonal part of the core-diagonal matrix $H$ are based on the assumption that the multiscale basis vectors determined by the MMF algorithm are as close to eigenvectors as possible. In other words, the filtration defined in~\cref{eq:filtration} is a sequence of subspaces that is as close to a spectral filtration as possible. Indeed, MMF approximately diagonalizes a symmetric matrix using $O(n)$ rotations, where $n$ is the dimension of the matrix. We consider this intuition as our starting off point to extend MMF to skew-symmetric matrices.

Note that a matrix $A\in\R^{n\times n}$ is skew-symmetric if $A^T = -A$. Skew-symmetric matrices have complex eigenvalues and are therefore not diagonalizable in real space. Thus, a skew-symmetric matrix can never be perfectly represented using the same form of MMF as for a symmetric matrix over reals. In order to overcome this issue, we consider a well known observation~\citep{murnaghan1931canonical} that a skew-symmetric matrix, under a unitary transformation, assumes the so-called Murnaghan canonical form,\begin{align}
\begin{bmatrix}
\Lambda_1 & 0 & \ldots & 0\\
0 & \Lambda_2 & \ldots & 0\\
\vdots & \vdots & \ddots & \vdots  \\
0 & \ldots & 0 & \Lambda_L
\end{bmatrix}
\label{eqn:murnaghan}
\end{align}
where $\Lambda_i, i=1,\ldots, L$ are $2\times 2$ matrices of the form \[\Lambda_i = \begin{pmatrix} 0 & \lambda_i \\ -\lambda_i & 0\end{pmatrix}\]
where $\lambda_i \in \R$.

This observation affords a multiresolution factorization for skew-symmetric matrices identical to that for symmetric matrices with the difference that the core-diagonal matrix is allowed to have the following form:
\begin{align}
H = \begin{bmatrix}
H_\text{core} & 0 & \ldots & \ldots & 0\\
0 & \Lambda_1 & 0 & \ldots & 0\\
0 & 0 & \Lambda_2 & \ldots & 0\\
\vdots & \vdots & \ddots & \ddots & \vdots \\
0 & \ldots & \ldots & 0 & \Lambda_L
\end{bmatrix}
\label{eqn:asymmetric_core_diagonal}
\end{align}
where $\Lambda_i, i=1,\ldots, L$ are $2\times 2$ matrices as defined above.

A couple of observations based on this form are in order. First,  notice that the skew-symmetricity of the matrix in its MMF is always preserved.  Second, this form also allows for interaction between wavelets at adjacent levels. This feature is distinct from the original MMF procedure in which no such interactions occur. To obtain the factorization, the procedure is similar to the \textsc{GreedyJacobi} algorithm from~\cite{kondor2014multiresolution}, with the difference that in in the last step, instead of zeroing out off-diagonal elements, we sparsify the matrix into the Murnaghan normal form as described above.

Thus, the multiresolution decomposition of a square matrix can be done by factorizing each of the two parts from~\cref{eq:square_decomp} separately and writing $A$ as a sum of two MMFs. In other words, we may write
\begin{align*}
A \approx& P_1^T\ldots P_L^T H_\text{sym}\ts P_L \ldots P_1  \\ & +  Q_1^T\ldots Q_L^T H_\text{skew}\ts Q_L \ldots Q_1
\end{align*}
where
\begin{align}
\label{eqn:asymmetric_MMF}
\frac{A + A^T}{2} &\approx P_1^T\ldots P_L^T H_\text{sym} P_L \ldots P_1 \\
\frac{A - A^T}{2} &\approx Q_1^T\ldots Q_L^T H_\text{skew} Q_L \ldots Q_1
\end{align}

where $H_\text{skew}$ has the form defined in~\cref{eqn:asymmetric_core_diagonal}.

In the next section, we present another approach for multiresolution factorization.

\subsection{MMF via direct factorization}

In this approach, we aim to capture hierarchical structure by operating on the square matrix directly rather than decomposing it into an additive form. Recall that we would like to identify clusters of coordinates that a matrix $A$ more closely couples than others, and determine an appropriate multiresolution analysis of $\R^{n}$. When the matrix is symmetric, the linear transformation that it represents is self-adjoint. In the asymmetric case, depending on whether we are transforming $x\rightarrow x^TA$ or $x \rightarrow Ax$, we may end up with different ways in which $A$ might couple the coordinates.

We consider two different ways of performing multiresolution analysis of $\R^n$, one that is based on $A$, and another on $A^T$. Suppose $P\in\R^{n\times n}$ represents the wavelet transform in the former case, and $Q\in\R^{n\times n}$ of the latter. Thus, we have

\[ H = P^TAQ\]

which is the matrix $A$ in the bases $P$ and $Q$. In the symmetric case, $H$ takes a core-diagonal form, based on the fact that the eigenvalue decomposition converts a symmetric matrix to diagonal, and MMF uses orthonormal transformations that are designed to be as close to eigenvectors as possible. In the asymmetric case, we relax this assumption and allow some of the off-diagonal elements of $H$ to be nonzero. This is motivated by the fact that the frequencies or smoothness coefficients filtered by the left and right hand sides may not align with each other.

\begin{remark}
    When $A$ represents the adjacency matrix of a directed graph, we are essentially identifying hierarchical cluster structures considering 1) only the outgoing edges, and 2) only the incoming edges. The matrix $H$ gives a mapping between these two cluster structures.
\end{remark}

We define asymmetric MMF to be a multiresolution factorization of the following form
\begin{align}
A = P_1P_2\ldots,P_L H Q_L^TQ_{L-1}^T\ldots Q_1^T
\end{align}, where
\begin{itemize}
    \item Each $P_i, Q_i, i=1,\ldots,L$ are highly sparse, orthogonal rotation matrices similar to the rotations in a symmetric MMF
    \item $H\in\R^{n\times n}$ is a core-sparse matrix, that is, under an appropriate permutation of row and columns, $H$ would be a $2\times 2$ block-diagonal matrix with the upper left block being dense, and the other 3 blocks would be highly sparse
    \item When $P_i = Q_i$ and $H$ is in core-diagonal form with a symmetric core, the above factorization would be a symmetric MMF.
\end{itemize}

The computation of the asymmetric MMF is obtained using an approach similar to the symmetric MMF. The algorithm we propose is designed to minimize the Frobenius norm error
\[\| A - P_1P_2\ldots,P_L H Q_L^TQ_{L-1}^T\ldots Q_1^T\|_{F} \]
We determine the left and right rotations in a greedy manner. That is, we compute a sequence of rotated version of $A$
\[A \rightarrow P_1^TAQ_1 \rightarrow \ldots \rightarrow P_L^T\ldots P_1^TAQ_1\ldots Q_L.\]
Each of the rotations is computed by identifying sets of similar rows (for the left rotations) and columns (for the right rotations), and determining an appropriate $k$-point rotation. This requires us to compute Gram matrices on both the rows as well as the columns. Finally, we sparsify the final rotated version of $A$ to determine the core-sparse matrix $H$. We explore three ways of sparsification
\begin{enumerate}
    \item \textsc{CoreDiagonal}: retain only the diagonal elements of $H$
    \item \textsc{TopN}: retain the top $n$ elements in terms of magnitude
    \item \textsc{GreedyTopN}: select $n$ elements by magnitude greedily such that no two of the selected elements fall in the same row/column
\end{enumerate}
The full procedure is summarized in~\Cref{alg:asymmetric_mmf}.

The running time of this algorithm is $O(n^3)$, similar to $\textsc{GreedyJacobi}$ routine used in the original symmetric MMF. Nevertheless, computation can be made faster by similarly extending the parallel MMF algorithm to maintain row and column Gram matrices and computing separate left and right rotations. It must be noted that applying the left rotations to $A$ does not affect the column Gram matrix, and applying the right rotations does not affect the row Gram matrix.

\begin{algorithm}
    \caption{Asymmetric MMF via direct factorization}
    \begin{algorithmic}
        \REQUIRE core size $d$, matrix $A_0=A\in\R^{n\times n}$
        \STATE Set of active rows $S^r_0 \gets [n]$
        \STATE Set of active columns $S^c_0 \gets [n]$
        \FOR{$\ell = 1$ \TO $n-d$}
        \STATE Select a random row $i$ from the set $S^r_\ell$
        \STATE $G \gets A_{S^r_\ell,S^c_\ell}A_{i,S^c_\ell}^T$
        \STATE $j \gets S^r_\ell[\arg\max(G)]$
        \STATE $P_\ell \gets \text{Jacobi rotation on rows } i,j$
        \STATE $A \gets P_\ell^T A$
        \STATE Remove $\arg\min_{t\in\lbrace i,j\rbrace}\norm{A_{t,S^c_\ell}}_2$ from $S^r_\ell$
        \STATE
        \STATE Select a random column $i^\prime$ from the set $S^c_\ell$
        \STATE $G^\prime \gets A_{S^r_\ell,S^c_\ell}^TA_{S^r_\ell,i^\prime}$
        \STATE $j^\prime \gets S^c_\ell[\arg\max(G^\prime)]$
        \STATE $Q_\ell \gets \text{Jacobi rotation on rows } i^\prime,j^\prime$
        \STATE $A \gets Q_\ell$
        \STATE Remove $\arg\min_{t\in\lbrace i^\prime,j^\prime\rbrace}\norm{A_{S^r_\ell,t}}_2$ from $S^c_\ell$
        \ENDFOR
        \STATE $H = \textsc{CoreDiagonal}(A)$ or $\textsc{TopN}(A)$ or $\textsc{GreedyTopN}(A)$
        \RETURN $P_1,\ldots,P_{n-d}, H, Q_1, \ldots, Q_{n-d}$
    \end{algorithmic}
    \label{alg:asymmetric_mmf}
\end{algorithm}
\section{Conclusion}
In this paper, we have presented approaches to extend Multiresolution Matrix Factorization to nonsymmetric matrices. In general, efficiently estimating hierarchical structure in matrices is an increasingly important problem for which there currently exist very few practically useful methods. For future work we aim to use ideas from wavelet theory on more general spaces to define MMFs for general square matrices in which the MRA interpretation is clear.

\subsection{Numerical experiments}
\label{sec:asymmetric_experiments}
Similar to the evaluation of MMF in~\cite{teneva2016multiresolution}, we compare our asymmetric MMF formulations with the low-rank-based CUR decomposition~\citep{mahoney2009cur} for the task of compressing square matrices. The CUR decomposition is a widely used alternative to the singular value decomposition to compress large matrices. To reduce a matrix $A\in\R^{n\times n}$ to rank-$k$, the method proceeds by carefully selecting $k$ rows ($R\in\R^{n\times n}$) and columns ($C\in\R^{k\times n}$) and determines a matrix $U\in\R^{k\times k}$ such that the product $CUR$ is as close to the best rank-$k$ approximation of $A$ as possible.

To set up a comparison, we sample 70 square matrices from the UFlorida sparse matrix repository~\citep{davis2011university}. The matrices in this repository were sourced from various fields of science and engineering and contain data arising from circuit simulations, social networks, chemical processes, 2D/3D problems, etc. All the sampled matrices were chosen such that the numerical symmetry was less than $25\%$, and had a maximum dimensionality of $100,000$.

\subsubsection{Results on MMF via an additive decomposition}

We first present results comparing the asymmetric MMF obtained by the additive decomposition to CUR.
\Cref{tbl:mmf_cur_additive} shows the percentage of times the asymmetric MMF had a better compression error over CUR over 65 matrices. Since both the CUR and MMF algorithms are randomized, we report results averaged over 3 trials. CUR performs better than MMF in the majority of the cases.

\begin{table*}[!htbp]
    \small
    \centering
    \begin{tabular}{lcccccc}
        \toprule
        Matrix kind & num. matrices & 1\% & 10\% & 25\% & 50\% & 75\% \\
        \midrule
        \multicolumn{7}{c}{\textit{problems with underlying 2D/3D geometry}}\\
        structural problem&4&50.0&50.0&50.0&0.0&0.0\\
        computational fluid dynamics problem&3&0.0&0.0&0.0&0.0&0.0\\
        2D/3D problem&5&80.0&80.0&60.0&60.0&60.0\\
        materials problem&2&0.0&0.0&0.0&0.0&0.0\\
        \midrule
        \multicolumn{7}{c}{\textit{problems with no underlying geometry}}\\
        economic problem&11&18.2&9.1&9.1&9.1&9.1\\
        directed graph&19&5.3&5.3&5.3&0.0&0.0\\
        power network problem &3&0.0&0.0&0.0&0.0&0.0\\
        circuit simulation problem&2&0.0&0.0&0.0&0.0&0.0\\
        chemical process simulation problem&23&0.0&0.0&8.7&0.0&0.0\\
        statistical/mathematical problem&1&100.0&0.0&0.0&0.0&0.0\\
        counter-example problem&1&0.0&0.0&0.0&0.0&0.0\\
        \midrule
        total wins: & &13.5  &10.8  &12.2  &5.4  &5.4 \\
        \bottomrule
    \end{tabular}
    \caption{Matrix compression using asymmetric MMF (additive). Each entry of the rightmost 5 columns is the percentage of matrices on which MMF performed better than CUR, as measured by Frobenius norm compression error. Each column corresponds to the percentage size (w.r.t original size) to which a matrix is compressed.}
    \label{tbl:mmf_cur_additive}
\end{table*}

\subsubsection{Results on MMF via direct factorization}

Next, we report results on the direct factorization variant of the asymmetric MMF. In particular, we consider two different versions of the method based on what elements are retained in the core-sparse matrix $H$. In the first case, we retain the compressed $k\times k$ dense core, and $n-k$ elements chosen greedily according to magnitude and made sure that no two of these $n-k$ elements fall in the same row or column.

\Cref{tbl:mmf_cur_top_N_greedy} shows the percentage times this version of the asymmetric MMF wins over CUR in compression error.
\begin{table*}[!htb]
    \small
    \centering
    \begin{tabular}{lcccccc}
        \toprule
        Matrix kind & num. matrices & 1\% & 10\% & 25\% & 50\% & 75\% \\
        \midrule
        \multicolumn{7}{c}{\textit{problems with underlying 2D/3D geometry}}\\
        structural problem&4&50.0&50.0&25.0&0.0&0.0\\
        materials problem&1&0.0&0.0&0.0&0.0&0.0\\
        2D/3D problem&4&100.0&75.0&75.0&75.0&0.0\\
        computational fluid dynamics problem&3&100.0&100.0&0.0&0.0&0.0\\
        \midrule
        \multicolumn{7}{c}{\textit{problems with no underlying geometry}}\\
        statistical/mathematical problem&1&100.0&100.0&100.0&0.0&0.0\\
        economic problem&9&77.8&22.2&11.1&33.3&11.1\\
        power network problem&2&100.0&0.0&50.0&0.0&0.0\\
        directed graph&15&80&20&6.7&0.0&0.0\\
        chemical process simulation problem&21&80.9&23.8&14.3&0.0&0.0\\
        circuit simulation problem&2&100.0&0.0&0.0&0.0&0.0\\
        counter-example problem&1&0.0&0.0&0.0&0.0&0.0\\
        \midrule
        total wins: & &79.4 &30.2 &17.5 &9.5 &1.6 \\
        \bottomrule
    \end{tabular}
    \caption{Matrix compression using asymmetric MMF (\textsc{TopN}). See~\Cref{tbl:mmf_cur_additive} for a description of the table entries. }
    \label{tbl:mmf_cur_top_N_greedy}
\end{table*}
We observe that this version of MMF is significantly better than the additive version, and is able to capture structure in both problems with underlying known geometry and without underlying geometry.

Relaxing the structure of the matrix $H$ further, we report the results for simply storing the top $n-k$ elements in the core-sparse matrix in~\Cref{tbl:mmf_cur_sparse}.
\begin{table*}[!htb]
    \small
    \centering
    \begin{tabular}{lcccccc}
        \toprule
        Matrix kind & num. matrices & 1\% & 10\% & 25\% & 50\% & 75\% \\
        \midrule
        \multicolumn{7}{c}{\textit{problems with underlying 2D/3D geometry}}\\
        structural problem&4&100.0&100.0&100.0&50.0&50.0\\
        materials problem&1&100.0&100.0&100.0&100.0&100.0\\
        2D/3D problem&5&100.0&100.0&100.0&80.0&60.0\\
        computational fluid dynamics problem&3&100.0&100.0&100.0&0.0&0.0\\
        \midrule
        \multicolumn{7}{c}{\textit{problems with no underlying geometry}}\\
        directed graph&18&100.0&50.0&22.3&16.7&0.0\\
        chemical process simulation problem&21&100.0&90.5&67.7&57.1&38.1\\
        circuit simulation problem&2&100.0&100.0&100.0&50.0&0.0\\
        power network problem&3&100.0&100.0&100.0&66.7&33.3\\
        counter-example problem&1&100.0&100.0&100.0&100.0&100.0\\
        economic problem&10&100.0&100.0&100.0&70.0&50.0\\
        statistical/mathematical problem&1&100.0&100.0&100.0&100.0&0.0\\
        \midrule
        total wins: & &100.0 &84.1 &69.6&49.3 &29.0 \\
        \bottomrule
    \end{tabular}
    \caption{Matrix compression using asymmetric MMF (\textsc{GreedyTopN}). See~\Cref{tbl:mmf_cur_additive} for a description of the table entries. }
    \label{tbl:mmf_cur_sparse}
\end{table*}
Here, we see that MMF is able to capture structure and compress nearly all kinds of matrices at all compression levels.

From the numerical results, it seems that allowing interaction between wavelet levels is critical to achieving good approximation errors. This indicates that hierarchical structure in nonsymmetric matrices is more complex than that uncovered by simply extending the symmetric MMF analogous to an SVD.

For smaller compression ratios, CUR seems to be preferable for some types of problems such as directed graphs. It is possible that some matrices may have global-scale behavior that is superimposed over the hierarchical structure. This is possible, for instance, when the data is noisy. In the next part, we explore ways of combining MMF and low-rank approaches to achieve even better compression errors.

\subsubsection{Matrix compression via a combination of CUR and MMF}\label{sec:combination}

In order to better motivate this section, we first consider~\Cref{fig:mmferrors}, which shows the error of MMF approximation of a matrix $A$ as a function of the decay rate of its singular values.

\begin{figure}[htb]
    \includegraphics[width=\linewidth]{mmf_errors.png}
    \caption{Approximation error of symmetric MMF with different rates of decay in the eigenvalues. The $200 \times 200$ random matrices were generated by computing $Q D Q^T$, where $Q\in\R^{200\times 200}$ is a fixed, randomly generated orthogonal matrix. $D$ is a diagonal matrix whose entries are given by $\frac{1-e^{t(x-1)}}{1-e^{t}}$, where $t$ is the decay coefficient and $x$ takes $200$ uniformly spaced values between $0$ and $1$. The X-axis corresponds to $t$.}
    \label{fig:mmferrors}
\end{figure}

We observe that when the singular values decay at a super-linear rate, MMF does not approximate the matrix well. However, in many real-world matrices, the decay of singular values may not be uniform and may fluctuate to very high and low values. An example of such a spectrum is shown in~\Cref{fig:ex18}. In this case we see that although there are segments where the decay rate is sub-linear, the large decay rate around singular value 1500 may significantly impact the effectiveness of MMF on this matrix. Thus, it might be well motivated to first compress the matrix using low-rank methods and then use MMF on the residual. We describe this technique in~\Cref{alg:mmf_cur_combined}.

\begin{figure}[htb]
    \includegraphics[width=\linewidth]{ex8_svd.png}
    \caption{Plot showing the singular values of the `ex8' matrix from the UFlorida sparse matrix repository in descending order. Although the matrix is full-rank, the sharp drop in the singular values around index 800 imparts a low-rank nature to the matrix. (Note that the y-axis is on log-scale). Image sourced from UFlorida Sparse Matrix Repository~\citep{davis2011university}.}
    \label{fig:ex18}
\end{figure}

\begin{algorithm}
    \caption{Matrix compression using a combination of low-rank and multiresolution methods}
    \label{alg:mmf_cur_combined}
    \begin{algorithmic}
        \REQUIRE Square matrix $A$, compression size $k$, and $r$
        \STATE [C,U,R] = CUR(A, r)
        \STATE [P, H, Q] = MMF(CUR, k)
        \RETURN $\norm{PHQ^T}_F/\norm{A}_F$
    \end{algorithmic}
\end{algorithm}

For various values of $k$, we report the errors for compressing various matrices to 5\% of their original size in~\Cref{fig:mmf_cur_combined}.

\begin{figure}[htb]
    \centering
    \includegraphics[width=0.45\linewidth]{mmf_cur_combined/b_dyn.png}
    \includegraphics[width=0.45\linewidth]{mmf_cur_combined/b2_ss.png} \\
    \includegraphics[width=0.45\linewidth]{mmf_cur_combined/CSphd.png}
    \includegraphics[width=0.45\linewidth]{mmf_cur_combined/cz2548.png} \\
    \includegraphics[width=0.45\linewidth]{mmf_cur_combined/ex8.png}
    \includegraphics[width=0.45\linewidth]{mmf_cur_combined/extr1.png} \\
    \caption{Frobenius norm error as a function of CUR compression rank for compressing the matrix to 5\% of its original size using the combined low-rank + multiresolution method. Solid blue line indicates error of compression with only CUR to 5\% matrix size, while the dotted blue line indicates the same, but with MMF.}
    \label{fig:mmf_cur_combined}
\end{figure}

In most of the matrices, we observe that there exists at least one rank to which the matrix can be first compressed using CUR before compressing with MMF, that results in lower compression errors than each of CUR and MMF methods individually. In practice, there is no method to determine what this rank would be for a given matrix, however, it is an interesting and worthwhile problem to be able to remove a small global-scale component of the matrix, after which the hierarchical structure may be easily extracted.

In all the experiments, the running time of MMF is equal or slightly slower than that of low-rank-based methods. However, we note that our algorithms can be made significantly faster by straightforwardly adapting parallel versions of MMF~\citep{kondor2015parallel} to the asymmetric case. We expect speedups similar to those achieved in the symmetric case~\citep{teneva2016multiresolution}.
\section{Introduction}

A wide variety of naturally occurring data, represented as large matrices, often has complex hierarchical structure. For instance, this is the case in data originating from social networks, complex biological interactions, circuit simulations and so on. With the increasing size of such matrices, there is a need to represent and compress them efficiently. Towards this end, a number of methods have been proposed~\citep{mahoney2009cur,coifman2006diffusion,hackbusch1999sparse, halko2009}. All such methods make an implicit or explicit assumption about the nature of such matrices, which serves as a prior to facilitate efficient factorization and compression. Frequently, the assumption in question is that the matrices are of low rank. For example, Principal Component Analysis (PCA) decomposes a symmetric matrix $A$ into the form \begin{align*}A = Q^T D\ts Q\end{align*} where $Q$ is a orthogonal matrix and $D$ is diagonal. Notably, such methods are single-level algorithms. Moreover, the columns of $Q$ are usually dense, which does not adequately capture strong locality properties that might be exhibited, for instance, in real world graphs.

Taking a different tack, Multiresolution Matrix Factorization (MMF)~\citep{kondor2014multiresolution} framed matrix factorization as a type of multiresolution analysis (MRA) over a discrete metric space whose metric structure is represented by a self-adjoint operator. Thus, in contrast to low-rank approaches, MMF factorizes a symmetric matrix $A$ in the following multilevel manner: \begin{align*}A = Q_1^T Q_2^T \dots Q_L^T H\ts Q_L \dots Q_2 Q_1\end{align*} where each of the carefully selected $Q_i$ matrices are orthogonal and extremely sparse, and the matrix $H$ is close to diagonal. MMF is able to discover soft hierarchical arrangements of localized structures in the data represented by the matrix.  It is amenable to parallelization and fast implementations~\citep{kondor2015parallel}. The wavelet bases uncovered by MMF also provide a natural sparse approximation for functions defined over the space. For instance, if the matrix $A$ is the normalized graph Laplacian, then the factorization provides us with wavelet bases for approximating functions on graphs.

While MMF has been shown to be useful in various practical settings~\citep{teneva2016multiresolution,ding2017multiresolution,ithapu2017decoding}, one notable drawback of the method is that it is wholly defined using a self-adjoint operator such as the graph Laplacian. Therefore, MMF can only handle a multi-level factorization of symmetric matrices. However, many naturally occurring data are represented more concisely as an nonsymmetric matrix; the underlying space might only have a quasi-metric structure. This is the case for instance, when we are working with directed graphs. For data living on such spaces, hierarchical or multiscale structure is as important, albeit more complicated, and thus there is much to be gained by defining appropriate sparse wavelet bases for function approximation.

Defining such wavelet bases, in particular for functions on directed graphs, is an active area of research~\citep{sevi2018harmonic, satoshi2019, mhaskar2018}. However, there is currently no consensus on the correct approach for doing so. Further, existing methods are difficult to implement in practice for downstream applications. In this paper, we do not make an attempt to directly extend multiresolution analysis (MRA) in the spirit of MMF to such spaces, but rather provide practical methods to leverage intuitions used in MMF to make it operable on general square matrices. We show experimentally that our methods can perform very well in various real world matrix compression tasks.

To conclude this section, our contributions in this paper are twofold: 1) we provide two practical ways to adapt MMF to nonsymmetric matrices, 2) we argue that extending MRA to nonsymmetric matrices is not analogous to developing a multi-level SVD; in particular, we show experimentally that it is important to consider interactions between different levels in the hierarchical structure and lastly 3) we also show that a combined low-rank and MMF approach, which amounts to removing a small global-scale component of the matrix and then extracting hierarchical structure from the residual, is even more effective than each of the two complementary methods for matrix compression

After a description of notation used in this paper, in~\Cref{sec:mmf}, we give a brief introduction to Multiresolution Matrix Factorization.

\subsubsection*{\bibname}}

\renewcommand{\todo}[1]{{\color{red} TODO: #1}}

\bibliographystyle{apalike}

\begin{document}

\twocolumn[

\aistatstitle{Asymmetric Multiresolution Matrix Factorization}

\aistatsauthor{ Pramod Kaushik Mudrakarta \And Shubhendu Trivedi \And  Risi Kondor }

\aistatsaddress{ University of Chicago \And  CSAIL, MIT \And Flatiron Institute, NY \\ University of Chicago} ]

\input{abstract.tex}
\input{introduction.tex}
\input{notation.tex}
\input{mmf.tex}
\input{relatedwork.tex}
\input{asymmetricmmf.tex}
\input{experiments.tex}
\input{conclusion.tex}

\section{Multiresolution Matrix Factorization}\label{section:mmf}
\label{sec:mmf}

Given a measurable space $X$ and square integrable functions $L_2(X)$ on $X$, multiresolution analysis (MRA) constructs a sequence of spaces of functions of increasing smoothness:
\[
\def1.5{1.5}
\begin{array}{ccccccccc}
L_2(X) = V_0 & \rightarrow & V_1 &\rightarrow &V_2 &\rightarrow &\ldots &\rightarrow &V_L \\
& \searrow & & \searrow & & \searrow & & \searrow & \\
& & W_1 & & W_2 & & \ldots & & W_L
\end{array}\] where each $V_\ell$ is split into a smoother space $V_{\ell+1}$ and a rougher space $W_{\ell+1}$, such that $V_\ell = V_{\ell+1} \oplus W_{\ell+1}$. The length-scale over which a function $f\in V_\ell$ varies increases with $\ell$ and thus makes this decomposition an analysis of $L_2(X)$ multiscale/multiresolution.

\cite{kondor2014multiresolution} extends MRA to functions on discrete spaces (e.g. undirected graphs) by interpreting a given symmetric matrix as a discrete version of a continuous self-adjoint operator. Specifically, for $\vert X \vert = n$,~\cite{kondor2014multiresolution} defines multiresolution analysis with respect to a symmetric matrix $A\in\R^{n\times n}$ as a successive filtration \begin{align}\label{eq:filtration}V_L \subset \ldots \subset V_1 \subset V_1 \subset V_0 = L(X) \cong \R^n\end{align} where each $V_\ell$ has an orthonormal basis $\Phi_\ell := \lbrace \phi_m^\ell \rbrace$, and its complementary space $W_\ell$ has an orthonormal basis $\Psi_\ell := \lbrace \psi_m^\ell \rbrace$ that satisfy the following three axioms:
\begin{description}
    \item[MRA1] The sequence~\cref{eq:filtration} is a filtration of $\R^n$ in terms of smoothness. In particular, \[ \eta_\ell = \sup_{v\in V_\ell} \frac{\inner{v,Av}}{\inner{v,v}} \] decays with $\ell$.
    \item[MRA2] The basis vectors of $W_\ell$ are localized in the sense that \[ \mu_l = \max_{m\in\lbrace 1,\ldots,d_l\rbrace} \norm{\psi_m^\ell}_0 \] increases no faster than a certain rate.
    \item[MRA3] Suppose the matrix $U_\ell$ represents the matrix expressing $\Psi_l \cup \Phi_l$ in terms of the previous basis $\Phi_{\ell-1}$. Then, $U_\ell$ is sparse. ($\Phi_0$ is taken to be the standard basis.)
\end{description}

Once an overcomplete multiscale basis satisfying MRA1-3 for $\R^n$, i.e., $U_\ell, \ell=1,\ldots, L$, is determined, it can be used to compress the matrix $A$. Applying each of the $U_\ell$'s sequentially, we obtain versions of $A$ compressed to various scales:

\[A\mapsto \underbrace{U_1 A\ts U_1^T}_{A_1} \mapsto \ldots
\mapsto \underbrace{U_L\ldots U_2 U_1 A\ts U_1^T U_2^T\ldots U_L^T}_{A_L = H}\]

Since the choice of the $U_\ell$'s aligns with MRA1,~\cite{kondor2014multiresolution} argues that the matrices $A_1,\ldots, A_L$ and $H$ are approximately in core-diagonal form. That is, up to a permutation, each of those matrices has a block-diagonal form, with one block being a diagonal matrix.

Formally, Multiresolution Matrix Factorization is defined as an approximate multilevel factorization of the form
\begin{align}
\label{eq:mmf}
A \approx Q_1^T Q_2^T \ldots Q_L^T H\tts Q_L \ldots Q_2 Q_1,
\end{align}
where the matrices $Q_1,\ldots,Q_L$ and $H$ obey the following conditions:
\begin{enumerate}[~1.]
    \item Each $Q_\ell \in \R^{n\times n}$ is orthogonal and highly sparse.
    In the simplest case, each $Q_\ell$ is a Givens rotation, i.e., a matrix which differs from the identity in
    just the four matrix elements
    \begin{align*}
    [Q_\ell]_{i,i} = \cos \theta,&\qqquad [Q_\ell]_{i,j} = - \sin \theta,\\
    [Q_\ell]_{j,i} = \sin \theta,&\qqquad [Q_\ell]_{j,j} = \cos \theta,
    \end{align*}
    for some pair of indices $(i,j)$ and rotation angle $\theta$.
    Multiplying a vector with such a matrix rotates it counter-clockwise by $\theta$ in the $(i,j)$ plane.
    More generally, $Q_\ell$ is a so-called
    $k$-point rotation,
    which rotates not just two, but $k$ coordinates.
    \item Typically, in MMF factorizations $L\<=O(n)$, and the size of the active part of the \m{Q_\ell} matrices decreases according to a set schedule $n = \delta_0 \ge \delta_1 \ge \ldots \ge \delta_L$.
    More precisely, there is a nested sequence of sets $[n] = S_0 \supseteq S_1 \supseteq \ldots \supseteq S_L$
    such that the $[Q_\ell]_{\overline{S_{\ell-1}},\overline{S_{\ell-1}}}$
    part of each rotation is the $n-\delta_{\ell-1}$
    dimensional identity. $S_\ell$ is called the \textbf{active set} at level $\ell$. In the simplest case,  $\delta_\ell=n-\ell$.
    \item $H$ is an $S_L$-core-diagonal matrix, which means that it
    is block diagonal with two blocks: $H_{S_L,S_L}$, called the core, which is dense, and
    $H_{\wbar{S_L},\wbar{S_L}}$ which is diagonal. In other words, $H_{i,j}\!=\!0$ unless $i,j\tin S_L$ or $i\!=\!j$.
\end{enumerate}

Given a symmetric matrix, the factorization is carried out by minimizing the Frobenius norm error between the original matrix and its factorized form. Algorithms such as \textsc{GreedyJacobi}~\citep{kondor2014multiresolution} or pMMF~\citep{teneva2016multiresolution} are typically used in practical applications~\citep{teneva2016multiresolution,ding2017multiresolution,mudrakarta2017generic}

Having discussed the core ideas behind MMF, we briefly review some related art before moving forward.

\ignore{
\section{Multiscale structure in nonsymmetric matrices}

In this section, I want to write about why its not trivial to extend MMF to nonsymmetric matrices, and also explain a little bit about the intuition behind the formulations in the next sections. Two points to address:
\begin{itemize}
    \item How well is MRA1 satisfied in the formulations proposed? i.e., what measure of smoothness are we using?
    \item
\end{itemize}

\section*{Notes (just for guidance - not intended to be part of the paper)}
\begin{itemize}
    \item The broad goal here is to efficiently compress and estimate large matrices arising in the real world
    \item This process involves making assumptions on the matrix -- the most common one being the low-rank assumption, which is typically modeled via PCA
    \item The problem with PCA is that the eigenvectors are always dense, which does not capture cases where the matrix ``more closely couples certain clusters of nearby coordinates than those farther apart w.r.t. some underlying topology''. Sparse PCA solves the locality problem, but does not capture global scale structure.
    \item MMF is a method that aims to capture multiscale structure in discrete spaces such as graphs.  Matrices are considered as operators on vectors, and MMF essentially decomposes that operator in terms of a multiscale basis.
    \item Given a continuous operator such as a Laplacian. The application of this operator on a function can be expressed in terms of the eigenfunctions of the operator. In the case of the Laplacian, the eigenfunctions are the Fourier basis functions.

    Similarily, when the symmetric matrix is the operator, the application of this operator (i.e., matrix-vector product) can be expressed in terms of eigenvectors of the matrix. Thus, in this way the eigenvalue decomposition is an analogue of the Fourier decomposition.
    \item Multiresolution Analysis (MRA) captures multiscale structure in a given space of functions by repeatedly reducing the space so that the smoothness of functions increases in some sense. For functions on $\mathbb{R}$, Mallat (1989) defines MRA in terms of the wavelets constructed via dilation and translation of certain canonical functions.

    \pramod{Question: how does Mallat show that this form of constructing wavelets correspond to the idea of compressing the space by smoothness of functions? In particular, what exactly is the measure of smoothness that he uses, and how is it related to symmetricity/asymmetricity of an operator that is used to determine the smoothness of a function?}
    \item The MMF paper defines MRA on discrete spaces similarly, in terms of successively compressing a given sucspace. Note that each subpsace here would be defined by a finite set of basis vectors. Three axioms are defined that charactersize the smoothness of each space (MRA1), the localization (MRA2), and the existence of fast wavelet transform (MRA3).

    \pramod{What does it mean to relax the criterion that each $V_\ell$ must be a strict subset of $V_{\ell-1}$?}

    \pramod{How is smoothness defined in the asymmetric case?}

    \item Now that we can determine a wavelet basis in matrix form $U_1, \ldots U_L$ , the given operator $A$ can be transformed into it by performing
    $$H = U_L\ldots U_1 A\ts U_1^T \ldots U_L^T$$

    In the symmetric case, the wavelet functions are assumed to be close to eigenvectors of $A$, so that the MRA1 criterion ensures that each subspace is smoother than the previous. This makes the structure of $H$ core-diagonal. MMF-factorizable matrices are defined to be those for which applying the wavelet transformation results in $H$ being core-diagonal.

    \pramod{In reality, $H$ never is core-diagonal. We manually zero out entries of the wavelet-transformed matrix so that it comes to that form. That makes MMF an approximate factorization.}

    \pramod{In some of the asymmetric versions we develop, this criterion is relaxed, thereby violating the MRA1 axiom a bit, but that results in better compression.}
\end{itemize}
}
\section{Notation}
We denote $[n] = {1, 2, \ldots, n}$. Inner products between vectors are denoted by the operator $\inner{\cdot,\cdot}$. The $(i,j)$-th entry of a matrix $A$ is denoted by $A_{i,j}$. If $S\subseteq [n]$ is a set of indices, $A_{S,S}$ represents the submatrix of $A$ corresponding to those indices. $\overline{S}$ denotes the complement of a set. When $S$ can be interpreted as an ordered set, for instance, if it is a set of indices, then $S[i]$ represents the $i$-th index from that set.
\section{Related Work}

The work most closely related to some of the ideas in this paper is that on defining wavelets on functions on directed graphs. Two notable recent works on the theme are that of \cite{sevi2018harmonic} and \cite{mhaskar2018}. \cite{sevi2018harmonic} did so by using a random walk operator, while \cite{mhaskar2018} used an extension of diffusion polynomial frames~\citep{mhaskarmaggioni}, using a different operator. In fact, the influential work on diffusion wavelets~\citep{coifman2006diffusion}, provides a more general theory in which symmetry of the diffusion operator is not necessary. However, it is unclear how to lift these ideas for defining a principled asymmetric MMF that has also a clear MRA interpretation. On the side of matrix factorization, works that have some resemblance to our approach are \cite{LiYangButterfly, LiYangInterpolate}, in which the left and right matrices don’t have to be related by transposition. However these approaches don't have a clear wavelet based motivation.

Numerical Analysis (math.NA); Machine Learning (cs.LG); Machine Learning (stat.ML)