Counting primitive subsets and other statistics of the divisor graph of $\{1,2, \ldots n\}$

Abstract: Let $Q(n)$ denote the count of the primitive subsets of the integers $\{1,2\ldots n\}$. We give a new proof that $Q(n) = \alpha^{(1+o(1))n}$ which allows us to give a good error term and to improve upon the lower bound for the value of this constant $\alpha$. We also show that the method developed can be applied to many similar problems that can be stated in terms of the divisor graph, including other questions about primitive sets, geometric-progression-free sets, and the divisor graph path-cover problem.

\section{Previous work on counting primitive sets}

A set of integers is called a \textit{primitive set} if no integer in the set is a divisor of another.  For example the prime numbers form a primitive set, as do the integers with exactly $k$ prime factors for any fixed $k$.

In 1990, as part of a paper \cite{camerd} filled with conjectures and problems related to subsets of the integers with various properties, {Cameron}  and {Erd\H{o}s} considered the counting function $Q(n)$ of the number of primitive subsets of the integers up to $n$.  They note that the bounds $ (\sqrt{2})^n  < Q(n) < 2^n$ can be observed immediately from the observation that for fixed $n$, the set of integers $\left \{ \left \lfloor \frac{n}{2} \right\rfloor+1,\left \lfloor \frac{n}{2} \right\rfloor +2,\ldots, n\right \}$ is primitive (a set of size $\lceil \frac{n}{2}\rceil$), as are any of the $2^{\lceil\frac{n}{2}\rceil}$ subsets of this set.

Based on this observation, they conjecture that $\displaystyle{\lim_{n\to \infty} Q(n)^{1/n}}$ exists. In addition to this conjecture, they outline a proof that the bounds above can be improved to \begin{equation}
     1.55967^n \ll Q(n) \ll 1.60^n. \label{bds:ErdCam}
\end{equation}

Many of the conjectures in \cite{camerd} have attracted a substantial amount of attention, especially those of an additive nature.  More recently some of the multiplicative questions have begun to attract attention.  Recently, their conjecture above about the count of primitive sets was proven by Angelo \cite{angelo}.
\begin{theorem}[Angelo, 2017] \label{thm:angelo}
     The limit $\displaystyle{\lim_{n\to \infty} Q(n)^{1/n}}$ exists.  Equivalently, there exists a constant $\alpha$ such that $Q(n) = \alpha^{(1+o(1))n}$.
\end{theorem}

The proof proceeds by considering subsets of the integers $[1,n]$ in which the ratio between any two included elements is not an ($s$-smooth) integer (an integer without any prime factors larger than $s$) for fixed $s$, and then allowing $s$ to tend to infinity.  Unfortunately the proof was not effective in the sense that it doesn't give a way to improve the bounds in \eqref{bds:ErdCam}.

More recently Vijay \cite{vijay} considered the problem of counting primitive subsets of $[1,n]$ of maximum size, called \textit{maximum primitive sets}.  A pidgeonhole principle argument shows that for any $n$ the maximum size of a primitive subset of the integers up to $n$ is $\lceil \frac{n}{2}\rceil$.  He considers only even $n$, although the result extends immediately to all integers $n$.  In particular he shows that the count $M(n)$ of the number of primitive subsets of $[1,n]$ of size $\lceil \frac{n}{2}\rceil$ satisfies the bounds
\[1.303^n \ll M(2n) \ll 1.408^n\]
or, in terms of $n$,
\[1.141^n \ll M(n) \ll 1.187^n.\]

These questions have also recently been investigated independently by Liu, Pach and Palincza \cite{lpp}, and some of the results of that paper overlap with the results presented here.  While many of the ideas are similar, we take a slightly different approach that gives an error term and computationally seems to provide better lower bounds, though is not so good at giving numerical upper bounds.  They present a method to effectively compute $\alpha$, and improve the bounds in \eqref{bds:ErdCam} to

\[1.571068^n \ll Q(n) \ll 1.574445^n.\]
They also consider maximal primitive sets, and show that the limit $\lim_{n \to \infty} M(n)^{1/n}$ exists, (which we denote $\beta$)  show that this constant can be effectively computed, and improve the bounds above to
\[1.148172^n \ll M(n) \ll 1.148230^n \text{ \hspace{1cm} or \hspace{1cm}}  1.3183^n \ll M(2n) \ll 1.31843^n.\]

\section{results and other applications}

We show that these two constants $\alpha$ and $\beta$ can be given explicitly in terms of an infinite product described in Section \ref{sec:countprim}. This observation allows us to prove the following.

\begin{theorem} \label{thm:alpha}
    For any $\epsilon>0$, and $n$ sufficiently large, the number of primitive subsets of $[1,n]$ is \[Q(n) = \alpha^{n\left(1+O\left(\exp\left(-\sqrt{\left(\frac{1}{6}-\epsilon\right)\log n \log \log n}\right)\right)\right)}.\]
    or equivalently,
    \[\log Q(n) = n\log \alpha + O\left(n\exp\left(-\sqrt{\left(\tfrac{1}{6}-\epsilon\right)\log n \log \log n}\right)\right)\]
    and the constant $\alpha$ is effectively computable.
    \end{theorem}

This result is a corollary of a our main theorem, which can be used to compute a wide variety of statistics about subsets of $\{1,2,\ldots n\}$ with certain multiplicative structure.   Before we can state our main theorem however we must introduce a definition. Consider the divisor graph of a set of integers, the graph obtained by treating each number as a vertex and connecting two integers by an edge if one divides another.  In this context, for example, a primitive subset would correspond to an independent set of vertices.

We also consider the divisor graphs of the integers in an interval $[a,n]$.  In general these graphs won't be connected, and we will be interested primarily in the component of this graph connected to the vertex $a$.  We say that a function $f(a,n)$ \textit{depends only on the connected component of $a$ in the divisor graph of the interval $[a,n]$} if $f(a,n)=f(b,m)$ whenever the connected component of $b$ in the divisor graph of $[b,m]$ is isomorphic to that of $a$ in $[a,n]$ (with the vertex of $a$ corresponding to $b$ in the isomorphism).  With this, we can state our main theorem.

\begin{theorem}[Main Theorem] \label{thm:main} Suppose $\epsilon>0$, $A\geq0$ and $f(a,n)$ is a bounded function $|f(a,n)|\leq A$ that depends only on the connected component of $a$ in the divisor graph of the interval $[a,n]$.  Then there exists a constant \[C_f = \sum_{i=1}^\infty  \sum_{\substack{d\\ P^+(d)\leq i}} \sum_{t \in [id,(i+1)d)} \left(\frac{f(d,t)}{t(t+1)}\prod_{p\leq i} \frac{p-1}{p}\right)\]
such that \[\sum_{a=1}^n f(a,n) = nC_f + O_A\left(n\exp\left(-\sqrt{\left(\tfrac{1}{6}{-}\epsilon\right)\log n \log \log n}\right)\right).\]
\end{theorem}
As we will see, many questions about subsets with multiplicative structure can be regarded as questions about this divisor graph, and we demonstrate several different applications of this result.  First we apply it to several related problems about primitive sets. For example, we obtain the corresponding result for counting maximum primitive subsets as well.
\begin{theorem} \label{thm:beta}
The limit $\lim_{n \to \infty} M(n)^{1/n} = \beta$ exists.  Furthermore, for any $\epsilon>0$ and $n$ sufficiently large,
\[M(n) = \beta^{n\left(1+O\left(\exp\left(-\sqrt{\left(\frac{1}{6}-\epsilon\right)\log n \log \log n}\right)\right)\right)}\] and the constant $\beta$ is effectively computable.
\end{theorem}

We also use it to count maximal primitive subsets of $[1,n]$, where a primitive subset is maximal if no additional integer from the interval can be added to the set without having one integer divide another in the subset. Note that all maximum primitive subsets are maximal, but not all maximal primitive subsets are a maximum primitive set.  For example, $\{2,3,5,7\}$ is a maximal primitive subset of the integers up to $9$, but not a maximum primitive set, as $\{5,6,7,8,9\}$ is larger.  We obtain analogues of these theorems for $m(n)$, the count of the maximal primitive subsets of $[1,n]$.

\begin{theorem} \label{thm:eta}
The limit $\lim_{n \to \infty} m(n)^{1/n} = \eta$ exists.  Furthermore, for any $\epsilon>0$ and $n$ sufficiently large,
\[m(n) = \eta^{n\left(1+O\left(\exp\left(-\sqrt{\left(\frac{1}{6}-\epsilon\right)\log n \log \log n}\right)\right)\right)}\] and the constant $\eta$ is effectively computable.
\end{theorem}

By estimating the relevant constant from Theorem \ref{thm:main} we can numerically approximate each of these constants to arbitrary precision.  In Section \ref{sec:numerics} we describe the computations performed to compute partial sums of this product, which give us bounds for the size of these constants.

\begin{theorem} \label{thm:numalpha}
The constants $\alpha$, $\beta$, and $\eta$  satisfy the bounds \[1.572939<\alpha\]
\[1.148192<\beta \hspace{3mm} (\text{or } 1.318345<\beta^2 ) \]
\[1.212500<\eta < 1.240904.\]
\end{theorem}

The lower bounds for $\alpha$ and $\beta$ are stronger than those  appearing in \cite{lpp} however our method has not yet been able to improve upon the upper bounds obtained by their methods.  Combining their result with ours gives the new known ranges
\[ 1.572939 < \alpha < 1.574445.\]
\[ 1.148192 < \beta < 1.148230 \hspace{3mm} (\text{or }  1.318345 < \beta^2 < 1.31843).\]

The constant $\eta$ and the problem of counting maximal primitive subsets of the integers does not appear to have been considered before in the literature, so in this situation we have included the upper bound obtained by our method as well.

Despite our method's success at counting primitive subsets with a variety of properties, it isn't clear that it can be used to count primitive subsets of sizes other than $\left\lceil\frac{n}{2}\right\rceil$.
It would be interesting, for example, to study the distribution of the sizes of the primitive subsets of $\{1,2 \ldots n\}$.  As a partial result in this direction we use the bounds for $\alpha$ and $\eta$ from Theorem \ref{thm:numalpha} to bound the median size of such a subset.

\begin{theorem} \label{thm:median}
Let $\nu(n)$ denote the median of the sizes of the primitive subsets of $\{1,2,...n\}$.  Then for sufficiently large $n$, we have \[0.168153n <\nu(n) < 0.391752n.\]
\end{theorem}

It is natural to conjecture from this that $\nu(n) \sim vn$ for some constant $v$, this and other questions are posed in Section \ref{sec:questions}.

We briefly introduce a few additional applications of the main theorem to problems outside of primitive sets.  In Section \ref{sec:gpf} we consider geometric progression free subsets of the integers up to $n$.   Here we consider geometric progressions of the form $a,ar,ar^2$ with $a$ a positive integer, and $r$ an integer ratio greater than 1.  Unlike the case of primitive sets, where it is easy to see that the largest primitive subset of $\{1,2,\ldots n\}$ has size $\left\lceil\frac{n}{2}\right\rceil$, the problem of determining the largest subset of these integers avoiding 3-term geometric progressions is not so clear.  Various authors have considered the problem of determining the greatest possible density of a set of integers that is free of 3-term geometric progressions.  (See \cite{rankin}, \cite{riddell}, \cite{bbhs}, \cite{NO}, \cite{mcnewgpf}).  Let $G(n)$ denote the size of the largest subset of the integers up to $n$ avoding a 3-term geometric progression with integral ratio.  The argument for the bounds for the upper density of a subset of integers avoiding such progressions in \cite{mcnewgpf} shows that \[0.81841n \leq G(n) \leq 0.81922n\] and that $G(n)\sim bn$ for an effectively computable constant $b$.
The main theorem allows us to reprove this result with an error term.

\begin{theorem} \label{thm:gpf}
For any $\epsilon>0$, $G(n)$ the size of the largest geometric progression free subset of the integers up to $n$ satisfies \[G(n) = bn\left(1+O\left(\exp\left(-\sqrt{\left(\tfrac{1}{6}-\epsilon\right)\log n \log \log n}\right)\right)\right).\].
\end{theorem}

We can likewise count the geometric progression free subsets of these integers, which we denote by $H(n)$.

\begin{theorem} \label{thm:theta}
The limit $\lim_{n \to \infty} H(n)^{1/n} = \theta$ exists.  Furthermore, for any $\epsilon>0$ and $n$ sufficiently large,
\[H(n) = \theta^{n\left(1+O\left(\exp\left(-\sqrt{\left(\frac{1}{6}-\epsilon\right)\log n \log \log n}\right)\right)\right)}.\] The constant $\theta$ is effectively computable and satisfies the bounds
\[1.901448 < \theta < 1.925556.\]
\end{theorem}

In Section \ref{sec:pathcov} we consider the problem of covering the divisor graph of $\{1,2,\ldots n\}$ with as few vertex-disjoint paths as possible. (A minimal path cover of the graph)  Various authors have studied the paths in this divisor graph.  Pomerance \cite{PDG} showed that the length of the longest vertex-disjoint path in this divisor graph has length $o(n)$ and now, due to Erd\H{o}s and Saias \cite{erdsai}, we know that the length of this longest path can be bounded above and below by positive constants multiplied by $\frac{n}{\log n}$.

Let $C(n)$ denote the minimal number of vertex disjoint paths required to cover the integers up to $n$.  For example $C(7)=2$, as the divisor graph can be covered by the two paths $\{7,1,5\}$ and $\{3,6,2,4\}$ but it is not possible to include all the vertices in a single path.   This problem has been considered in several papers after being introduced by Saias and Erd\H{o}s \cite{erdsai}.  Saias \cite{saiasdiv} showed that $\frac{n}{6} \leq C(n) \leq \frac{n}{4} $ for sufficiently large $n$. Mazet \cite{mazet} improves this, showing that $C(n) \sim cn$ for some constant $c$ satisfying $0.1706\leq c \leq 0.2289$, and Chadozeau \cite{chad} gives the error term $C(n) = cn\left(1+O\left(\frac{1}{\log \log n \log \log \log n}\right)\right)$.  Our main theorem allows us to improve this error term dramatically and also to improve the lower bound for $c$.

\begin{theorem} \label{thm:pathcover}
The minimum number of paths required to cover the divisor graph of the integers up to $n$ satisfies \[C(n) = cn\left(1+O\left(\exp\left(-\sqrt{\left(\tfrac{1}{6}-\epsilon\right)\log n \log \log n}\right)\right)\right).\]  The constant $c$ is effectively computable, and satisfies the improved lower bound $0.176488<c$.
\end{theorem}

Finally, in Section \ref{sec:proof} we conclude with several lemmas from analytic number theory which lead to the proof of the main theorem.

Smooth numbers play an important role throughout the paper.  We will denote by $P^+(n)$ the largest prime divisor of $n$ (and take the convention that $P^+(1)=1$), and by $P^-(n)$ the smallest prime divisor of $n$.

\section{Counting Primitive Sets} \label{sec:countprim}

We first show how the main theorem, Theorem \ref{thm:main}, can be used to estimate $Q(n)$, the count of the primitive subsets of the integers up to $n$.  We fix $n$, and count the possible primitive subsets of $\{1,2,\ldots n\}$ by working backward from the end.  Recall, any subset of the integers in $\left(\frac{n}{2},n\right]$ is primitive.  Since each element of this set can either be included or not so $Q(n) \geq 2^{n/2}$.  We can make this bigger by considering integers those integers in $\left(\frac{n}{3},\frac{n}{2}\right]$.   For each such $k$ in this range we can either include $k, 2k$ or neither.   Thus for each such integer there are 3 possibilities, replacing the two possibilities when only $2k$ was considered.  Thus $\displaystyle{Q(n)\geq 2^{n/2}\left(\tfrac{3}{2}\right)^{n/6} = 2^{n/3}3^{n/6} \approx 1.5131^n}$.

We generalize this to any $k\leq n$ by defining \begin{equation}
    r(k,n) = \frac{\# \text{Primitive subsets of $[k,n]$}}{\# \text{Primitive subsets of $[k+1,n]$}} \label{eqn:rkn}
\end{equation}
to be the contribution of $k$ to our potential primitive set working backward from the end.

With this definition we can see that
\[Q(n) = \prod_{k=1}^n r(k,n)\]
as this product telescopes, leaving only $Q(n)=\frac{\# \text{Primitive subsets of [1,n]}}{\# \text{Primitive subsets of }\varnothing }$.

We can apply Theorem \ref{thm:main} by taking $f(k,n)=\log(r(k,n))$. To do so, it is necessary to see that $f(i,n)$ depends only on the connected component of $k$ in the divisibility graph of $[k,n]$.  This is easily seen however, as the inclusion of $k$ (or any integer in the interval $[k,n]$) in a primitive set depends only on whether its multiples (or divisors) are already included.  The count of the number of primitive subsets of an interval can therefore be computed as the product of the number of primitive subsets (or independent sets of vertices when viewed as a graph) of each connected component of the divisibility graph.  The contribution from each component of the divisibility graph that doesn't contain $k$ will cancel in the numerator and denominator of \eqref{eqn:rkn}.  This leaves only the ratio of the number of primitive (independent) subsets of the component connected component of $k$ to the number of primitive (independent) subsets of this component with the element $k$ removed.  As this is purely a graph theoretic question, it is clear that the answer will be the same for any other isomorphic divisibility graph.

Note also that $1\leq r(k,n)< 2$, since every primitive subset of $[k+1,n]$ is also a primitive subset of $[k,n]$, and at most the number of primtive sets could double after $k$ is considered if it is possible to add $k$ to every primitive subset of $[k+1,n]$ and still get a primitive set.  Thus we have $0 \leq f(k,n) \leq \log 2$.

Applying Theorem \ref{thm:main} we have
\begin{align*}
    Q(n) & = \prod_{i=1}^n r(i,n) = \exp\left(\sum_{i=1}^n f(i,n)\right) \\
    &= \exp\left(n\log \alpha +  O\left(n\exp\left(-\sqrt{\left(\tfrac{1}{6}{-}\epsilon\right)\log n \log \log n}\right)\right)\right)
\end{align*}
with \begin{align} \alpha &= \exp\left(\sum_{i=1}^\infty  \sum_{\substack{d\\ P^+(d)\leq i}} \sum_{t \in [id,(i+1)d)} \left(\frac{f(d,t)}{t(t{+}1)}\prod_{p\leq i} \frac{p{-}1}{p}\right)\right) \nonumber \\
&= \prod_{i=1}^\infty  \prod_{\substack{d\\ P^+(d)\leq i}} \prod_{t \in [id,(i+1)d)} \left(\frac{r(d,t)}{t(t{+}1)}\prod_{p\leq i} \frac{p{-}1}{p}\right). \label{eqn:alpha} \end{align}
This proves Theorem \ref{thm:alpha}.

The proofs of Theorems \ref{thm:beta} and \ref{thm:eta} are very similar.  We define
\begin{equation}
    s(k,n) = \frac{\# \text{Primitive subsets of $[k,n]$ of maximum size}}{\# \text{Primitive subsets of $[k+1,n]$ of maximum size}} \label{eqn:skn}
\end{equation}
and
\begin{equation}
    w(k,n) = \frac{\# \text{Maximal primitive subsets of $[k,n]$}}{\# \text{Maximal primitive subsets of $[k+1,n]$}}. \label{eqn:tkn}
\end{equation}
It follows that these functions depend only on the connected component of $k$ in the divisibility graph of $[k,n]$ by the same argument as for $r(k,n)$.  Unlike the case of $r(k,n)$, however, some care is required to show that these functions are bounded so that Theorem \ref{thm:main} can be applied.

First note that if $k>\frac{n}{2}$ then $s(k,n)=w(k,n)=1$, since in this situation, $k$ is the only vertex in the component connected to $k$ in the divisor graph of $[k,n]$, and so the element $k$ must be included in a maximal/maximum primitive subset of $[k,n]$. Thus we can restrict our attention to the case $k\leq \frac{n}{2}$.

For either $s(k,n)$ or $w(k,n)$, we find that every maximum/maximal primitive subset of $[k+1,n]$ is also a maximum/maximal primitive subset of $[k,n]$ of maximal size, since one of the multiples of $2k$ (and thus of $k$) must have been included in the primitive set if it were to be maximal.  Thus both $s(k,n)$ and $w(k,n)$ are always at least 1.

To get upper bounds for these functions, we produce an injection from maximal/maximum primitive subsets of $[k,n]$ containing $k$ to maximal/maximum primitive subsets of $[k+1,n]$. First note that any maximum primitive subset of $[k,n]$ that contains $k$ can be put into correspondence with a primitive subset of $[k+1,n]$ of the same size by replacing $k$ by $2k$. (Note that every neighbor of $2k$ in the divisor graph is also a neighbor of $k$).  This is clearly injective, so $s(k,n)\leq 2$.  For maximal primitive sets, simply replacing $k$ by $2k$ might not produce a maximal primitive subset if it is possible also to add other (odd) multiples of $k$ once $k$ is removed.  We therefore remove $k$ and instead include $pk$ for each prime number $p$ with $pk\leq n$ that can be added to the set while still producing a primitive set.  Since it is always possible to add the element $2k$ once $k$ is removed, it is clear that this again is an injection and so $w(k,n)\leq 2$ as well.

We then set
\[M(n) = \prod_{k=1}^n s(k,n),
\hspace{2cm} m(n) = \prod_{k=1}^n w(k,n)\]
which each telescope in the same way as the product for $Q(n)$, and apply Theorem \ref{thm:main}, proving Theorems \ref{thm:beta} and \ref{thm:eta} with
\begin{equation} \beta = \prod_{i=1}^\infty  \prod_{\substack{d\\ P^+(d)\leq i}} \prod_{t \in [id,(i+1)d)} \left(\frac{s(d,t)}{t(t{+}1)}\prod_{p\leq i} \frac{p{-}1}{p}\right) \label{eqn:beta} \end{equation}

and

\begin{equation} \eta = \prod_{i=1}^\infty  \prod_{\substack{d\\ P^+(d)\leq i}} \prod_{t \in [id,(i+1)d)} \left(\frac{w(d,t)}{t(t{+}1)}\prod_{p\leq i} \frac{p{-}1}{p}\right). \label{eqn:eta} \end{equation}

\section{Numerical Estimates} \label{sec:numerics}
Using the product formulas for $\alpha$, $\beta$ and $\eta$, given in \eqref{eqn:alpha}, \eqref{eqn:beta} and \eqref{eqn:eta} respectively, it is possible to compute each constant to arbitrary precision by computing sufficiently many terms in the product.  Replacing any unestimated terms by an appropriate upper or lower bound for the term gives a corresponding upper or lower bound for the value of the constant.

The constants were estimated using computations done using Sagemath \cite{sagemath}. Several optimizations were made to improve the speed of convergence.
\begin{observation} \label{obs:consecutivet}
For a fixed value of $i$ and $d$, if $t+1 \in (id,(i+1)d)$ is not an $i$-smooth number then $r(d,t)=r(d,t+1)$.
\end{observation}
This follows because the connected component of $d$ in the divisibility graph of the interval $[d,t+1]$ does not include the integer $t+1$ in that case.  Thus for each pair $i$, $d$ it is only necessary to compute $r(d,t)$ for the $i$-smooth values of $t$ in the interval $[id,(i+1)d)$.

\begin{observation} \label{obs:primes}
Fix $i,d$ and $t \in [id,(i+1)d)$. If, for some $p\leq i$, every term in the connected component of $pd$ in the divisibility graph of $[pd,pt]$ is divisible by $p$, then $r(pd,pt)=r(d,t)$.  Furthermore, in this situation $r(p^jd,p^jt)=r(d,t)$ for all $j>0$.
\end{observation}
\begin{proof}
If every term in the connected component of $pd$ in the divisibilty graph of this interval is divisible by $p$, then it is clear that this component is isomorphic to the component obtained by dividing everything by $p$.

Now, suppose every term in this component is divisible by $p$, and consider the interval $[p^2d,p^2t]$.  Clearly the component connected to $p^2d$ contains every term in the component of the divisibilty graph of $[d,t]$ connected to $d$ multiplied by $p^2$.  Now suppose for contradiction that the connected component of $dp^2$ contained an additional term $a>dp^2$ not divisible by $p^2$.  Then this term must be a divisor of some term $bp^2$ in the connected component of $dp^2$ where $p\nmid b$ (or if not, it could be replaced by one that is, since there must exist a path in this graph connecting $a$ to $dp^2$).

This means the ratio between $a$ and $bp^2$ is divisible by $p$, and so \[bp^2/p = bp \geq a >dp^2.\]  Thus we have $b>dp$ and so $b$ would be an element of the connected component of $dp$ in the divisor graph of $[pd,pt]$ which is a contradiction.
\end{proof}

Observation \ref{obs:primes} means that the sum (or product) over $d$ becomes finite for values of $i\leq 4$, because values of $d>48$ will always result in a connected graph component with a common prime factor in this range.  Thus the challenge is to numerically estimate the terms for $i\geq 5$.

Determining the optimal order in which to consider the pairs of $i$ and $d$ in the infinite product is challenging because $r(d,t)$ becomes more difficult to compute both for larger values of $d$ and $t$.  At the same time, the value of $r(d,t)$ tends to one rapidly as $i=\lfloor\frac{t}{d}\rfloor$ gets large.  Experimentally, the best lower bounds obtained by the author were obtained by considering potential pairs $i$,$d$ in increasing order of the value of the expression $d\times i^5$.

By computing $r(d,t)$ For all pairs of $i,d$ with $d$ being an $i$-smooth integer satisfying $di^5\leq 10^8$, (as well as the extended ranges $d<11250000$ when $i=5$, $d<2400000$ for $i=6$ and $d<27440$ when $i=7$) and all $i$-smooth values of $t\in [id,(i+1)d)$, and using these terms in the product \eqref{eqn:alpha} along with the additional terms obtained by applying Observation \ref{obs:primes}, the lower bound  is obtained.

In a similar fashion one can compute initial terms for $\beta$ and $\eta$.  The computation for $\beta$ involved computing $s(d,t)$ for all values of $d$ and $i$ satisfying $di^5<10^8$. For $\eta$ a more involved range of values was taken into account in order to get decent upper and lower bounds.  Table \ref{tab:maxbds} gives the bounds for the $d$ taken into account when computing $\eta$.  Again, in both computations Observations \ref{obs:consecutivet} and \ref{obs:primes} were used to speed up the computation and take into account additional pairs $d,t$.

\section{Conjectural Improved Upper Bounds} \label{sec:conj}

We give here a different method of counting primitive sets which would give a much better upper bound for the constant $\alpha$ if Conjecture \ref{conj:gmul} were proven, and may be of some independent interest.

Here we count primitive subsets of $\{1,2,\ldots n\}$ by working forward from 1, rather than backward from $n$ as was done in the previous section.  We define $g(1)=1$ and for $k>1$ \begin{align*}
    g(k) &= \frac{Q(k)}{Q(k-1)}-1 = \frac{\# \text{Primitive subsets of $[1,k]$ that include $k$}}{\# \text{Primitive subsets of $[1,k-1]$}}.
\end{align*}

The reason we defined $g(k)$ this way, as the ratio of primitive sets up to $k$ that include the integer $k$ to the primitive sets up to $k$ that do not, is that the resulting function appears to be submultiplicative in the following sense.

\begin{conjecture} \label{conj:gmul}
    The function $g(n)$ is submultiplicative.   If (n,m)=1 then \[g(nm)\leq g(n)g(m).\]
    Furthermore, for any prime $p$, $g(p^{i+1})\leq g(p^{i})$.
\end{conjecture}

The conjecture has been verified for all values of $n<899$.

If one assumes the conjecture, then a substantially improved upper bound for $\alpha$ can be computed. Using the computed values of $g(n)$, $n<899$, and taking into account all 31-smooth numbers gives the upper bound $c<1.573487$.

\section{The median size of a primitive subset}

The computations obtained in the previous section will also allow us to give upper and lower bounds for $\nu(n)$, the median size of a primitive subset of the integers up to $n$.  First we need a lemma (see for example \cite{knuth}) on sums of binomial coefficients, which gives the number of subsets of a sets of size $n$ into subsets of size at most $\lambda n$.
    \begin{lemma} \label{lem:sumbin}
    Fix $0 <\lambda < \frac{1}{2}$.  The partial sum of binomial coefficients satisfies
    \[\sum_{i=1}^{\lfloor \lambda n \rfloor} \binom{n}{i} = 2^{nH(\lambda)+O(\log n)}\]
    where $H(\lambda)=-\lambda\log_2 \lambda +(\lambda-1)\log_2(1-\lambda)$ and $\log_2$ is the logarithm base 2.
    \end{lemma}

Using this, and the numerical computations of $\alpha$ and $\beta$, we can bound the median size of a primitive subset of $\{1,2,\ldots n\}$.

\begin{proof}[Proof of Theorem \ref{thm:median}]
Let $\nu(n)$ denote the median size of the primitive subsets of the integers up to $n$.  Since half of the primitive subsets of this set must have size at most $\nu(n)$, and the total number of primitive subsets is approximately $\alpha^n$, we can obtain a lower bound for $\nu(n)$ by supposing that all small subsets of the integers up to $n$ are primitive.  Combining this with Lemma \ref{lem:sumbin} we get that

\[\sum_{i=1}^{\nu(n)} \binom{n}{i} = 2^{nH(\nu(n)/n)+O(\log n)} \geq \frac{1}{2}Q(n) =  \alpha^{n\left(1+O\left(\exp\left(-\sqrt{\left(\frac{1}{4}-\epsilon\right)\log n \log \log n}\right)\right)\right)}. \]

Taking logs gives \[H(\nu(n)/n) \geq \frac{\log \alpha }{\log 2} +O\left(\exp\left(-\sqrt{\left(\frac{1}{4}-\epsilon\right)\log n \log \log n}\right)\right)\] or \[\nu(n) \geq nH^{-1}\left(\frac{\log \alpha }{\log 2}\right)(1+o(1)).\]

Using the lower bound $1.572939<\alpha$ gives $\nu(n) \geq 0.168153n$ for $n$ sufficiently large.

To get an upper bound, we must use in addition the lower bound obtained for the constant $\eta$.  Every primitive set is contained in a maximal primitive set (but not necessarily a maximum one).  We count those primitive sets obtained by starting with a maximal primitive set $S$ and removing some subset $T$ from it.  Since half of the primitive subsets up to $n$ have size at least $\nu(n)$, and every maximal primitive set has size at most $\frac{n}{2}$, it must be possible to construct at least half of the primitive subsets up to $n$ by starting with a maximal primitive set and removing at most $\frac{n}{2}-\nu(n)$ terms from it.  Thus the median size of a primitive subset must be sufficiently less than $\frac{n}{2}$ to produce $\frac{n}{2}Q(n)$ sets in this way.  In particular we have the inequality \[m(n)\sum_{i=0}^{\frac{n}{2}-\nu(n)} \binom{n/2}{i}\geq \frac{1}{2}Q(n).\]
Using Lemma \ref{lem:sumbin} and Theorems \ref{thm:alpha} and \ref{thm:eta}, and taking logs, gives
\[H\left(1-\frac{2\nu(n)}{n}\right) \geq \frac{2(\log \alpha-\log \eta)}{\log 2} +O\left(\exp\left(-\sqrt{\left(\frac{1}{4}-\epsilon\right)\log n \log \log n}\right)\right)\]
so that $\nu(n)\leq \frac{n}{2}\left(1-H^{-1}\left(\frac{2(\log \alpha-\log \eta)}{\log 2}\right)+o(1)\right)$.  Using the upper bound for $\alpha$, and the lower bound for $\eta$, we find that $\nu(n) \leq 0.391752n$ for sufficiently large $n$.
\end{proof}

\section{Geometric Progression Free Sets} \label{sec:gpf}
Geometric progression free sets are similar to primitive sets, and so it is not surprising that the methods used to study primitive sets apply to this situation as well.  In this situation the entire divisor graph isn't necessary, only a subgraph that contains 3-term geometric progressions.  Unlike primitive sets, we don't know precisely the size $G(n)$ of the largest subset of $\{1,2,\ldots n\}$ avoiding three term geometric progressions with integral ratio.  It is known that $G(n) \sim bn$ for some constant $b$ with $0.81841<b<0.81922$, and that $b$ is effectively computable.  While we can't improve these bounds using the current method, we can get in Theorem \ref{thm:gpf} an improved error term  $G(n) = bn\left(1+O\left(\exp\left(-\sqrt{\left(\tfrac{1}{6}-\epsilon\right)\log n \log \log n}\right)\right)\right)$ for any $\epsilon>0$.
\begin{proof}
In this situation we have a sum, not a product so there is no need to take logs, however the set-up is the same.  We let $G(d,t)$ denote the size of the largest geometric progression free subset of the integers $[d,t]$, and let $g(d,t) = G(d,t)-G(d+1,t)$, so that we can write a telescoping sum, \[G(n) = G(1,n) = \sum_{k\leq n} g(k,n).\]
The main theorem then applies since $g(d,t)$ depends only on the divisor graph of the interval $[d,t]$, and only ever takes the values 0 or 1.  The result follows immediately.
\end{proof}

We can likewise use the main theorem to count (Theorem \ref{thm:theta}) the number $H(n)$ of subsets of the integers up to $n$ that avoid 3-term-geometric progressions with integral ratio.  The proof is identical to that of Theorem \ref{thm:alpha}.  To obtain the bounds on the constant $\theta$, we find that the analogous version of Observation \ref{obs:primes} for geometric progressions allows us to take into account the contribution from all values of $d$ for $1\leq i \leq 11$, as well as the contribution from those values of $d$ up to those listed in Table \ref{tab:gpf} in order to get the bounds
\[1.901448 < \theta < 1.925556.\]
\section{An optimal path covering of the divisor graph} \label{sec:pathcov}
As mentioned in the introduction, it is known due to Mazet \cite{mazet} and Chadozeau \cite{chad} that $C(n)$, the minimum number of disjoint paths necessary to cover the divisor graph of the integers up to $n$ satisfies
\[C(n)=cn\left(1 +O\left(\frac{1}{\log \log n \log \log \log n}\right)\right).\]
for a constant $0.1706\leq c\leq 0.2289$.
Using our main theorem, we show the following.
\begin{theorem*}[Theorem \ref{thm:pathcover}]
The minimum number of paths required to cover the divisor graph of the integers up to $n$ satisfies \[C(n) = cn\left(1+O\left(\exp\left(-\sqrt{\left(\tfrac{1}{6}-\epsilon\right)\log n \log \log n}\right)\right)\right).\]  The constant $c$ is effectively computable, and satisfies the improved lower bound $0.176448<c$.
\end{theorem*}
\begin{proof}
As in previous examples, we set up a telescoping sum.  Define  \[V(k,n)=\#\{\text{disjoint paths needed to cover the divisor graph of }[k,n]\} \] and define $v(k,n)=V(k,n)-V(k+1,n)$ so that $C(n)=\sum_{k=1}^n v(k,n)$.  Note that $v(k,n) \in \{-1,0,1\}$, since in the worst case scenario, we can cover the additional vertex $k$ using a single new path, and in the best case scenario, we are able to join paths from two different components of the divisor graph of $[k+1,n]$ through the vertex $k$.  Since $k$ can only be part of one path however, it is never possible to decrease the number of paths required by more than 1.

It is also clear that $v(k,n)$ depends only on the connected component of $k$ in the divisor graph of $[k,n]$, so we can apply the main theorem to $C(n)$, which completes the proof, with
\[c= \sum_{i=1}^\infty  \sum_{\substack{d\\ P^+(d)\leq i}} \sum_{t \in [id,(i+1)d)} \left(\frac{v(d,t)}{t(t{+}1)}\prod_{p\leq i} \frac{p{-}1}{p}\right). \]
Note that $v(1,1)=1$, but is otherwise nonpositive for any of the terms included in the sum above.  (If $i\geq 2$, then when computing $v(d,t)$, the element $2d$ is a vertex of the graph, and any neighbor of $2d$ is also a neighbor of $d$.  Thus it is always possible to cover the vertex $d$ by including it in the same path that is used to cover $2d$.)

In practice it seems that $v(d,t)$ is generally equal to $-1$ unless $d$ is divisible by $6$ in which case it is generally 0.  There are a few exceptions, however, in particular, it was computed that $v(d,t)=-1$ for all values of $d,t$ arising when $i\leq 5$; when $d=6$ and $t=50,51$; when $d=24$ and $225\leq t \leq 239$; and when $d=30$ and either $250\leq t \leq 269$ or $280\leq t \leq 299$.  The range of $d$ searched for each value of $i$ is listed in Table \ref{tab:pathid}.  In addition, a few sporadic values of equal to 0 were found when $d$ was not divisible by 6.  These are listed in Table \ref{tab:pathid2}.   Using these computed values where $v(d,t)=0$, along with observations \ref{obs:consecutivet} and \ref{obs:primes}, and assuming $v(d,t)=-1$ for all other values, we obtain the improved lower bound $0.176488<c$.
\end{proof}
\section{Questions} \label{sec:questions}

Despite the wide range of problems that can be tackled using the main theorem, this work leaves several open questions that will require new techniques.  In addition to the conjecture described in Section \ref{sec:conj}, we pose two questions that seem interesting enough to study further.

\begin{question}
Is the median size $\nu(n)$ of the primitive subsets of the integers up to $n$ asymptotic to $vn$ for some constant $v$? If so, is there an algorithm to compute $v$ to arbitrary precision?
\end{question}

In Section \ref{sec:gpf} we considered sets that avoided 3-term geometric progressions with integral ratio.  The results in \cite{mcnewgpf} imply that the largest subset of the integers up to $n$ avoiding 3-term geometric progressions with \textit{rational} ratio is also asymptotic to a different effectively computable constant times $n$.
\begin{question}
Is it possible to give an error term analogous to Theorem \ref{thm:gpf} for the size of the largest subset of the integers up to $n$ avoiding 3-term geometric progressions with rational ratio? Can we count the number of such subsets analogously to Theorem \ref{thm:theta}?
\end{question}

\section{Proof of the Main Theorem} \label{sec:proof}

It remains to prove the main theorem upon which the results of this paper rely.  First we need a lemma categorizing the connected components of the divisor graph of an interval, as well as  several lemmas from analytic number theory.

    \begin{lemma} \label{lem:isodivgra}
    Fix a positive integer $n$, suppose that $0<a\leq n$, and set $i= \left\lfloor \frac{n}{a}\right\rfloor$.  Let $d|a$ be the largest $i$-smooth divisor of $a$, and $\ell =\frac{a}{d}$ the ``$i$-rough'' part of $a$.  Finally let $t=\left\lfloor \frac{n}{\ell}\right\rfloor$.  Then the connected component of $a$ in the divisor graph of $[a,n]$ is isomorphic to the connected component of $d$ in the divisor graph of $[d,t]$ (with the vertex of $a$ corresponding to the vertex of $d$).
    \end{lemma}
    \begin{proof}  Suppose that $a\leq b<c\leq n$ are any two connected vertices in the connected component of $a$ in this divisor graph, and let $r=\frac{c}{b}$ be the (necessarily integral) ratio between them.  Since $r\leq \frac{n}{a}<i+1$, $r$ cannot have any prime factors greater than $i$.  So $b$ and $c$ are divisible by all of the same prime factors greater than $i$ to the same powers.

    Recall that $\ell$ is divisible only by primes greater than $i$ and $\ell|a$ so $\ell$ divides all of the integers in the connected component of $a$.  We defined $t=\left\lfloor \frac{n}{\ell}\right\rfloor$ so that $t\ell $ is the largest integer less than or equal to $n$ divisible by $\ell$, and thus the largest number from this interval that could possibly be part of the component of the divisor graph connected to $a$.  So the connected component of $a$ in $[a,n]$ is the same as the connected component of $a$ in $[a,t\ell]$.

    Now we can divide each integer in this connected component by $\ell$, and see that the connected component of $a$ in $[a,t\ell]$ is the same as the connected component of $\left[\frac{a}{\ell},t\right]=[d,t]$ with the isomorphism being multiplication by $\ell$.
    \end{proof}

    \begin{lemma} \label{lem:sieveinterval}
    Let $\epsilon>0$, $I$ an interval of length $X>y>2$ and define $u=\frac{\log X}{\log y}$. The number of integers in $I$ free of prime divisors up to $y$ is \[X\left(\prod_{p\leq y}\frac{p-1}{p}+O\left(\log y\exp\left((-1+o(1))u\log u\right)\right) \right) +O_\epsilon(X^\epsilon)\]
    as $u \to \infty$.

    \end{lemma}
    \begin{proof}
    This follows from the ``Fundamental Lemma'' of Brun's sieve, see for example Theorem 6.12 of \cite{FI}, taking the level of distribution to be $X$.  The error term $R(I,D)$ can be bounded by $\Psi(X,y)$, which means that the number of such integers in this interval is \[X\prod_{p\leq y}\frac{p-1}{p}\left(1+O\left(\exp\left((-1+o(1))u\log u\right)\right) \right) +O(\Psi(X,y)).\]  Now we can bound $\Psi(X,y) \ll_\epsilon Xu^{-u}+X^\epsilon$  \cite[Section III.5]{IAPNT} and the result follows by approximating $\prod_{p\leq y} \frac{p-1}{p} = O\left(\frac{1}{\log y}\right)$.
    \end{proof}

    We will also need estimates of Tenenbaum \cite{Tenenbaum-LFC} for the function $\Theta(x,y,z)$ which counts integers $n\leq x$ with a $y$-smooth divisor  $d>z$, as well as the related function $$S(y,z)= \sum_{\substack{d>z\\P^+(d)<y}} \frac{1}{d}.$$
    \begin{lemma} \label{lem:recipsmooth} For all $\epsilon>0$, $x, y\geq 2$ and $z<\exp \exp (\log y)^{(3/5-\epsilon)}$
    \[\Theta(x,y,z) = (1+o(1))xS(y,z)\prod_{p<y}\frac{p-1}{p} \ll \frac{x\rho(u)}{\log (u)}.\]
\end{lemma}

    \begin{proof}[Proof of main theorem]
    Fix $\epsilon>0$, $A\geq 0$ and suppose $f(k,n)$ is bounded in absolute value by $A$ and depends only on the connected component of $k$ in the divisor graph of $[k,n]$. Our goal is to estimate $\sum_{k=1}^n f(k,n)$ as $nC_f$ where
    \[C_f = \sum_{i=1}^\infty  \sum_{\substack{d\\ P^+(d)\leq i}} \sum_{t \in [id,(i+1)d)} \left(\frac{f(d,t)}{t(t+1)}\prod_{p\leq i} \frac{p-1}{p}\right).\]

We use Lemma \ref{lem:isodivgra} to group together the equal terms in this sum.  We first group together those $k$ having the same value of $i=\left\lfloor \frac{n}{k} \right\rfloor$, followed by those whose largest $i$-smooth divisor $d$ is the same, and finally those with the same value of $t=\left\lfloor\frac{nd}{k}\right\rfloor$.  The possible values of $t$ are thus each of the integers in the interval $[id,(i+1)d)$.

Pulling this all together we have \begin{align*}
        \sum_{k=1}^n f(k,n) &= f(1,n)+\sum_{i=1}^{\lfloor n/2 \rfloor }  \sum_{\substack{d<\frac{n}{i}\\ P^+(d)<i}} \sum_{t \in [id,(i+1)d)}
        \sum_{\substack{k \in \left(\frac{dn}{t+1},\frac{dn}{t}\right]\\ d\mid k \\ P^{-}\left(\frac{k}{d}\right)> i}} f(k,n) \\
        &=f(1,n)+\sum_{i=1}^{\lfloor n/2 \rfloor } \sum_{\substack{d<\frac{n}{i}\\ P^+(d)<i}} \sum_{t \in [id,(i+1)d)}\left( f(d,t)
        \sum_{\substack{\ell \in \left(\frac{n}{(t+1)},\frac{n}{t}\right] \\ P^-(\ell)> i}} 1  \right).
\end{align*}

        We truncate the outer two sums at some levels $N$ and $M$ to be determined later, giving us
        \begin{align*}
        \sum_{k=1}^n f(k,n) &= \sum_{i=1}^{N } \sum_{\substack{d<M\\ P^+(d)<i}} \sum_{t \in [id,(i+1)d)}
        \left(f(d,t)\sum_{\substack{\ell \in \left(\frac{n}{(t+1)},\frac{n}{t}\right]\\ P^-(\ell)\geq i}} 1\right)\hspace{-1mm}{+}O_A\hspace{-1mm}\left(\frac{n}{N}+\Theta(n,N,M)\right).
        \end{align*}
        In doing so, it was necessary to introduce two error terms, $\frac{n}{N}$ to account for all of those integers $k<\frac{n}{N}$ in the product for $Q(n)$ that were ommitted when truncating the initial product at $N$, and $\Theta(n,N,M)$, which counts those integers up to $n$ which have an $N$-smooth divisor greater than $M$, and hence may have been ignored when truncating the second sum.

        This means that the intervals being sifted of primes up to $i$ when choosing $\ell$ have size $\frac{n}{t(t+1)}$ and so, as long as both $N$ and $M$ are chosen not too large (meaning that $t<NM$ is not too large), these intervals are sufficiently long to apply Lemma \ref{lem:sieveinterval} to approximate the count of such integers.  Doing so gives

        \begin{align}
        &\sum_{k=1}^n f(k,n) \nonumber \\&=\sum_{i=1}^{N}
        \sum_{\substack{d<M\\ P^+(d)<i}} \sum_{t \in [id,(i+1)d)}
        \left(\frac{nf(d,t)}{t(t{+}1)}\left(\prod_{p<i} \frac{p-1}{p}+O\left(\frac{\log i}{\exp\left((1{+}o(1))u_t \log u_t \right)}\right)\right)\right. \nonumber \\
        &\hspace{6cm} \left. \vphantom{\left(\prod_{p<i} \frac{p-1}{p}\right)} +O_{A,\epsilon}\left(\frac{n^\epsilon}{t^{2\epsilon}}\right)\right)\nonumber+O_A\left(\displaystyle{\frac{n}{N}}+\Theta(n,N,M)\right) \nonumber\\
        &=\sum_{i=1}^{N }
        \sum_{\substack{d<M\\ P^+(d)<i}} \sum_{t \in [id,(i+1)d)}
        {\frac{f(d,t)n}{t(t+1)}\prod_{p<i} \frac{p-1}{p}} \nonumber \\
        &\hfill +\displaystyle{ O_A\hspace{-1mm}\left(\sum_{i=1}^N \sum_{\substack{d<M\\ P^+(d)<i}} \sum_{t \in [id,(i+1)d)} \hspace{-2mm} \left(
        \frac{n\log i}{t^2\exp\left((1{+}o(1))u_t \log u_t \right)}{+}\frac{n^\epsilon}{t^{2\epsilon}}\right){+} \frac{n}{N}+\Theta(n,N,M)\right)}\hspace{8mm} \label{eqn:firstapprox}
        \end{align}
       as long as $u_t = \frac{\log \frac{n}{t(t+1)}}{\log i}$, corresponding to the length of the interval in which Lemma  \ref{lem:sieveinterval} was applied, tends to infinity. Thus it will be necessary to choose $N, M$ such that \begin{equation}
            \log N =o\left(\log \frac{n}{M^2}\right) \label{ineq:utinfty}
        \end{equation}
        as $n\to \infty$.

        We now extend both of the initial two sums in the main term of this expression to infinity, so that we can replace it with $nC_f$.

        This introduces another error term of the form $O_A\left(\frac{n}{N}\right)$, to account for all of the new terms $i>N$ in the now infinite sum, and also an error term to account for those additional terms with with $i<N$ but where $d>M$. This error term has the form
         \begin{align*}
            O_A\left(\sum_{i=1}^N  \sum_{\substack{d\geq M\\ P^+(d)<i}} \sum_{t \in [id,(i+1)d)} \frac{n}{t(t+1)}\prod_{p<i}\frac{p-1}{p}\right) &=O_A\left(\sum_{i=3}^N\frac{n}{i^2\log i }  \sum_{\substack{d\geq M\\ P^+(d)<i}} \frac{1}{d}\right)\\
            &=O_A\left(\frac{n}{\log N} \sum_{\substack{d\geq M\\ P^+(d)<N}} \frac{1}{d}\right)\\
            &=O_A\left(\frac{n\rho\left(v\right)}{\log v}\right).
        \end{align*}
        Here $v = \frac{\log M}{\log N}$ and we have used Lemma \ref{lem:recipsmooth} to bound the sum (assuming that $N$ and $M$ satisfy the conditions of that lemma.)  Also by Lemma \ref{lem:recipsmooth} this same expression bounds the size of $\Theta(n,N,M)$, so putting this into our estimate \eqref{eqn:firstapprox} the sum over $f(k,n)$ becomes
        \begin{equation}
        nC_f +
        O_A\left(\sum_{i=1}^N \sum_{\substack{d<M\\ P^+(d)<i}} \sum_{t \in [id,(i+1)d)}\hspace{-2mm} \left(
        \frac{n\log i}{t^2\exp\left((1{+}o(1))u_t \log u_t \right)}{+}\frac{n^\epsilon}{t^{2\epsilon}}\right){+} \frac{n}{N}{+}\frac{n\rho(v)}{\log v}\right).
        \label{eqn:approx2}
        \end{equation}

        We can write the first term inside the innermost sum above as \[ \log i \left(\frac{n}{t^2}\right)^{1-(1+o(1))\frac{\log u_t}{\log i}} < \log N \left(\frac{n}{t^2}\right)^{1-(1+o(1))\frac{\log u_t}{\log N}} < \log N \left(\frac{n}{t^2}\right)^{1-(1+o(1))\frac{\log u'}{\log N}}\]
        where $u'=\frac{\log \frac{n}{N^2M^2}}{\log N}$ is obtained by taking both $i$ and $t$ as large as possible in $u_t$, $i=N$, $t=NM$.
        So long as $N$ and $M$ are chosen so that $\frac{\log u'}{\log N} \to 0$ then this term will dominate the second term in the sum for sufficiently large $n$ (assuming $\epsilon <1$).  We can thus bound the triple sum as
        \begin{align}
        \sum_{i=1}^N \sum_{\substack{d<M\\ P^+(d)<i}} & \sum_{t \in [id,(i+1)d)} \left(
        \frac{n\log i}{t^2\exp\left((1{+}o(1))u_t \log u_t \right)}+\left(\frac{n}{t^2}\right)^\epsilon\right) \nonumber \\
        &\ll n^{1-(1+o(1))\frac{\log u'}{\log N}} \log N\sum_{i=1}^N \sum_{\substack{d<M\\ P^+(d)<i}} \sum_{t \in [id,(i+1)d)} t^{-2+(2+o(1))\frac{\log u'}{\log N}}  \nonumber\\
        &\ll n^{1-(1+o(1))\frac{\log u'}{\log N}} \log N\sum_{i=1}^N i^{-2+(2+o(1))\frac{\log u'}{\log N}} \sum_{\substack{d<M\\ P^+(d)<i}}  d^{-1+(2+o(1))\frac{\log u'}{\log N}} \nonumber\\
        &\ll n^{1-(1+o(1))\frac{\log u'}{\log N}}\log N \sum_{\substack{d<M}}  d^{-1+(2+o(1))\frac{\log u'}{\log N}} \nonumber \\
        &\ll  n^{1-(1+o(1))\frac{\log u'}{\log N}} M^{(2+o(1))\frac{\log u'}{\log N}} \nonumber\\
        &\ll n\exp\left((-1+o(1))(\log n-2\log M) \frac{\log u'}{\log N}+\log \log N\right).
        \end{align}
        Putting this into \eqref{eqn:approx2} and approximating $\rho(v)=\exp((-1+o(1))v\log v)$,
        \begin{align}
        &\sum_{k=1}^n f(k,n)  =nC_f \nonumber \\
        &+ O_A\left(\frac{n}{\exp\left((1{+}o(1))\log \left(\frac{n}{M^2}\right) \frac{\log u'}{\log N}-\log \log N\right)} + \frac{n}{N}+\frac{n}{\exp\left((1{+}o(1))v \log v \right)}\right).
        \end{align}
        Finally, we optimize the error term in the expression above by taking $M=n^{1/3-\epsilon}$, and $N=\exp\left(\sqrt{\left(\tfrac{1}{6}-\epsilon\right)\log n \log \log n}\right)$.  Note that this choice satisfies \eqref{ineq:utinfty} as well as the hypotheses of Lemma \ref{lem:recipsmooth}.  This choice gives $\log u' = \left(\frac{1}{2}+o(1)\right)\log \log n$, so, for sufficiently large $n$,
        \begin{align*}
            (1{+}o(1))(\log n -2\log M) \frac{\log u'}{\log N} - \log \log N &=(1+o(1))\frac{\frac{1}{6}{+} \epsilon}{\sqrt{\frac{1}{6} -\epsilon}} \sqrt{\log n \log \log n} \\
            &>\sqrt{\frac{1}{6}\log n \log \log n}
        \end{align*}

        and \[(1+o(1))v\log v=\frac{(1{+}o(1))\left(\frac13{-}\epsilon\right)\log n}{\sqrt{\left(\frac  16 {-}\epsilon\right) \log n \log \log n}}\left(\frac 12 \log \log n\right) > \sqrt{\left(\tfrac{1}{6}-\epsilon\right)\log n \log \log n}.\]
        This gives us our result, namely
        \[\sum_{k=1}^n f(k,n) = nC_f+
         O_{A}\left(n\exp\left(-\sqrt{\left(\tfrac{1}{6}-\epsilon\right)\log n \log \log n}\right)\right).\]

\end{proof}

\bibliographystyle{amsplain}

Number Theory (math.NT); Combinatorics (math.CO)


Nonlinear Matrix Approximation with Radial Basis Function Components

Abstract: We introduce and investigate matrix approximation by decomposition into a sum of radial basis function (RBF) components. An RBF component is a generalization of the outer product between a pair of vectors, where an RBF function replaces the scalar multiplication between individual vector elements. Even though the RBF functions are positive definite, the summation across components is not restricted to convex combinations and allows us to compute the decomposition for any real matrix that is not necessarily symmetric or positive definite. We formulate the problem of seeking such a decomposition as an optimization problem with a nonlinear and non-convex loss function. Several modern versions of the gradient descent method, including their scalable stochastic counterparts, are used to solve this problem. We provide extensive empirical evidence of the effectiveness of the RBF decomposition and that of the gradient-based fitting algorithm. While being conceptually motivated by singular value decomposition (SVD), our proposed nonlinear counterpart outperforms SVD by drastically reducing the memory required to approximate a data matrix with the same L2 error for a wide range of matrix types. For example, it leads to 2 to 6 times memory save for Gaussian noise, graph adjacency matrices, and kernel matrices. Moreover, this proximity-based decomposition can offer additional interpretability in applications that involve, e.g., capturing the inner low-dimensional structure of the data, retaining graph connectivity structure, and preserving the acutance of images.

\section{Introduction}

There is an extensive body of work that aims to approximate large multi-dimensional data via simple low-dimensional components. Such approximations save the storage memory and expose simpler underlying structures in the data. The singular value decomposition (SVD) for matrices is arguably the standard example of such a low-rank approximation, while the canonical polyadic (CP) decomposition~\cite{carrollAnalysisIndividualDifferences1970,harshmanFoundationsPARAFACProcedure1970a}, where every component is represented by the outer product of two or more vectors, is the generalization for higher-order tensors. Much less is known for approximations that are low-complexity but not necessarily low-rank. In this work, we propose one such approximation by generalizing the outer product between vectors. By \emph{generalizing}, we mean to replace the scalar product between vector elements by a nonlinear function, \eg the one-dimensional radial basis function (RBF) in this case. Just like a component formed by an outer product, each nonlinear component is indexed by two vectors with potentially one additional scaling factor. However, the nonlinear relationship introduced between the vectors leads to more expressive components as manifested by, for example, their higher rank. This in turn yields better overall approximations with less explicit components comparing to the SVD in many situations.

The generalization of the outer product using RBF functions stems from their use in kernel machines~\cite{hofmannKernelMethodsMachine2008}. In that context, a kernel is a positive definite function that evaluates the inner product between data points after implicitly mapping them to a higher-dimensional vector space. Specifically, the RBF kernel between two feature vectors $\B{u}, \B{v} \in \mathbb{R}^n$ is a real-valued function of the form
\begin{equation}\label{eq:rbf-kernel}
k_\text{RBF}(\B{u}, \B{v}) = \exp\left(-\frac{\|\B{u} - \B{v}\|^2}{2h^2}\right),
\end{equation}
where $h$ is a scalar parameter. The mapping, which is nonlinear, introduces flexibility for modeling functions and decision boundaries that would otherwise be difficult to learn in the original feature space. Moreover, the RBF kernel can be conveniently interpreted as a similarity measure, \ie  the value of the kernel is small if input vectors are far away from each other.

In our work, an \emph{RBF component} created from two vectors $\B{u} = (u_1, \ldots, u_n) \in \mathbb{R}^n$, $\B{v} = (v_1, \ldots, v_m) \in \mathbb{R}^m$ is a kernel matrix $K = \left(k_{ij}\right) \in \mathbb{R}^{n \times m}$ with the elements defined as
\begin{equation}\label{eq:rbf-comp}
    k_{ij} = \exp\left( -\left|u_i - v_j\right|^2 \right).
\end{equation}
 An important difference here, as opposed to the case in \eqref{eq:rbf-kernel}, is that the vectors $\B{u}, \B{v}$ are not known inputs but rather \emph{learnable} parameters for expressing a given target data matrix. In particular, there is no need to specify the length-scale parameter $h$ as in the standard RBF definition~\eqref{eq:rbf-kernel}, since optimizing $\B{u}$ and $\B{v}$ can achieve an equivalent effect.  Similarly to the RBF function~\eqref{eq:rbf-kernel}, an RBF component~\eqref{eq:rbf-comp} exposes the underlying geometric structures in the data matrices based on the proximity between the elements of the learned vectors.
 However, we emphasize that the approximations are not specific to any special geometry-related features in the data. Linear combinations of such kernel matrices can act as universal approximators of general matrices while competing successfully in efficiency with SVD decomposition.

The task of finding and interpreting an optimal RBF approximation results in unique challenges distinct from SVD-based approximation algorithms. This includes the absence of known direct algorithms to find the best decomposition, the lack of orthogonality between the individual components, and the lack of a clear ordering of the components based on their norm. However, we show that trading-in the linearity of the components has great advantages in memory efficiency and additional expressivity of the decomposition due to the Euclidean distance in the structure of the RBF kernels. We also demonstrate that iterative gradient-based optimization methods make the problem of finding a good approximation tractable.

This paper is structured as follows. In Section~\ref{sec:motivating-example}, we present a synthetic case where an RBF approximation is naturally optimal because the matrix to be approximated is created from the difference between two RBF components. This sets the scene for a formal description of our approximation method, including a discussion of the success and sophistication of the learning process, as presented in Section~\ref{sec:method}. In Section~\ref{sec:efficiency}, we provide extensive empirical evidence of the general applicability of the method to a variety of natural matrix models without \textit{a priori} underlying structures that make them particularly amenable to the RBF approximation. In Section~\ref{sec:interpretability}, we discuss how additional knowledge about the structure in data can be obtained using the learned RBF components in several situations, including a manifold learning example, a graph structure learning example, and an image compression example. In \Cref{sec:related}, we compare the proposed approach with related works.
In \Cref{sec:conclusions}, we conclude the paper with a discussion of multiple future directions related to the analysis and applications of the proposed RBF approximation.

\section{A Motivating Example}\label{sec:motivating-example}

Let us consider a special case when an RBF component defined in \eqref{eq:rbf-comp} is square and symmetric, namely, $\bf{u} = \bf{v}$ and
\begin{equation}\label{eq:rbf-component-sym}
\tilde k_{ij} \coloneqq \exp\left[-(u_i - u_j)^2\right], \quad \text{for each entry } \tilde k_{ij} \text{ in } \tilde K \in \mathbb{R}^{n \times n}.
\end{equation}  Depending on the vector $\B{u}$, a symmetric RBF component can have vastly different structures from a rank one constant matrix to a full-rank near identity matrix. In the case where $\B{u}$ is randomly sampled from a regular enough distribution, the spectrum of the normalized matrix  $n^{-1}K$ is concentrated around the spectrum of its expectation $n^{-1}\E K$~\cite{koltchinskii2000random}. For example, when the vector elements are \iid standard Gaussians, with high probability, the RBF component has $9$ eigenvalues above $10^{-5}$.

A fact that we take as an essential advantage for approximating general matrices is that a linear combination of two or more RBF components is not necessarily semi-positive definite. Figure~\ref{fig:motivating} shows one such example of a two-component matrix $K_{exact2} = 5K^{(1)} - 4K^{(2)}$, where $K^{(1)}$ and $K^{(2)}$ are the RBF components defined per \eqref{eq:rbf-component-sym}. Without loss of generality, we take $\B{u}^{(1)}$ and $\B{u}^{(2)}$ to be standard normal vectors smoothed by one-dimensional Gaussian filters with standard deviations of 3 and 6, respectively. Even though the matrix can be perfectly reconstructed from only $2n$ parameters that encode $\B{u}^{(1)}$ and $\B{u}^{(2)}$, the best rank-4 SVD approximation has a mean square error above $0.01$ and delivers a visibly imperfect reconstruction using $4n$ parameters.

\begin{figure}
    \includegraphics[width=\textwidth]{fig-motivation.pdf}
	\caption{The best rank 1, 2 and 4 approximations of $K_{exact2}$ matrix are visibly not ideal, even though the latter one requires twice more parameters than the exact functional expression of $K_{exact2} = 5K^{(1)} - 4K^{(2)}$ where the matrices $K^{(i)}$ are given by the form \eqref{eq:rbf-component-sym}.}
    \label{fig:motivating}
\end{figure}

In this paper, we are motivated by the following \emph{inverse} questions: Would this be possible to recover vectors $\B{u}^{(1)}$ and $\B{u}^{(2)}$ if we were just given a matrix $K_{exact2}$, similar to how the SVD reconstruction in \Cref{fig:motivating} was found? More generally, given a matrix $K$ that is known to be equal to, or well-approximated by a linear combination of $r$ general RBF components as per \eqref{eq:rbf-comp}, can we learn the vectors ${\bf u}^{(k)}, \B{v}^{(k)}$ and scalar coefficients that define this linear combination? Further, can we find a linear combination of the RBF components to approximate a generic data matrix $K$ with no initial structure assumed? This is parallel to finding an SVD approximation of a given rank for a target matrix but poses a much more challenging optimization problem due to the nonlinearity of the RBF components. In this paper, we give promising answers to all three key questions above. In Figure~\ref{fig:iteration}, we demonstrate that for a matrix $K_{exact2}$, the component vectors $\B{u}^{(1)}$ and $\B{u}^{(2)}$ can be found by an efficient iterative optimization process. In the next section, we dive into the specifics of this training process that allows us to learn the RBF component vectors and coefficients for a given data matrix.
\begin{figure}
    \includegraphics[width=\textwidth]{fig-iteration.pdf}    \caption{Learning process via Adam method of the two RBF component approximation of $K_{exact2}$ matrix: initial matrix is nearly constant (first row), since randomly initialized component vectors $\B{u^{(1)}}$ and $\B{u^{(2)}}$ are chosen to have small coordinates all about 1e-2 (second row), and, as the optimization proceeds, after 5, 20 and 100 steps, the component vectors gradually learn the shape of the true component vectors, and the learned matrix approximates $K_{exact2}$.}
    \label{fig:iteration}
\end{figure}

\section{The Optimization Problem}\label{sec:method}
For a matrix $K =(k_{ij}) \in \BB{R}^{n\times m}$, constructing an $r$-component RBF approximation is a problem of finding a sequence of column vectors $\{\BF{u}^{(k)} \in \BB{R}^n, k = 1, \ldots, r\}$ and row vectors $\{\BF{v}^{(k)} \in \BB{R}^m, k = \{1, \ldots, r\}$, a vector of coefficients $\BF{a} \in \BB{R}^r$ and a scalar offset $b$ such that
\begin{equation}
    k_{ij} \approx k_{ij}^0 := b + \sum_{k=1}^r a_k \exp \left[- \left(u_i^{(k)} - v_j^{(k)} \right)^2 \right]
\end{equation}

This leads to the following optimization problem
\begin{equation}
    \underset{\{\BF{u}^{(k)}\}, \{\BF{v}^{(k)}\}, \BF{a}, b}{\mathrm{argmin}} \mathcal{L}(K, K^0), \quad \text{ where } \ \mathcal{L}(K, K^0) = \frac{1}{nm}\sum_{i,j =1, 1}^{n,m} \left(k_{ij} - k_{ij}^0 \right)^2
 \label{eq:optimization-problem}
\end{equation}
is the mean squared error loss function. Although other losses could be considered depending on an application, we choose an $L_2$-based loss to establish direct comparison with SVD-based approximations, which is the best low-rank approximation for the loss as guaranteed by the Eckart–Young–Mirsky theorem \cite{eckart1936approximation}.

\textbf{Global minimum structure.} The problem \eqref{eq:optimization-problem} is nonlinear, non-convex, and potentially of very high dimension $\sim \mathcal{O}\left((n+m) k\right)$. In particular, the global minimum may not be unique as multiple equivalent solutions exist due to the permutation of the RBF components. It is only obvious in the one-component case, where the entries of $K$ correspond to one-dimensional proximities between the vector elements as discussed above, that the set of global minima consists of the ground truth vector $\B{u}$ uniquely determined up to reflection and a constant shift. Empirically, we can see in~\Cref{fig:iteration} how two learned component vectors differ from true component vectors only by isometries.

\textbf{Adaptive iterative gradient-based optimization: Adam.} We turn to \emph{gradient-based optimization techniques} for solving the problem~\eqref{eq:optimization-problem}. It is known that the stochastic gradient descent method used with adaptive step size algorithms can successfully handle complex non-convex objectives~\cite{wilson2017marginal, o2019behavior, de2018convergence}. Their practical use in the deep network community goes way beyond their established theoretical guarantees. As visualized in~\Cref{fig:optimization}, the learning dynamic of the two-component matrix $K$, introduced in Section~\ref{sec:motivating-example}, suggests that Adam~\cite{kingma2014adam} can generally outperform other popular gradient descent algorithms. Therefore, we use it for all subsequent experiments presented in the paper.

\begin{figure}
    \includegraphics[width=\textwidth]{fig-optimization.pdf}    \caption{Learning two (exactly-parametrized regime) and four (over-parametrized regime) RBF component approximations of $K_{exact2}$: (A) and (B) show that standard adaptive gradient-based learning methods, such as Adam, AdamW and Adagrad, can find approximation $K_{exact2}$ with the MSE loss below 1e-5 in about $1000$ iterations of $100$ independent runs, and that convergence tends to be quicker in the over-parametrized regime. (C) and (D) show the distribution of the resulting MSE approximation loss over $100$ independent randomly started runs after $10K$ iteration steps of the Adam algorithm, depending on the starting random distribution of the component vector coefficients (standard Gaussian, or 10 times damped). The initialization with the smaller random values, and especially the over-parametrization, lead to additional robustness of the learning process.}
    \label{fig:optimization}
\end{figure}

\textbf{Vanishing gradients and small magnitude initialization.} The problem of vanishing gradient may halt the training process when entries of the learned vectors have large mutual distances. To illustrate this issue, let us consider the problem of learning the vector $\B{u}$ from a matrix $K$ of the simplest one-component symmetric form \eqref{eq:rbf-component-sym}. The corresponding loss function is
\begin{equation}\label{eq:one-component-loss}
\mathcal{L} = \frac{1}{n^2}\sum_{ij} \left[\exp(-(u_i - u_j)^2) - k_{ij} \right]^2
\end{equation}
and its $(i,j)$-th gradient is given by $-2/n^2(u_i - u_j)\exp(-(u_i - u_j)^2)$, which approaches zero both for very large and especially small values of $|u_i - u_j|$. In particular, the training stalls (or stops) if we start with the points that are too far from each other. We partially alleviate this issue by making sure to initialize the parameters in component vectors $\B{u}$ with small enough values. The experimental evidence confirms that this makes the training process more consistent: the histograms in \Cref{fig:optimization} show the distributions of the losses achieved on the batch of $100$ runs, all started randomly with the independent standard normal entries, multiplied by a factor of $1$ and $0.1$ respectively. We can see that smaller initialization values lead to a larger fraction of independent training runs resulting in a global minimum. We can also see in~\Cref{fig:iteration} how the magnitude of the parameters gradually grows during the learning process.

\textbf{Batch training: only the best-run matters.} However, the histograms in \Cref{fig:optimization} show a non-negligible fraction of runs that do not converge to a global minimum, for both initialization options. The crucial advantage of our problem comparing to the majority of typical machine learning problems (when a gradient-based approach is used for training the model) is that we always know the ground truth matrix $K$, and we do not need the learned parameters to generalize to the unseen data. Thus, we can track the error during the training process. The following process can be employed to get a consistent convergence result: after training a batch of gradient descent processes with various random starts, we keep only the best one (or several), and restart all the others. In this paper, we employ a simpler approach: we simply run a \emph{batch of independent learning processes} concurrently and remember that it is enough to get just one good run to learn an efficient RBF approximation. Essentially, our task requires an algorithm design that will result in a non-negligible (and not necessarily high) probability of successful training.

\textbf{Random initialization and over-parametrization.} It is known that the gradient-based methods are very sensitive to the initialization, and it is always advantageous to have a warm start for the training. Although the magnitudes of the entries of $K$ give some information about the distances between the coordinates of the component vectors, such effect is hard to quantify especially for fitting an approximation with multiple RBF components. Here, we do not tune the starting distribution for the specific data matrix, but rather initialize randomly. The promise of random initialization was proposed in several recent papers, including~\cite{wang2020beyond}, where the authors demonstrate that random initialization works for a tensor CP-fitting problem in the over-parametrized regime. In our RBF fitting problem, over-parametrization also leads to the increased speed and robustness of the training process, as we can see on the right two displays in \Cref{fig:optimization}.

\textbf{Stochastic counterparts.} \emph{Stochastic} gradient descent is often used in training deep neural networks to reduce the computational cost while also providing the ability to overcome local minima as a side effect of the introduced randomization. The key here is to estimate the gradient of the loss function with respect to the learnable parameters using a small subset of the training examples chosen randomly during each gradient descent iteration. In the context of solving~\eqref{eq:optimization-problem} for the RBF decomposition, this translates into evaluating the loss function over a subset of the elements of the target matrix. In other words, we evaluate the loss function as
\begin{equation}
    \tilde{\mathcal{L}} = \frac{1}{|I|}\sum_{i, j \in I} (k_{ij}- k^0_{ij})^2,
\end{equation}
where $I$ is an index set that contains a random subset of the elements of the matrix coordinates and $|I|$ denotes its cardinality.

Let us conclude this section with a disclaimer that it is still an important direction for future work to analyze the loss landscape specific to the RBF fitting problem and to develop tailored initialization and optimization processes for this particular shape of the loss function.

\section{Efficiency of the RBF approximation}\label{sec:efficiency}
As we discussed in Section~\ref{sec:motivating-example}, an RBF approximation is more efficient than any low-rank approximation for at least \emph{some} matrices. Further, observing that \emph{every} matrix is exactly a linear combination of several rank-one matrices, we claim that a natural analog holds: every matrix can be approximated by a linear combination of the RBF components given by ~\eqref{eq:rbf-comp}. One can notice that by constructing $u_{i'} = v_{j'}$ for a fixed pair of indices $(i',j')$ and setting all other elements very large so that $|u_i - v_j|, i \neq i' \text{or} j \neq j'$ is greater than multiple length scales of the RBF function, we obtain a matrix $\tilde K$ very close to being having exactly one non-zero entry. This gives a trivial, albeit memory-heavy, universal approximation for any matrix in $\R^{n \times m}$ using at most $nm$ components of type \eqref{eq:rbf-comp}. Of course, approximating only one matrix element using an RBF component is completely impractical, since storing a truncated SVD decomposition or even storing the elements of the original matrix in verbatim would take less memory. This motivates the next key question we address in this paper: \emph{For what natural models is the RBF approximation of a general data matrix K more successful than one based on SVD?}

In this section, we compare the efficiency of the approximations by RBF and SVD components in terms of reconstruction quality as measured by the MSE loss~\eqref{eq:optimization-problem} over a variety of the data matrix models.  Note that in the symmetric case, right and left singular vectors coincide, and so the SVD-based low-rank approximation depends on twice fewer parameters. Parallel to this, we use symmetric RBF components of form~\eqref{eq:rbf-component-sym} to approximate symmetric matrices. Finally, it is worth noting that there are ``adversarial" settings where SVD approximations will show better results than the RBF ones. For example, a rank $1$ matrix can be approximated exactly by one SVD component, but generally not by one RBF component. However, as the rank $r$ of the target matrix grows, we can see that the rank $r' \lesssim r$ approximations by the RBF components quickly become more efficient. See also the discussion about the correct functional form below in Section~\ref{sec:conclusions}.

\subsection{Synthetic Data}

First, we consider the following artificial random matrix models: 1) a square matrix of random Gaussian noise, 2) a rectangular matrix of random Gaussian noise, 3) the adjacency matrix of the Erd\H{o}s-R\'{e}nyi graph, and 4) the adjacency matrix of the Barab\'{a}si-Albert graph~\cite{barabasi1999emergence}. We show that RBF component approximations are more efficient than truncated SVD approximations even for these highly unstructured models.
This suggests that the RBF approximation is not merely more efficient due to the exploitation of special structures in the data, \eg as exemplified in \Cref{fig:motivating}. The RBF components learn \emph{general} data matrices better while being defined by the same number of parameters as the standard outer products because they are more informative and flexible as manifested by the higher number of ranks that each component possesses.

\Cref{fig:random_efficiency} (A)/(B) shows the decay of the mean square loss as we increase the number $r$ of RBF components. For the random matrices with independent standard Gaussian entries, approximately twice the number of SVD ranks are required to achieve the same mean square error with the RBF approximations.

\Cref{fig:random_efficiency}(C)/(D) show the decay rate of reconstruction loss for the adjacency matrices of a Erd\H{o}s-R\'{e}nyi random graph and a Barab\'{a}si-Albert random graph. The Erd\H{o}s-R\'{e}nyi graph has $40$ vertices and a probability of $0.5$ for each independent undirected edge. The Barab\'{a}si-Albert random graph also has $40$ vertices and is formed by the preferential attachment model so that each new vertex is attached with $3$ undirected edges. See, \eg, \cite{barabasi1999emergence} for further details on the construction. For very small $r$, the MSE loss of the RBF and SVD models is similar for both graph models while the RBF approximations have a small advantage. However, the RBF model exhibits much a faster reduction of loss as the number of components grow. In particular, for the Erd\H{o}s-R\'{e}nyi model, exact reconstruction with an MSE error below $10^{-5}$ requires $18$ RBF components and $39$ SVD components, respectively. For the preferential attachment model, $8$ RBF and $37$ SVD components are needed for the exact reconstruction.

\begin{figure}
    \includegraphics[width=\textwidth]{fig-efficiency.pdf}
	\caption{A comparison of MSE error achieved by RBF and SVD approximations depending on the number of components/ranks. Various full-rank random models considered: (A) rectangular and (B) square standard asymmetric Gaussian matrices and (C) and (D) adjacency matrices of random graphs. Consistenly over these models, several times less parameters are needed to achieve the same MSE accuracy with the RBF approximation.}
    \label{fig:random_efficiency}
\end{figure}

\subsection{Molecular Distance Matrix}

In the next example, we try to approximate a distance matrix between a set of molecules in the reproducing kernel Hilbert space (RKHS) induced by a marginalized graph kernel~\cite{kashimaMarginalizedKernelsLabeled2003,tangPredictionAtomizationEnergy2019,xiangPredictingSingleSubstancePhase2021}. Here, each molecule is represented by a graph where nodes represent atoms and edges represent inter-atomic interactions. The inner product between two molecules under the graph kernel is the expectation of the path similarities between a simultaneous random walk on the two corresponding molecular graphs. Given a graph kernel $\langle \cdot, \cdot \rangle$ and a pair of molecular graphs $G$ and $G'$, the distance is computed using the geometric definition
\begin{equation}\label{eq:rkhs-distance}
d_{G, G'}^2 = \langle G - G', G - G' \rangle = \langle G, G \rangle + \langle G', G' \rangle - 2 \langle G, G' \rangle,
\end{equation}
where the subtractions in the intermediate step are to be understood as happening in the feature space.

The matrix in \Cref{fig:molecular-distance}(A) is computed from a random subset of 256 molecules from the QM7 dataset~\cite{blum970MillionDruglike2009}. \Cref{fig:molecular-distance}(B)/(C) show the MSE losses in log scale of the RBF approximations and the SVD approximation with the same number of ranks. To further demonstrate the advantage of the RBF approximation, in Table~\ref{table:kernel}, we collect the number of components required to achieve certain approximation accuracy, in particular, there is more than seven times memory save to reach a $0.0005$ MSE loss.
\begin{center}
\begin{tabular}{ |c|c|c| }
 \hline
 MSE & \# SVD ranks & \# RBF components \\
 \hline
 $<$ 0.01 & 3 & 1 \\
  0.001 & 30 & 7 \\

 0.0005 & 77 & 10  \\
 \hline
\end{tabular}
\captionof{table}{\TODO{Problem: the data here does not seem to be consistent with \Cref{fig:molecular-distance}. In that figure, for 0.001 MSE, we need rougly twice as many ranks for SVD, but not 4x as in this table.}\label{table:kernel}}
\end{center}

\begin{figure}
    \includegraphics[width=\textwidth]{fig-efficiency-grpah-kernel.pdf}
	\caption{A comparison of RBF and SVD approximations, as well as different solution algorithms for RBF approximations, of a molecular kernel distance matrix in terms of accuracy and time consumption. (A) the molecular distance matrix; (B) construction loss of RBF and SVD approximations; (C) zoom-in comparison of the performance of full-batch gradient descent (GD) and mini-batch stochastic gradient descent (SGD) algorithms; (D) comparison of GD and SGD in terms of time-to-accuracy.
	\label{fig:molecular-distance}}
\end{figure}
Green plots in \Cref{fig:molecular-distance} refer to the RBF approximation obtained by the stochastic optimization approach (discussed above at the end of Section~\ref{sec:method}) and black plots refer to the learning process based on the full gradient. In \Cref{fig:molecular-distance} (C), we can see that a non-stochastic training achieves slightly smaller MSE loss for the same parameter set-up, and \Cref{fig:molecular-distance} (D) shows that it costs us a $2.5$-$3$ times longer training time. Here, the batch size is $100$, the learning rate is $0.1$ and the number of Adam iteration steps is chosen to be $10000k$ for $k$ component approximation since this is approximately the number of iterations required for the best run of a non-stochastic training process to stabilize at some loss level. Stochastic version mini-batch size is set to be $5kn$, where $k$ is the number of components learned and $n = 256$ is the size of the matrix. Moreover, a stochastic training process can be administered significantly lighter than described, allowing to search for the RBF approximation under the limited memory resources. So, in \Cref{fig:molecular-distance} (B), we demonstrate that stochastic Adam can find an RBF approximation with $k$ being up to $128$ that significantly outperforms corresponding SVD-based approximation. For this experiment, the batch size is taken to be $50$, there is a hard limit to $50000$ iterations and mini-batch size is set to be $8n$ for every number of components $k$. We note that $64$ RBF components are observed to be enough for an almost perfect RBF approximation (with the MSE loss below 1e-4).

\section{Interpretation of the resulting decomposition}\label{sec:interpretability}

RBF components of form \eqref{eq:rbf-comp} and \eqref{eq:rbf-component-sym} can discover one-dimensional distance-related relationships because the underlying radial basis function is a distance-based positive definite kernel. As we have discussed in the previous section, the RBF approximation can reach the same $L_2$-error with the SVD approximation using significantly fewer components and parameters across a variety of natural matrix models. However, the advantages of the RBF approximation go beyond memory savings. Just like any approximation of a large data object using a few simpler and lower-dimensional components, the RBF approximation provides a means for capturing simpler underlying structures in the data. In this section, we illustrate this on a sequence of diverse applications, \eg, learning one-dimensional geometry in a three-dimensional manifold, uncovering structures in graphs, and compressing image data.

Before we proceed with the details, let us note that the applications that we consider are among the most popular and widely studied applied math problems. Consequently, each of them is associated with multiple specialized and deeply studied solution approaches. Here, our aim is \emph{not} to compare the RBF-based approach with the state-of-the-art solutions specialized to any of these applications, but rather to demonstrate the universality of the RBF approach over several diverse tasks. We hope that this can motivate the competitive promise of the RBF approximations and set a stage for further exploration.

\textbf{One-dimensional manifold learning.} Consider an S-shaped surface
\begin{equation*}
    \left\{ \left(\sin t, 2y, \sgn(t)(\cos t - 1)\right) \mid t \in [-\frac{3}{2}\pi, \frac{3}{2}\pi], y \in [0, 2] \right\}
\end{equation*}
in a 3D space, where $t$ represents the ``time" coordinate along the curve while $y$ represents the coordinate along the ``width'' dimension of the letter S. A noisy dataset $X = (\B{x}_1, \ldots, \B{x}_{1000})$ is drawn from this surface by sampling $t$ and $y$ uniformly $1000$ times and then perturbing the points by an additive Gaussian noise $Z \sim \mathcal{N}(0, \delta \mathds{I})$.

The approximation target is the soft distance matrix $K$ for the dataset, namely, its $(i,j)$-th entry is defined as $k_{ij} = \exp(-\|\B{x}_i - \B{x}_j\|^2/2)$ for $i,j, = 1, \ldots, 1000$. One symmetric RBF component as per definition~\eqref{eq:rbf-component-sym} is fitted to approximate $K$, resulting in a vector $\B{u} \in \R^{1000}$ that approximately recovers the time coordinate $t$. See the relationship between the ground truth order along the curve ($\B{t} = (t_1, \ldots , t_{1000})$) and learned order ($\B{u}$) for four noise levels $\delta$ from $0$ (no noise) to $0.6$ in Figure~\ref{fig:s-curve}.
\begin{figure}
    \includegraphics[width=\textwidth]{fig-manifold.pdf}
	\caption{First row: Visualization of the S-curve dataset with various added noise levels. Second row: Linear relation between Ground truth ($\B{t}$) ``time" coordinate parametrizing given S dataset and Learned embedding ($\B{u}$) demonstrates that the increasing or decreasing order of the coordinates of $\B{u}$ can be used to uncover main one-dimensional structure of the dataset.}
    \label{fig:s-curve}
\end{figure}

\textbf{Community detection.} The stochastic block model (SBM) is a random graph model with planted clusters, widely used for community detection and clustering tasks. See, \eg, \cite{abbe2017community} for a survey of modern approaches to the SBM-based clustering, as well as their limitations. One of the most standard approaches to find $k$ clusters in an SBM matrix is spectral clustering using the entries in the $k$ top leading eigenvectors. In contrast, only one RBF component can suffice to successfully detect multiple clusters. As demonstrated in \Cref{fig:sbm}, we consider an $80$-vertex graph with $5$ communities each containing $8, 12, 16, 20$ and $24$ vertices, respectively. All clusters have an intra-cluster probability of $0.8$ and an inter-cluster probability of $0.2$. Like in the previous example, we fit one symmetric RBF component~\eqref{eq:rbf-component-sym} to the adjacency matrix of the graph. A one-dimensional clustering is carried out on the learned coordinates of the component vector $u$ to assign the respective graph vertices into $5$ clusters.

\begin{figure}
\includegraphics[width=\textwidth]{fig-sbm.pdf}
	\caption{Left: adjacency matrix of an SBM random graph with $5$ communities with $8, 12, 16, 20$ and $24$ vertices, sampled from the model with intra-cluster probability of $0.8$ and an inter-cluster probability of $0.2$. Middle: a one component RBF approximation visually captures all $5$ communities whereas the rank one SVD-based approximation fails. Right: One-dimensional clustering of the coordinates of the RBF approximation component vector allows to recover community attribution for all the vertices of the given graph.}
    \label{fig:sbm}
\end{figure}

\textbf{Approximation of graph adjacency matrices.} We return to the example of the Erd\H{o}s-R\'{e}nyi $(40, 0.5)$ random graph considered in Section~\ref{sec:efficiency} with the observation that the $L_2$-based MSE loss is probably not the most important measure of a meaningful graph compression. Instead, what one might want to preserve in the approximation is the connectivity of the graph. In \Cref{fig:er-inter}, we can visually compare the difference in how well the RBF and SVD approximation models can capture the structural patterns in the adjacency matrix. To quantify this observation, we formalize the following \emph{edge prediction algorithm}: we threshold the real-valued entries of the approximation matrix by some cutoff value $\varepsilon \in [0,1]$ and predict an edge if the element of the approximation matrix is greater than $\varepsilon$ (thus, it is close enough to its value in the binary adjacency matrix). We confirm the advantage of the RBF approximation in retaining the true edge structure by the difference in the ROC curve and AUC score (\cite{fawcett2006introduction}) for the SVD-based and RBF-based edge prediction algorithms with $2$ and $7$ components (see Figure~\ref{fig:er-inter}, right).
\begin{figure}
    \includegraphics[width=\textwidth]{fig-er-analysis.pdf}
	\caption{Left: adjacency matrix of an Erd\H{o}s-R\'{e}nyi random graph on $40$ vertices with edge probability $0.5$. Middle: a 2 component RBF approximation captures visibly more local structure comparing to the rank $2$ SVD approximation. Right: RBF-based edge prediction algorithm dominates its SVD-based counterpart for $2$-component and $7$-component approximations. Seven component RBF-based approximation allows for an almost exact edge reconstruction.}
    \label{fig:er-inter}
\end{figure}

\textbf{Image data compression.} It is common to consider natural image data as inherently low-rank, this assumption makes it both theoretically and practically amenable for various denoising and compressive sensing tasks. Our preliminary experiments suggest that, unlike exactly low-rank matrices, real images can be approximated with the RBF components at least just as well as with the SVD-based components. Moreover, RBF-based approximation is even better at preserving visual acutance using the same number of parameters. Here, we compare the quality of RBF and SVD approximations on two images: 1) a baboon picture that is widely used in the image processing literature, and 2) a poster from a 2009 documentary ``Space Tourists''. For RBF approximations, we fit the images using multiple components of the general asymmetric form~\eqref{eq:rbf-comp}.

For the ``baboon'' image, MSE-losses are slightly better in the RBF approximation as shown in \Cref{fig:image}, while the visual quality of the compressed image is notably higher.
For the ``Space Tourists" poster, MSE-losses are very similar in approximations with the same number of components, however, we can also observe the advantages of the RBF compression in the visual quality both in the text and the portrait parts of the poster. It is an important follow-up direction of the current work to define the quantitative measure that captures the differences in the sharpness of the image compression better than the generic $L_2$-based MSE loss.
\begin{figure}
\includegraphics[width=\textwidth]{fig-image.pdf}
	\caption{Visual comparison of the RBF and SVD based approximations for  ''Space tourists" poster and ''Baboon" image with $32$ and $64$ components. Low rank approximations tend to be smoother and less noisy, while RBF components produce sharper images, better preserving local features of an image, such as textures and edges.}
    \label{fig:image}
\end{figure}

\section{Related Work}\label{sec:related}

The literature on the low-rank approximation of matrices is truly abundant.

While SVD is known to give the best $L_2$ norm approximation of a given rank, numerous treatments develop efficient algorithms for computing and approximating SVD, making the computations scalable and robust, and use them to do dimension reduction, denoising, and learning from data. For a survey, see, e.g., \cite{kannan2009spectral, halko2011finding}. Let us briefly note here that slightly more sophisticated low-rank approximation tasks, such as low-rank tensor approximations, or non-negative factorizations, rapidly increase in complexity of training, as well as existence and uniqueness questions. It is common to turn to the iterative optimization algorithms to solve them (e.g., \cite{cichocki2009nonnegative}).

Beyond the low-rank approaches, special properties of the underlying data can be used to learn its approximation by some simple components. One classical example is approximation of kernel matrices. Many approaches were introduced to make kernel trick scalable, among the most prominent ones is the approximation of a kernel matrix by an scalar product of higher-dimensional vectors, namely $K(\B{x}^i, \B{x}^j) \approx \sum_k \phi(\B{x}^i)_k \phi(\B{x}^j)_k$. The function $\phi$ here can be constructed using randomly samples from the Fourier transformation of the kernel, versions of the Nystr\"{o}m method, or using polynomial approximation \cite{rahimiRandomFeaturesLargescale2007,avron2017random, wangNumericalRankRadial2018}. All these methods use positive semi-definiteness of the kernel matrix to be approximated, and sometimes its concrete functional form, and neither of which is imposed in our approach.

The idea to use kernel functions \emph{for approximation} had several appearances in the recent literature, usually still limited to the approximation of positive semi-definite or otherwise very special matrices. For example, in \cite{duvenaudStructureDiscoveryNonparametric2013}, compositions of kernels used for nonparametric
regression. In \cite{smolik2021radial,zabran2020radial}, the authors consider approximations of functions defined on a known discrete dataset in $\mathbb{R}^d$ by linear combinations of $d$-dimensional gaussian RBF functions, as well as by more sophisticated building blocks. Although our setting is crucially different from this one, where only the coefficients are learned, but the component matrices are known by design, which results in a linear system to solve instead of a highly non-convex optimization problem.

A method that is intimately related to our current approximation approach is the Radial-Basis-Function Network. An RBF network as defined in~\cite{parkUniversalApproximationUsing1991} is a three layer neural network with RBF nonlinearities in its hidden layer. Effectively, it learns, \ie approximates, an unknown function as a linear combination of multivariate Gaussian kernel functions. RBF networks are known for the universal approximation properties \cite{parkUniversalApproximationUsing1991}. The conceptual difference between our RBF approximation with neural networks trained by a supervised learning task is that we also learn the input data. From a neural network perspective, we treat the input training data as a set of learnable parameters in addition to the weights. Moreover, in the optimization process, we are not concerned with the generalizability of the learned model because a different set of parameters are learned for \emph{each} concrete data matrix $K$. This makes our target different: we aim to make the model structure as light as possible. Clearly, an approximation exists for any data with enough overfitting, as illustrated in~\Cref{sec:efficiency}. Our primary interest is in the most compact one.

\section{Conclusion and future work}\label{sec:conclusions}
We present a novel framework for matrix data approximation based on nonlinear RBF components. We restrict the building blocks to the one-dimensional kernels, which gives us a simple way to control complexity\footnote{Moreover, multi-dimensional kernels can be approximated by one-dimensional ones, just like any other data matrix, so, considering heavier multi-dimensional components in our framework is probably redundant.}. Unlike composite kernel learning, we consider arbitrary linear combinations of kernel matrix component and deliberately do not constrain the resulting matrix to be semi-positive definite. This relaxation enhances the flexibility of the approximation and improves efficiency. We demonstrate the advantage of the proposed approximation, both in terms of memory saving and interpretability, across various datasets. We employ gradient-based method for learning the nonlinear RBF components, discuss the particulars of the learning setup and demonstrate convergence to a low-component RBF approximation.

\bigskip

This is an initial exploratory treatment of a new method, which gives rise  to an abundance of natural follow up questions and possibilities of future extensions. For example, our current symmetric matrix fitting is bound to have ones in the diagonal. A simple correction of the functional form, such as multiplying the symmetric outer product to the RBF component matrix, can be used to approximate symmetric matrices with non-constant diagonal while keeping the number of parameters the same. More generally, changing the functional form brings in more flexibility to the method. It would be even more interesting to learn the correct functional form from the data. Another direction of a follow-up work is to formalize similar RBF-based approximations for multi-order tensor data, \eg, a multi-time series dataset, a color image, or a video clip.

Our current optimization strategy is quite generic and not specialized for a particular functional form. If the nonlinear components degenerate into standard outer products, \eg as in the case of SVD and tensor CP decomposition, then better ways are known to solve the approximation problem much faster than gradient descent based approaches. Unfortunately, the applocation of linear solver-based approaches are likely ruled out by the Gaussian nonlinearity. Nonetheless, it would be very interesting to leverage more general optimization strategies beyond gradient descent that can still benefit from some structural assumptions about the model.

Compressed data in the form of an RBF approximation can be used as an intermediate representation in many machine learning pipelines. For example, an RBF approximation can compress neural networks models or massive training datasets. \TODO{The last sentence is a bit vague} Finally, it is a very natural future direction to further investigate particular simple data features that nonlinear approximation preserves in various applications.

\section{Acknowledgment}

This work was supported by the Laboratory Directed Research and Development Program of Lawrence Berkeley National Laboratory under U.S. Department of Energy Contract No. DE-AC02-05CH11231.

Machine Learning (cs.LG); Machine Learning (stat.ML)