Some Kähler structures on products of 2-spheres

Abstract: We consider a family of Kähler structures on products of 2-spheres, arising from complex Bott manifolds. These are obtained via iterated $\mathbb P^1$-bundle constructions, generalizing the classical Hirzebruch surfaces. We show that the resulting Kähler structures all have identical Chern classes. We construct Bott diagrams, which are rooted forests with an edge labelling by positive integers, and show that these classify these Kähler structures up to biholomorphism.

\section{Introduction}

In complex geometry, it is interesting to study the class of complex structures (or K\"ahler structures) supported on a fixed smooth
oriented manifold $M$. Since the basic invariants of a complex manifolds are the Chern classes, it is tempting to try and use these
to distinguish complex structures on $M$. In complex dimension two, the Hirzebruch surfaces $\mathbb F _m$
are topologically either diffeomorphic to $S^2 \times S^2$ (if $m$ is even), or to $\mathbb P^2 \# \overline{\mathbb P^2}$ (if $m$ is odd).
Focusing on the Hirzebruch surfaces diffeomorphic to $S^2 \times S^2$, a celebrated result of Hirzebruch \cite{hirzebruch}
shows that all the $\mathbb F_{2k}$
are distinct as complex manifolds, even though they have identical Chern classes.

In the present paper, we extend Hirzebruch's result, by considering {\it $\mathbb Z$-trivial complex Bott manifolds} (see Bott and
Samelson \cite{BS}). In complex dimension
two, these are precisely the Hirzebruch surfaces $\mathbb F _{2k}$. In complex dimension $n$, these are compact K\"ahler manifolds
diffeomorphic to $S^2 \times \cdots \times S^2 = (S^2)^n$.
To each $n$-dimensional $\mathbb Z$-trivial complex Bott manifold $M$,
we associate a {\it Bott diagram}, which is a rooted forest equipped with an edge labelling by positive integers. Our main result is the following:

\begin{main*}
Every $n$-vertex rooted forest equipped with an edge labelling by positive integers arises as the Bott diagram of some
$n$-dimensional $\mathbb Z$-trivial complex Bott manifold. Moreover, for an arbitrary pair of $n$-dimensional $\mathbb Z$-trivial
complex Bott manifolds $M_1, M_2$, we have:
\begin{enumerate}
\item $M_1$ is biholomorphic to $M_2$ if and only if their Bott diagrams are isomorphic, but
\item there is a diffeomorphism $\phi: M_1\rightarrow M_2$ with the property that $\phi^*(c(M_2))=c(M_1)$, where $c$
denotes the total Chern classes.
\end{enumerate}
\end{main*}

Our result provides a combinatorial classification of a certain family of K\"ahler structures on the products
$S^2 \times \cdots \times S^2 = (S^2)^n$. When $n=2$, the only K\"ahler structures on $S^2\times S^2$
are those arising from the Hirzebruch surfaces. When $n\geq 3$, we do not know whether these products of
$2$-spheres support any other K\"ahler structures. Our result also shows that these K\"ahler structures are
indistinguishable as far as Chern classes are concerned.

\vskip 5pt

Recall that the rational Pontrjagin classes of a smooth manifold are defined using the smooth structure. A celebrated result
of Novikov \cite{Novikov} shows however that these classes in fact only depend on the underlying topological structure (other
proofs were given in \cite{Gromov}, \cite{Ranicki}, \cite{RY}, and \cite{RW}). More precisely, if one
has a pair of homeomorphic smooth manifolds, then the homeomorphism can be chosen to take the total rational Pontrjagin class
to the total rational Pontrjagin class. Because of the lack of known counterexamples, we would like to raise the following analogous
question for Chern classes:

\begin{question*}
Let $X$ be a compact smooth manifold and let ${\mathcal M}_0$ be the set of biholomorphism classes of all K\"ahlerian complex structures
on $X$.  Do the total Chern classes $c(J)\in H^{2\ast }(M, {\mathbb Z})$ (for all $J\in {\mathcal M}_0$) belong to a single orbit for the action
of the diffeomorphism group on the cohomology ring?
\end{question*}

Part of the difficulty in addressing this question is the lack of examples.  Indeed, several classes of manifolds are known to support
unique K\"ahler structures -- see for instance the rigidity results of Hirzebruch-Kodaira \cite{HK}, Yau \cite{yau} and Siu \cite{siu}.
In contrast, there are very few smooth manifolds that are known to support multiple distinct K\"ahler structures.
Our main theorem shows that the answer to this question is {\it yes} for a particular family of K\"ahler structures
on $X= (S^2)^n$.

We also point out that there are examples of manifolds where the answer to this question is {\it no}. Notably, it follows
from work of Kotschick \cite{Kotschick1} \cite{Kotschick2}, that there exist a pair of K\"ahler $3$-folds $M_1, M_2$ which are diffeomorphic,
but have distinct Chern numbers $c_1^3(M_1)\neq c_1^3(M_2)$. A consequence is that their first Chern classes must lie in distinct orbits of
the diffeomorphism group action on second cohomology. Similar examples exist in higher dimensions. Nevertheless, the question might
have an affirmative answer for some manifolds, particularly in the presence of a high degree of symmetry.

\vskip 5pt

Our paper is structured as follows. We review some background material in Section \ref{sec-backgrnd}, and prove our
main theorem in Section \ref{sec-main-thm}. Our argument requires a reconstruction result for labelled rooted forests,
which is explained in Section \ref{sec-reconstruction}.

\section{Background material}\label{sec-backgrnd}

\subsection*{Bott manifolds.}
Recall that a Bott manifold $M^n$ is one that admits a {\em Bott Tower,} namely, $M^n=B_n$ and
\begin{equation}
\label{tower} B_n \xrightarrow {\pi_n} B_{n\!-\!1} \xrightarrow {\pi_{n\!-\!1}} \cdots \xrightarrow {\pi_2} B_1 \xrightarrow{\pi_1} B_0=\{ \text{a  point} \}
\end{equation}
where for each $1\leq j\leq n$, $B_j={\mathbb P}({\mathcal O}\oplus S_j)$ is the projectivization of the direct sum of the trivial line bundle with a holomorphic line bundle $S_j$ over $B_{j\!-\!1}$, with $\pi_j$ the projection map.

Clearly, $B_1={\mathbb P}^1$, and $B_2$ is a Hirzebruch surface ${\mathbb F}_m = {\mathbb P}({\mathcal O}_{{\mathbb P}^1} \oplus {\mathcal O}_{{\mathbb P}^1}(-m))$ over ${\mathbb P}^1$, where $m$ is any nonnegative integer. It is well-known that all ${\mathbb F}_{2k}$ are diffeomorphic to ${\mathbb F}_0=\mathbb P^1 \times \mathbb P^1 \cong S^2\!\times \!S^2$, while all ${\mathbb F}_{2k-1}$ are diffeomorphic to ${\mathbb F}_1={\mathbb P}^2 \# \overline{{\mathbb P}^2}$, the one point blow up of $ {\mathbb P}^2$.

\begin{definition}
A Bott manifold $M^n$ is called ${\mathbb Z}$-trivial, if it is diffeomorphic to $({\mathbb P}^1)^n$, the product of $n$ copies of the complex projective line.
\end{definition}

By the work of Choi and Masuda \cite{ChoiMasuda}, a Bott manifold $M^n$ is ${\mathbb Z}$-trivial if its integral cohomology ring is isomorphic (as a graded ring) to that of $({\mathbb P}^1)^n$. In fact, they show that every graded ring isomorphism between the cohomology rings of two ${\mathbb Q}$-trivial Bott manifolds is induced by a diffeomorphism. Here $M^n$ is ${\mathbb Q}$-trivial means that the cohomology ring of $M$ with ${\mathbb Q}$-coefficients is isomorphic (as a graded ring) to that of $({\mathbb P}^1)^n$.

\subsection*{Projectivization of vector bundles.} Let us recall some general facts concerning projectivizations of vector bundles.

Let $E$ be a holomorphic vector bundle of rank $r$ over a compact complex manifold $B$, let $\pi : M={\mathbb P}(E)\rightarrow B$
be the  projectivization of $E$, where $\pi$ is the projection map. We adopt the algebro-geometric convention here, namely,
$\pi^{-1}(x)={\mathbb P}(E_x)$ is the set of all the {\em hyperplanes} (instead of {\em lines}) through the origin in the fiber
$E_x\cong {\mathbb C}^{r\!-\!1}$.  $M$ is again a compact complex manifold, a holomorphic fiber bundle with fiber
${\mathbb P}^{r\!-\!1}$ over $B$.

Denote by $L$ the dual of the tautological line bundle, then we have the following two short exact sequences of holomorphic vector bundles
over $M$:
\begin{eqnarray}
\label{euler}  &&  0 \rightarrow  {\mathcal O}_M \rightarrow  \pi^{\ast } E^{\ast } \otimes L \rightarrow  T_{M|B} \rightarrow  0 \\
 \label{relativetangent} &&  0 \rightarrow  T_{M|B} \rightarrow  T_{M} \rightarrow  \pi^{\ast } T_{B} \rightarrow  0
\end{eqnarray}
where $T_{M|B}$ is the relative tangent bundle, namely, the kernel of the differential of $\pi$. The first Chern class $\xi = c_1(L)$
satisfies the Grothendieck equation
\begin{equation*}
 f(\xi) : = \xi^r - \xi^{r\!-\!1} \cdot \pi^{\ast }c_1(E) + \xi^{r\!-\!2} \cdot \pi^{\ast }c_2(E) - \cdots + (-1)^r \pi^{\ast }c_r(E) = 0,
\end{equation*}
while the cohomology ring (or the Chow ring) of $M$ is generated by the pull back of that of $B$ and $\xi$:
\begin{equation}
H^{\ast }(M, {\mathbb Z})  : = \pi^{\ast }H^{\ast }(B, {\mathbb Z})\  [\xi ]  \ / \  ( f(\xi )) .
\end{equation}

Recall that a {\em section} of $\pi$ is a complex submanifold $Z\subseteq M$ such that $\pi|_Z: Z \rightarrow B$ is a biholomorphism.
Equivalently, a section of $\pi$ is given by a holomorphic map $i : B \rightarrow M$ such that $\pi \circ i = id_B$. In this case the image
$i(B)$ is the submanifold in $M$ isomorphic to $B$. Note that the sections of $\pi$ correspond to quotient line bundles of $E$.

To see this, let $Q$ be a holomorphic line bundle on $B$ which is a quotient bundle of $E$. As we are using the hyperplane convention for
projectivizations, so ${\mathbb P}(Q)\cong B$ is a submanifold of ${\mathbb P}(E)=M$, which gives a section of $\pi$. Conversely, given a
section $i: B\rightarrow M$ of $\pi$, since the tautological line bundle $L^{\ast}$ is a subbundle of $\pi^{\ast}E^{\ast }$ on $M$, $Q=i^{\ast }L$
would be a quotient line bundle of $i^{\ast} \pi^{\ast }E = E$ on $B$.

Next, let us specialize to the situation when the vector bundle on $B$ is $E={\mathcal O}\oplus S$, the sum of the trivial line bundle with
another line bundle $S$. Writing $s= - \pi^{\ast}c_1(S)$, the above short exact sequences (\ref{euler}) , (\ref{relativetangent}), along with the
Grothendieck equation, gives us
\begin{equation}
c_1(T_{M|B})=2\xi + s, \ \ c(M) = (1+ 2\xi +s ) \cdot \pi^{\ast }c(B), \ \ \mbox{and} \ \xi^2 = - \xi \cdot s
\end{equation}
in the cohomology (or the Chow) ring $H^{\ast }(M,{\mathbb Z})$.

\subsection*{Cohomology ring of Bott manifolds.}

Now let us apply these formula to the $j$-th stage $\pi_j: B_j\rightarrow B_{j\!-\!1}$, which is the projectivization of the splitting rank $2$ vector bundle ${\mathcal O}\oplus S_j$ on $B_{j\!-\!1}$, then we get the following:
\begin{eqnarray*}
&& H^{\ast }(B_j,{\mathbb Z}) = \pi_j^{\ast } H^{\ast }(B_{j\!-\!1},{\mathbb Z}) [\xi_j] / (\xi_j^2+\xi_js_j)  \\
&& c(B_j) = (1+ 2\xi_j + s_j ) \cdot \pi_j^{\ast }c(B_{j\!-\!1}),
\end{eqnarray*}
where $-s_j$ and $\xi_j$ are the first Chern class of $\pi_j^{\ast }S_j$ and $L_j={\mathcal O}_{\pi_j}(1)$, the dual of the tautological line bundle on $B_j$.

Given a Bott manifold $M^n$ with Bott tower \eqref{tower}, let us write
\begin{eqnarray*}
x_j & = &  (\pi_{j\!+\!1}\circ \cdots \circ \pi_n)^{\ast } \xi_j \\
h_j & = &  (\pi_{j\!+\!1}\circ \cdots \circ \pi_n)^{\ast } s_j
\end{eqnarray*}
for each $1\leq j\leq n$. Note that $x_1$ is the first Chern class of the pull back to $M$ of ${\mathcal O}_{{\mathbb P}^1} (1)$  on $B_1$, and $h_1=0$. By an inductive argument, we obtain the following:
\begin{eqnarray}
&&
H^{\ast }(M, {\mathbb Z}) = {\mathbb Z} [ x_1, \ldots , x_n] / ( x_1^2, x_2^2+x_2h_2, \ldots , x_n^2+x_nh_n ) \\
&& c(M) = (1+2x_1)(1+2x_2+h_2) \cdots (1+2x_n+h_n)
\end{eqnarray}
where $x_1, \ldots , x_n$ is a set of generators for $H^2(M,{\mathbb Z})\cong {\mathbb Z}^n$, and each $h_j$ satisfies
\begin{equation}
h_j = a_{j1}x_1 + a_{j2}x_2 + \cdots + a_{j,j\!-\!1}x_{j\!-\!1}
\end{equation}
where all $a_{jk}$ are integers.

\begin{example*}
In the special case where all the line bundles $S_j$ are trivial, we get the product $P=({\mathbb P}^1)^n$ of $n$-copies of the complex projective line ${\mathbb P}^1$. In this case, all $h_j=0$ and we will denote the corresponding $x_j$ by $y_j$. The above computations give us:
\begin{eqnarray}
&& H^{\ast }(P,{\mathbb Z}) = {\mathbb Z}[y_1, \ldots , y_n] / (y_1^2, \ldots , y_n^2) \\
&&  c(P)=(1+2y_1)\cdots (1+2y_n)
\end{eqnarray}
\end{example*}

\vskip 10pt

\section{Proof of the Main Theorem.}\label{sec-main-thm}

This entire section is devoted to the proof of the Main Theorem.

 \subsection{The structure of ${\mathbb Z}$-trivial Bott manifolds.} We start by analyzing how the ${\mathbb Z}$-triviality condition
 affects the cohomology elements $h_j$.

For a given a Bott tower on $M^n$, assume that $2\mid h_j$ and $h_j^2=0$ for all $j$. Write $z_j=x_j + \frac{1}{2}h_j$. Since $h_1=0$, and
for each $2\leq j\leq n$ the corresponding $h_j$ is generated by $x_1, \ldots , x_{j-1}$, it follows that $\{ z_1, \ldots , z_n\} $ generates
$H^{\ast}(M,{\mathbb Z})$. Also, each $z_j^2 =0$ by the Grothendieck equation. So defining $\phi (y_j)=z_j$ gives a graded ring isomorphism
$\phi : H^*(P; \mathbb Z) \rightarrow H^*(M^n ; \mathbb Z)$. By the result of Choi and Masuda \cite{ChoiMasuda}, there is a diffeomorphism
$\Phi: M^n\rightarrow P\cong (S^2)^n$, which induces $\Phi^* = \phi$. It follows that
$M^n$ is ${\mathbb Z}$-trivial. Moreover, by the Chern class formula, we see that $\phi (c(P)) = c(M)$.

Conversely, if there exists an isomorphism $\phi : H^{\ast}(P,{\mathbb Z}) \rightarrow H^{\ast}(M,{\mathbb Z})$, then we claim that
$2\mid h_j$ and $h_j^2=0$ for all $j$. To see this, let us write $\phi (y_j)=z_j$. We have
$$ H^{\ast }(M,{\mathbb Z}) = {\mathbb Z} [z_1, \ldots , z_n] / ( z_1^2, \ldots , z_n^2).$$
For each $1\leq k\leq n$, the group $H^{2k}(M,{\mathbb Z})$ is a free abelian group generated by products
$z_I = z_{i_1}  \cdots z_{i_k}$ for all multi-indices $I= (i_1\cdots i_k)$ of length $k$, where $1\leq i_1 < i_2 < \cdots < i_k\leq n$.
Note that for any integer linear combination $z=a_1z_1+ \cdots + a_nz_n$, if $z^2=0$, then $a_ia_j=0$ for all $i\neq j$,
thus at most one of these $a_i$ could be non-zero.

Now we proceed to show that $2\mid h_j$ and $h_j^2=0$, by induction on $j$, where $j\in A:=\{ 1, 2, \ldots , n\}$. First we have $h_1=0$.
For $j=2$, since $x_1^2=0$, we know that there must be a unique $i_1 \in A$ such that $x_1=\varepsilon_1z_{i_1}$, where
$\varepsilon_1=\pm 1$ since $x_1$ is a primitive element in $H^2(M,{\mathbb Z})$. Write $x_2=az_{i_1}+z$, where $z$ is a
linear combination of $z_j$ for $j\in A\setminus \{ i_1\}$. We have $h_2=bz_{i_1}$ since $h_2$ is a multiple of $x_1$. Since
$x_2(x_2+h_2)=0$, we have
$$ (2a+b)z_{i_1}z + z^2 = 0. $$
Since $H^4(M, {\mathbb Z})$ is a free abelian group with generators $z_iz_j$ for $1\leq i<j\leq n$, we conclude from the above
equality that $2a+b=0$ and $z^2=0$. So $2\mid h_2$, $h_2^2=0$, and $z=x_2+\frac{1}{2}h_2$ satisfies $z^2=0$, thus equals to
$\varepsilon_2z_{i_2}$ for some $i_2\neq i_1$, and $\varepsilon_2=\pm 1$.

Now assume that for a fixed $2\leq k<n$, we already have $2\mid h_j$, $h_j^2=0$ for each $j\leq k$, and
$x_j':= x_j+\frac{1}{2}h_j=\varepsilon_j z_{i_j}$ where $i_1 $, ..., $i_k$ are all distinct in $A$ and $\varepsilon_j=\pm 1$.
Since $h_{k+1}$ is a linear combination of $x_1', \ldots , x_k'$, we can write
$$ h_{k+1} = b_1z_{i_1} + \cdots + b_k z_{i_k} $$
Also, let us write $x_{k+1}=a_1z_{i_1} + \cdots + a_k z_{i_k} + z $, where $z$ is a combination of those $z_j$ for $j$ in
$A\setminus \{ i_1, \ldots , i_k\}$. Now by applying the Grothendieck equation, namely, $x_{k+1}(x_{k+1}+ h_{k+1})=0$, we
get the equation
$$ \sum_{j=1}^k (2a_j + b_j)z_{i_j}z + z^2 + \sum_{j,l=1}^k a_l(a_j+ b_j)z_{i_j} z_{i_l} =0. $$
Since $z$ cannot be zero, we know that $b_j=-2a_j$ for each $j\leq k$, so $2\mid h_{k+1}$ and $h_{k+1}^2=0$. Furthermore,
$x_{k+1}'=z$ is a square zero primitive element, thus must be of the form $\pm z_{i_{k+1}}$ for some $i_{k+1}$ in
$A\setminus \{ i_1, \ldots , i_k\}$.

To summarize, we have established the following (also independently obtained by J. H. Kim \cite{kim}):

\begin{lemma}\label{diffeo-orbit}
If $M^n$ is a Bott manifold and $\phi$ is an isomorphism between the integral cohomology rings of $P=({\mathbb P}^1)^n$ and $M$,
then for any Bott tower \eqref{tower} with $M=B_n$, we have $2\mid h_j$ and $h_j^2=0$ for each $j$, and $\phi(c(P)) = c(M)$.
\end{lemma}

Note that for any holomorphic line bundle $Q$ on $B$, the projectivizations ${\mathbb P}(E)$ and ${\mathbb P}(E\otimes Q)$ are isomorphic to each other. In particular, for $B_j={\mathbb P}({\mathcal O}\oplus S_j)$ over $B_{j-1}$, one can replace $S_j$ by its dual $S_j^{\ast }$, as
$$ {\mathcal O} \oplus S_j \cong (S_j^{\ast } \oplus {\mathcal O})\otimes S_j. $$
This replacement will not change $B_j$, but will affect the choice of  sections $L_j$ thus affecting $x_j$, while $h_j$ is replaced by $-h_j$.

By the proof of the lemma above, we know that for any ${\mathbb Z}$-trivial Bott manifold $M^n$ and any Bott tower \eqref{tower} on
$M$, if we write  $z_j=x_j+\frac{1}{2}h_j$, then $\{ z_1, \ldots , z_n\}$ is a set of   generators for the cohomology ring, with $z_j^2=0$
for each $j$. For any $2\leq j\leq n$, since $h_j^2=0$, we know that either $h_j=0$, or $h_j=  2q_j z_{\sigma (j)}$ for some positive
integer $q_j$ and $\sigma (j) < j$. Here we used the fact that we can replace $S_j$ by $S_j^{\ast }$ without changing the Bott tower
to ensure that these $q_j$ be positive.

From now on, we will make these choices, so $q_j>0$ whenever $h_j\neq 0$. That is, under our choices of these $S_j$, each
$x_j$ is represented by the central sections of $\pi_j$, and each $z_j=x_j+ \frac{1}{2}h_j$ is represented by an effective divisor. We
have shown

\begin{lemma}
For any Bott tower \eqref{tower} we can choose the generator sets $\{ x_1, \ldots , x_n\}$ and $\{ z_1 , \ldots , z_n\}$ so that
(i) each $z_j^2=0$, (ii) each $z_j$ is represented by an effective divisor, and (iii) each $x_j$ is represented by a smooth hypersurface,
which is iteself a Bott manifold of dimension $n-1$.
\end{lemma}

Obviously, for a given Bott manifold $M^n$, there are many Bott towers on it. So to sort out all distinct complex structures on $P=(S^2)^n$
given by the Bott manifolds, we need to find canonical representations for the ${\mathbb Z}$-trivial Bott manifolds. This is the goal of the
next section.

\subsection{Bott diagrams.}

Let us denote by $A=\{ 1, 2, \ldots , n\}$ and write
$A_0=\{ j\in A \mid h_j=0\}$.
When $A_0\neq A$, we have a map $\sigma: A\setminus A_0 \rightarrow A$ satisfying $\sigma (j) < j$, given by the equation $h_j= 2q_j z_{\sigma (j)}$. Let us denote by $A_1=\sigma^{-1}(A_0)$, $A_2=\sigma^{-1}(A_1)$, and so on. It is easy to see that there exists some positive integer $r$ such that $A$ is the disjoint union of non-empty sets $A_0$, $A_1$, ... , $A_r$.

We will say that the {\em level} of $j\in A$ is $k$ if $j\in A_k$. It takes $\sigma$ exactly $k$ times to send a level $k$ element into $A_0$.

\begin{definition}
For a given Bott tower \eqref{tower}, we define its {\em Bott diagram} to be the following data: each element of $A$ gives a vertex, each $j\in A \setminus A_0$ gives a vertical edge from $j$ to $\sigma (j)$, marked with a positive integer $q_j$.
\end{definition}

In other words, a Bott diagram $G$ in dimension $n$ is a disjoint union $A=A_0\cup A_1\cup \cdots \cup A_r$ into $r+1$ nonempty subsets,  along with maps $A_r \rightarrow A_{r-1} \rightarrow \cdots \rightarrow  A_1 \rightarrow  A_0$ and a map $q:A\setminus A_0\rightarrow {\mathbb Z}^+$. Here $A$ is the set of $n$ elements and $r\geq 0$.

Two Bott diagrams are considered isomorphic, if there is a bijection from $A$ to $A$ which commutes with the partition of $A$ and the maps.

We can arrange all the dots in $A_k$ at the same height, and will refer to that (imagined) horizontal line the level $k$ line (when $k=0$ we will
also call it the base line). The diagram is a graph, with finitely many connected components which are {\it trees}. Each tree has a distinguished
vertex, lying in $A_0$, which is the {\it root} of the tree. Thus from a combinatorial viewpoint, a Bott diagram is a {\it rooted forest}. Clearly,
the Bott manifold is a product of lower dimensional ones, with each factor corresponds to a connected component of the Bott diagram. So
$M^n$ is irreducible (in the sense that it is not the product of lower dimensional Bott manifolds) if and only if the Bott diagram is connected,
which occurs if and only if $A_0$ contains only one element.

\begin{example*}
To illustrate how these diagrams work, let us first consider the case $n=2$. In this case we have only two possibilities for the Bott diagram: the first one just has two dots lying horizontally, with no edges, representing the surface ${\mathbb P}^1\times {\mathbb P}^1$; and the second one is two dots with a vertical edge connecting them, marked by a positive integer $q$. This corresponds to the Hirzebruch surface ${\mathbb F}_{2q}$.
\end{example*}

\begin{example*}
For $n=3$, we have three horizontal dots, corresponding to ${\mathbb P}^1\times {\mathbb P}^1 \times {\mathbb P}^1$; two dots on the base line, the third dot on top of the right one with a vertical edge marked mark $q$, corresponding to ${\mathbb P}^1 \times {\mathbb F}_{2q}$; one dot on the base line, two dots on the level $1$ line joining the base point by edges marked with $q$ and $p$, which corresponds to the fiber product $ {\mathbb F}_{2p}\times_{{\mathbb P}^1}  {\mathbb F}_{2q}$; and finally, we have three dots lined up in a vertical line, with two edges marked with $p$ and $q$ (with $p$ on top). In this case the threefold is $M^3={\mathbb P}( {\mathcal O}_B \oplus {\mathcal O}_B(-2p(C_0+qF)) ) $, where $B={\mathbb F}_{2q}$ is the Hirzebruch surface with $F$ the ruling and $C_0$ the central section (so $C_0^2=-2q$).
\end{example*}

\subsection{Bott diagrams determine biholomorphism type.}  Our goal here is to complete the proof of our main theorem, by showing
the following:

\begin{theorem} \label{theorem}
Two Bott manifolds of dimension $n$ are biholomorphic to each other if and only if they have isomorphic Bott diagram.
\end{theorem}
Since one can build up a Bott tower from the data of a Bott diagram, we just need to prove the ``only if" part of the statement,
namely, if $f: M'\rightarrow M$ is a biholomorphism, then $M'$ and $M$ must have isomorphic Bott diagrams.

Let us fix a Bott tower on $M$. By our previous discussion, we know that each $x_j$ is represented by a smooth hypersurface $X_j$
in the sense that $x_j=c_1(X_j)$, where the divisor $X_j$ is identified with the line bundle associated with it, and each $z_j$ is
represented by an effective divisor. To be more precise, for any $j\in A_0$ of level $0$, $Z_j=X_j$ is irreducible. For any
$j\in A_1$, $Z_j=X_j + q_jX_{\sigma (j)}$. For any $j\in A_2$, we have
$$ Z_j = X_j + q_j ( X_l + q_l X_{\sigma (l)}), \ \ \ \mbox{where} \ l=\sigma (j) \in A_1. $$
Note that each $X_j$ is itself a Bott manifold of dimension $n-1$, and the support of each $Z_j$ is a normal crossing divisor.

In the case of a Bott tower, $X_1$ is a fiber of the composition map $\pi= \pi_2\circ \pi_3 \circ \cdots \circ \pi_n$ from $M^n$ to
${\mathbb P}^1$, and $M$ is covered by the pencil of the divisor $X_1$. Given a Bott diagram, for any $j\in A_0$, we can
choose a Bott tower on $M$ so that $j$ corresponds to the bottom layer, so we know that $M$ is covered by a pencil $|X_j|$
of the smooth hypersurface $X_j$, denote as $Y_t$ for $t\in {\mathbb P}^1$. These $Y_t$  do not intersect with each other.

\vskip 5pt

We claim that, if $D$ is any effective divisor in $M$ homologous to $X_j$, then $D\in |X_j|$, namely, $D$ is a member of the pencil.
To see this, first consider the special case when $D$ is irreducible. If $D$ is not in $|X_j|$, we may choose a member $Y$ in the
pencil so that $D\cap Y \neq \phi$. This codimension $2$ subvariety is homologous to $0$, since $D\sim X_j$ and $x_j^2=0$, which
is a contradiction since $M$ is projective. The same argument works when $D$ is just effective.

\vskip 5pt

Now suppose $f: M' \rightarrow M$ is a biholomorphism. It induces a graded isomorphism $\phi = f^{\ast }$ between the cohomology rings.
We want to show that $f$ induces an isomorphism between the Bott diagrams  as well. First we claim that $f$ induces a bijection
between $A_0'$ and $A_0$, the set of level $0$ vertices. Let $\{ z_1, \ldots , z_n\}$ and $\{ z_1', \ldots , z_n'\}$ be generators on $M$
and $M'$ as before. Then $\phi (z_j)= z_{\tau (j)}'$ for some permutation $\tau$ on $A$ (the sign is positive since $z_j$, $z_j'$ are all
represented by non-trivial effective divisors). If $j\in A_0$, then $M$ is covered by the pencil $X_j=Z_j$ of non-intersecting divisors.
Consider the effective divisor $D=f(Z_{\tau (j)}')$ in $M$. $D$ represents $z_j$, thus is homologous to $Z_j$. By the claim above,
we know that $D$ must be irreducible and is a member of the pencil $|X_j|$. This means that $\tau (j)$ lies in $A_0'$. So the level
$0$ sets of $M$ and $M'$ are bijective to each other.

Note that in the above argument, we furthermore obtained the fact that $f(X_{\tau (j)}') = X_j$ for $j\in A_0$. For $j\in A_0$, the smooth
hypersurface $X_j$ is itself an $(n-1)$-dimensional Bott manifold. Its Bott diagram is obtained from that of $M^n$ by deleting the vertex
corresponding to $j$, and pulling down one level in the tree above this vertex, while keeping everything else unchanged. We will call this
new Bott diagram the {\em card} at vertex $j$. Now since $f(X_{\tau (j)}') = X_j$, the two Bott $(n-1)$-manifolds $X_{\tau (j)}'$ and $X_j$
are biholomorphic, so by induction on the dimension of the Bott manifolds, we see that the card of the Bott diagram $G'$ at vertex $\tau (j)$
must be isomorphic to the card of the Bott diagram $G$ at the vertex $j$.

When $A_0$ has more than one elements, we have at least two cards, and we can use the set of cards to reconstruct the Bott diagram,
see Proposition \ref{reconstruction} and Remark \ref{reconstruction-labelled}. This implies that $G$ and $G'$ must be isomorphic to each other. When $A_0$ has only one element, the Bott diagrams agree as graphs, but we additionally need to show that the marking numbers $q_j$ for
$j\in A_1$ should match those on $M'$ (see Remark \ref{reconstruction-labelled}).

Without loss of generality, let us assume that $A_0=\{ 1\}=A_0'$. We already know that $f(X_1')=X_1$, and the card of the Bott graph $G$ at
vertex $1$ is isomorphic to the card of the Bott graph $G'$ at vertex $1$. So $f$ gives bijection between $A_1$ and $A_1'$. Again without
loss of generality, let us assume that $\phi (z_2) = z_2'$, where $2\in A_1$ and $2\in A_1'$. We have
$$ Z_2=X_2+ q_2X_1, \ \ Z_2' = X_2' + q_2'X_1'. $$
Consider the irreducible divisor $D=f(X_2')$. We have $D+q_2'X_1 \sim X_2 + q_2X_1$. If $D\neq X_2$, then $D\cap X_2$ is an effective
cycle of codimension $2$ (could be trivial), and the intersection
$$ DX_2 + q_2' X_1X_2 \sim Z_2X_2 \sim Z_2(Z_2-q_2X_1) \sim -q_2 Z_2X_1 \sim -q_2 X_2X_1, $$
so the non-trivial effective cycle $D\cap X_2 + (q_2+q_2')X_1\cap X_2$ would be homologous to $0$ -- which is impossible since $M$ is
projective. So $D$ must be equal to $X_2$, forcing $q_2'=q_2$. So the Bott diagrams of $M$ and $M'$ are indeed isomorphic to each other.
This completes the proof of the theorem.

\section{Appendix: Reconstruction of rooted forests}\label{sec-reconstruction}

A famous problem in graph theory is the {\it reconstruction conjecture}. This conjecture asserts that finite graphs with
at least three vertices are completely determined by their collection of vertex deleted graphs, see \cite{Harary}, \cite{BH}, \cite{Manvel}.
In this short appendix, we formulate and establish an analogue for rooted forests.

\begin{definition}
A contractable connected graph is called a {\it tree}. If such a graph is marked with a distinguished vertex (the {\it root}),
we call it a {\it rooted tree}. If every connected component of a graph is a (rooted) tree, then we call the graph a (rooted)
forest.
\end{definition}

Given a rooted tree $T$ with $n$ vertices, we can form an associated rooted forest with $n-1$ vertices by deleting the root $v$ of $T$
(and all incident edges). This leaves a forest with connected components $T_1, \ldots , T_k$, and we can pick a root on each tree $T_i$
to be the unique vertex $v_i$ of $T_i$ that was incident to $v$. We denote this rooted forest by $\hat T$, and call its individual trees
the {\it children} of the original tree $T$.

\begin{definition}
Let $F$ be a rooted forest, with connected components the rooted trees $T_1, \ldots , T_k$. Given a component $T_i$, we define
the associated {\it card} to be the rooted forest with components $T_1, \ldots, T_{i-1}, \hat T_i , T_{i+1}, \ldots ,T_k$, i.e. we replace
the rooted tree $T_i$ by its collection of children. The forest $F$ has $k$ associated cards, each of which is a rooted forest.
\end{definition}

A graph is finite if it has finitely many vertices. We can now establish the reconstruction conjecture for rooted forests.

\begin{proposition}\label{reconstruction}
Let $F$ be a finite rooted forest. Then the set of cards of $F$ uniquely determines the forest $F$.
\end{proposition}

In other words, if one has a pair of forests $F_1, F_2$, and a bijection between the set of cards of $F_1$ and those of $F_2$, which
has the property that corresponding cards are isomorphic (as rooted forests), then the original forests have to be isomorphic.

\begin{proof}
We prove the statement using mathematical induction on the number of cards.
Note that the number of cards coincides with the number of roots (and hence the number of connected components) in the original
rooted forest $F$.

\vskip 5pt

\noindent \underline{Base case}: When there is only one card, we know that there is only one tree $T$ in the original forest $F$. The
individual trees in the single card are the children of $T$. We can thus reconstruct $T$ by taking a root vertex $v$, and for each of the
children of $T$, connecting its root to the vertex $v$. The resulting rooted tree is the single tree in the forest $F$.

\vskip 5pt

\noindent \underline{Inductive step}: Let there be $n\geq 2$ cards in total. We run through the $n$ cards and locate a maximal tree (i.e. with
the maximal number of vertices) among all the trees appearing on all the cards. Of course there could be more than one such tree, but we
pick one of them. Let us call this chosen maximal tree $T$. Our claim is that $T$ must be a rooted tree present in the original forest. If not,
then $T$ appeared on the card after eliminating the root of one of the original trees $T_i$ of the forest. This means that $T$ is a proper
subgraph of $T_i$, and that $T_i$ contains more vertices than $T$ (as the root $v_i$ of $T_i$ is not in $T$). Since $n\geq 2$, there is at least
one other card, arising from the deletion of another root $v_j$. The corresponding card contains $T_i$ as a rooted tree, contradicting
the fact that $T$ was a maximal tree from all the cards. Note that this argument also shows that $T$ is not a child of any of the rooted trees
in the original rooted forest.

Now that we have established $T$ is one of the original trees in the forest we are trying to rebuild, let us try and identify the multiplicity
with which it occurs in the forest. Assume the forest consists of $n$ rooted trees, and that $r$ of them are isomorphic to $T$ (where
$1\leq r \leq k$. Then there are precisely $r$ cards that contain $r-1$ copies of $T$, and $n-r$ cards that contain $r$ copies of $T$.
Thus, we may compute the integer $r$ from the set of cards.

Let $F'$ denote the forest obtained from the original forest $F$ by removing the $r$ copies of $T$. If we can reconstruct $F'$, then by
adding in $r$ copies of $T$, we will have reconstructed $F$. But note that the cards of $F'$ are easy to identify: just take the $n-r$
cards of $F$ that contain exactly $r$ copies of $T$, and remove from each of these cards the $r$ copies of $T$. The resulting $n-r$
rooted forests are the cards of $F'$. Since $r\geq 1$, the rooted forest $F'$ has $n-r<n$ cards, so by the inductive hypothesis,
$F'$ can be reconstructed from its cards. Adding in $r$ disjoint copies of $T$ then produces $F$, and completes the proof of
the Proposition.
\end{proof}

\begin{remark}\label{reconstruction-labelled}
Note that the proof of the proposition also holds for {\it labelled} rooted forests, where the cards are equipped with the natural
induced labelling. In this setting, you need to additionally assume that the number of cards is $n\geq 2$ (i.e. this is the base case
of the induction, and is argued exactly like the inductive step above). When $n=1$, the only indeterminacy lies in the labels
for the edges in the rooted tree which are connected to the root vertex. These are obviously not recoverable from the single
corresponding card. This is the reason for the additional argument at the end of the proof of Theorem \ref{theorem}.
\end{remark}

Differential Geometry (math.DG); Algebraic Geometry (math.AG); Complex Variables (math.CV)


Alternating Randomized Block Coordinate Descent

Abstract: Block-coordinate descent algorithms and alternating minimization methods are fundamental optimization algorithms and an important primitive in large-scale optimization and machine learning. While various block-coordinate-descent-type methods have been studied extensively, only alternating minimization -- which applies to the setting of only two blocks -- is known to have convergence time that scales independently of the least smooth block. A natural question is then: is the setting of two blocks special?We show that the answer is "no" as long as the least smooth block can be optimized exactly -- an assumption that is also needed in the setting of alternating minimization. We do so by introducing a novel algorithm AR-BCD, whose convergence time scales independently of the least smooth (possibly non-smooth) block. The basic algorithm generalizes both alternating minimization and randomized block coordinate (gradient) descent, and we also provide its accelerated version -- AAR-BCD. As a special case of AAR-BCD, we obtain the first nontrivial accelerated alternating minimization algorithm.

\section{Introduction}\label{sec:intro}

First-order methods for minimizing smooth convex functions are a cornerstone of large-scale optimization and machine learning. Given the size and heterogeneity of the data in these applications, there is a particular interest in designing iterative methods that, at each iteration, only optimize over a subset of the decision variables~\cite{wright2015coordinate}.

This paper focuses on two classes of methods that constitute important instantiations of this idea. The first class is that of {\it block-coordinate descent methods}, i.e., methods that partition the set of variables into $n \geq 2$ blocks and perform a gradient descent step on a single block at every iteration, while leaving the remaining variable blocks fixed. A paradigmatic example of this approach is the randomized Kaczmarz algorithm of~\cite{SV} for linear systems and its generalization~\cite{nesterov2012efficiency}.
The second class is that of {\it alternating minimization methods}, i.e., algorithms that partition the variable set into only $n=2$ blocks and alternate between {\it exactly optimizing} one block or the other at each iteration (see, e.g.,~\cite{beck2015convergence-AM} and references therein).

Besides the computational advantage in only having to update a subset of variables at each iteration, methods in these two classes are also able to exploit better the structure of the problem, which, for instance, may be computationally expensive only in a small number of variables.
To formalize this statement, assume that the set of variables is partitioned into $n \leq N$ mutually disjoint blocks, where the $i^{\mathrm{th}}$ block of variable $\mathbf{x}$ is denoted by $\mathbf{x}^{i}$, and the gradient corresponding to the $i^{\mathrm{th}}$ block is denoted by $\nabla_i f(\mathbf{x})$. Each block $i$ will be associated with a smoothness parameter $L_i,$ I.e., $\forall \mathbf{x}, \mathbf{y} \in \mathbb{R}^N$:
\begin{equation}\label{eq:smoothness-grad}
\|\nabla_i f(\mathbf{x} + I_N^i\mathbf{y}) - \nabla_i f(\mathbf{x})\|_* \leq L_i \|\mathbf{y}^i\|,
\end{equation}
where $I_N^i$ is a diagonal matrix whose diagonal entries equal one for coordinates from block $i$, and are zero otherwise.

In this setting, the convergence time of standard randomized block-coordinate descent methods, such as those in~\cite{nesterov2012efficiency}, scales as $O\left(\frac{\sum_i L_i}{\epsilon}\right)$, where $\epsilon$ is the desired additive error. By contrast, when $n=2,$ the convergence time of the alternating minimization method~\cite{beck2015convergence-AM} scales as $O\left(\frac{L_{\min}}{\epsilon}\right),$ where $L_{\min}$ is the minimum smoothness parameter of the two blocks.
This means that one of the two blocks can have arbitrarily poor smoothness (including $\infty$), as long it is easy to optimize over it. Some important examples with a nonsmooth block (with smoothness parameter equal to infinity) can be found in~\cite{beck2015convergence-AM}. Additional examples of problems for which exact optimization over the least smooth block can be performed efficiently are provided in Appendix~\ref{app:efficient-iterations}.

In this paper, we address the following open question, which was implicitly raised by~\cite{beck2013convergence-BCD}: can we design algorithms that combine the features of randomized block-coordinate descent and alternating minimization? In particular, assuming we can perform exact optimization on block $n$, can we construct a block-coordinate descent algorithm whose running time scales with $O(\sum_{i=1}^{n-1} L_i),$ i.e., independently of the smoothness $L_n$ of the $n^{\mathrm{th}}$ block?  This would generalize both existing block-coordinate descent methods, by allowing one block to be optimized exactly, and existing alternating minimization methods, by allowing $n$ to be larger than $2$ and requiring exact optimization only on a single block.

We answer these questions in the affirmative by presenting a novel algorithm: alternating randomized block coordinate descent (\ref{eq:AR-BCD}). The algorithm alternates between an exact optimization over a fixed, possibly non-smooth block, and a gradient descent or exact optimization over a randomly selected block among the remaining blocks.
For two blocks, the method reduces to the standard alternating minimization, while when the non-smooth block is empty (not optimized over), we get randomized block coordinate descent (RCDM) from~\cite{nesterov2012efficiency}.

Our second contribution is \ref{eq:AAR-BCD}, an accelerated version of \ref{eq:AR-BCD}, which achieves the accelerated rate of $\frac{1}{k^2}$ without incurring any dependence on the smoothness of block $n$.  Furthermore,  when the non-smooth block is empty, \ref{eq:AAR-BCD} recovers the fastest known convergence bounds for block-coordinate descent~\cite{qu2016coordinate,allen2016even, nesterov2012efficiency, lin2014accelerated,nesterov-Stich2017efficiency}.
Another conceptual contribution is our extension of the approximate duality gap technique of~\cite{thegaptechnique}, which leads to a general and more streamlined analysis. Finally, to illustrate the results, we perform a preliminary experimental evaluation of our methods against existing block-coordinate algorithms and discuss how their performance depends on the smoothness and size of the blocks.

\paragraph{Related Work}
Alternating minimization and cyclic block coordinate descent are old and fundamental algorithms~\cite{ortega1970iterative} whose convergence (to a stationary point) has been studied even in the non-convex setting, in which they were shown to converge asymptotically under the additional assumptions that the blocks are optimized exactly and their minimizers are unique~\cite{bertsekas1999nonlinear}. However, even in the non-smooth convex case, methods that perform exact minimization over a fixed set of blocks may converge arbitrarily slowly. This has lead scholars to focus on the case of smooth convex minimization, for which nonasymptotic convergence rates were obtained recently in~\cite{beck2013convergence-BCD,beck2015convergence-AM,sun2015improved,saha2013nonasymptotic}. However, prior to our work, convergence bounds that are independent of the largest smoothness parameter were only known for the setting of two blocks.

Randomized coordinate descent methods, in which steps over coordinate blocks are taken in a non-cyclic randomized order (i.e., in each iteration one block is sampled with replacement) were originally analyzed in~\cite{nesterov2012efficiency}. The same paper~\cite{nesterov2012efficiency} also provided an accelerated version of these methods. The results of~\cite{nesterov2012efficiency} were subsequently improved and generalized to various other settings (such as, e.g., composite minimization) in \cite{lee2013efficient,allen2016even,nesterov-Stich2017efficiency,richtarik2014iteration,fercoq2015accelerated,lin2014accelerated}. The analysis of the different block coordinate descent methods under various sampling probabilities (that, unlike in our setting, are non-zero over all the blocks) was unified in \cite{qu2016coordinate} and extended to a more general class of steps within each block in \cite{ GowerR15, qu2016sdna}.

Our results should be carefully compared to a number of proximal block-coordinate methods that rely on different assumptions~\cite{tseng2009coordinate,richtarik2014iteration,lin2014accelerated,fercoq2015accelerated}. In this setting, the function $f$ is assumed to have the structure $f_0(\mathbf{x})+ \Psi(\mathbf{x}),$ where $f_0$ is smooth, the non-smooth convex function $\Psi$ is separable over the blocks, i.e., $\Psi(\mathbf{x}) = \sum_{i=1}^n \Psi_i(\mathbf{x}_i)$, and we can efficiently compute the proximal operator of each $\Psi_i$. This strong assumption allows these methods to make use of the standard proximal optimization framework.
By contrast, in our paper, the convex objective can be taken to have an arbitrary form, where the non-smoothness of a block need not be separable, though the function is assumed to be differentiable.

\section{Preliminaries}\label{sec:prelims}
We assume that we are given oracle access to the gradients of a continuously differentiable convex function $f: \mathbb{R}^N \rightarrow \mathbb{R}$, where computing gradients over only a subset of coordinates is computationally much cheaper than computing the full gradient. We are interested in minimizing $f(\cdot)$ over $\mathbb{R}^N$, and we denote $\mathbf{x}_* = \argmin_{\mathbf{x} \in \mathbb{R}^N}f(\mathbf{x})$. We let $\|\cdot\|$ denote an arbitrary (but fixed) norm, and $\|\cdot\|_*$ denote its dual norm, defined in the standard way: $\|\mathbf{z}\|_* = \sup_{\mathbf{x} \in \mathbb{R}^N: \|\mathbf{x}\|=1}\innp{\mathbf{z}, \mathbf{x}}$.\footnote{Note that the analysis extends in a straightforward way to the case where each block is associated with a different norm (see, e.g.,~\cite{nesterov2012efficiency}); for simplicity of presentation, we take the same norm over all blocks.}

 Let $I_N$ be the identity matrix of size $N$, $I_N^{i}$ be a diagonal matrix whose diagonal elements $j$ are equal to one if variable $j$ is in the $i^{\mathrm{th}}$ block, and zero otherwise. Notice that $I_N = \mathop{\textstyle\sum}_{i=1}^n I_N^i$.  Let $S_{i}(\mathbf{x}) = \{\mathbf{y} \in \mathbb{R}^N : (I_N - I_N^i)\mathbf{y} = (I_N - I_N^i)\mathbf{x}\}$, that is, $S_i$ contains all the points from $\mathbb{R}^N$ whose coordinates differ from those of $\mathbf{x}$ only over block $i$.

We denote the smoothness parameter of block $i$ by $L_i$, as defined in Equation~\eqref{eq:smoothness-grad}. Equivalently,
$\forall \mathbf{x}, \mathbf{y} \in \mathbb{R}^N$:
\begin{equation}\label{eq:smoothness-2nd-order}
f(\mathbf{x} + I_N^i \mathbf{y}) \leq f(\mathbf{x}) + \innp{\nabla_i f(\mathbf{x}), \mathbf{y}^i} + \frac{L_i}{2}\|\mathbf{y}^i\|^2.
\end{equation}

The gradient step over block $i$ is then defined as:
\begin{equation}\label{eq:grad-step-i}
\begin{aligned}
&T_i(\mathbf{x})\\
&\hspace{.2cm}= \argmin_{\mathbf{y} \in {S}_i(\mathbf{x})} \Big\{ \innp{\nabla f(\mathbf{x}), \mathbf{y} - \mathbf{x}} + \frac{L_i}{2}\|\mathbf{y} - \mathbf{x}\|^2 \Big\}.
\end{aligned}
\end{equation}
By standard arguments (see, e.g., Exercise 3.27 in \cite{boyd2004convex}):
\begin{equation}\label{eq:gradient-progress}
f(T_i(\mathbf{x}))-f(\mathbf{x}) \leq -\frac{1}{2L_i}\|\nabla_i f(\mathbf{x})\|_*^2.
\end{equation}

Without loss of generality, we will assume that the $n^{\mathrm{th}}$ block has the largest smoothness parameter and is possibly non-smooth (i.e., it can be $L_n = \infty$). The standing assumption is that exact minimization over the $n^{\mathrm{th}}$ block is ``easy'', meaning that it is computationally inexpensive and possibly solvable in closed form; for some important examples that have this property, see Appendix~\ref{app:efficient-iterations}. Observe that when block $n$ contains a small number of variables, it is often computationally inexpensive to use second-order optimization methods, such as, e.g., interior point method.

We assume that $f(\cdot)$ is strongly convex with parameter $\mu \geq 0$, where it could be $\mu = 0$ (in which case $f(\cdot)$ is not strongly convex). Namely, $\forall \mathbf{x}, \mathbf{y}$:
\begin{equation}\label{eq:strong-convexity}
f(\mathbf{y}) \geq f(\mathbf{x}) + \innp{\nabla f(\mathbf{x}), \mathbf{y} - \mathbf{x}} + \frac{\mu}{2}\|\mathbf{y} - \mathbf{x}\|^2.
\end{equation}
When $\mu > 0$, we take $\|\cdot\| = \|\cdot\|_2$, which is customary for smooth and strongly convex minimization~\cite{Bube2014}.

Throughout the paper, whenever we take unconditional expectation, it is with respect to all randomness in the algorithm

\subsection{Alternating Minimization}
In (standard) alternating minimization (AM), there are only two blocks of coordinates, i.e., $n = 2$. The algorithm is defined as follows.
\noindent\vsepfbox{\begin{minipage}{.47\textwidth}
\begin{equation}\label{eq:alt-min}\tag{AM}
\begin{gathered}
\mathbf{\hat{x}}_{k} = \argmin_{\mathbf{x} \in S_1(\mathbf{x}_{k-1})} f(\mathbf{x}),\\
\mathbf{x}_{k} = \argmin_{\mathbf{x} \in S_2(\mathbf{\hat{x}}_k)} f(\mathbf{x}),\\
\mathbf{x}_1 \in \mathbb{R}^N \text{ is an arbitrary initial point.}
\end{gathered}
\end{equation}
\end{minipage}}
We note that for the standard analysis of alternating minimization~\cite{beck2015convergence-AM}, the exact minimization step over the smoother block can be replaced by a gradient step (Equation~(\ref{eq:grad-step-i})), while still leading to convergence that is only dependent on the smaller smoothness parameter.

\subsection{Randomized Block Coordinate (Gradient) Descent}

The simplest version of randomized block coordinate (gradient) descent (RCDM) can be stated as~\cite{nesterov2012efficiency}:

\noindent\vsepfbox{\begin{minipage}{.47\textwidth}
\begin{equation}\label{eq:standard-BCD}\tag{RCDM}
\begin{gathered}
\text{Select } i_k\in \{1,\dots,n\} \text{ w.p. }p_{i_k} > 0,\\
\mathbf{x}_{k} = T_{i_k}(\mathbf{x}_{k-1}),\\
\mathbf{x}_1\in \mathbb{R}^N \text{ is an arbitrary initial point,}
\end{gathered}
\end{equation}
\end{minipage}}
where $\sum_{i=1}^n p_i = 1$.
A standard choice of the probability distribution is $p_i \sim L_i$, leading to the convergence rate that depends on the sum of block smoothness parameters.

\section{AR-BCD}\label{sec:standard-algo}

The basic version of alternating randomized block coordinate descent (AR-BCD) is a direct generalization of (\ref{eq:alt-min}) and (\ref{eq:standard-BCD}): when $n=2$, it is equivalent to (\ref{eq:alt-min}), while when the size of the $n^{\mathrm{th}}$ block is zero, it reduces to (\ref{eq:standard-BCD}). The method is stated as follows:
\noindent\vsepfbox{
\begin{minipage}{.47\textwidth}
\begin{equation}\label{eq:AR-BCD}\tag{AR-BCD}
\begin{gathered}
\text{Select } i_k\in \{1,\dots,n-1\} \text{ w.p. }p_{i_k} > 0,\\
\mathbf{\hat{x}}_k = T_{i_k}(\mathbf{x}_{k-1}),\\
\mathbf{x}_{k} = \argmin_{\mathbf{x} \in S_n(\mathbf{\hat{x}}_k)}f(\mathbf{x}),\\
\mathbf{x}_1\in \mathbb{R}^N \text{ is an arbitrary initial point,}
\end{gathered}
\end{equation}
\end{minipage}}
where $\sum_{i=1}^{n-1} p_i = 1$. We note that nothing will change in the analysis if the step $\mathbf{\hat{x}}_{k} = T_{i_k}(\mathbf{x}_{k-1})$ is replaced by $\mathbf{\hat{x}}_{k} = \argmin_{\mathbf{x} \in S_{i_k}(\mathbf{x}_{k-1})}f(\mathbf{x})$, since $\min_{\mathbf{x} \in S_{i_k}(\mathbf{x}_{k-1})}f(\mathbf{x}) \leq f(T_{i_k}(\mathbf{x}_{k-1}))$.

In the rest of the section, we show that (\ref{eq:AR-BCD}) leads to a convergence bound that interpolates between the convergence bounds of (\ref{eq:alt-min}) and (\ref{eq:standard-BCD}): it depends on the sum of the smoothness parameters of the first $n-1$ blocks, while the dependence on the remaining problem parameters is the same for all these methods.

\subsection{Approximate Duality Gap}

To analyze (\ref{eq:AR-BCD}), we extend the approximate duality gap technique~\cite{thegaptechnique} to the setting of randomized block coordinate descent methods. The approximate duality gap $G_k$ is defined as the difference of an upper bound $U_k$ and a lower bound $L_k$ to the minimum function value $f(\mathbf{x}_*)$. For (\ref{eq:AR-BCD}), we choose the upper bound to simply be $U_k = f(\mathbf{x}_{k+1})$.

The generic construction of the lower bound is as follows. Let $\mathbf{x}_1, \mathbf{x}_2,..., \mathbf{x}_k$ be any sequence of points from $\mathbb{R}^N$ (in fact we will choose them to be exactly the sequence constructed by (\ref{eq:AR-BCD})). Then, by (strong) convexity of $f(\cdot)$, $f(\mathbf{x}_*)\geq f(\mathbf{x}_j) + \innp{\nabla f(\mathbf{x}_j), \mathbf{x}_* - \mathbf{x}_j} + \frac{\mu}{2}\|\mathbf{x}_* - \mathbf{x}_j\|^2$, $\forall j \in \{1, \dots, k\}$. In particular, if $a_j > 0$ is a sequence of (deterministic, independent of $i_j$) positive real numbers and $A_k = \sum_{j=1}^k a_j$, then:
\begin{align}
f(\mathbf{x}_*) \geq & \frac{\mathop{\textstyle\sum}_{j=1}^k a_j f(\mathbf{x}_j) + \mathop{\textstyle\sum}_{j=1}^k a_j \innp{\nabla f(\mathbf{x}_j), \mathbf{x}_* - \mathbf{x}_j}}{A_k}\notag\\
& + \frac{\frac{\mu}{2}\sum_{j=1}^k a_j\|\mathbf{x}_* - \mathbf{x}_j\|^2}{A_k}\stackrel{\mathrm{\scriptscriptstyle def}}{=}  L_k. \label{eq:lb-non-acc}
\end{align}

\subsection{Convergence Analysis}

The main idea in the analysis is to show that $\mathbb{E}[A_k G_k - A_{k-1}G_{k-1}] \leq E_k$, for some deterministic $E_k$. Then, using linearity of expectation, $\mathbb{E}[f(\mathbf{x}_{k+1})] - f(\mathbf{x}_*)\leq \mathbb{E}[G_k] \leq \frac{\mathbb{E}[A_1 G_1]}{A_k} + \frac{\sum_{j=2}^k E_j}{A_k}$.
The bound in expectation can then be turned into a bound in probability, using well-known concentration bounds. The main observation that allows us not to pay for the non-smooth block is:
\begin{observation}\label{obs:grad-ns-zero}
For $\mathbf{x}_k$'s constructed by (\ref{eq:AR-BCD}), $\nabla_n f(\mathbf{x}_k) = \mathbf{0}$, $\forall k$, where $\mathbf{0}$ is the vector of all zeros.
\end{observation}
This observation is essentially what allows us to sample $i_k$ only from the first $n-1$ blocks, and holds due to the step $\mathbf{x}_{k} = \argmin_{\mathbf{x} \in S_n(\mathbf{\hat{x}}_k)}f(\mathbf{x})$ from (\ref{eq:AR-BCD}).

Denote $R_{\mathbf{x}_*^i} = \max_{\mathbf{x}\in \mathbb{R}^N}\{\|I_N^i(\mathbf{x}_* - \mathbf{x})\|^2: f(\mathbf{x}) \leq f(\mathbf{x}_1)\}$, and let us bound the initial gap $A_1G_1$.
\begin{proposition}\label{prop:ARBCD-initial-gap}
$\mathbb{E}[A_1G_1] \leq E_1$, where $E_1 = a_1\mathop{\textstyle\sum}_{i=1}^{n-1} \left(\frac{L_i}{2p_i}-\frac{\mu}{2}\right)R_{\mathbf{x}_*^i}$.
\end{proposition}
\begin{proof}
By linearity of expectation, $\mathbb{E}[A_1G_1] = \mathbb{E}[A_1U_1] - \mathbb{E}[A_1L_1].$ The initial lower bound is deterministic, and, by  $\nabla_n f(\mathbf{x}_1) = \mathbf{0}$ and duality of norms, is bounded as:
\begin{equation*}
\begin{aligned}
\mathbb{E}[A_1L_1]\geq & a_1f(\mathbf{x}_1) - a_1 \mathop{\textstyle\sum}_{i=0}^{n-1} \|\nabla_i f(\mathbf{x}_1)\|_*\|\mathbf{x}_*^i - \mathbf{x}_1^i\|\\
& + a_1\frac{\mu}{2}\|\mathbf{x}_* - \mathbf{x}_1\|^2.
\end{aligned}
\end{equation*}
Using (\ref{eq:gradient-progress}), if $i_2 = i$, then:
\begin{equation*}
U_1 = f(\mathbf{x}_2) \leq f(\mathbf{\hat{x}}_2) \leq f(\mathbf{x}_1) - \frac{1}{2L_i}\|\nabla_i f(\mathbf{x}_1)\|_*^2.
\end{equation*}
Since block $i$ is selected with probability $p_i$ and $A_1 = a_1$:
\begin{align*}
\mathbb{E}[A_1 U_1] \leq & a_1 f(\mathbf{x}_1) - \mathop{\textstyle\sum}_{i=1}^{n-1} \frac{a_1p_i}{2L_i}\|\nabla_i f(\mathbf{x}_i)\|_*^2.
\end{align*}
Since the inequality $2ab - a^2 \leq b^2$ holds $\forall a, b$, we have:
\begin{equation*}
\begin{aligned}
& a_1 \|\nabla_i f(\mathbf{x}_1)\|_*\|\mathbf{x}_*^i - \mathbf{x}_1^i\| - \frac{a_1p_i}{2L_i}\|\nabla_i f(\mathbf{x}_i)\|_*^2\\
&\hspace{1cm}\leq \frac{a_1L_i}{2p_i}\|\mathbf{x}_*^i - \mathbf{x}_1^i\|^2, \; \forall i \in \{1,\dots, n-1\}
\end{aligned}
\end{equation*}
Hence, when $\mu = 0$, $\mathbb{E}[A_1 G_1] \leq \sum_{i=1}^{n-1}\frac{a_1L_i}{2p_i}\|\mathbf{x}_*^i - \mathbf{x}_1^i\|^2$. When $\mu > 0,$ since in that case we are assuming $\|\cdot\| = \|\cdot\|_2$ (Section~\ref{sec:prelims}), $\|\mathbf{x}_* - \mathbf{x}_1\|^2 \geq \sum_{i=1}^{n-1}\|\mathbf{x}_*^i - \mathbf{x}_1^i\|^2$, leading to $\mathbb{E}[A_1 G_1] \leq a_1\sum_{i=1}^{n-1}\left(\frac{L_i}{2p_i}- \frac{\mu}{2}\right)\|\mathbf{x}_*^i - \mathbf{x}_1^i\|^2$.
\end{proof}

We now show how to bound the error in the decrease of the scaled gap $A_kG_k$.

\begin{lemma}\label{lemma:ARBCD-gap-decrease}
$\mathbb{E}[A_kG_k - A_{k-1}G_{k-1}] \leq E_k$, where $E_k = a_k\sum_{i=1}^{n-1}\left(\frac{{a_k}L_i}{2A_kp_i}-\frac{\mu}{2}\right)R_{\mathbf{x}_*^i}$.
\end{lemma}
\begin{proof}
Let $\mathcal{F}_k$ denote the natural filtration up to iteration $k$. By linearity of expectation and $A_{k}L_k - A_{k-1}L_{k-1}$ being measurable w.r.t. $\mathcal{F}_k$,
\begin{align*}
&\mathbb{E}[A_kG_k - A_{k-1}G_{k-1}|\mathcal{F}_{k}]\\
&\hspace{.2cm}= \mathbb{E}[A_kU_k - A_{k-1}U_{k-1}|\mathcal{F}_{k}] - (A_k L_k - A_{k-1}L_{k-1}).
\end{align*}
With probability $p_i$ and as $f(\mathbf{x}_{k+1}) \leq f(\mathbf{\hat{x}}_{k+1})$, the change in the upper bound is:
\begin{align}
A_k U_k - A_{k-1}U_{k-1} \leq & A_k f(\mathbf{\hat{x}}_{k+1}) - A_{k-1}f(\mathbf{x}_k)\notag\\
\leq &  a_k f(\mathbf{x}_k) - \frac{A_k}{2L_i}\|\nabla_i f(\mathbf{x}_k)\|_*^2\notag,
\end{align}
where the second line follows from $\mathbf{\hat{x}}_{k+1} = T_{i_k}(\mathbf{x}_k)$ and Equation (\ref{eq:gradient-progress}). Hence:
\begin{align*}
&\mathbb{E}[A_k U_k - A_{k-1}U_{k-1}|\mathcal{F}_{k}]\\
&\hspace{1cm}\leq a_k f(\mathbf{x}_k) - A_k\mathop{\textstyle\sum}_{i=1}^{n-1}\frac{p_i}{2L_i}\|\nabla_i f(\mathbf{x}_k)\|_*^2.
\end{align*}
On the other hand, using the duality of norms, the change in the lower bound is:
\begin{align*}
&A_kL_k - A_{k-1}L_{k-1}\\
&\hspace{1cm}\geq a_k f(\mathbf{x}_k) - a_k \mathop{\textstyle\sum}_{i=1}^{n-1} \|\nabla_i f(\mathbf{x}_k)\|_*\|\mathbf{x}_*^i - \mathbf{x}_k^i\|\\
&\hspace{1.4cm}+ a_k\frac{\mu}{2}\|\mathbf{x}_* - \mathbf{x}_k\|^2\\
&\hspace{1cm}\geq a_k f(\mathbf{x}_k) - a_k \mathop{\textstyle\sum}_{i=1}^{n-1} \|\nabla_i f(\mathbf{x}_k)\|_* \sqrt{R_{\mathbf{x}_*^i}}\\
&\hspace{1.4cm}+ a_k\frac{\mu}{2}\|\mathbf{x}_* - \mathbf{x}_k\|^2.
\end{align*}
By the same argument as in the proof of Proposition~\ref{prop:ARBCD-initial-gap}, it follows that: $\mathbb{E}[A_k G_k - A_{k-1}G_{k-1}|\mathcal{F}_k] \leq a_k\sum_{i=1}^{n-1}\left(\frac{L_i{a_k}}{2A_kp_i}-\frac{\mu}{2}\right)R_{\mathbf{x}_*^i} = E_k$. Taking expectations on both sides, as $E_k$ is deterministic, the proof follows.
\end{proof}

We are now ready to prove the convergence bound for (\ref{eq:AR-BCD}), as follows.

\begin{theorem}\label{thm:ARBCD-convergence}
Let $\mathbf{x}_k$ evolve according to (\ref{eq:AR-BCD}). Then, $\forall k \geq 1$:
\begin{enumerate}
\item If $\mu = 0:$
$
\mathbb{E}[f(\mathbf{x}_{k+1})] - f(\mathbf{x}_*) \leq \frac{2\mathop{\textstyle\sum}_{i=1}^{n-1}\frac{L_i}{p_i}R_{\mathbf{x}_*^i}}{k+3}.
$
In particular, for $p_i = \frac{L_i}{\sum_{i' = 1}^{n-1} L_{i'}},$ $1\leq i \leq n-1$:
$$
\mathbb{E}[f(\mathbf{x}_{k+1})] - f(\mathbf{x}_*) \leq \frac{2(\sum_{i'=1}^{n-1}L_{i'}) \mathop{\textstyle\sum}_{i=1}^{n-1}R_{\mathbf{x}_*^i}}{k+3}.
$$
Similarly, for $p_i = \frac{1}{n-1}$, $1\leq i\leq n-1:$
$$
\mathbb{E}[f(\mathbf{x}_{k+1})] - f(\mathbf{x}_*) \leq \frac{2(n-1)\mathop{\textstyle\sum}_{i=1}^{n-1}{L_i}R_{\mathbf{x}_*^i}}{k+3}
$$
\item If $\mu > 0$, $p_i = \frac{L_i}{\sum_{i' = 1}^{n-1}L_{i'}}$ and $\|\cdot\| = \|\cdot\|_2:$
\begin{align*}
&\mathbb{E}[f(\mathbf{x}_{k+1})] - f(\mathbf{x}_*)\\
&\hspace{1cm}\leq \Big(1 - \frac{\mu}{\sum_{i'=1}^{n-1}L_{i'}}\Big)^k\\
& \hspace{1.5cm}\cdot \frac{(\sum_{i'=1}^{n-1}L_{i'})\|(I_N - I_N^n)(\mathbf{x}_* - \mathbf{x}_1)\|^2}{2}.
\end{align*}
\end{enumerate}
\end{theorem}
\begin{proof}
From Proposition~\ref{prop:ARBCD-initial-gap} and Lemma~\ref{lemma:ARBCD-gap-decrease}, by linearity of expectation and the definition of $G_k$:
\begin{equation}
\mathbb{E}[f(\mathbf{x}_{k+1})] - f(\mathbf{x}_*) \leq \mathbb{E}[G_k] \leq \frac{\sum_{j=1}^k E_j}{A_k},
\end{equation}
where $E_j = \frac{{a_j}^2}{A_j}\sum_{i=1}^{n-1}\frac{L_i}{2p_i}R_{\mathbf{x}_*^i}$.

Notice that the algorithm does not depend on the sequence $\{a_j\}$ and thus we can choose it arbitrarily. Suppose that $\mu = 0$. Let $a_j = \frac{j+1}{2}$. Then $\frac{{a_j}^2}{A_j} = \frac{(j+1)^2}{j(j+3)}\leq 1$, and thus:

\frac{\sum_{j=1}^k E_j}{A_k} \leq \frac{2\mathop{\textstyle\sum}_{i=1}^{n-1}\frac{L_i}{p_i}R_{\mathbf{x}_*^i}}{k+3},

which proves the first part of the theorem, up to concrete choices of $p_i$'s, which follow by simple computations.

For the second part of the theorem, as $\mu > 0$, we are assuming that $\|\cdot\| = \|\cdot\|_2$, as discussed in Section~\ref{sec:prelims}. From Lemma~\ref{lemma:ARBCD-gap-decrease}, $E_j = a_j \sum_{i=1}^{n-1} \left(\frac{a_j L_i}{2A_jp_i} - \frac{\mu}{2}\right)R_{\mathbf{x}_*^i}$, $\forall j \geq 2$. As $p_i = \frac{L_i}{\sum_{i'=1}^{n-1}L_{i'}}$, if we take $\frac{a_j}{A_j} = \frac{\mu}{\sum_{i'=1}^{n-1}L_{i'}}$, it follows that $E_j = 0$, $\forall j \geq 2$. Let $a_1 = A_1 = 1$ and $\frac{a_j}{A_j} = \frac{\mu}{\sum_{i'=1}^{n-1}L_{i'}}$ for $j \geq 2$. Then:

\mathbb{E}[f(\mathbf{x}_{k+1})] - f(\mathbf{x}_*) \leq \mathbb{E}[G_k]
\leq  \frac{\mathbb{E}[A_1 G_1]}{A_k}.
$
As $\frac{A_1}{A_k} = \frac{A_1}{A_2}\cdot \frac{A_2}{A_3}\cdot \dots \cdot \frac{A_{k-1}}{A_k}$ and $\frac{A_{j-1}}{A_j} = 1 - \frac{a_j}{A_j}$:

\mathbb{E}[f(\mathbf{x}_{k+1})] - f(\mathbf{x}_*)
\leq  \Big(1 - \frac{\mu}{\sum_{i'=1}^{n-1}L_{i'}}\Big)^{k-1}\mathbb{E}[G_1].
$
It remains to observe that, from Proposition~\ref{prop:ARBCD-initial-gap}, $\mathbb{E}[G_1] \leq \big(1 - \frac{\mu}{\sum_{i'=1}^{n-1}L_{i'}}\big)\frac{(\sum_{i'=1}^{n-1}L_{i'})\|(I_N - I_N^n)(\mathbf{x}_* - \mathbf{x}_1)\|^2}{2}$.
\end{proof}

We note that when $n=2$, the asymptotic convergence of~\ref{eq:AR-BCD}  coincides with the convergence of alternating minimization~\cite{beck2015convergence-AM}. When $n^{\mathrm{th}}$ block is empty (i.e., when all blocks are sampled with non-zero probability and there is no exact minimization over a least-smooth block), we obtain the convergence bound of the standard randomized coordinate descent method~\cite{nesterov2012efficiency}.
\section{Accelerated AR-BCD}\label{sec:acc-algo}

In this section, we show how to accelerate (\ref{eq:AR-BCD})  when $f(\cdot)$ is smooth. We believe it is possible to obtain similar results in the smooth and strongly convex case, which we defer to a future version of the paper.
Denote:
\begin{gather}
\Delta_k = I_N^{i_k} \nabla f(\mathbf{x}_k)/p_{i_k},\label{eq:Delta_k-def}\notag\\
\mathbf{v}_k = \argmin_{\mathbf{u}}\Big\{\mathop{\textstyle\sum}_{j=1}^k a_j \innp{\Delta_j, \mathbf{u}
\notag\\
\hspace{3cm}
+ \mathop{\textstyle\sum}_{i=1}^n \frac{\sigma_i}{2}\|\mathbf{u}^i - \mathbf{x}_1^i\|^2\Big\},\label{eq:arg-min-lb}
\end{gather}
where $\sigma_i > 0$, $\forall i$, will be specified later.
Accelerated AR-BCD (AAR-BCD) is defined as follows:
\vsepfbox{
\begin{minipage}{.47\textwidth}
\begin{equation}\label{eq:AAR-BCD}\tag{AAR-BCD}
\begin{gathered}
\text{Select }i_k \text{ from } \{1,\dots, n-1\} \text{ w.p. } p_{i_k},\\
\mathbf{\hat{x}}_k = \frac{A_{k-1}}{A_k}\mathbf{y}_{k-1} + \frac{a_k}{A_k}\mathbf{v}_{k-1},\\
\mathbf{x}_k = \argmin_{\mathbf{x} \in S_n(\mathbf{\hat{x}}_k)}f(\mathbf{x}),\\
\mathbf{y}_k = \mathbf{x}_k + \frac{a_k}{p_{i_k}A_k}I_N^{i_k}(\mathbf{v}_{k}-\mathbf{v}_{k-1}),\\%T_{i_k}(\mathbf{x}_k),\\%
\mathbf{x}_1 \text{ is an arbitrary initial point,}
\end{gathered}
\end{equation}
\end{minipage}}
where $\sum_{i=1}^{n-1}p_i = 1$, $p_i > 0$, $\forall i \in \{1,\dots, n-1\}$, and $\mathbf{v}_k$ is defined by (\ref{eq:arg-min-lb}). To seed the algorithm, we further assume that
$\mathbf{y}_1 = \mathbf{x}_1 + I_N^{i_1}\frac{1}{{p_{i_1}}}(\mathbf{v}_1 - \mathbf{x}_1)$.

\begin{remark}\label{remark:iteration-complexity}
Iteration complexity of (\ref{eq:AAR-BCD}) is dominated by the computation of $\mathbf{\hat{x}}_k,$ which requires updating an entire vector. This type of an update is not unusual for accelerated block coordinate descent methods, and in fact appears in all such methods we are aware of~\cite{nesterov2012efficiency,lee2013efficient,lin2014accelerated,fercoq2015accelerated,allen2016even}. In most cases of practical interest, however, it is possible to implement this step efficiently (using that $\mathbf{v}_k$ changes only over block $i_k$ in iteration $k$). More details are provided in Appendix~\ref{app:efficient-iterations}
\end{remark}

To analyze the convergence of \ref{eq:AAR-BCD}, we will need to construct a more sophisticated duality gap than in the previous section, as follows.

\subsection{Approximate Duality Gap}

\begin{figure*}[ht!
\begin{equation}\label{eq:rand-lb}
\Lambda_k = \frac{\sum_{j=1}^k a_j f(\mathbf{x}_j) + \min_{\mathbf{u} \in \mathbb{R}^N}\left\{\sum_{j=1}^k a_j\innp{\Delta_j, \mathbf{u} - \mathbf{x}_j}
+ \mathop{\textstyle\sum}_{i=1}^{n-1} \frac{\sigma_i}{2}\|\mathbf{u}^i - \mathbf{x}_1^i\|^2\right\}- \mathop{\textstyle\sum}_{i=1}^{n-1} \frac{\sigma_i}{2}\|\mathbf{x}_*^i - \mathbf{x}_1^i\|^2}{A_k}.
\end{equation}
\vspace{-15pt}
\end{figure*}

We define the upper bound to be $U_k = f(\mathbf{y}_k)$.
The constructed lower bound $L_k$ from previous subsection is not directly useful for the analysis of (\ref{eq:AAR-BCD}). Instead, we will construct a random variable $\Lambda_k$, which in expectation is upper bounded by $f(\mathbf{x}^*)$. The general idea, as in previous subsection, is to show that some notion of approximate duality gap decreases in expectation.

Towards constructing $\Lambda_k,$ we first prove the following technical proposition, whose proof is in Appendix~\ref{app:omitted-proofs}.
\begin{proposition}\label{prop:expect-Delta_k}
Let $\mathbf{x}_k$ be as in (\ref{eq:AAR-BCD}). Then: $$
\mathbb{E}[\mathop{\textstyle\sum}_{j=1}^k a_j \innp{\Delta_j, \mathbf{x}_* - \mathbf{x}_j}] = \mathbb{E}[\mathop{\textstyle\sum}_{j=1}^k a_j \innp{\nabla f(\mathbf{x}_j), \mathbf{x}_* - \mathbf{x}_j}].
$$
\end{proposition}
Define the randomized lower bound as in Eq.~(\ref{eq:rand-lb}), and observe that (\ref{eq:arg-min-lb}) defines $\mathbf{v}_k$ as the argument of the minimum from $\Lambda_k$.
The crucial property of $\Lambda_k$ is that it lower bounds $f(\mathbf{x}_*)$ in expectation, as shown in the following lemma.

\begin{lemma}\label{lemma:Lambda_k-is-lb}
Let $\mathbf{x}_k$ be as in (\ref{eq:AAR-BCD}). Then
$
f(\mathbf{x}_*) \geq \mathbb{E}[\Lambda_k].
$
\end{lemma}
\begin{proof
By convexity of $f(\cdot),$ for any sequence  $\{\tilde{\mathbf{x}}_j\}$ from $\mathbb{R}^N$, $f(\mathbf{x}_*)\geq \frac{\sum_{j=1}^k a_j (f(\tilde{\mathbf{x}}_j) + \innp{\nabla f(\tilde{\mathbf{x}}_j), \mathbf{x}_* - \tilde{\mathbf{x}}_j})}{A_k}$. Since the statement holds for any sequence $\{\tilde{\mathbf{x}}_j\},$ it also holds if $\{\tilde{\mathbf{x}}_j\}$ is selected according to some probability distribution. In particular, for $\{\tilde{\mathbf{x}}_j\} = \{\mathbf{x}_j\}$:
\begin{align*}
f(\mathbf{x}_*)\geq &\mathbb{E}\Big[\frac{\sum_{j=1}^k a_j (f(\mathbf{x}_j) + \innp{\nabla f(\mathbf{x}_j), \mathbf{x}_* - \mathbf{x}_j})}{A_k}\Big].
\end{align*}
By linearity of expectation and Proposition~\ref{prop:expect-Delta_k}:
\begin{equation}\label{eq:lb-in-exp}
\begin{aligned}
f(\mathbf{x}_*)\geq \mathbb{E}\Big[\frac{\sum_{j=1}^k a_j (f(\mathbf{x}_j) + \innp{\Delta_j, \mathbf{x}_* - \mathbf{x}_j})}{A_k} \Big].
\end{aligned}
\end{equation}
Adding and subtracting (deterministic) $\mathop{\textstyle\sum}_{i=1}^{n-1} \frac{\sigma_i}{2}\|\mathbf{x}_*^i - \mathbf{x}_1^i\|^2$ to/from (\ref{eq:lb-in-exp}) and using that:
\begin{equation*}
\begin{aligned}
&\mathop{\textstyle\sum}_{j=1}^k a_j \innp{\Delta_j, \mathbf{x}_* - \mathbf{x}_j} + \mathop{\textstyle\sum}_{i=1}^{n-1} \frac{\sigma_i}{2}\|\mathbf{x}_*^i - \mathbf{x}_1^i\|^2\\
&\hspace{.5cm}\geq \min_{\mathbf{u}}\Big\{ \mathop{\textstyle\sum}_{j=1}^k a_j \innp{\Delta_j, \mathbf{u} - \mathbf{x}_j}+\mathop{\textstyle\sum}_{i=1}^{n-1} \frac{\sigma_i}{2}\|\mathbf{u}^i - \mathbf{x}_1^i\|^
\Big\}\\
&\hspace{.5cm}= \min_{\mathbf{u}} m_k(\mathbf{u}),
\end{aligned}
\end{equation*}
where $m_k(\mathbf{u}) = \mathop{\textstyle\sum}_{j=1}^k a_j \innp{\Delta_j, \mathbf{u} - \mathbf{x}_j}+ \mathop{\textstyle\sum}_{i=1}^{n-1} \frac{\sigma_i}{2}\|\mathbf{u}^i - \mathbf{x}_1^i\|^
$, it follows that:
\begin{align*}
&f(\mathbf{x}_*) \geq \mathbb{E}\Big[\frac{\sum_{j=1}^k a_j f(\mathbf{x}_j) - \mathop{\textstyle\sum}_{i=1}^{n-1} \frac{\sigma_i}{2}\|\mathbf{x}_*^i - \mathbf{x}_1^i\|^2}{A_k}\\
&\hspace{1.7cm} + \frac{\min_{\mathbf{u} \in \mathbb{R}^N} m_k(\mathbf{u})}{A_k}\Big],
\end{align*}
which is equal to $\mathbb{E}[\Lambda_k],$ and completes the proof.
\end{proof}
Similar as before, define the approximate gap as $\Gamma_k = U_k - \Lambda_k$.
Then, we can bound the initial gap as follows.
\begin{proposition}\label{prop:AAR-BCD-init-gap}
If $a_1 = \frac{{a_1}^2}{A_1} \leq \frac{\sigma_i{p_i}^2}{L_i}$, $\forall i\in\{1,..., n-1\}$, then $\mathbb{E}[A_1\Gamma_1] \leq \mathop{\textstyle\sum}_{i=1}^{n-1} \frac{\sigma_i}{2}\|\mathbf{x}_* - \mathbf{x}_1\|^2$.
\end{proposition}
\begin{proof}
As $a_1 = A_1$ and $\mathbf{y}_1$ differs from $\mathbf{x}_1$ only over block $i = i_1$, by smoothness of $f(\cdot)$:
\begin{align*}
&A_1 U_1 = A_1f(\mathbf{y}_1)\\
&\leq a_1 f(\mathbf{x}_1) + a_1\innp{\nabla_i f(\mathbf{x}_1), \mathbf{y}_1^i - \mathbf{x}_1^i} + \frac{a_1 L_i}{2}\|\mathbf{y}_1^i - \mathbf{x}_1^i\|^2.
\end{align*}
On the other hand, the initial lower bound is:
\begin{align*}
A_1\Lambda_1 =& a_1 (f(\mathbf{x}_1) + \innp{\Delta_1, \mathbf{v}_1 - \mathbf{x}_1}) \\
&+ \mathop{\textstyle\sum}_{i=1}^{n-1} \frac{\sigma_i}{2}\|\mathbf{v}_1^i - \mathbf{x}_1^i\|^2
- \mathop{\textstyle\sum}_{i=1}^{n-1} \frac{\sigma_i}{2}\|\mathbf{x}_*^i - \mathbf{x}_1^i\|^2.
\end{align*}
Recall that $\mathbf{y}_1^i = \mathbf{x}_1^i + \frac{1}{p_i}(\mathbf{v}_1^i - \mathbf{x}_1^i)$. Using  $A_1\Gamma_1  = A_1U_1 - A_1\Lambda_1$ and the bounds on $U_1,\, \Lambda_1$ from the above:
$
A_1\Gamma_1 \leq  \mathop{\textstyle\sum}_{i=1}^{n-1} \frac{\sigma_i}{2}\|\mathbf{x}_*^i - \mathbf{x}_1^i\|^2,
$
as $a_1 \leq {p_i}^2\frac{\sigma_i}{L_i}$, and, thus, $\mathbb{E}[A_1\Gamma_1] \leq  \mathop{\textstyle\sum}_{i=1}^{n-1} \frac{\sigma_i}{2}\|\mathbf{x}_*^i - \mathbf{x}_1^i\|^2$.
\end{proof}

The next part of the proof is to show that $A_k\Gamma_k$ is a supermartingale. The proof is provided in Appendix~\ref{app:omitted-proofs}.

\begin{lemma}\label{lemma:AkGammak-supermartingale}
If $\frac{{a_k}^2}{A_k}\leq \frac{{p_i}^2\sigma_i}{L_i}$, $\forall i \in\{1, \dots,n-1\}$, then $\mathbb{E}[A_k\Gamma_k|\mathcal{F}_{k-1}] \leq A_{k-1}\Gamma_{k-1}$.
\end{lemma}

Finally, we bound the convergence of~(\ref{eq:AAR-BCD}).
\begin{theorem}\label{thm:AAR-BCD-convergence}
Let $\mathbf{x}_k,\, \mathbf{y}_k$ evolve according to (\ref{eq:AAR-BCD}), for $\frac{{a_k}^2}{A_k} = \min_{1\leq i\leq n-1}\frac{\sigma_i{p_i}^2}{L_i} = \mathrm{const}$. Then, $\forall k \geq 1$:
$$
\mathbb{E}[f(\mathbf{y}_k)] - f(\mathbf{x}_*) \leq \frac{\sum_{i=1}^{n-1} \sigma_i \|\mathbf{x}_*^i - \mathbf{x}_a^i\|^2}{2A_k}.
$$
In particular, if $p_i = \frac{\sqrt{L_i}}{\sum_{i'=1}^{n-1}\sqrt{L_{i'}}}$, $\sigma_i = (\sum_{i'=1}^{n-1}\sqrt{L_{i'}})^2$, and $a_1 = 1$, then:
$$
\mathbb{E}[f(\mathbf{y}_k)] - f(\mathbf{x}_*) \leq \frac{2(\sum_{i'=1}^{n-1}\sqrt{L_{i'}})^2 \sum_{i=1}^{n-1}\|\mathbf{x}_*^i - \mathbf{x}_1^i\|^2}{k(k+3)}.
$$
Alternatively, if $p_i = \frac{1}{n-1}$, $\sigma_i = L_i$, and $a_1 = \frac{1}{(n-1)^2}$:
$$
\mathbb{E}[f(\mathbf{y}_k)] - f(\mathbf{x}_*) \leq \frac{2(n-1)^2\sum_{i=1}^{n-1}L_i \|\mathbf{x}_*^i - \mathbf{x}_1^i\|^2}{k(k+3)}.
$$
\end{theorem}
\begin{proof}
The first part of the proof follows immediately by applying Proposition~\ref{prop:AAR-BCD-init-gap} and Lemma~\ref{lemma:AkGammak-supermartingale}. The second part follows by plugging in the particular choice of parameters and observing that $a_j$ grows faster than $\frac{j+1}{2}$ in the former, and faster than $\frac{j+1}{2(n-1)^2}$ in the latter case
\end{proof}

Finally, we make a few remarks regarding Theorem~\ref{thm:AAR-BCD-convergence}. In the setting without a non-smooth block (when $n^{\mathrm{th}}$ block is empty), (\ref{eq:AAR-BCD}) with sampling probabilities $p_i \sim \sqrt{L_i}$ has the same convergence bound as the NU\_ACDM algorithm~\cite{allen2016even} and the ALPHA algorithm for smooth minimization~\cite{qu2016coordinate}. Further, when the sampling probabilities are uniform, (\ref{eq:AAR-BCD}) converges at the same rate as the ACDM algorithm \cite{nesterov2012efficiency} and the APCG algorithm applied to non-composite functions \cite{lin2014accelerated}.

\section{Numerical Experiments}\label{sec:experiments}

\begin{figure*}[ht!]
\centering
\hspace{.3cm}
\subfigure[$N/n = 5$]{\includegraphics[width=0.22\textwidth]{smoothness_bf_5.png}\label{fig:smooth-5-blog}}\hspace{.3cm}
\subfigure[$N/n = 10$]{\includegraphics[width=0.22\textwidth]{smoothness_bf_10.png}\label{fig:smooth-10-blog}}\hspace{.3cm}
\subfigure[$N/n = 20$]{\includegraphics[width=0.22\textwidth]{smoothness_bf_20.png}\label{fig:smooth-20-blog}}\hspace{.3cm}
\subfigure[$N/n = 40$]{\includegraphics[width=0.22\textwidth]{smoothness_bf_40.png}\label{fig:smooth-40-blog}}\hspace{-.1cm}
 \subfigure[$N/n = 5$]{\includegraphics[width=0.24\textwidth]{BCD_grad_med_5_lip.png}\label{fig:lip-5-blog}}
 \subfigure[$N/n=10$]{\includegraphics[width=0.24\textwidth]{BCD_grad_med_10_lip.png}\label{fig:lip-10-blog}}
 \subfigure[$N/n=20$]{\includegraphics[width=0.24\textwidth]{BCD_grad_med_20_lip.png}\label{fig:lip-med-20}}
 \subfigure[$N/n=40$]{\includegraphics[width=0.24\textwidth]{BCD_grad_med_40_lip.png}\label{fig:lip-med-40}}
\subfigure[$N/n = 5$]{\includegraphics[width=0.24\textwidth]{Acc_BCD_grad_med_5_lip-min.png}\label{fig:acc-lip-5-blog}}
 \subfigure[$N/n=10$]{\includegraphics[width=0.24\textwidth]{Acc_BCD_grad_med_10_lip-min.png}\label{fig:acc-lip-10-blog}}
 \subfigure[$N/n=20$]{\includegraphics[width=0.24\textwidth]{Acc_BCD_grad_med_20_lip-min.png}\label{fig:acc-lip-med-20}}
 \subfigure[$N/n=40$]{\includegraphics[width=0.24\textwidth]{Acc_BCD_grad_med_40_lip-min.png}\label{fig:acc-lip-med-40}}
\caption{Comparison of different block coordinate descent methods: \protect\subref{fig:smooth-5-blog}-\protect\subref{fig:smooth-40-blog} distribution of smoothness parameters over blocks, \protect\subref{fig:lip-5-blog}-\protect\subref{fig:lip-med-40} comparison of non-accelerated methods, and \protect\subref{fig:acc-lip-5-blog}-\protect\subref{fig:acc-lip-med-40} comparison of accelerated methods. Block sizes $N/n$ increase going left to right.}
\label{fig:AR-BCD}
\end{figure*}

To illustrate the results, we solve the least squares problem on the BlogFeedback Data Set~\cite{buza2014feedback} obtained from UCI Machine Learning Repository~\cite{Lichman:2013}. The data set contains 280 attributes and 52,396 data points. The attributes correspond to various metrics of crawled blog posts. The data is labeled, and the labels correspond to the number of comments that were posted within 24 hours from a fixed basetime. The goal of a regression method is to predict the number of comments that a blog post receives.

What makes linear regression with least squares on this dataset particularly suitable to our setting is that the smoothness parameters of individual coordinates in the least squares problem take values from a large interval, even when the data matrix $\mathbf{A}$ is scaled by its maximum absolute value (the values are between 0 and $\sim$354).\footnote{We did not compare \ref{eq:AR-BCD} and \ref{eq:AAR-BCD} to other methods on problems with a non-smooth block ($L_n = \infty$), as no other methods have any known theoretical guarantees in such a setting.} The minimum eigenvalue of $\mathbf{A}^T\mathbf{A}$ is zero (i.e., $\mathbf{A}^T\mathbf{A}$ is not a full-rank matrix), and thus the problem is not strongly convex.

We partition the data into blocks as follows. We first sort the coordinates by their individual smoothness parameters. Then, we group the first $N/n$ coordinates (from the sorted list of coordinates) into the first block, the second $N/n$ coordinates into the second block, and so on. The chosen block sizes $N/n$ are 5, 10, 20, 40, corresponding to $n=\{56, 28, 14, 7\}$ coordinate blocks, respectively.

The distribution of the smoothness parameters over blocks, for all chosen block sizes, is shown in Fig.~\ref{fig:smooth-5-blog}-\ref{fig:smooth-40-blog}. Observe that as the block size increases (going from left to right in Fig.~\ref{fig:smooth-5-blog}-\ref{fig:smooth-40-blog}), the discrepancy between the two largest smoothness parameters increases.

In all the comparisons between the different methods, we define an epoch to be equal to $n$ iterations (this would correspond to a single iteration of a full-gradient method). The graphs plot the optimality gap of the methods over epochs, where the optimal objective value $f^*$ is estimated via a higher precision method and denoted by $\hat{f}^*$. All the results are shown for 50 method repetitions, with bold lines  representing the median\footnote{We choose to show the median as opposed to the mean, as it is well-known that in the presence of outliers the median is a robust estimator of the true mean~\cite{hampel2011robust}.} optimality gap over those 50 runs. The norm used in all the experiments is $\ell_2$, i.e., $\|\cdot\| = \|\cdot\|_2$.

\paragraph{Non-accelerated methods} We first compare \ref{eq:AR-BCD} with a gradient step to \ref{eq:standard-BCD}~\cite{nesterov2012efficiency} and standard cyclic BCD -- C-BCD (see, e.g.,~\cite{beck2013convergence-BCD}). To make the comparison fair, as \ref{eq:AR-BCD} makes two steps per iteration, we slow it down by a factor of two compared to the other methods (i.e., we count one iteration of \ref{eq:AR-BCD} as two). In the comparison, we consider two cases for RCDM and C-BCD: (i) the case in which these two algorithms perform gradient steps on the first $n-1$ blocks and exact minimization on the $n^{\mathrm{th}}$ block (denoted by RCDM and C-BCD in the figure), and (ii) the case in which the algorithms perform gradient steps on all blocks (denoted by RCDM-G and C-BCD-G in the figure). The sampling probabilities for RCDM and AR-BCD are proportional to the block smoothness parameters. The permutation for C-BCD is random, but fixed in each method run.

Fig.~\ref{fig:lip-5-blog}-\ref{fig:lip-med-40} shows the comparison of the described non-accelerated algorithms, for block sizes $N/n \in\{5, 10, 20, 40\}$. The first observation to make is that adding exact minimization over the least smooth block speeds up the convergence of both C-BCD and RCDM, suggesting that the existing analysis of these two methods is not tight. Second, \ref{eq:AR-BCD} generally converges to a lower optimality gap. While RCDM makes a large initial progress, it stagnates afterwards due to the highly non-uniform sampling probabilities, whereas \ref{eq:AR-BCD} keeps making progress.

\paragraph{Accelerated methods}
Finally, we compare \ref{eq:AAR-BCD} to NU\_ACDM~\cite{allen2016even}, APCG~\cite{lin2014accelerated}, and accelerated C-BCD (ABCGD) from~\cite{beck2013convergence-BCD}. As \ref{eq:AAR-BCD} makes three steps per iteration (as opposed to two steps normally taken by other methods), we slow it down by a factor 1.5 (i.e., we count one iteration of \ref{eq:AAR-BCD} as 1.5). We chose the sampling probabilities of NU\_ACDM and \ref{eq:AAR-BCD} to be proportional to $\sqrt{L_i}$, while the sampling probabilities for APCG are uniform\footnote{The theoretical results for APCG were only presented for uniform sampling~\cite{lin2014accelerated}.}. Similar as before, each full run of ABCGD is performed on a random but fixed permutation of the blocks.

The results are shown in Fig.~\ref{fig:acc-lip-5-blog}-\ref{fig:acc-lip-med-40}. Compared to APCG (and ABCGD), NU\_ACDM and \ref{eq:AAR-BCD} converge much faster, which is expected, as the distribution of the smoothness parameters is highly non-uniform and the meethods with non-uniform sampling are theoretically faster by factor of the order $\sqrt{n}$~\cite{allen2016even}. As the block size is increased (going left to right), the discrepancy between the smoothness parameters of the least smooth block and the remaining blocks increases, and, as expected, \ref{eq:AAR-BCD} exhibits more dramatic improvements compared to the other methods.

\section{Conclusion}\label{sec:conclusion}

We presented a novel block coordinate descent algorithm \ref{eq:AR-BCD} and its accelerated version for smooth minimization \ref{eq:AAR-BCD}. Our work answers the open question of~\cite{beck2013convergence-BCD} whether the convergence of block coordinate descent methods intrinsically depends on the largest smoothness parameter over all the blocks by showing that such a dependence is not necessary, as long as exact minimization over the least smooth block is possible. Before our work, such a result only existed for the setting of two blocks, using the alternating minimization method.

There are several research directions that merit further investigation. For example, we observed empirically that exact optimization over the non-smooth block improves the performance of RCDM and C-BCD, which is not justified by the existing analytical bounds. We expect that in both of these methods the dependence on the least smooth block can be removed, possibly at the cost of a worse dependence on the number of blocks. Further, \ref{eq:AR-BCD} and \ref{eq:AAR-BCD} are mainly useful when the discrepancy between the largest block smoothness parameter and the remaining smoothness parameters is large, while under uniform distribution of the smoothness parameters it can be slower than other methods by a factor 1.5-2. It is an  interesting question whether there are modifications to \ref{eq:AR-BCD} and \ref{eq:AAR-BCD} that would make them uniformly better than the alternatives.

\section{Acknowledgements}
Part of this work was done while the authors were visiting the Simons Institute for the Theory of Computing. It was partially supported by NSF grant \#CCF-1718342 and by the DIMACS/Simons Collaboration on Bridging Continuous and Discrete Optimization through NSF grant \#CCF-1740425.
\clearpage


Root separation for reducible monic polynomials of odd degree

Abstract: We study root separation of reducible monic integer polynomials of odd degree. Let h(P) be the naive height, sep(P) the minimal distance between two distinct roots of an integer polynomial P(x) and sep(P)=h(P)^{-e(P)}. Let e_r*(d)=limsup_{deg(P)=d, h(P)-> +infty} e(P), where the limsup is taken over the reducible monic integer polynomials P(x) of degree d. We prove that e_r*(d) <= d-2. We also obtain a lower bound for e_r*(d) for d odd, which improves previously known lower bounds for e_r*(d) when d = 5, 7, 9.

\section{Introduction}
All the polynomials that we deal with in this paper have integer coefficients. For any such polynomial,
we can look at how close two of its real or complex roots can be. Since we can always find polynomials
with distinct roots as close as desired, we need to introduce some measure of size for polynomials with
which we can compare this minimal separation of roots. This is done by bounding the degree and most
usually using the na\"\i ve height, that is, the maximum of the absolute values of the coefficients of a polynomial.

The problem of minimal root separation for polynomials with fixed degree has been completely solved
only in the trivial case of quadratic polynomials. Best possible separation exponent is also known
for nonmonic cubic polynomials (see \cite{Ev,Sch}). For monic cubic polynomials,
complete resolution would be equivalent to proving or disproving
the well known Hall's conjecture \cite{BuMi}. Therefore, resolving the problem completely for
polynomials of larger degree seems entirely out of reach.

However, if we restrict ourselves to reducible monic polynomials, then the cubic case becomes easy
and the quartic case has been solved by the authors \cite{DP11}. Thus, we are interested in
the separation properties of the reducible monic polynomials of degree at least $5$.

Let $P(x)$ be a polynomial of degree $d\geq 2$, na\"\i ve height $\ho(P)$ and with at least two distinct roots.
The \emph{polynomial root separation} of $P(x)$  is
\[ \sep(P):=\min_{\substack{P(\alpha)=P(\beta)=0,\\ \alpha\neq\beta}}|\alpha-\beta|. \]
The quantity $e(P)$ is defined by
\[ \sep(P)=\ho(P)^{-e(P)}. \]
Following the notation introduced in \cite{BuMi}, for $d\geq 2$, we set
\[ e(d):=\limsup_{\deg(P)=d,\, \ho(P)\to +\infty}e(P) \]
and
\[ \er(d):=\limsup_{\deg(P)=d,\, \ho(P)\to +\infty}e(P),  \]
where the latter limsup is taken over the reducible monic integer polynomials $P(x)$ of degree $d$.

There are some other variants of polynomial root separation problem like $p$-adic root separation,
where $|\alpha - \beta|$ is replaced by $|\alpha - \beta|_p$ (see \cite{Phazu}),
and absolute root separation, where $|\alpha - \beta|$ is replaced by $||\alpha| - |\beta||$ (see \cite{BDPS}),
but they will not be treated in this paper.

Obviously, we have $e(d)\geq \er(d)$. A classical result of Mahler \cite{Ma} says that $e(d)\leq d-1$ for every $d\geq 2$.
It is easy to see that $\er(2)=0 $ and $\er(3)=1$, while the main result of \cite{DP11} shows that
$\er(4)=2$. The best current lower bounds for the values we are interested in are
$\er(5)\geq 2$ from \cite{BuMi} and the following general result by Bugeaud and Dujella \cite{BD14}
\[
\er(d)\geq \frac{2d}{3}-1 \textrm{ for even } d\geq 6, \quad
\er(d)\geq \frac{2d}{3}-\frac{5}{3} \textrm{ for odd } d\geq 7.
\]
In particular, this implies that $\er(7)\geq 3$ and $\er(9)\geq \frac{13}{3}$.
The mentioned result from \cite{BD14} is obtained by constructing the parametric family
of polynomials $Q_{d,n}(x)=(x^2-(n^2+3n+1)x+(n+2))q_{d-2,n}(x)$, where $q_{d-2,n}(x)$ is a recursive sequence
of polynomials of even degree. For $d$ odd, the polynomials
$Q_{d,n}(x)=x(x^2-(n^2+3n+1)x+(n+2))q_{d-3,n}(x)$ were used, so it is not surprising that results
for odd degrees are sightly weaker and, in particular, for $d=5$ the lower bound $\er(5)\geq \frac{5}{3}$
from \cite{BD14} is weaker than the bound $\er(5)\geq 2$ from \cite{BuMi}.

In this paper, we mainly consider odd degree polynomials.
In Section \ref{sec:2} we improve the upper bound $\er(d)\leq d-1$ which follows from \cite{Ma}
by proving that $\er(d)\leq d-2$ for $d\geq 2$.
In Section \ref{sec:3}, we will construct a parametric family of reducible monic polynomials
of odd degree with good root separation properties. Although the obtained lower bound will be
asymptotically weaker compared with the bound from \cite{BD14}, it will improve all previously
known lower bounds on $\er(d)$ for $d\in\{5,7,9\}$. We show that
\[
  \er(5) \geq \frac{7}{3}, \quad
  \er(7) \geq \frac{17}{5}, \quad
  \er(9) \geq \frac{31}{7}.
\]

\section{Upper bounds}  \label{sec:2}

Our first goal is to we improve the upper bound $\er(d)\leq d-1$ which follows from \cite{Ma}
by proving that $\er(d)\leq d-2$ for $d\geq 2$. Note that we already know that $\er(d)= d-2$ for $d=2,3,4$,
so we may assume $d\geq 5$ here.

Let $P(x)$ be a monic polynomial of degree $d$, reducible over the rationals.
Gauss's Lemma shows that we can factor $P(x)$ into two nonconstant monic polynomials with integer coefficients.
If $P(x)=Q(x)R(x)$, where $Q(x)$ and $R(x)$ are integer polynomials of positive degrees $n$ and $m$, respectively, and $\alpha,\beta\in\C$ are such that $Q(\alpha)=R(\beta)=0$, then a version of Liouville's inequality (see \cite[Theorem A.1]{Bu}) ensures that
\begin{equation} \label{eq:liouville}
  |\alpha - \beta| \gg \ho(Q)^{-m} \ho(R)^{-n},
\end{equation}
where the constant implied in the Vinogradov symbol $\gg$ here and further on depends only on $m$ and $n$.

Gelfond's Lemma \cite[Lemma A.3]{Bu} says that
\begin{equation} \label{eq:gelfond}
  2^{-m-n}\ho(Q)\ho(R) \leq \ho(P) \leq 2^{m+n} \ho(Q)\ho(R).
\end{equation}
Now \eqref{eq:liouville} and \eqref{eq:gelfond} and the fact that $\ho(Q)\geq 1$, $\ho(R)\geq 1$ give
\[ |\alpha - \beta| \gg \ho(P)^{-\max\{m,n\}}. \]
Mahler's general result shows that
\begin{equation} \label{eq:Q}
  \sep(Q) \gg \ho(Q)^{-n+1} \gg \ho(P)^{-n+1}
\end{equation}
and
\begin{equation} \label{eq:R}
  \sep(R) \gg \ho(R)^{-m+1} \gg \ho(P)^{-m+1}.
\end{equation}
Combining the last three inequalities, we obtain
\begin{equation} \label{eq:P}
  \sep(P) \gg \ho(P)^{-\max\{m,n\}}.
\end{equation}

If $\min\{m,n\}\geq 2$, we have $\max\{m,n\}\leq d-2$ and \eqref{eq:P} shows that $\sep(P)\gg \ho(P)^{-d+2}$.

We see that the only case that needs to be considered more thoroughly in order to prove the upper bound
$\er(d)\leq d-2$ is when one of the polynomials $Q(x)$, $R(x)$ is linear and we have a root of $Q(x)$
and a root of $R(x)$ which are very close. Without loss of generality, assume that $Q(x)=x-c$,
$R(c)\neq 0$ and $R(c+\ep)=0$, where $c$ is an integer and $\ep$ is small, but nonzero, say $0<|\ep|<1$.

An easy bound $|c+\ep|\leq m\ho(R)$ is obtained by substituting $x=c+\ep$ into $R(x)$ and
comparing the leading term with the rest.

If $|c+\ep|\geq 3$, then $|c+\ep|\geq |c|-|\ep|\geq |c|-1\geq |c|/2$ since
$|c|\geq |c+\ep|-|\ep|\geq |c+\ep|-1\geq 2$. The first inequality in \eqref{eq:gelfond} together with $\ho(Q)=\max\{1,|c|\}$ and $|c|/2\leq |c+\ep| \leq m\ho(R)$ gives
\begin{equation} \label{eq:sqrtHP}
  |c|\ll \ho(P)^{1/2}.
\end{equation}
In case $|c+\ep|<3$, we have $|c|\leq |c+\ep|+|\ep|<4$ and since $\ho(P)\geq 1$, inequality \eqref{eq:sqrtHP} also holds.

Rolle's mean value theorem gives
\begin{equation} \label{eq:eps}
  1\leq |R(c)| = |R(c+\ep)-R(c)| = |R'(t)|\cdot |\ep|,
\end{equation}
where $t$ is between $c$ and $c+\ep$ and thus in the interval $(c-1,c+1)$. Since
\[ \ho(R')\leq  m\ho(R) \ll \ho(P) \quad\textrm{and}\quad
  |t|<|c|+1\ll \ho(P)^{1/2}, \]
we have
\[ |R'(t)|\ll \ho(P)^{\frac{m-1}{2}} + \ho(P)\cdot\ho(P)^{\frac{m-2}{2}} \ll \ho(P)^{\frac{m}{2}}, \]
where we used the fact that $R(x)$ is monic and $R'(x)$ is of degree $m-1$. Comparing with \eqref{eq:eps}, we obtain
\begin{equation} \label{eq:linear}
  |\ep|\gg \ho(P)^{-m/2} \gg \ho(P)^{-(d-1)/2} \gg \ho(P)^{-d+3}
\end{equation}
since $d\geq 5$ (we get $|\ep|\gg \ho(P)^{-(d-1)/2} \gg \ho(P)^{-d+2}$ for $d\geq 3$).

Hence, we proved the following theorem.

\begin{theorem} \label{th:2}
For $d\geq 2$, it holds that $\er(d)\leq d-2$.
\end{theorem}

\section{Lower bounds} \label{sec:3}
We would like to improve the lower bound $\er(5)\geq 2$ from \cite{BuMi}.
By \eqref{eq:linear}, we see that this bound cannot be improved by
considering degree $5$ polynomials which have a linear factor.
Thus, we have to consider products of two monic polynomials of degrees $2$ and $3$.

We made some experiments in order to find suitable degree $5$ polynomials with $e(P)>2$.
We consider monic cubic polynomials with coefficients of moderate size. For such a polynomial, we
choose one of its real roots $\gamma$
and then apply the LLL-algorithm to a matrix of the form
$$ \left(
  \begin{array}{ccc}
    1 & 0 & 0 \\
    0 & 1 & 0 \\
    N & \lfloor N \gamma \rceil & \lfloor N \gamma^2 \rceil \\
  \end{array}
\right),
$$
with suitably chosen large positive integer $N$,
to construct quadratic polynomials with a root close to $\gamma$
(the method is explained e.g. in \cite[Chapter 6]{Smart}).
Among the polynomials obtained in the experiments,
we have noted two collections of polynomials with $e(P)$ approaching $\frac{7}{3}$:
$$ s_n(x)=(x^3-2nx^2+(2-2n)x+2)(x^2+(-2n^2-2n)x+2n+2) $$
and
$$ p_n(x)=(x^3+nx-1)(x^2+n^2x-n). $$
Indeed, $s_n(x)$ has two close roots with asymptotic expansions
$$ \frac{1}{n}+\frac{1}{2n^4} -\frac{1}{2n^5}+ \frac{1}{2n^6} + O(\frac{1}{n^8}) \quad
\mbox{and} \quad  \frac{1}{n}+\frac{1}{2n^4} -\frac{1}{2n^5}+ \frac{1}{2n^6} + \frac{1}{4n^7} + O(\frac{1}{n^8}), $$  while $p_n(x)$ has two close roots with asymptotic expansions
$$ \frac{1}{n}-\frac{1}{n^4}+\frac{3}{n^7} + O(\frac{1}{n^{10}}) \quad
\mbox{and} \quad \frac{1}{n}-\frac{1}{n^4}+\frac{2}{n^7} + O(\frac{1}{n^{10}}). $$
Hence, we proved that $\er(5) \geq \frac{7}{3}$.
Moreover, it is easy to see that the polynomials $p_n(x)$
can be generalized to arbitrary odd degree.

Let $d\geq 5$ be an odd number. To obtain a lower bound on $\er(d)$,
we construct a family $(P_{d,n})_{n\geq 1}$
of reducible monic polynomials of degree $d$ (such that $P_{5,n}=p_n$),
depending on the parameter $n$,
with root separation asymptotically
\begin{equation} \label{eq:sepP}
  \sep(P_{d,n})\ll \ho(P_{d,n})^{-\frac{d^2-2d-1}{2d-4}},\quad n\to +\infty.
\end{equation}
This will give
\begin{equation} \label{eq:sepPb}
\er(d)\geq\frac{d^2-2d-1}{2d-4} = \frac{d}{2} - \frac{1}{2d-4}.
\end{equation}
The bound \eqref{eq:sepPb} is comparable with best known lower bounds
for separation of irreducible monic
and nonmonic polynomials ($e_{irr}^{*}(d) \geq \frac{d}{2} - \frac{1}{4}$,
$e_{irr}(d) \geq \frac{d}{2} + \frac{d-2}{4(d-1)}$ see \cite{BD14,BD11}).
Although it is asymptotically weaker than the bound $\er(d)\geq (2d-5)/3$ from \cite{BD14},
it is better than $\er(d)\geq (2d-5)/3$ for $d=5$, $d=7$ and $d=9$,
and it is also better than $\er(5)\geq 2$ from \cite{BuMi}.

For $k,n\in\Z_{\geq 2}$, let
\[ \begin{aligned}
   Q(x) &= Q_{k,n}(x) = x^2 + n^k x - n, \\
   R(x) &= R_{k,n}(x) = \frac{x^{2k+1}-Q(x)}{x^2-n} = x\cdot \frac{x^{2k}-n^k}{x^2-n} - 1 \\
     &= x(x^{2(k-1)} + nx^{2(k-2)} + \cdots + n^{k-1}) - 1.
\end{aligned} \]
We omit $k$ and $n$ from the index of polynomials $Q_{k,n}(x)$ and $R_{k,n}(x)$ for easier writing.
It is clear that $Q(x)$ and $R(x)$ are monic integer polynomials of degrees $2$ and $2k-1$ and heights $n^k$ and $n^{k-1}$, respectively. Thus $Q(x)R(x)$ is a reducible monic polynomial of degree $2k+1$ and height $n^{2k-1}-1$ which is attained only in the quadratic term.

Quadratic polynomial $Q(x)$ has roots $(-n^k\pm \sqrt{n^{2k} + 4n})/2$, so that one root is close to $-n^k$ and the other root, which we denote by
\[ \alpha = \frac{2n}{n^k+\sqrt{n^{2k}+4n}}, \]
is close to $n^{1-k}$, more precisely, $0 < \frac{1}{2}n^{1-k} < \alpha < n^{1-k}.$

We also have
\[ R(\alpha) = \frac{\alpha^{2k+1}}{\alpha^2 - n} <0, \quad
  R(2\alpha) = \frac{(2\alpha)^{2k+1} - 2\alpha^2 - n}{(2\alpha)^2 - n} >0 \]
since the terms other than $-n$ in the numerator and the denominator are very small.
Therefore, the polynomial $R(x)$ has a root $\beta$ in the interval $(\alpha,2\alpha)$.

Using the mean value theorem, we conclude that there is $t\in(\alpha,\beta)$ such that
\begin{equation} \label{eq:rolle}
   0 < R'(t) (\beta-\alpha) = R(\beta) - R(\alpha) = \frac{\alpha^{2k+1}}{n - \alpha^2}
     < 2n^{(2k+1)(1-k)-1}.
\end{equation}
It is easily seen that
$R'(t)>n^{k-1}$ and employing this inequality in \eqref{eq:rolle} gives
\[ 0 < \beta - \alpha < \frac{2n^{(2k+1)(1-k)-1}}{n^{k-1}} = 2n^{1-2k^2}. \]
This implies
\[ \sep(Q\cdot R) < 2\ho(Q\cdot R)^{\frac{1-2k^2}{2k-1}}.  \]

For an odd integer $d\geq 5$, we take $k=(d-1)/2$ and define $P_{d,n}(x):=Q_{k,n}(x)R_{k,n}(x)$ obtaining a family
that satisfies \eqref{eq:sepP}.

\begin{corollary} \label{kor:3}
It holds that
$$  \frac{7}{3} \leq \er(5) \leq 3, \quad
  \frac{17}{5} \leq \er(7) \leq 5, \quad
  \frac{31}{7} \leq \er(9) \leq 7. $$
\end{corollary}

\bigskip