Title: On the k-Means/Median Cost Function,
Abstract: In this work, we study the $k$-means cost function. The (Euclidean) $k$-means
problem can be described as follows: given a dataset $X \subseteq \mathbb{R}^d$
and a positive integer $k$, find a set of $k$ centers $C \subseteq
\mathbb{R}^d$ such that $\Phi(C, X) \stackrel{def}{=} \sum_{x \in X} \min_{c
\in C} ||x - c||^2$ is minimized. Let $\Delta_k(X) \stackrel{def}{=} \min_{C
\subseteq \mathbb{R}^d} \Phi(C, X)$ denote the cost of the optimal $k$-means
solution. It is simple to observe that for any dataset $X$, $\Delta_k(X)$
decreases as $k$ increases. We try to understand this behaviour more precisely.
For any dataset $X \subseteq \mathbb{R}^d$, integer $k \geq 1$, and a small
precision parameter $\varepsilon > 0$, let $\mathcal{L}_{X}^{k, \varepsilon}$
denote the smallest integer such that $\Delta_{\mathcal{L}_{X}^{k,
\varepsilon}}(X) \leq \varepsilon \cdot \Delta_{k}(X)$. We show upper and lower
bounds on this quantity. Our techniques generalize for the metric $k$-median
problem in arbitrary metrics and we give bounds in terms of the doubling
dimension of the metric. Finally, we observe that for any dataset $X$, we can
compute a set $S$ of size $O \left(\mathcal{L}_{X}^{k, \frac{\varepsilon}{c}}
\right)$ such that $\Delta_{S}(X) \leq \varepsilon \cdot \Delta_k(X)$ using the
$D^2$-sampling algorithm which is also known as the $k$-means++ seeding
procedure. In the previous statement, $c$ is some fixed constant. We also
discuss some applications of our bounds.