 0^{+}$, when $S_n$ is sampled from the manifold ${\mathcal{M}}$.

Diffusion $K$-means {#sec:diffusion_Kmeans}
===================

Recall that in our clustering model, $S_n=\{X_{1},X_2,\ldots,X_n\}$ is a sample of independent random variables taking values in $S$, where $S$ is the union of $K$ disjoint Riemannian submanifolds ${\mathcal{D}}_{1},\dots,{\mathcal{D}}_{K}$ embedded in the ambient space ${\mathbb{R}}^p$. The clustering problem is to divide these $n$ data points into $K$ clusters, so that points in the same cluster belongs to the same connected component in $S$, based on certain similarity measures between the points. In particular, the (classical) $K$-means clustering method minimizes the total intra-cluster squared Euclidean distances in ${\mathbb{R}}^p$ $$\min_{G_{1},\dots,G_{K}} \sum_{k=1}^{K} {1 \over |G_{k}|} \sum_{i,j \in G_{k}} \|X_{i}-X_{j}\|^{2}$$ over all possible partitions on $[n]$, where $|G_{k}|$ is the cardinality of $G_{k}$. Dropping the sum of squared norms $\sum_{i=1}^{n} \|X_{i}\|^{2}$, we see that the $K$-means clustering is equivalent to the maximization of the total within-cluster covariances $$\max_{G_{1},\dots,G_{K}} \sum_{k=1}^{K} {1 \over |G_{k}|} \sum_{i,j \in G_{k}} a_{ij}, \quad\mbox{with }a_{ij} = X_{i}^{T} X_{j}.$$ Here, $a_{ij}=X_{i}^{T} X_{j}$ can be viewed as a similarity measure specified by the Euclidean space inner product $\langle X_{i}, X_{j}\rangle_{{\mathbb{R}}^p}$. In general, we can replace the Euclidean inner product with any other inner product over $S_n$ [@ChenYang2018]. For manifold clustering, we replace it with the inner product induced from the empirical diffusion distance, that is, $$\begin{aligned}
\langle x,\, y\rangle_{D_{n,t}} = \
0^{+}$, when $ S_n$ is sampled from the manifold $ { \mathcal{M}}$. 

 Diffusion $ K$-means { # sec: diffusion_Kmeans } 
 = = = = = = = = = = = = = = = = = = = 

 Recall that in our clustering model, $ S_n=\{X_{1},X_2,\ldots, X_n\}$ is a sample of autonomous random variable star taking values in $ S$, where $ S$ is the coupling of $ K$ disjoint Riemannian submanifolds $ { \mathcal{D}}_{1},\dots,{\mathcal{D}}_{K}$ embed in the ambient space $ { \mathbb{R}}^p$. The clustering problem is to separate these $ n$ datum points into $ K$ clusters, so that points in the same cluster belong to the same connected part in $ S$, based on certain similarity measures between the points. In particular, the (classical) $ K$-means clustering method acting minimizes the total intra - cluster squared euclidian distances in $ { \mathbb{R}}^p$ $ $ \min_{G_{1},\dots, G_{K } } \sum_{k=1}^{K } { 1 \over |G_{k}| } \sum_{i, j \in G_{k } } \|X_{i}-X_{j}\|^{2}$$ over all possible partitions on $ [ n]$, where $ |G_{k}|$ is the cardinality of $ G_{k}$. Dropping the sum of squared norm $ \sum_{i=1}^{n } \|X_{i}\|^{2}$, we see that the $ K$-means clustering is equivalent to the maximization of the total within - cluster covariances $ $ \max_{G_{1},\dots, G_{K } } \sum_{k=1}^{K } { 1 \over |G_{k}| } \sum_{i, j \in G_{k } } a_{ij }, \quad\mbox{with } a_{ij } = X_{i}^{T } X_{j}.$$ Here, $ a_{ij}=X_{i}^{T } X_{j}$ can be watch as a similarity measure specified by the Euclidean space inner product $ \langle X_{i }, X_{j}\rangle_{{\mathbb{R}}^p}$. In general, we can replace the Euclidean inner product with any other inner product over $ S_n$ [ @ChenYang2018 ]. For manifold clustering, we replace it with the inner merchandise induced from the empirical dispersion distance, that is, $ $ \begin{aligned } 
 \langle x,\, y\rangle_{D_{n, t } } = \
 0^{+}$, wjen $S_n$ is sampled from tme manifold ${\matheql{M}}$.

Difhusion $I$-means {#sdc:diffusion_Kmeans}
===================

Recall that ib our clustering model, $S_n=\{X_{1},X_2,\udots,X_n\}$ id a sampoe oh independent raisom varlcbles bakiny talues in $S$, whete $S$ is the gnion of $K$ disboknc Riemannian submanifolds ${\mathcal{D}}_{1},\does,{\mathcsl{F}}_{K}$ embedded in the wmbisnt space ${\mathbb{R}}^p$. The clustering lroblem is to divide these $n$ data points into $N$ clksters, so that poijts in the wame xluster belovgs to the same connecjed component in $S$, based on certxin snmilarity mgcwurfv between tie poigts. In partigllar, tha (classocal) $K$-means clmstermng nethod minimizes the votal intra-cluster szuared Euwlndean distances in ${\marhvb{R}}^p$ $$\kin_{G_{1},\gots,E_{J}} \sjm_{k=1}^{I} {1 \ober |G_{k}|} \suj_{i,j \in G_{k}} \|S_{i}-X_{j}\|^{2}$$ over aol possible partitipnf on $[n]$, where $|G_{i}|$ is tre cardinality of $G_{k}$. Dropping the sum of vquzred norms $\sum_{i=1}^{n} \|X_{i}\|^{2}$, we wee that the $K$-means cpustering is equivalent to the maximization of the total whthin-rljsttr covafuajces $$\max_{G_{1},\dots,G_{K}} \sum_{k=1}^{K} {1 \over |G_{k}|} \sum_{i,j \in G_{k}} a_{it}, \qiac\mbox{with }a_{ij} = X_{i}^{T} X_{j}.$$ Herr, $w_{ik}=V_{i}^{T} X_{j}$ can be vieweb aa a similarity meadure spgcifiee by the Tuclicean space inner product $\labgle X_{i}, X_{j}\rauglw_{{\mathbb{R}}^p}$. In genercl, we can re'lace jhe Euvlidean inner product wnth ang other innfr producf over $S_n$ [@ChenYane2018]. Fpr manifold clustering, we re[lace it xith che innef prpduct ynduced frlm thc empirical diffusiln didtdnce, that ls, $$\begin{aligned}
\langle x,\, y\rangle_{V_{i,t}} = \
0^{+}$, when $S_n$ is sampled from the Diffusion {#sec:diffusion_Kmeans} =================== that in our sample independent random variables values in $S$, $S$ is the union of $K$ Riemannian submanifolds ${\mathcal{D}}_{1},\dots,{\mathcal{D}}_{K}$ embedded in the ambient space ${\mathbb{R}}^p$. The clustering problem is divide these $n$ data points into $K$ clusters, so that points in the cluster to same component in $S$, based on certain similarity measures between the points. In particular, the (classical) $K$-means method minimizes the total intra-cluster squared Euclidean distances ${\mathbb{R}}^p$ $$\min_{G_{1},\dots,G_{K}} \sum_{k=1}^{K} {1 |G_{k}|} \sum_{i,j \in G_{k}} \|X_{i}-X_{j}\|^{2}$$ all partitions on where is cardinality of $G_{k}$. the sum of squared norms $\sum_{i=1}^{n} \|X_{i}\|^{2}$, we see that the $K$-means clustering is equivalent to the of the covariances $$\max_{G_{1},\dots,G_{K}} {1 |G_{k}|} \in G_{k}} a_{ij}, = X_{i}^{T} X_{j}.$$ Here, $a_{ij}=X_{i}^{T} X_{j}$ as a similarity measure specified by the Euclidean inner product X_{i}, X_{j}\rangle_{{\mathbb{R}}^p}$. In general, we can the Euclidean inner product with any other inner over $S_n$ [@ChenYang2018]. For manifold clustering, we replace it with the inner product induced from diffusion distance, that is, \langle x,\, y\rangle_{D_{n,t}} \
 0^{+}$, when $S_n$ is sampled from the manIfold ${\mathcAl{M}}$.

DiFfuSioN $K$-MeanS {#sec:Diffusion_KmeanS}
===================

recaLl that in our clustering mOdel, $S_N=\{X_{1},x_2,\LdotS,x_n\}$ Is a saMple of iNDePENdeNt RaNdoM vARiAbles TakIng valuEs in $S$, where $s$ is ThE union of $K$ disJOiNt RiemanniAn sUbmanifolds ${\mAthCal{D}}_{1},\doTs,{\MatHCal{D}}_{K}$ EmbEdded In the aMBient sPace ${\mathbB{R}}^P$. the cluSTering pROBlEm is To divide these $n$ datA PoINts into $K$ clusteRs, so thAt POiNTS in The Same clusteR bElongS To the saME cONNEctED component in $S$, Based on certAIn sImilarItY meASures bEtweeN tHE poInts. In partiCulaR, the (classIcal) $K$-mEAns clusTEring meThod miNimIzeS the TOtAl IntRa-CLusTEr SquARed euclideaN dIsTanceS in ${\mATHBB{R}}^p$ $$\mIn_{G_{1},\Dots,g_{K}} \sum_{K=1}^{K} {1 \over |G_{k}|} \sum_{i,j \In G_{K}} \|X_{i}-X_{J}\|^{2}$$ OveR all pOssibLe paRtItionS on $[n]$, whEre $|G_{k}|$ Is The cardinality oF $G_{k}$. DRopping thE suM oF sqUaRed noRMs $\sum_{i=1}^{N} \|X_{i}\|^{2}$, We sEe that tHe $K$-meanS CluStERINg Is equivalent to the mAxIMIzAtion of tHe totaL WiThIN-cluster CoVarIancES $$\Max_{G_{1},\dOts,G_{k}} \SuM_{k=1}^{K} {1 \over |G_{K}|} \sum_{i,j \IN G_{K}} a_{Ij}, \quad\mBoX{with }a_{Ij} = x_{i}^{T} x_{j}.$$ HEre, $a_{iJ}=x_{i}^{T} X_{J}$ can be Viewed as A simiLArity measure spECified by the EuCLiDEAn SPace InnEr product $\laNgle x_{I}, X_{j}\rAnglE_{{\MaThbB{r}}^p}$. In gEneraL, wE CaN Replace the Euclidean InNer proDuct wIth any other inNer product OVER $S_n$ [@ChenYAng2018]. FOR mANifold clusteriNg, we rEplace it wiTH the inneR prodUct inducEd from the EMPirical dIffUsiOn dIstANCe, That is, $$\begin{alIGNed}
\lAnGle x,\, y\raNglE_{D_{n,t}} = \
 0^{+}$, when $S_n$ is sam pled fromthe m ani fol d${\m athc al{M}}$.

Diff u sion  $K$-means {#sec:diffu sion_ Km e ans} == ===== ======= = == = = 

R ec al l t ha t  i n our  cl usterin g model, $ S_n =\ {X_{1},X_2,\ l do ts,X_n\}$isa sample ofind epende nt  ra n dom v ari ables  takin g  value s in $S$,  w h ere $S $  is the u ni on o f $K$ disjoint Ri e ma n nian submanifo lds ${ \m a th c a l{D }}_ {1},\dots, {\ mathc a l{D}}_{ K }$ e m bed d ed in the amb ient space$ {\m athbb{ R} }^p $ . Theclust er i ngproblem isto d ivide the se $n$ data po i nts int o $K$clu ste rs,s oth atpo i nts in  th e  sa me clust er  b elong s to t h e  sam e c onne ctedcomponent in$S$ , ba s edon ce rtain  sim il arity  measu res b et ween the points . In  particul ar,  t he(c lassi c al) $K $-m ean s clust ering m e tho dm i n im izes the total int ra - c lu ster squ ared E u cl id e an dista nc esin $ { \ mathb b{R} } ^p $ $$\min _{G_{1 } ,\ do ts,G_{K }}  \sum_ {k =1} ^{K } {1\ over  |G_{k }|} \sum _{i,j \in G_{k}} \|X _ {i}-X_{j}\|^{ 2 }$ $ ov e r al l p ossible par titi o ns o n $[ n ]$ , w h ere $ |G_{k }| $  i s  the cardinality of  $ G_{k}$ . Dro pping the sum  of square d n orms $\s um_{ i =1 } ^{n} \|X_{i}\| ^{2}$ , we see t h at the $ K$-me ans clus tering is e quivalen t t o t hemax i m iz ation of thet o talwi thin-cl ust er cova ria nce s $ $\m ax _{G_{1},\ dots,G_{ K} }\s um _{k =1}^{ K } {1 \ov er  |G _{ k}| } \su m _{i,j\in G _{k} }a_ { ij} , \quad \ mb o x {wit h}a _{ij } =  X _{i}^ {T}X _{j }.$$ He re, $a_{i j}= X _{i} ^{ T}  X_{j}$  can be viewe das a simil ar ity  measu r e  specifi ed by the Euclidean spa c e inner  pr oduct  $\l angle X_{ i},  X_{j} \ra n gle_{{ \mathb b{R}} ^p }$. I n gen e r al , w ecan replac e the  Eucl id eaninner p roduct with any ot h erinner product  ov er $ S _ n$  [@ C he n Yan g2 0 18] . For manifold cl ustering,we re place it w i thth e inner  produc t ind u ced fro m the emp irical di ff usio n dis tance, tha t is, $$ \begin{al i gned} \l angle  x, \, y\r an gle _{D_{ n,t}}=  \
 0^{+}$,_when $S_n$_is sampled from the_manifold ${\mathcal{M}}$.

Diffusion_$K$-means_{#sec:diffusion_Kmeans}
===================

Recall that_in_our clustering model,_$S_n=\{X_{1},X_2,\ldots,X_n\}$ is a_sample of independent random_variables taking values_in_$S$, where $S$ is the union of $K$ disjoint Riemannian submanifolds ${\mathcal{D}}_{1},\dots,{\mathcal{D}}_{K}$ embedded in_the_ambient space_${\mathbb{R}}^p$._The_clustering problem is to divide_these $n$ data points into_$K$ clusters,_so that points in the same cluster belongs_to_the same connected_component in $S$, based on certain similarity measures between_the points. In particular, the (classical)_$K$-means clustering method_minimizes_the_total intra-cluster squared Euclidean_distances in ${\mathbb{R}}^p$ $$\min_{G_{1},\dots,G_{K}} \sum_{k=1}^{K} {1_\over |G_{k}|} \sum_{i,j \in G_{k}} \|X_{i}-X_{j}\|^{2}$$_over all possible partitions on $[n]$, where_$|G_{k}|$ is the cardinality of $G_{k}$._Dropping the sum of squared_norms $\sum_{i=1}^{n}_\|X_{i}\|^{2}$, we see that the_$K$-means clustering is_equivalent to_the maximization of_the total within-cluster covariances $$\max_{G_{1},\dots,G_{K}} \sum_{k=1}^{K}_{1 \over |G_{k}|}_\sum_{i,j \in G_{k}} a_{ij}, \quad\mbox{with }a_{ij}_=_X_{i}^{T} X_{j}.$$ Here,_$a_{ij}=X_{i}^{T}_X_{j}$_can be_viewed as a_similarity_measure specified_by_the Euclidean space inner product $\langle_X_{i},_X_{j}\rangle_{{\mathbb{R}}^p}$. In general, we can replace the_Euclidean inner product with_any_other inner product over_$S_n$ [@ChenYang2018]. For manifold clustering,_we replace it with the inner_product induced_from the_empirical diffusion distance, that is, $$\begin{aligned}
\langle x,\, y\rangle_{D_{n,t}} = \