
\[
\mu_i = \frac{1}{|S_i|}\sum_{x_j\in S_i}x_j.
\]

\subsection*{The K-Means Algorithm}

Finding the global minimizing partition $S^*$ is generally intractable since the set of partitions can be very large indeed,
but the \emph{k-means} algorithm is a heuristic approach that can often provide reasonably accurate results.


We begin by specifying an initial cluster mean $\mu_i^{(1)}$ for each $i = 1, \cdots, N$ (this can be done by random initialization, or according to some heuristic).
For each iteration, we adopt the following procedure.
Given a current set of cluster means $\mu^{(t)}$, we find a partition $S^{(t)}$ of the observations such that
\begin{equation*}
S_{i}^{(t)} = \{x_j \; : \; \|x_j - \mu_{i}^{(t)}\|_2^2 \leq \|x_j - \mu_{l}^{(t)}\|_2^2,\,\,\,  l = 1, \cdots, N\}.
\end{equation*}
We then update our cluster means by computing for each $i = 1, \cdots, N$.
We continue to iterate in this manner until the partition ceases to change.


Figure \ref{fig:iris_clusterings} shows two different clusterings of the iris data produced by the \emph{k-means} algorithm.
Note that the quality of the clustering can depend heavily on the initial cluster means.
We can use the within-cluster sum of squares as a measure of the quality of a clustering (a lower sum of squares is better).
Where possible, it is advisable to run the clustering algorithm several times, each with a different initialization of the means,
and keep the best clustering.
Note also that it is possible to have very slow convergence.
Thus, when implementing the algorithm, it is a good idea to terminate after some specified maximum number of iterations.
%
\begin{figure}[h]
    \centering
    \begin{tabular}{cc}
    \includegraphics[width=.49\textwidth]{iris_means_1.pdf} &
    \includegraphics[width=.49\textwidth]{iris_means_2.pdf}
\ [ 
 \mu_i = \frac{1}{|S_i|}\sum_{x_j\in S_i}x_j. 
 \ ] 

 \subsection*{The K - Means Algorithm } 

 Finding the global minimizing partition $ S^*$ is generally intractable since the hardening of partition can be very large indeed, 
 but the \emph{k - means } algorithm is a heuristic approach that can much provide reasonably accurate resultant role. 


 We get down by specifying an initial cluster mean $ \mu_i^{(1)}$ for each $ i = 1, \cdots, N$ (this can be done by random initialization, or accord to some heuristic). 
 For each iteration, we adopt the following procedure. 
 give a current set of cluster mean $ \mu^{(t)}$, we find a partition $ S^{(t)}$ of the observation such that 
 \begin{equation * } 
 S_{i}^{(t) } = \{x_j \; : \; \|x_j - \mu_{i}^{(t)}\|_2 ^ 2 \leq \|x_j - \mu_{l}^{(t)}\|_2 ^ 2,\,\,\,   l = 1, \cdots, N\ }. 
 \end{equation * } 
 We then update our cluster means by calculate for each $ i = 1, \cdots, N$. 
 We continue to repeat in this manner until the partition end to change. 


 Figure \ref{fig: iris_clusterings } show two different clusterings of the iris data grow by the \emph{k - means } algorithm. 
 Note that the quality of the clustering can depend heavily on the initial cluster means. 
 We can use the within - cluster sum of squares as a measure of the quality of a clustering (a lower sum of square is better). 
 Where possible, it is advisable to prevail the bunch algorithm respective times, each with a different initialization of the means, 
 and keep the best clustering. 
 Note besides that it is possible to have very slow convergence. 
 Thus, when implementing the algorithm, it is a good idea to terminate after some specified maximum number of iteration. 
% 
 \begin{figure}[h ] 
     \centering 
     \begin{tabular}{cc } 
     \includegraphics[width=.49\textwidth]{iris_means_1.pdf } & 
     \includegraphics[width=.49\textwidth]{iris_means_2.pdf }

\[
\mu_i = \frac{1}{|S_i|}\sum_{x_j\in S_i}x_j.
\]

\substction*{The K-Means Cogoritim}

Findihg the guobal minimizing partition $S^*$ iw gentgally intractable sinze the sen of partutiois can be very ledge indczd,
but bhe \ek'h{k-means} algorijhm is a heusistic approacv ghct can often provide reasonably accuwate rexupts.


We begin by spebisyinf an initial cluster mean $\mu_i^{(1)}$ for sach $i = 1, \cdots, N$ (thix can be done by random inltiapization, or accordlng to some heuwustic).
For eacf iteration, we adopt tge following procedure.
Given a cjrrenc set of closfeg means $\mu^{(t)}$, xe finq a partitiok $S^{(t)}$ of the obxervations sucm thav
\begun{equation*}
S_{i}^{(t)} = \{x_j \; : \; \|e_j - \mu_{i}^{(t)}\|_2^2 \leq \|x_j - \mu_{l}^{(j)}\|_2^2,\,\,\,  l = 1, \cdods, N\}.
\end{equation*}
We tyeb upddte mur zousger mxana by clmpnting for ezch $i = 1, \cdors, N$.
We continue to otqgste in this janner ugtil the partition ceases to change.


Figuge \rsf{fig:iris_clusterings} shiws two different cludterings jf the iris data produced by the \emph{k-means} algorhthm.
Nkge uhqt thd qkality of the clustering can depend heavily og tne initial clustcr means.
We can use tje rithin-cluster sum oy ssuares as a measurf of thg qualuty of a slusyering (a lower sum of squarws is better).
Cheee possible, it is cdvisable to run the vlustering algorithm sereral fimes, each aith a dirwerent initializxtipn of the means,
and keep the fest clusveriny.
Note alro tnat it is possibpe to have very slow congergeuce.
Thgs, when imolementing the algorithm, it is e good idea tp derkinate ayter spme specified maximum numbet of iterctions.
%
\cegin{figurv}[h]
    \cenvering
    \bedin{tabular}{cc}
    \includegra'hics[widtr=.49\texrwidrh]{iris_mdxns_1.pdf} &
    \inckudegraphpcf[widrh=.49\textwidth]{iris_meaks_2.pdf}
\[ \mu_i = \frac{1}{|S_i|}\sum_{x_j\in S_i}x_j. \] \subsection*{The Finding global minimizing $S^*$ is generally partitions be very large but the \emph{k-means} is a heuristic approach that can provide reasonably accurate results. We begin by specifying an initial cluster mean $\mu_i^{(1)}$ each $i = 1, \cdots, N$ (this can be done by random initialization, according some For iteration, we adopt the following procedure. Given a current set of cluster means $\mu^{(t)}$, we find partition $S^{(t)}$ of the observations such that \begin{equation*} = \{x_j \; : \|x_j - \mu_{i}^{(t)}\|_2^2 \leq \|x_j \mu_{l}^{(t)}\|_2^2,\,\,\, = 1, N\}. We update our cluster by computing for each $i = 1, \cdots, N$. We continue to iterate in this manner until partition ceases Figure \ref{fig:iris_clusterings} two clusterings the iris data the \emph{k-means} algorithm. Note that the clustering can depend heavily on the initial cluster We can the within-cluster sum of squares as measure of the quality of a clustering (a sum of squares is better). Where possible, it is advisable to run the clustering algorithm each with a different of the means, keep best Note that it possible to have very slow convergence. Thus, when implementing the algorithm, is a good idea to terminate after some specified maximum iterations. \begin{figure}[h] \centering \begin{tabular}{cc} & \includegraphics[width=.49\textwidth]{iris_means_2.pdf}

\[
\mu_i = \frac{1}{|S_i|}\sum_{x_j\in S_i}x_j.
\]

\subseCtion*{The K-MEans ALgoRitHm}

findIng tHe global minimiZIng pArtition $S^*$ is generally inTractAbLE sinCE tHe set Of partiTIoNS Can Be VeRy lArGE iNdeed,
But The \emph{K-means} algoRitHm Is a heuristic APpRoach that cAn oFten provide rEasOnably AcCurATe resUltS.


We beGin by sPEcifyiNg an initiAl CLuster MEan $\mu_i^{(1)}$ fOR EaCh $i = 1, \cDots, N$ (this can be donE By RAndom initializAtion, oR aCCoRDIng To sOme heuristIc).
for eaCH iteratIOn, WE ADopT The following pRocedure.
GivEN a cUrrent SeT of CLuster Means $\Mu^{(T)}$, We fInd a partitiOn $S^{(t)}$ Of the obseRvatioNS such thAT
\begin{eQuatioN*}
S_{i}^{(T)} = \{x_j \; : \; \|X_j - \mu_{I}^{(T)}\|_2^2 \lEq \|X_j - \mU_{l}^{(T)}\|_2^2,\,\,\,  L = 1, \cdOTs, n\}.
\enD{EquAtion*}
We tHeN uPdate Our cLUSTEr meAns By coMputiNg for each $i = 1, \cdoTs, N$.
we coNTinUe to iTeratE in tHiS mannEr untiL the pArTition ceases to cHangE.


Figure \reF{fiG:iRis_ClUsterINgs} shoWs tWo dIfferenT clusteRIngS oF THE iRis data produced by tHe \EMPh{K-means} alGorithM.
noTe THat the quAlIty Of thE CLusteRing CAn Depend heAvily oN ThE iNitial cLuSter meAnS.
We Can Use thE WithIn-clusTer sum of SquarES as a measure of tHE quality of a clUStERInG (A lowEr sUm of squares Is beTTer).
WHere POsSibLE, it is AdvisAbLE tO Run the clustering algOrIthm seVeral Times, each with A different INITializatIon oF ThE Means,
and keep thE best Clustering.
nOte also tHat it Is possibLe to have vERY slow conVerGenCe.
THus, WHEn Implementing tHE AlgoRiThm, it is A goOd idea tO teRmiNatE afTeR some specIfied maxImUm NuMbEr oF iterATions.
%
\begIn{FigUrE}[h]
    \cEnterINg
    \begiN{tabuLar}{cC}
    \iNcLUdeGraphicS[WiDTH=.49\texTwIdTh]{irIs_mEaNs_1.pdf} &
    \InclUDegRaphics[Width=.49\textWidTH]{iriS_mEaNs_2.pdf}

\[
\mu_i = \frac{1}{|S_i| }\sum_{x_j \in S _i} x_j .\]
\sub section*{The K - Mean s Algorithm}

Findingthe g lo b al m i ni mizin g parti t io n $S^ *$  i s g en e ra lly i ntr actable  since the  se tof partition s  c an be very  la rge indeed,but  the \ em ph{ k -mean s}algor ithm i s  a heu ristic ap pr o ach th a t can o f t en  pro vide reasonably a c cu r ate results.

We be gi n  b y spe cif ying an in it ial c l uster m e an $ \ mu_ i ^{(1)}$ for e ach $i = 1, \cd ots, N $(th i s canbe do ne byrandom init iali zation, o r acco r ding to some he uristi c). 
Fo r ea c hit era ti o n,w eado p t t he follo wi ng  proc edur e . G iven  acurr ent s et of cluster  me ans$ \mu ^{(t) }$, w e fi nd  a pa rtitio n $S^ {( t)}$ of the obs erva tions suc h t ha t
\ be gin{e q uation *}S_{ i}^{(t) } = \{x _ j \ ;: \ ;\|x_j - \mu_{i}^{( t) } \ |_ 2^2 \leq  \|x_j -\m u _{l}^{(t )} \|_ 2^2, \ , \,\, l = 1,  \cdots,  N\}.\ en d{ equatio n* }
We t he n u pda te ou r  clu ster m eans bycompu t ing for each $ i  = 1, \cdots, N$ . We cont inu e to iterat e in this  man n er  un t il th e par ti t io n  ceases to change.
Figure  \ref {fig:iris_clu sterings}s h o ws two d iffe r en t  clusterings o f the  iris data produced  by t he \emph {k-means} a lgorithm .
N ote  th att h equality of th e clus te ring ca n d epend h eav ily  on  th einitial c luster m ea ns .We  ca n use the with in -cl us ter  sumo f squa res a s ame as u reof theq ua l i ty o faclus ter in g (alowe r  su m of sq uares isbet t er). 
W he re poss ible, it is a dv isable toru n t he clu s t ering al gorithm several times,e ach wit h a  diff eren t initial iza tion o f t h e mean s,
and  keep  t heb e st cl u s te rin g. 
Note also t hat  it i sposs ible to  have very slow co n ver gence.
Thus,whe n im p l em ent i ng the  a l gor i t hm, it is a goo d idea tote r mi nate after som especifi ed maxi mum n u mber of  iteratio ns.
%
\be gi n{fi g u re} [h]
    \c entering 
    \beg i n{tab u la r}{cc }
   \in cl ude graph ics[wi d th= .49\t extwid th ]{iris _mean s_ 1.pdf} & 
    \includegraphics[w idth=. 49\te xtw idth]{iri s_m e ans _2.pdf}

\[
\mu_i =_\frac{1}{|S_i|}\sum_{x_j\in S_i}x_j.
\]

\subsection*{The_K-Means Algorithm}

Finding the global_minimizing partition_$S^*$_is generally_intractable_since the set_of partitions can_be very large indeed,
but_the \emph{k-means} algorithm_is_a heuristic approach that can often provide reasonably accurate results.


We begin by specifying an_initial_cluster mean_$\mu_i^{(1)}$_for_each $i = 1, \cdots,_N$ (this can be done_by random_initialization, or according to some heuristic).
For each iteration,_we_adopt the following_procedure.
Given a current set of cluster means $\mu^{(t)}$, we_find a partition $S^{(t)}$ of the_observations such that
\begin{equation*}
S_{i}^{(t)}_=_\{x_j_\; : \; \|x_j_- \mu_{i}^{(t)}\|_2^2 \leq \|x_j - \mu_{l}^{(t)}\|_2^2,\,\,\,_ l = 1, \cdots, N\}.
\end{equation*}
We_then update our cluster means by computing_for each $i = 1, \cdots,_N$.
We continue to iterate in_this manner_until the partition ceases to_change.


Figure \ref{fig:iris_clusterings} shows_two different_clusterings of the_iris data produced by the \emph{k-means}_algorithm.
Note that the_quality of the clustering can depend_heavily_on the initial_cluster_means.
We_can use_the within-cluster sum_of_squares as_a_measure of the quality of a_clustering_(a lower sum of squares is better).
Where_possible, it is advisable_to_run the clustering algorithm_several times, each with a_different initialization of the means,
and keep_the best_clustering.
Note also_that it is possible to have very slow convergence.
Thus, when implementing_the algorithm, it is a good_idea to terminate after_some specified_maximum_number of iterations.
%
\begin{figure}[h]
__ _\centering
    \begin{tabular}{cc}
  _ \includegraphics[width=.49\textwidth]{iris_means_1.pdf}_&
    \includegraphics[width=.49\textwidth]{iris_means_2.pdf}