 even if the latter are organized in a hierarchical way. Results are very robust and show less errors than the AP with its hard constraints.

Extension to semi-supervised clustering {#sec:semi}
=======================================

In case labels are provided for some data points, they can be exploited to enhance the algorithmic performance. We propose the following way: Identically labeled data are collected in [*macro-nodes*]{}, one for each label. Since macro-nodes are labeled, they do not need an exemplar, but they may serve as exemplars for other data. If there are $N$ [*unlabeled*]{} points and $m$ known labels, the exemplar mapping thus gets generalized to ${\bf c}: \{1,...,N\} \mapsto \{1,...,N,N+1,...,N+m\}$ where indexes $N+1,...,N+m$ correspond to macro-nodes. We define the similarity of an arbitrary unlabeled point to a macro-node as the maximum of similarities between the point and all elements of the macro-node [^2]. The Hamiltonian now becomes: $${\cal H}_2[{\bf c}] = -\sum_{\mu=1}^{N} S(\mu,c_\mu)
+p_1\sum_{\mu=1}^N\chi_{\mu}[{\bf c}]+p_2\sum_{\nu=N+1}^{N+m}
\chi_{\nu}[{\bf c}]
\label{eq:cost2}$$ Note that neither the sizes of the training set nor of the macro-nodes appear explicitly. They are implicitly present via the determination of the similarities between data and macro-nodes. In principle, we can choose different values of $p_1$ and $p_2$, more precisely $p_1>p_2$, to reduce the cost of choosing macro-nodes as exemplars as compared to normal data points. However, this usually forces data to choose the closest macro-node instead of making a collective choice using the geometrical information contained in the data set. We found $p_1=p_2=p$ to work best.

![Factor graph and message direction: Circles (variable nodes) are unlabeled data points, squares (factor nodes) constraints due to unlabeled (light) and macro-nodes (dark). Similarities act as $(N+m-1)$-dimensional external fields on the unlabeled data points. Messages are exchanged between all connected pairs of data points and
even if the latter are organized in a hierarchical way. Results are very robust and picture less error than the AP with its hard constraints. 

 Extension to semi - supervised clustering { # sec: semi } 
 = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = 

 In lawsuit labels are provided for some datum points, they can be exploited to enhance the algorithmic performance. We project the following way: Identically labeled datum are collected in [ * macro - nodes * ] { }, one for each label. Since macro - nodes are labeled, they do not necessitate an exemplar, but they may serve as exemplars for other data. If there are $ N$ [ * unlabeled * ] { } point and $ m$ known labels, the exemplar mapping therefore gets generalized to $ { \bf c }: \{1,... ,N\ } \mapsto \{1,... ,N, N+1,... ,N+m\}$ where index $ N+1,... ,N+m$ correspond to macro - nodes. We specify the similarity of an arbitrary unlabeled point to a macro - node as the maximum of similarities between the point and all element of the macro - node [ ^2 ]. The Hamiltonian now becomes: $ $ { \cal H}_2[{\bf c } ] = -\sum_{\mu=1}^{N } S(\mu, c_\mu) 
 + p_1\sum_{\mu=1}^N\chi_{\mu}[{\bf c}]+p_2\sum_{\nu = N+1}^{N+m } 
 \chi_{\nu}[{\bf c } ] 
 \label{eq: cost2}$$ Note that neither the sizes of the training set nor of the macro - nodes appear explicitly. They are implicitly present via the determination of the similarities between data and macro - nodes. In principle, we can choose different values of $ p_1 $ and $ p_2 $, more precisely $ p_1 > p_2 $, to reduce the monetary value of choosing macro - nodes as exemplar as compare to normal datum points. However, this usually forces data to choose the closest macro - node instead of making a corporate choice using the geometrical information contained in the data set. We found $ p_1 = p_2 = p$ to work best. 

! [ Factor graph and message direction: circle (variable node) are unlabeled data points, squares (factor nodes) constraint due to unlabeled (light) and macro - nodes (dark). Similarities work as $ (N+m-1)$-dimensional external fields on the unlabeled data points. Messages are exchange between all connected couple of data points and
 evfn if the latter are orgxnized in a hietaechican way. Desults xre very robust and show lesd wrrorw than the AP with its hard condtraints.

Wxteision to semi-supxdvised gjustsving {#vxc:semi}
=======================================

In case lsbels are [rovided for smmd bata points, they can be exploited to enhancr hhe algorithmis pegfjrmahbe. We propose the following way: Identibally labeled dats are collected in [*macro-nofes*]{}, lne for each label. Since macri-nodqw are labelea, they do not need an gxemplar, but they may serve as ebemplcrs for othgx dahd. If there ere $N$ [*lnlabeled*]{} poikns and $k$ known labels, the excmplac mapping thus gets generelized to ${\bf c}: \{1,...,N\} \mapfto \{1,...,N,N+1,...,N+m\}$ fhzre indexes $N+1,...,N+m$ correwpind tm mawro-nuees. We dxfihe the sijilarity or an arbitrqry unlabeled point tj a macro-node zs the mwximum of similarities between the poinu and all elements of the maxro-node [^2]. The Hamiltonlan now bqcomes: $${\cal H}_2[{\bf c}] = -\sum_{\mu=1}^{N} S(\mu,c_\mu)
+p_1\sum_{\mu=1}^N\chi_{\mu}[{\bf c}]+p_2\sgm_{\nu=N+1}^{I+m}
\zhi_{\um}[{\nf c}]
\uqbfl{eq:cost2}$$ Note that neither the sizes of the twzimikg set nor of thc macro-nodes appeat fxljicitly. They xre im'miditly present via hhe detgrminarion of tre somilarities between data ane macro-nodes. Un principle, we cau choose difyerent valurs of $p_1$ and $p_2$, more precnsely $l_1>p_2$, to reducf the cosf of choosing macfo-npdas as extoplars as compareq to normel daca pointr. Hoeever, ehis usualpy fovwes data to choose the elosevt macro-nofe instead of making a collectitx choice usinb dhe geometrncal ikformation contwined in the dcta set. Ce fouvd $p_1=p_2=p$ to sork bevt.

![Factor grwph and messacg direction: Cmrcles (vawiaboe nides) ard unlabeled dats points, squares (faxtor nodes) constralnts ahe to unlabeled (litht) and macro-noces (dawk). Smmilawhties act as $(N+m-1)$-dkmevxionau external nieuds pn the unlabeled datd pojnts. Messages are rxghanged bgtween alj connected psirs of data pointd and
even if the latter are organized in way. are very and show less its constraints. Extension to clustering {#sec:semi} ======================================= case labels are provided for some points, they can be exploited to enhance the algorithmic performance. We propose the way: Identically labeled data are collected in [*macro-nodes*]{}, one for each label. Since are they not an exemplar, but they may serve as exemplars for other data. If there are $N$ [*unlabeled*]{} and $m$ known labels, the exemplar mapping thus generalized to ${\bf c}: \mapsto \{1,...,N,N+1,...,N+m\}$ where indexes $N+1,...,N+m$ to We define similarity an unlabeled point to macro-node as the maximum of similarities between the point and all elements of the macro-node [^2]. The now becomes: c}] = S(\mu,c_\mu) c}]+p_2\sum_{\nu=N+1}^{N+m} c}] \label{eq:cost2}$$ Note the sizes of the training set macro-nodes appear explicitly. They are implicitly present via determination of similarities between data and macro-nodes. In we can choose different values of $p_1$ and more precisely $p_1>p_2$, to reduce the cost of choosing macro-nodes as exemplars as compared to points. However, this usually data to choose closest instead making collective choice the geometrical information contained in the data set. We found $p_1=p_2=p$ work best. ![Factor graph and message direction: Circles (variable nodes) data squares (factor nodes) due to unlabeled (light) macro-nodes Similarities act as $(N+m-1)$-dimensional on unlabeled are between connected pairs of data and
 even if the latter are organizEd in a hieraRchicAl wAy. REsUlts Are vEry robust and shOW lesS errors than the AP with itS hard CoNStraINtS.

ExteNsion to SEmI-SUpeRvIsEd cLuSTeRing {#sEc:sEmi}
=======================================

In caSe labels arE prOvIded for some dATa Points, they Can Be exploited tO enHance tHe AlgORithmIc pErforMance. WE ProposE the folloWiNG way: IdENticallY LAbEled Data are collected iN [*MaCRo-nodes*]{}, one for eAch labEl. sInCE MacRo-nOdes are labElEd, theY Do not neED aN EXEmpLAr, but they may sErve as exempLArs For othEr DatA. if therE are $N$ [*UnLAbeLed*]{} points anD $m$ knOwn labels, The exeMPlar mapPIng thus Gets geNerAliZed tO ${\Bf C}: \{1,...,N\} \MapStO \{1,...,n,N+1,...,N+M\}$ WhEre INdeXes $N+1,...,N+m$ coRrEsPond tO macRO-NODes. WE deFine The siMilarity of an aRbiTrarY UnlAbeleD poinT to a MaCro-noDe as thE maxiMuM of similarities BetwEen the poiNt aNd All ElEmentS Of the mAcrO-noDe [^2]. The HaMiltoniAN noW bECOMeS: $${\cal H}_2[{\bf c}] = -\sum_{\mu=1}^{N} S(\mu,c_\Mu)
+P_1\SUm_{\Mu=1}^N\chi_{\mu}[{\Bf c}]+p_2\suM_{\Nu=n+1}^{N+M}
\Chi_{\nu}[{\bf c}]
\LaBel{Eq:coST2}$$ note tHat nEItHer the siZes of tHE tRaIning seT nOr of thE mAcrO-noDes apPEar eXpliciTly. They aRe impLIcitly present vIA the determinaTIoN OF tHE simIlaRities betweEn daTA and MacrO-NoDes. iN prinCiple, We CAn CHoose different valueS oF $p_1$ and $p_2$, More pRecisely $p_1>p_2$, to rEduce the coST OF choosinG macRO-nODes as exemplars As comPared to norMAl data poInts. HOwever, thIs usually FORces data To cHooSe tHe cLOSeSt macro-node inSTEad oF mAking a cOllEctive cHoiCe uSinG thE gEometricaL informaTiOn CoNtAinEd in tHE data set. we FouNd $P_1=p_2=p$ To worK Best.

![FaCtor gRaph AnD mESsaGe direcTIoN: cIrclEs (VaRiabLe nOdEs) are UnlaBEleD data poInts, squarEs (fACtor NoDeS) constrAints due to unlAbEled (light) aNd MacRo-nodeS (DArk). SimilArities act as $(N+m-1)$-dimensionAL externAl fIelds On thE unlabeleD daTa poinTs. MESsages Are excHangeD bEtwEEN all cONNeCteD pAirs of data POIntS and
 even if the latter are or ganized in  a hi era rch ic al w ay.Results are ve r y ro bust and show less err ors t ha n  the AP  with  its ha r dc o nst ra in ts. 
E xt ensio n t o semi- supervised  cl us tering {#sec : se mi}
====== === ============ === ====== == === = ===
Incaselabels are pr ovided fo rs ome da t a point s ,  t heycan be exploitedt oe nhance the alg orithm ic pe r f orm anc e. We prop os e the followi n gw a y : I d entically lab eled data a r e c ollect ed  in [*macr o-nod es * ]{} , one for e achlabel. Si nce ma c ro-node s  are la beled,  th eydo n o tne edan exe m pl ar, but  they ma yse rve a s ex e m p l arsfor  oth er da ta. If thereare  $N$ [*u nlabe led*] {} p oi nts a nd $m$  know nlabels, the exe mpla r mapping  th us  ge ts  gene r alized  to  ${ \bf c}:  \{1,.. . ,N\ }\ m a ps to \{1,...,N,N+1,. .. , N +m \}$ wher e inde x es  $ N +1,...,N +m $ c orre s p ond t o ma c ro -nodes.We def i ne  t he simi la rity o fanarb itrar y  unl abeled  point t o a m a cro-node as th e  maximum of s i mi l a ri t iesbet ween the po inta nd a ll e l em ent s  of t he ma cr o -n o de [^2]. The Hamilt on ian no w bec omes: $${\cal  H}_2[{\bf c } ] = -\su m_{\ m u= 1 }^{N} S(\mu,c_ \mu)+p_1\sum_{ \ mu=1}^N\ chi_{ \mu}[{\b f c}]+p_2 \ s um_{\nu= N+1 }^{ N+m }
\ c h i_ {\nu}[{\bf c} ] \lab el {eq:cos t2} $$ Note  th atnei the rthe sizes  of thetr ai ni ng  se t nor of the m ac ro- no des  appe a r expl icitl y. T he ya reimplici t ly p rese nt  v ia t hede termi nati o n o f the s imilariti esb etwe en  d ata and  macro-nodes.  I n principl e,  we  can c h o ose diff erent values of $p_1$ a n d $p_2$ , m ore p reci sely $p_1 >p_ 2$, to  re d uce th e cost  of c ho osi n g  macr o - no des  a s exemplar s ascompa re d to  normal  data points. Howe v er,  this usually  fo rces d at a t o  c h oos et hec l osest macro-nod e insteadof ma king a col l ect iv e choic e using  theg eometri cal infor mation co nt aine d inthe data s et. We f ound $p_1 = p_2=p $  t o wor k b est.
![ Fac tor g raph a n d m essag e dire ct ion: C ircle s(variabl e nodes) are unlabeleddata p oints , s quares (f act o r n odes) con stra ints due t o u nla beled  (l i ght)andm ac ro- n odes(dar k ). Simila r it ies a ct  as $(N+m-1 ) $ - dim ensio nal extern al f ields on the unla b eled data poin ts.M e ssa ges areex changed betwee n a ll c onnected  p airs of dat a points  a n d
 even_if the_latter are organized in_a hierarchical_way._Results are_very_robust and show_less errors than_the AP with its_hard constraints.

Extension to_semi-supervised_clustering {#sec:semi}
=======================================

In case labels are provided for some data points, they can be exploited_to_enhance the_algorithmic_performance._We propose the following way:_Identically labeled data are collected_in [*macro-nodes*]{},_one for each label. Since macro-nodes are labeled,_they_do not need_an exemplar, but they may serve as exemplars for_other data. If there are $N$_[*unlabeled*]{} points and_$m$_known_labels, the exemplar mapping_thus gets generalized to ${\bf c}:_\{1,...,N\} \mapsto \{1,...,N,N+1,...,N+m\}$ where indexes $N+1,...,N+m$_correspond to macro-nodes. We define the similarity_of an arbitrary unlabeled point to_a macro-node as the maximum_of similarities_between the point and all_elements of the_macro-node [^2]._The Hamiltonian now_becomes: $${\cal H}_2[{\bf c}] = -\sum_{\mu=1}^{N}_S(\mu,c_\mu)
+p_1\sum_{\mu=1}^N\chi_{\mu}[{\bf c}]+p_2\sum_{\nu=N+1}^{N+m}
\chi_{\nu}[{\bf c}]
\label{eq:cost2}$$_Note that neither the sizes of_the_training set nor_of_the_macro-nodes appear_explicitly. They are_implicitly_present via_the_determination of the similarities between data_and_macro-nodes. In principle, we can choose different_values of $p_1$ and_$p_2$,_more precisely $p_1>p_2$, to_reduce the cost of choosing_macro-nodes as exemplars as compared to_normal data_points. However,_this usually forces data to choose the closest macro-node instead of_making a collective choice using the_geometrical information contained in_the data_set._We found $p_1=p_2=p$_to_work best.

![Factor_graph and message direction: Circles (variable nodes)_are unlabeled_data points, squares (factor nodes) constraints_due to unlabeled (light)_and_macro-nodes (dark). Similarities act as $(N+m-1)$-dimensional_external fields on the unlabeled data_points. Messages are exchanged between_all_connected_pairs of data points and