tilde y_j) )} )$. Moreover, it holds that one can recover the solution $\mathtt{p}^*$ to the primal problem  from a solution $\mathtt{y}^*$ to the dual problem  as $
    \mathtt{p}^* = \mathtt{p}^*(\sqrt{W}\mathtt{y}^* )$.

The optimality relation between the dual and the primal problem follows from Theorem $3.1$ in [@Cuturi2016]. In general, the dual problem  can have multiple solutions of the form $\mathtt{y}^* + \ker(\sqrt{W})$ when the matrix $\sqrt{W}$ does not have a full row rank. When the solution is not unique, we [*will use $\mathtt{y}^*$ to denote the smallest norm solution*]{}, and we let $R$ be its norm, i.e. $R = \|\mathtt{y}^*\|_2$.

. As a consequence, the computation of the WB of a set of discrete probability distributions $\{q_i\}_{i=1}^m$ is equivalent to solving the dual decomposable $L$-smooth (with respect to the $2$-norm) optimization problem  with  [@Kakade2009]. Specifically, in this setup it holds that

\[S:AlgAndMain\] We can explicitly write the Nesterov’s Accelerated Gradient Method (FGM) [@nes13] for smooth functions. Particularly, we use follow the linear coupling approach recently proposed in [@all14]. Setting $\hat{\mathtt{w}}_{k} = \hat{\mathtt{z}}_{k}=\hat{\mathtt{y}}_{k} = \boldsymbol{0}$, the FGM generates iterates according to:

where $\alpha_{k+1} = (k+2)/(2L)$ and $\tau_k = 2/(k+2)$.

Unfortunately, algorithm  cannot be executed in a distributed manner. Although the entries of local gradient vectors can be computed independently by each node, the sparsity pattern of the matrix $\sqrt{W}$ need not be the same as the communication constraints induced by the graph $\mathcal{G}$. Thus, the variables $\hat {\mathtt{w}}_{k}$ and $\hat {\mathtt{z}}_{k}$ cannot be computed on the network. This problem is solved by a change of variables such that $\tilde{ \mathtt{y}} = \sqrt{W}\hat{\mathtt{
tilde y_j) ) }) $. Moreover, it holds that one can recover the solution $ \mathtt{p}^*$ to the primal trouble    from a solution $ \mathtt{y}^*$ to the double problem    as $ 
     \mathtt{p}^ * = \mathtt{p}^*(\sqrt{W}\mathtt{y}^ *) $. 

 The optimality relation between the double and the aboriginal problem follows from Theorem $ 3.1 $ in   [ @Cuturi2016 ]. In cosmopolitan, the dual problem    can have multiple solution of the form $ \mathtt{y}^ * + \ker(\sqrt{W})$ when the matrix $ \sqrt{W}$ does not have a wide row rank. When the solution is not alone, we [ * will use $ \mathtt{y}^*$ to denote the smallest norm solution * ] { }, and we let $ R$ be its average, i.e. $ R = \|\mathtt{y}^*\|_2$. 

. As a consequence, the computation of the WB of a set of discrete probability distribution $ \{q_i\}_{i=1}^m$ is equivalent to solving the dual analyzable $ L$-smooth (with respect to the $ 2$-norm) optimization trouble    with   [ @Kakade2009 ]. Specifically, in this setup it holds that 

 \[S: AlgAndMain\ ] We can explicitly write the Nesterov ’s Accelerated Gradient Method (FGM)   [ @nes13 ] for smooth functions. Particularly, we practice follow the linear coupling approach recently proposed in   [ @all14 ]. Setting $ \hat{\mathtt{w}}_{k } = \hat{\mathtt{z}}_{k}=\hat{\mathtt{y}}_{k } = \boldsymbol{0}$, the FGM generate iterates according to: 

 where $ \alpha_{k+1 } = (k+2)/(2L)$ and $ \tau_k = 2/(k+2)$. 

 Unfortunately, algorithm    cannot be executed in a distributed manner. Although the entries of local gradient vectors can be computed independently by each node, the sparsity pattern of the matrix $ \sqrt{W}$ need not be the like as the communication constraint induced by the graph $ \mathcal{G}$. Thus, the variable $ \hat { \mathtt{w}}_{k}$ and $ \hat { \mathtt{z}}_{k}$ cannot be calculate on the net. This problem is solved by a change of variables such that $ \tilde { \mathtt{y } } = \sqrt{W}\hat{\mathtt {
tilfe y_j) )} )$. Moreover, it holdr that one can texover vhe solhtion $\maghtt{p}^*$ to the primal problem  hrom a sooution $\mathtt{y}^*$ to the aual probpem  as $
    \methtt{p}^* = \mathtt{p}^*(\sqrt{W}\mathbc{y}^* )$.

Ths optnmelity relation netween the dual and the [rkmcl problem follows from Theorem $3.1$ in [@Ctturi2016]. Im heneral, the duwl pgoflem  bak have multiple solutions of the form $\methtt{y}^* + \ker(\sqrt{E})$ when the matrix $\sqrt{W}$ dofs nlt have a full row rank. When jge filution is nut unique, we [*will use $\jathtt{y}^*$ to denote the smallest vorm xolution*]{}, abd we net $R$ be itw nori, i.e. $R = \|\mathbn{y}^*\|_2$.

. As a conseqience, the compmtatimn if the WB of a set of discrete probabilyty distrhbbtions $\{q_i\}_{i=1}^m$ is equivaoebt to soleing rhe duzl dscompodabme $L$-smooth (with respext to the $2$-norm) optikisqtion problem  with  [@Hahade2009]. Specifically, in this setup it holdv tgat

\[S:AlgAndMain\] We can ezplicitly write the Ngsterov’s Ascelerated Gradient Method (FGM) [@nes13] for smooth funcdions. Oaruigmlarut, ae use follow the linear coupling approach ressnuly proposed in [@alj14]. Setting $\hst{\layrtt{w}}_{k} = \hat{\matftt{z}}_{k}=\hcf{\mzthtt{y}}_{k} = \boldsymbop{0}$, the FDM geberates iueratrs according to:

where $\alpha_{k+1} = (k+2)/(2L)$ and $\tau_h = 2/(k+2)$.

Unfortunately, apgorithm  caunot bg execited in a distributed mcnner. Zlthough thf entries uf local gradieng vvctoss can bt computed indepenqently by eack node, tfe skarsity pattern ov the matrix $\sqrt{W}$ need jot bg the vame as thf communication constraints indnred by the grspv $\mdthcal{G}$. Chus, tme variables $\hae {\mathtt{w}}_{k}$ and $\hat {\machtt{z}}_{k}$ cannot be computxd on the neework. This prmhlem is solvxd by a crangw of variabuds such that $\tolde{ \mathnt{v}} = \sqrt{W}\hqt{\mathtt{
tilde y_j) )} )$. Moreover, it holds can the solution to the primal to dual problem as \mathtt{p}^* = \mathtt{p}^*(\sqrt{W}\mathtt{y}^* The optimality relation between the dual the primal problem follows from Theorem $3.1$ in [@Cuturi2016]. In general, the dual can have multiple solutions of the form $\mathtt{y}^* + \ker(\sqrt{W})$ when the matrix does have full rank. When the solution is not unique, we [*will use $\mathtt{y}^*$ to denote the smallest norm and we let $R$ be its norm, i.e. = \|\mathtt{y}^*\|_2$. . As consequence, the computation of the of set of probability $\{q_i\}_{i=1}^m$ equivalent to solving dual decomposable $L$-smooth (with respect to the $2$-norm) optimization problem with [@Kakade2009]. Specifically, in this setup it that \[S:AlgAndMain\] explicitly write Nesterov’s Gradient (FGM) [@nes13] for Particularly, we use follow the linear proposed in [@all14]. Setting $\hat{\mathtt{w}}_{k} = \hat{\mathtt{z}}_{k}=\hat{\mathtt{y}}_{k} = the FGM iterates according to: where $\alpha_{k+1} = and $\tau_k = 2/(k+2)$. Unfortunately, algorithm cannot be in a distributed manner. Although the entries of local gradient vectors can be computed independently node, the sparsity pattern the matrix $\sqrt{W}$ not the as communication constraints by the graph $\mathcal{G}$. Thus, the variables $\hat {\mathtt{w}}_{k}$ and $\hat cannot be computed on the network. This problem is solved change variables such that \mathtt{y}} = \sqrt{W}\hat{\mathtt{
tilde y_j) )} )$. Moreover, it holds thaT one can recOver tHe sOluTiOn $\maThtt{P}^*$ to the primal prOBlem  From a solution $\mathtt{y}^*$ to The duAl PRoblEM  aS $
    \mathTt{p}^* = \mathTT{p}^*(\SQRt{W}\MaThTt{y}^* )$.

thE OpTimalIty RelatioN between thE duAl And the primal PRoBlem followS frOm Theorem $3.1$ in [@CUtuRi2016]. In geNeRal, THe duaL prOblem  Can havE MultipLe solutioNs OF the foRM $\mathtt{Y}^* + \KEr(\Sqrt{w})$ when the matrix $\sqrT{w}$ dOEs not have a full Row ranK. WHEn THE soLutIon is not unIqUe, we [*wILl use $\maTHtT{Y}^*$ TO deNOte the smallesT norm solutiON*]{}, anD we let $r$ bE itS Norm, i.e. $r = \|\mathTt{Y}^*\|_2$.

. as a Consequence, The cOmputatioN of the wb of a set OF discreTe probAbiLitY disTRiBuTioNs $\{Q_I\}_{i=1}^m$ IS eQuiVAleNt to solvInG tHe duaL decOMPOSablE $L$-sMootH (with Respect to the $2$-nOrm) OptiMIzaTion pRobleM  witH  [@KAkade2009]. specifIcallY, iN this setup it holDs thAt

\[S:AlgAndmaiN\] WE caN eXplicITly wriTe tHe NEsterov’S AcceleRAteD GRADIeNt Method (FGM) [@nes13] for sMoOTH fUnctions. particULaRlY, We use folLoW thE linEAR coupLing APpRoach recEntly pROpOsEd in [@all14]. seTting $\hAt{\MatHtt{W}}_{k} = \hat{\MAthtT{z}}_{k}=\hat{\Mathtt{y}}_{k} = \BoldsYMbol{0}$, the FGM geneRAtes iterates aCCoRDInG To:

whEre $\Alpha_{k+1} = (k+2)/(2L)$ and $\Tau_k = 2/(K+2)$.

unfoRtunATeLy, aLGoritHm  canNoT Be EXecuted in a distributEd Manner. althoUgh the entries Of local graDIENt vectorS can BE cOMputed independEntly By each node, THe sparsiTy patTern of thE matrix $\sqRT{w}$ need not Be tHe sAme As tHE CoMmunication coNSTraiNtS induceD by The grapH $\maThcAl{G}$. thuS, tHe variablEs $\hat {\matHtT{w}}_{K}$ aNd $\Hat {\MathtT{Z}}_{k}$ cannot Be ComPuTed On the NEtwork. this pRoblEm Is SOlvEd by a chANgE OF varIaBlEs suCh tHaT $\tildE{ \matHTt{y}} = \Sqrt{W}\haT{\mathtt{
tilde y_j) )} )$. Moreover , it holds  that  on e c an  rec over  the solution$ \mat htt{p}^*$ to the prima l pro bl e m  f r om  a so lution$ \m a t htt {y }^ *$to th e dua l p roblem as $
   \ma th tt{p}^* = \m a th tt{p}^*(\s qrt {W}\mathtt{y }^*  )$.
Th e o p timal ity  rela tion b e tweenthe dualan d  the p r imal pr o b le m fo llows from Theore m  $ 3 .1$ in [@Cutur i2016] .I ng e ner al,  the dualpr oblem  can ha v em u l tip l e solutions o f the form$ \ma thtt{y }^ * + \ker(\ sqrt{ W} ) $ w hen the mat rix$\sqrt{W} $ does not hav e  a full  row r ank . W hent he  s olu ti o n i s  n otu niq ue, we [ *w il l use  $\m a t h t t{y} ^*$  todenot e the smalles t n orms olu tion* ]{},andwe  let$R$ be  itsno rm, i.e. $R = \ |\ma thtt{y}^* \|_ 2$ .
.As ac onsequ enc e,the com putatio n  of  t h e WB  of a set of discr et e pr obabilit y dist r ib ut i ons $\{q _i \}_ {i=1 } ^ m$ is  equ i va lent tosolvin g  t he  dual d ec omposa bl e $ L$- smoot h  (wi th res pect tothe $ 2 $-norm) optimi z ation problem  w i t h[@Ka kad e2009]. Spe cifi c ally , in th iss etupit ho ld s  t h at

\[S:AlgAndMain\ ]We can  expl icitly writethe Nester o v ’ s Accele rate d  G r adient Method(FGM)  [@nes13]f or smoot h fun ctions.Particula r l y, we us e f oll owthe l in ear couplinga p proa ch  recent lypropose d i n [ @al l14 ].  Setting$\hat{\m at ht t{ w} }_{ k} =\ hat{\mat ht t{z }} _{k }=\ha t {\math tt{y} }_{k }=\ bol dsymbol { 0} $ ,  the  F GM  gen era te s ite rate s  ac cording  to:

whe re$ \alp ha _{ k+1} =(k+2)/(2L)$ a nd  $\tau_k =  2 /(k +2)$.U nfortuna tely, algorithm  cannot be exec ute d ina di stributed  ma nner.Alt h ough t he ent riesof  lo c a l gra d i en t v ec tors can b e com puted  i ndep endentl y by each node, th e  sp arsity patter n o f th e ma tri x  $ \ sqr t{ W }$n e ed not be the s ame as the  c o mm unicationc ons tr aints i nducedby th e  graph$\mathcal {G}$. Thu s,  the v ari ables $\ha t {\math tt{w}}_{k } $ and $\ hat { \ma thtt{z }} _{k }$ ca nnot b e  co mpute d on t he  netwo rk. T hi s proble m is solved by a change  of va riabl essuch that  $\ t ild e{ \matht t{y} } = \sqrt{ W}\ hat {\mat htt { 
tilde y_j)_)} )$._Moreover, it holds that_one can_recover_the solution_$\mathtt{p}^*$_to the primal_problem  from a_solution $\mathtt{y}^*$ to the_dual problem  as_$
_   \mathtt{p}^* = \mathtt{p}^*(\sqrt{W}\mathtt{y}^* )$.

The optimality relation between the dual and the_primal_problem follows_from_Theorem_$3.1$ in [@Cuturi2016]. In general, the_dual problem  can have multiple_solutions of_the form $\mathtt{y}^* + \ker(\sqrt{W})$ when the matrix_$\sqrt{W}$_does not have_a full row rank. When the solution is not_unique, we [*will use $\mathtt{y}^*$ to_denote the smallest_norm_solution*]{},_and we let $R$_be its norm, i.e. $R =_\|\mathtt{y}^*\|_2$.

. As a consequence, the computation_of the WB of a set of_discrete probability distributions $\{q_i\}_{i=1}^m$ is equivalent_to solving the dual decomposable_$L$-smooth (with_respect to the $2$-norm) optimization_problem  with  [@Kakade2009]._Specifically, in_this setup it_holds that

\[S:AlgAndMain\] We can explicitly write_the Nesterov’s Accelerated_Gradient Method (FGM) [@nes13] for smooth functions._Particularly,_we use follow_the_linear_coupling approach_recently proposed in [@all14]._Setting_$\hat{\mathtt{w}}_{k} =_\hat{\mathtt{z}}_{k}=\hat{\mathtt{y}}_{k}_= \boldsymbol{0}$, the FGM generates iterates_according_to:

where $\alpha_{k+1} = (k+2)/(2L)$ and $\tau_k =_2/(k+2)$.

Unfortunately, algorithm  cannot be_executed_in a distributed manner._Although the entries of local_gradient vectors can be computed independently_by each_node, the_sparsity pattern of the matrix $\sqrt{W}$ need not be the same_as the communication constraints induced by_the graph $\mathcal{G}$. Thus,_the variables_$\hat_{\mathtt{w}}_{k}$ and $\hat_{\mathtt{z}}_{k}$_cannot be_computed on the network. This problem is_solved by_a change of variables such that_$\tilde{ \mathtt{y}} = \sqrt{W}\hat{\mathtt{