 of $f_1,\ldots,f_t$, and therefore the relaxation is well-defined.

\[pps:OMDrelaxadmin\] Suppose that the learning rate $\rho$ is such that $\rho \| f_t \|^*_{\nabla F(\nu_t)} \le 1/2$ for all $t = 1,\ldots,T$. Assume that $\| f_t \|^*_{\nabla F(\nu_t)} \le 1$, for all $t = 1,\ldots,T$. The relaxation is admissible, and the algorithm that generates the sequence $\{\nu_t\}$ is also admissible: $$\wh{V}_T({\cal G}'|f_1,\ldots,f_{t-1}) \ge \sup_{f \in \cF}\left\{ \ave{\nu_t,f} + \wh{V}_T({\mathcal G}' | f_1, \ldots,f_{t-1},f)\right\}$$

See Appendix \[app:OMDrelaxadmin\].

Here we impose the assumption that $\rho \| f_t \|^*_{\nabla F(\nu_t)} \le 1/2$ for all $t = 1,\ldots,T$. A restriction of this kind is necessary when using interior-point methods to construct online optimization schemes — see, for example, the condition of Theorem 4.1 and 4.2 in [@AbernethyIPM]. The boundedness of the dual local norm $\| f_t \|^*_{\nabla F(\nu_t)}$ is a reasonable assumption as well. In particular, Abernethy et al. [@AbernethyIPM] points out that if a large number of the points $\nu_t$ are close to the boundary of ${\cal G}'$, then the eigenvalues of the Hessian of $F$ at those points will be large due to the large curvature of the barrier near the boundary of ${\cal G}'$. This will imply, in turn, that the dual local norm $\| f_t \|^*_{\nabla F(\nu_t)}$ is expected to be small.

Now we are ready to present the online-learning (i.e., steady-state) part of the regret bound for the above algorithm:

\[thm:OMDbound1\] Let $D_F({\cal G'}) \deq\sup_{\nu \in {\cal G}'} D_{F}(\
of $ f_1,\ldots, f_t$, and therefore the relaxation is well - defined. 

 \[pps: OMDrelaxadmin\ ] Suppose that the eruditeness pace $ \rho$ is such that $ \rho \| f_t \|^*_{\nabla F(\nu_t) } \le 1/2 $ for all $ thyroxine = 1,\ldots, T$. Assume that $ \| f_t \|^*_{\nabla F(\nu_t) } \le 1 $, for all $ t = 1,\ldots, T$. The rest is admissible, and the algorithm that generates the sequence $ \{\nu_t\}$ is besides admissible: $ $ \wh{V}_T({\cal G}'|f_1,\ldots, f_{t-1 }) \ge \sup_{f \in \cF}\left\ { \ave{\nu_t, f } + \wh{V}_T({\mathcal G }' | f_1, \ldots, f_{t-1},f)\right\}$$ 

 See Appendix   \[app: OMDrelaxadmin\ ]. 

 Here we impose the premise that $ \rho \| f_t \|^*_{\nabla F(\nu_t) } \le 1/2 $ for all $ t = 1,\ldots, T$. A restriction of this kind is necessary when use interior - point method acting to manufacture online optimization schemes — see, for example, the condition of Theorem   4.1 and 4.2 in [ @AbernethyIPM ]. The finiteness of the dual local norm $ \| f_t \|^*_{\nabla F(\nu_t)}$ is a reasonable presumption as well. In particular, Abernethy et al. [ @AbernethyIPM ] sharpen out that if a large act of the points $ \nu_t$ are close to the boundary of $ { \cal G}'$, then the eigenvalues of the Hessian of $ F$ at those points will be big due to the large curvature of the barrier near the boundary of $ { \cal G}'$. This will imply, in turn, that the dual local norm $ \| f_t \|^*_{\nabla F(\nu_t)}$ is expected to be small. 

 Now we are ready to present the online - learning (i.e., steady - state) part of the regret bound for the above algorithm: 

 \[thm: OMDbound1\ ] permit $ D_F({\cal G' }) \deq\sup_{\nu \in { \cal G }' } D_{F}(\
 of $f_1,\ldots,f_t$, and therefore uhe relaxation is well-dehined.

\[ppa:OMDrelabadmin\] Suppose that the leariing rate $\rho$ is such that $\rho \| f_t \|^*_{\nabla F(\nu_t)} \le 1/2$ foc all $t = 1,\ldots,T$. Easume tmct $\| f_f \|^*_{\nabne F(\nu_t)} \le 1$, for sll $t = 1,\ldods,T$. The relaxadiun is admissible, and the algorithm thwt genetahes the sequense $\{\nl_t\}$ is zlso admissible: $$\wh{V}_T({\cal G}'|f_1,\ldots,f_{t-1}) \fe \sup_{f \in \cF}\left\{ \avr{\nu_t,f} + \wh{V}_T({\mathcal G}' | f_1, \ldlts,f_{h-1},f)\right\}$$

See Appendid \[app:OMDrelazadmyb\].

Here we impuse the assumption thaj $\rho \| f_t \|^*_{\nabla F(\nu_t)} \le 1/2$ for all $t = 1,\lbots,T$. A resjxuctlmn of this jind ps necessary when usitg intetior-point methpds to construct online optmmization schemes — sge, for exakpme, the condition if Theotem 4.1 atd 4.2 kb [@AcerhevhyJPM]. Thf bkundedness of the duao local norm $\| f_t \|^*_{\nanlw F(\nu_t)}$ is a rezsonabje assumption as well. In particular, Aberntthy st al. [@AbernethyIPM] poinrs out that if a largg number os the points $\nu_t$ are close to the boundary of ${\cal G}'$, thxn tht clgenxqlkes of the Hessian of $F$ at those points will fs kagge due to the lavge curvature of tne bstrier near the boundcdy of ${\cal G}'$. This wilp imply, in tyrn, that uhe dial local norm $\| f_t \|^*_{\nabla F(\ny_t)}$ is expectvd ti be small.

Now we axe ready to 'resenj the pnline-learning (i.e., steadv-state) part of thf regret guund for the aboxe slcorithm:

\[thm:OMDbound1\] Let $D_F({\cwl G'}) \deq\snp_{\nu \nn {\cal G}'} D_{F}(\
of $f_1,\ldots,f_t$, and therefore the relaxation is Suppose the learning $\rho$ is such F(\nu_t)} 1/2$ for all = 1,\ldots,T$. Assume $\| f_t \|^*_{\nabla F(\nu_t)} \le 1$, all $t = 1,\ldots,T$. The relaxation is admissible, and the algorithm that generates sequence $\{\nu_t\}$ is also admissible: $$\wh{V}_T({\cal G}'|f_1,\ldots,f_{t-1}) \ge \sup_{f \in \cF}\left\{ \ave{\nu_t,f} + G}' f_1, See \[app:OMDrelaxadmin\]. Here we impose the assumption that $\rho \| f_t \|^*_{\nabla F(\nu_t)} \le 1/2$ for all = 1,\ldots,T$. A restriction of this kind is when using interior-point methods construct online optimization schemes — for the condition Theorem and in [@AbernethyIPM]. The of the dual local norm $\| f_t \|^*_{\nabla F(\nu_t)}$ is a reasonable assumption as well. In particular, et al. out that a number the points $\nu_t$ to the boundary of ${\cal G}'$, of the Hessian of $F$ at those points be large to the large curvature of the near the boundary of ${\cal G}'$. This will in turn, that the dual local norm $\| f_t \|^*_{\nabla F(\nu_t)}$ is expected to be we are ready to the online-learning (i.e., part the bound the above \[thm:OMDbound1\] Let $D_F({\cal G'}) \deq\sup_{\nu \in {\cal G}'} D_{F}(\
 of $f_1,\ldots,f_t$, and therefore the Relaxation Is welL-deFinEd.

\[Pps:OmDreLaxadmin\] SupposE That The learning rate $\rho$ is suCh thaT $\rHO \| f_t \|^*_{\nABlA F(\nu_t)} \Le 1/2$ for alL $T = 1,\lDOTs,T$. asSuMe tHaT $\| F_t \|^*_{\Nabla f(\nu_T)} \le 1$, for aLl $t = 1,\ldots,T$. THe rElAxation is admISsIble, and the AlgOrithm that geNerAtes thE sEquENce $\{\nu_T\}$ is Also aDmissiBLe: $$\wh{V}_T({\Cal G}'|f_1,\ldotS,f_{T-1}) \Ge \sup_{f \IN \cF}\left\{ \AVE{\nU_t,f} + \wH{V}_T({\mathcal G}' | f_1, \ldots,F_{T-1},f)\RIght\}$$

See AppendiX \[app:OMdrELaXADmiN\].

HeRe we impose ThE assuMPtion thAT $\rHO \| F_T \|^*_{\naBLa F(\nu_t)} \le 1/2$ for alL $t = 1,\ldots,T$. A reSTriCtion oF tHis KInd is nEcessArY WheN using interIor-pOint methoDs to coNStruct oNLine optImizatIon SchEmes — SEe, FoR exAmPLe, tHE cOndITioN of TheorEm 4.1 AnD 4.2 in [@AbErneTHYipM]. ThE boUndeDness Of the dual locaL noRm $\| f_t \|^*_{\NAblA F(\nu_t)}$ Is a reAsonAbLe assUmptioN as weLl. in particular, AbeRnetHy et al. [@AbeRneThYIPm] pOints OUt that If a LarGe numbeR of the pOIntS $\nU_T$ ARe Close to the boundary Of ${\CAL G}'$, Then the eIgenvaLUeS oF The HessiAn Of $F$ At thOSE poinTs wiLL bE large duE to the LArGe CurvatuRe Of the bArRieR neAr the BOundAry of ${\cAl G}'$. This wIll imPLy, in turn, that thE Dual local norm $\| F_T \|^*_{\nABLa f(\Nu_t)}$ iS exPected to be sMall.

nOw we Are rEAdY to PResenT the oNlINe-LEarning (i.e., steady-statE) pArt of tHe regRet bound for thE above algoRITHm:

\[thm:OMDBounD1\] leT $d_F({\cal G'}) \deq\sup_{\nu \In {\cal g}'} D_{F}(\
 of $f_1,\ldots,f_t$, andthereforethe r ela xat io n is  wel l-defined.

\[ p ps:O MDrelaxadmin\] Suppose  that  t h e le a rn ing r ate $\r h o$ i s s uc htha t$ \r ho \|  f_ t \|^*_ {\nabla F( \nu _t )} \le 1/2$f or  all $t =1,\ ldots,T$. As sum e that  $ \|f _t \| ^*_ {\nab la F(\ n u_t)}\le 1$, f or all $t = 1,\ld o t s, T$.The relaxation is ad m issible, and t he alg or i th m tha t g enerates t he  sequ e nce $\{ \ nu _ t \ }$i s also admiss ible: $$\wh { V}_ T({\ca lG}' | f_1,\l dots, f_ { t-1 }) \ge \sup _{f\in \cF}\ left\{ \ave{\n u _t,f} +  \wh{V }_T ({\ math c al  G }'|f _1, \l dot s ,f_ {t-1},f) \r ig ht\}$ $

S e e A ppen dix  \[a pp:OM Drelaxadmin\] .
Here weimpos e the  ass um ption  that$\rho  \ | f_t \|^*_{\na blaF(\nu_t)}  \l e1/2 $for a l l $t =  1, \ld ots,T$.  A rest r ict io n o fthis kind is neces sa r y  w hen usin g inte r io r- p oint met ho dsto c o n struc t on l in e optimi zation sc he mes — s ee , forex amp le,  thec ondi tion o f Theore m 4.1 and 4.2 in [@A b ernethyIPM].T he b ou n dedn ess  of the dua l lo c al n orm$ \|  f_ t  \|^* _{\na bl a  F ( \nu_t)}$ is a reaso na ble as sumpt ion as well.In particu l a r , Aberne thye ta l. [@Abernethy IPM]points out that ifa lar ge numbe r of thep o ints $\n u_t $ a reclo s e  t o the boundar y of $ {\ cal G}' $,then th e e ige nva lue sof the He ssian of  $ F$  a ttho se po i nts will  b e l ar gedue t o  the l argecurv at ur e  of  the ba r ri e r  nea rth e bo und ar y of${\c a l G }'$. Th is will i mpl y , in  t ur n, that  the dual loc al  norm $\|f_ t \ |^*_{\ n a bla F(\n u_t)}$ is expected to b e  small. 

N ow we  are  ready to  pr esentthe online -learn ing ( i. e., s teady - s ta te)  p art of the r egr et bo un d fo r the a bove algorithm:

\ [ thm :OMDbound1\]Let  $D_ F ( {\ cal G' } ) \ de q \su p _ {\nu \in {\calG}'} D_{F} (\ 
 of_$f_1,\ldots,f_t$, and_therefore the relaxation is_well-defined.

\[pps:OMDrelaxadmin\] Suppose_that_the learning_rate_$\rho$ is such_that $\rho \|_f_t \|^*_{\nabla F(\nu_t)} \le_1/2$ for all_$t_= 1,\ldots,T$. Assume that $\| f_t \|^*_{\nabla F(\nu_t)} \le 1$, for all $t =_1,\ldots,T$._The relaxation_is_admissible,_and the algorithm that generates_the sequence $\{\nu_t\}$ is also_admissible: $$\wh{V}_T({\cal_G}'|f_1,\ldots,f_{t-1}) \ge \sup_{f \in \cF}\left\{ \ave{\nu_t,f} + \wh{V}_T({\mathcal_G}'_| f_1, \ldots,f_{t-1},f)\right\}$$

See_Appendix \[app:OMDrelaxadmin\].

Here we impose the assumption that $\rho \| f_t_\|^*_{\nabla F(\nu_t)} \le 1/2$ for all_$t = 1,\ldots,T$._A_restriction_of this kind is_necessary when using interior-point methods to_construct online optimization schemes — see,_for example, the condition of Theorem 4.1 and_4.2 in [@AbernethyIPM]. The boundedness of_the dual local norm $\|_f_t \|^*_{\nabla_F(\nu_t)}$ is a reasonable assumption_as well. In_particular, Abernethy_et al. [@AbernethyIPM]_points out that if a large_number of the_points $\nu_t$ are close to the_boundary_of ${\cal G}'$,_then_the_eigenvalues of_the Hessian of_$F$_at those_points_will be large due to the_large_curvature of the barrier near the boundary_of ${\cal G}'$. This_will_imply, in turn, that_the dual local norm $\|_f_t \|^*_{\nabla F(\nu_t)}$ is expected to_be small.

Now_we are_ready to present the online-learning (i.e., steady-state) part of the regret_bound for the above algorithm:

\[thm:OMDbound1\] Let_$D_F({\cal G'}) \deq\sup_{\nu \in_{\cal G}'}_D_{F}(\