R}}^d$ generates a sequence of points $\{{\mathbf{x}}_t\}$ satisfying $${\mathbf{x}}_{t+1} = {\mathbf{x}}_t + \eta_{{\mathbf{x}}_1,\dots,{\mathbf{x}}_t} \cdot (\nabla f({\mathbf{x}}_t)+{\boldsymbol{\xi}}_t),$$ where ${\boldsymbol{\xi}}_t$ are some random noise vectors. We further assume that the step sizes $\eta_{{\mathbf{x}}_1,\dots,{\mathbf{x}}_t}$ are deterministic functions of the norms and inner products between ${\mathbf{x}}_1,\dots,{\mathbf{x}}_t, \nabla f({\mathbf{x}}_1)+{\boldsymbol{\xi}}_1,\dots, \nabla f({\mathbf{x}}_t)+{\boldsymbol{\xi}}_t$ and may also depend on the exact second-order information $\nabla^2 f({\mathbf{x}}_1),\dots, \nabla^2 f({\mathbf{x}}_t)$. Then for any $\rho,\Delta,\sigma \in {\mathbb{R}}_{++}$ and $T\in\mathbb{N}$ there exists a function $f:{\mathbb{R}}^{T}\mapsto {\mathbb{R}}$ with $\rho$-Lipschitz Hessian, ${\mathbf{x}}_1\in {\mathbb{R}}^T$, and independent random variables ${\boldsymbol{\xi}}_t$ with ${\mathbb{E}}[{\boldsymbol{\xi}}_t]=0$ and ${\mathbb{E}}[\|{\boldsymbol{\xi}}_t\|^2]=\sigma^2$ such that $\forall t\in[T]$ $$\begin{aligned}
        & f({\mathbf{x}}_1) - f({\mathbf{x}}_t) \overset{\text{a.s.}}{\leq} \Delta, \\
        & \|\nabla f({\mathbf{x}}_t)\|^2 \overset{\text{a.s.}}{=} {\boldsymbol{\gamma}},
    \end{aligned}$$ where ${\boldsymbol{\gamma}}\in {\mathbb{R}}^T$ is a vector that satisfies $$\|{\boldsymbol{\gamma}}\|^2 = \frac{\sigma}{2}\left(\frac{\rho \Delta ^2}{ (T-1)^2}\right)^{1/3}.$$

We proceed as in the proof of [Thm. \[thm:aggregation\_step\]]{}, taking for $G$ the positive value that satisfies $$G^2 

R}}^d$ generates a sequence of points $ \{{\mathbf{x}}_t\}$ satisfying $ $ { \mathbf{x}}_{t+1 } = { \mathbf{x}}_t + \eta_{{\mathbf{x}}_1,\dots,{\mathbf{x}}_t } \cdot (\nabla f({\mathbf{x}}_t)+{\boldsymbol{\xi}}_t),$$ where $ { \boldsymbol{\xi}}_t$ are some random randomness vector. We further assume that the gradation size $ \eta_{{\mathbf{x}}_1,\dots,{\mathbf{x}}_t}$ are deterministic functions of the norms and inside products between $ { \mathbf{x}}_1,\dots,{\mathbf{x}}_t, \nabla f({\mathbf{x}}_1)+{\boldsymbol{\xi}}_1,\dots, \nabla f({\mathbf{x}}_t)+{\boldsymbol{\xi}}_t$ and may also depend on the accurate second - decree information $ \nabla^2 f({\mathbf{x}}_1),\dots, \nabla^2 f({\mathbf{x}}_t)$. Then for any $ \rho,\Delta,\sigma \in { \mathbb{R}}_{++}$ and $ T\in\mathbb{N}$ there exist a function $ f:{\mathbb{R}}^{T}\mapsto { \mathbb{R}}$ with $ \rho$-Lipschitz Hessian, $ { \mathbf{x}}_1\in { \mathbb{R}}^T$, and independent random variable $ { \boldsymbol{\xi}}_t$ with $ { \mathbb{E}}[{\boldsymbol{\xi}}_t]=0 $ and $ { \mathbb{E}}[\|{\boldsymbol{\xi}}_t\|^2]=\sigma^2 $ such that $ \forall t\in[T]$ $ $ \begin{aligned } 
         & f({\mathbf{x}}_1) - f({\mathbf{x}}_t) \overset{\text{a.s.}}{\leq } \Delta, \\ 
         & \|\nabla f({\mathbf{x}}_t)\|^2 \overset{\text{a.s.}}{= } { \boldsymbol{\gamma } }, 
     \end{aligned}$$ where $ { \boldsymbol{\gamma}}\in { \mathbb{R}}^T$ is a vector that meet $ $ \|{\boldsymbol{\gamma}}\|^2 = \frac{\sigma}{2}\left(\frac{\rho \Delta ^2 } { (T-1)^2}\right)^{1/3}.$$ 

 We proceed as in the proof of [ Thm.   \[thm: aggregation\_step\ ] ] { }, taking for $ G$ the positive value that satisfies $ $ G^2
R}}^d$ henerates a sequence of koints $\{{\mathbf{x}}_t\}$ scrisfyiig $${\mathgf{x}}_{t+1} = {\maghbf{x}}_t + \eta_{{\mathbf{x}}_1,\dots,{\mathbf{e}}_t} \ceot (\nqbla f({\mathbf{x}}_t)+{\boldsymbul{\xi}}_t),$$ whege ${\boldsynbol{\ei}}_t$ are some ranvkm noisc vecflrs. Ce further assumg that the sdep sizes $\eta_{{\mdtfby{x}}_1,\dots,{\mathbf{x}}_t}$ are deterministic funstions pf the norms and inntr [rodhbtw between ${\mathbf{x}}_1,\dots,{\mathbf{x}}_t, \nabla h({\mathbf{x}}_1)+{\boldsymnol{\xi}}_1,\dots, \nabla f({\mathbf{x}}_t)+{\blldsjmbol{\xi}}_t$ and may apso depend in trw exact secovd-order information $\nagla^2 f({\mathbf{x}}_1),\dots, \nabla^2 f({\mathbf{x}}_g)$. Theu for any $\ryo,\Eelhd,\sigma \in {\methbb{R}}_{++}$ and $T\in\mathnn{N}$ these exisys a function $n:{\mathub{R}}^{T}\napsto {\mathbb{R}}$ with $\rio$-Lipschitz Hessian, ${\iathbf{x}}_1\in {\mcthbb{R}}^T$, and independebt randmm vdriacoes ${\bomdxyjbol{\xi}}_h$ wmth ${\mathbb{E}}[{\goldsymbol{\xu}}_t]=0$ and ${\mathbb{E}}[\|{\boldsumfil{\xi}}_t\|^2]=\sigma^2$ sudh thae $\sorall t\in[T]$ $$\begin{aligned}
        & f({\mathbx{x}}_1) - f({\mathbf{x}}_t) \overset{\text{a.w.}}{\leq} \Delta, \\
        & \|\nwbla f({\matrbf{x}}_t)\|^2 \overset{\text{a.s.}}{=} {\boldsymbol{\gamma}},
    \end{aligned}$$ wherx ${\colbwymbou{\talma}}\in {\mathbb{R}}^T$ is a vector that satisfies $$\|{\bolqaykbpl{\gamma}}\|^2 = \frac{\slgma}{2}\left(\frac{\rho \Dektw ^2}{ (J-1)^2}\right)^{1/3}.$$

We procegd as iu tge proof of [Thm. \[thm:wggregajion\_stwp\]]{}, taking for $G$ the positive value that watisfies $$G^2 

R}}^d$ generates a sequence of points $\{{\mathbf{x}}_t\}$ = + \eta_{{\mathbf{x}}_1,\dots,{\mathbf{x}}_t} (\nabla f({\mathbf{x}}_t)+{\boldsymbol{\xi}}_t),$$ where vectors. further assume that step sizes $\eta_{{\mathbf{x}}_1,\dots,{\mathbf{x}}_t}$ deterministic functions of the norms and products between ${\mathbf{x}}_1,\dots,{\mathbf{x}}_t, \nabla f({\mathbf{x}}_1)+{\boldsymbol{\xi}}_1,\dots, \nabla f({\mathbf{x}}_t)+{\boldsymbol{\xi}}_t$ and may also depend on the second-order information $\nabla^2 f({\mathbf{x}}_1),\dots, \nabla^2 f({\mathbf{x}}_t)$. Then for any $\rho,\Delta,\sigma \in {\mathbb{R}}_{++}$ and there a $f:{\mathbb{R}}^{T}\mapsto with $\rho$-Lipschitz Hessian, ${\mathbf{x}}_1\in {\mathbb{R}}^T$, and independent random variables ${\boldsymbol{\xi}}_t$ with ${\mathbb{E}}[{\boldsymbol{\xi}}_t]=0$ and ${\mathbb{E}}[\|{\boldsymbol{\xi}}_t\|^2]=\sigma^2$ such that t\in[T]$ $$\begin{aligned} & f({\mathbf{x}}_1) - f({\mathbf{x}}_t) \overset{\text{a.s.}}{\leq} \Delta, & \|\nabla f({\mathbf{x}}_t)\|^2 \overset{\text{a.s.}}{=} \end{aligned}$$ where ${\boldsymbol{\gamma}}\in {\mathbb{R}}^T$ is vector satisfies $$\|{\boldsymbol{\gamma}}\|^2 \frac{\sigma}{2}\left(\frac{\rho ^2}{ We proceed as the proof of [Thm. \[thm:aggregation\_step\]]{}, taking for $G$ the positive value that satisfies $$G^2
R}}^d$ generates a sequence of poiNts $\{{\mathbf{x}}_T\}$ satiSfyIng $${\MaThbf{X}}_{t+1} = {\maThbf{x}}_t + \eta_{{\mathbF{X}}_1,\dotS,{\mathbf{x}}_t} \cdot (\nabla f({\matHbf{x}}_t)+{\BoLDsymBOl{\Xi}}_t),$$ whEre ${\boldSYmBOL{\xi}}_T$ aRe SomE rANdOm noiSe vEctors. WE further asSumE tHat the step siZEs $\Eta_{{\mathbf{x}}_1,\DotS,{\mathbf{x}}_t}$ are DetErminiStIc fUNctioNs oF the nOrms anD Inner pRoducts beTwEEn ${\mathBF{x}}_1,\dots,{\mATHbF{x}}_t, \nAbla f({\mathbf{x}}_1)+{\boldsYMbOL{\xi}}_1,\dots, \nabla f({\mAthbf{x}}_T)+{\bOLdSYMboL{\xi}}_T$ and may alsO dEpend ON the exaCT sECONd-oRDer informatioN $\nabla^2 f({\mathBF{x}}_1),\dOts, \nabLa^2 F({\maTHbf{x}}_t)$. THen foR aNY $\rhO,\Delta,\sigma \In {\maThbb{R}}_{++}$ and $T\In\mathBB{N}$ there EXists a fUnctioN $f:{\mAthBb{R}}^{T}\MApStO {\maThBB{R}}$ wITh $\Rho$-lIpsChitz HesSiAn, ${\MathbF{x}}_1\in {\MATHBb{R}}^T$, And IndePendeNt random variaBleS ${\bolDSymBol{\xi}}_T$ with ${\MathBb{e}}[{\boldSymbol{\Xi}}_t]=0$ anD ${\mAthbb{E}}[\|{\boldsymboL{\xi}}_t\|^2]=\Sigma^2$ such ThaT $\fOraLl T\in[T]$ $$\bEGin{aliGneD}
        & f({\mAthbf{x}}_1) - f({\Mathbf{x}}_T) \OveRsET{\TExT{a.s.}}{\leq} \Delta, \\
        & \|\nabla f({\mAtHBF{x}}_T)\|^2 \overset{\Text{a.s.}}{=} {\BOlDsYMbol{\gammA}},
    \eNd{aLignED}$$ Where ${\BoldSYmBol{\gamma}}\In {\mathBB{R}}^t$ iS a vectoR tHat satIsFieS $$\|{\boLdsymBOl{\gaMma}}\|^2 = \fraC{\sigma}{2}\leFt(\fraC{\Rho \Delta ^2}{ (T-1)^2}\right)^{1/3}.$$

wE proceed as in tHE pROOf OF [Thm. \[Thm:Aggregation\_Step\]]{}, TAkinG for $g$ ThE poSItive Value ThAT sATisfies $$G^2 

R}}^d$ generates a sequenc e of point s $\{ {\m ath bf {x}} _t\} $ satisfying $ $ {\ma thbf{x}}_{t+1} = {\mat hbf{x }} _ t +\ et a_{{\ mathbf{ x }} _ 1 ,\d ot s, {\m at h bf {x}}_ t}\cdot ( \nabla f({ \ma th bf{x}}_t)+{\ b ol dsymbol{\x i}} _t),$$ where  ${ \bolds ym bol { \xi}} _t$  aresome r a ndom n oise vect or s . We f u rther a s s um e th at the step sizes $\ e ta_{{\mathbf{x }}_1,\ do t s, { \ mat hbf {x}}_t}$ a re  dete r ministi c  f u n c tio n s of the norm s and inner pro ductsbe twe e n ${\m athbf {x } }_1 ,\dots,{\ma thbf {x}}_t, \ nablaf ({\math b f{x}}_1 )+{\bo lds ymb ol{\ x i} }_ 1,\ do t s,\ na bla f({ \mathbf{ x} }_ t)+{\ bold s y m b ol{\ xi} }_t$  andmay also depe ndon t h e e xactsecon d-or de r inf ormati on $\ na bla^2 f({\mathb f{x} }_1),\dot s,\n abl a^ 2 f({ \ mathbf {x} }_t )$. The n for a n y $ \r h o , \D elta,\sigma \in {\ ma t h bb {R}}_{++ }$ and $T \i n \mathbb{ N} $ t here e xists  a f u nc tion $f: {\math b b{ R} }^{T}\m ap sto {\ ma thb b{R }}$ w i th $ \rho$- Lipschit z Hes s ian, ${\mathbf { x}}_1\in {\ma t hb b { R} } ^T$,  an d independe nt r a ndom  var i ab les ${\bo ldsym bo l {\ x i}}_t$ with ${\math bb {E}}[{ \bold symbol{\xi}}_ t]=0$ and$ { \ mathbb{E }}[\ | {\ b oldsymbol{\xi} }_t\| ^2]=\sigma ^ 2$ suchthat$\forall  t\in[T]$ $ $\begin{ ali gne d}    & f({\mathbf { x }}_1 )- f({\m ath bf{x}}_ t)\ov ers et{ \t ext{a.s.} }{\leq}\D el ta ,\\      & \|\n ab laf( {\m athbf { x}}_t) \|^2\ove rs et { \te xt{a.s. } }{ = }  {\b ol ds ymbo l{\ ga mma}} ,
  \e nd{alig ned}$$ wh ere ${\b ol ds ymbol{\ gamma}}\in {\ ma thbb{R}}^T $isa vect o r  that sa tisfies $$\|{\boldsymbo l {\gamma }}\ |^2 =  \fr ac{\sigma }{2 }\left (\f r ac{\rh o \Del ta ^2 }{  (T - 1 )^2}\ r i gh t)^ {1 /3}.$$

We p roc eed a sin t he proo f of [Thm. \[thm:a g gre gation\_step\ ]]{ }, t a k in g f o r$ G$th e  po s i tive value that  satisfies  $ $ G^ 2 

R}}^d$ generates_a sequence_of points $\{{\mathbf{x}}_t\}$ satisfying_$${\mathbf{x}}_{t+1} =_{\mathbf{x}}_t_+ \eta_{{\mathbf{x}}_1,\dots,{\mathbf{x}}_t}_\cdot_(\nabla f({\mathbf{x}}_t)+{\boldsymbol{\xi}}_t),$$ where_${\boldsymbol{\xi}}_t$ are some_random noise vectors. We_further assume that_the_step sizes $\eta_{{\mathbf{x}}_1,\dots,{\mathbf{x}}_t}$ are deterministic functions of the norms and inner products between ${\mathbf{x}}_1,\dots,{\mathbf{x}}_t,_\nabla_f({\mathbf{x}}_1)+{\boldsymbol{\xi}}_1,\dots, \nabla_f({\mathbf{x}}_t)+{\boldsymbol{\xi}}_t$_and_may also depend on the_exact second-order information $\nabla^2 f({\mathbf{x}}_1),\dots,_\nabla^2 f({\mathbf{x}}_t)$._Then for any $\rho,\Delta,\sigma \in {\mathbb{R}}_{++}$ and $T\in\mathbb{N}$_there_exists a function_$f:{\mathbb{R}}^{T}\mapsto {\mathbb{R}}$ with $\rho$-Lipschitz Hessian, ${\mathbf{x}}_1\in {\mathbb{R}}^T$, and independent_random variables ${\boldsymbol{\xi}}_t$ with ${\mathbb{E}}[{\boldsymbol{\xi}}_t]=0$ and_${\mathbb{E}}[\|{\boldsymbol{\xi}}_t\|^2]=\sigma^2$ such that_$\forall_t\in[T]$_$$\begin{aligned}
   _    & f({\mathbf{x}}_1)_- f({\mathbf{x}}_t) \overset{\text{a.s.}}{\leq} \Delta, \\
 _      &_\|\nabla f({\mathbf{x}}_t)\|^2 \overset{\text{a.s.}}{=} {\boldsymbol{\gamma}},
  _ \end{aligned}$$ where ${\boldsymbol{\gamma}}\in {\mathbb{R}}^T$_is a_vector that satisfies $$\|{\boldsymbol{\gamma}}\|^2 =_\frac{\sigma}{2}\left(\frac{\rho \Delta ^2}{_(T-1)^2}\right)^{1/3}.$$

We proceed_as in the_proof of [Thm. \[thm:aggregation\_step\]]{}, taking for $G$_the positive value_that satisfies $$G^2 
