 the data.

A large body of literature has been devoted to studying codes that are minimax optimal with respect to (\[Redundancy\]), exactly [@Topsoe79; @Shtarkov87en2; @Haussler97] or asymptotically [@BarronRissanenYu98; @Grunwald07]. Let us notice that if the minimax expected Shannon redundancy $$\begin{aligned}
  \label{MinimaxRed}
  \min_{C} \sup_{P\in\mathcal{M}} {\textbf{E}\, _}{x\sim P} {\left[ |C(x)|+\log P(x) \right]}\end{aligned}$$ or the minimax regret $$\begin{aligned}
  \label{MinimaxReg}
  \min_{C} \sup_{P\in\mathcal{M}} \max_{x} {\left[ |C(x)|+\log P(x) \right]}\end{aligned}$$ are finite, plausibly bounded in terms of the data length, and achieved by a unique code $C$ then the corresponding minimax properties appear a plausible rationale to argue for code $C$’s optimality against data typical of a class of distributions $\mathcal{M}$.

Things change when (\[MinimaxRed\]) or (\[MinimaxReg\]) are infinite since then every code is a minimizer. Infinite or unbounded minimax values appear in fact in many statistical models: (i) There are no universal redundancy rates for stationary ergodic processes [@Shields93]. (ii) Even in the parametric iid case, like Poisson or geometric, one often has to restrict the parameter range to a compact subset to have a reasonable minimax code [@Grunwald07 Theorem 7.1 and Sections 11.1.1–2]. In a surprising contrast, the redundancy for computable parameters can be very small, which is known as superefficient estimation/compression [@LeCam53; @Vovk91en2; @BarronHengartner98].

The minimax values (\[MinimaxRed\]) or (\[MinimaxReg\]) may be infinite because there is no worst case of data rather than no intuitively good code. Often there exists an intuitively good code but to single it out with the minimax criterion, we have to modify the score (\[Redundancy\])
the data. 

 A   large body of literature has been devoted to study code that are minimax optimal with respect to (\[Redundancy\ ]), exactly [ @Topsoe79; @Shtarkov87en2; @Haussler97 ] or asymptotically [ @BarronRissanenYu98; @Grunwald07 ]. get us notice that if the minimax expected Shannon redundancy $ $ \begin{aligned } 
   \label{MinimaxRed } 
   \min_{C } \sup_{P\in\mathcal{M } } { \textbf{E}\, _ } { x\sim P } { \left [ |C(x)|+\log P(x) \right]}\end{aligned}$$ or the minimax sorrow $ $ \begin{aligned } 
   \label{MinimaxReg } 
   \min_{C } \sup_{P\in\mathcal{M } } \max_{x } { \left [ |C(x)|+\log P(x) \right]}\end{aligned}$$ are finite, plausibly bounded in terms of the datum duration, and achieved by a   singular code $ C$ then the corresponding minimax properties look a   plausible rationale to argue for code $ C$ ’s optimality against data typical of a   class of distributions $ \mathcal{M}$. 

 Things change when (\[MinimaxRed\ ]) or (\[MinimaxReg\ ]) are infinite since then every code is a   minimizer. countless or unbounded minimax values appear in fact in many statistical model: (i) There are no cosmopolitan redundancy rates for stationary ergodic process [ @Shields93 ]. (ii) Even in the parametric iid case, like Poisson or geometric, one much has to restrict the parameter compass to a   compact subset to have a   reasonable minimax code [ @Grunwald07 Theorem 7.1 and Sections 11.1.1–2 ]. In a   surprising contrast, the redundancy for computable parameters can be very small, which is known as superefficient estimation / compression [ @LeCam53; @Vovk91en2; @BarronHengartner98 ]. 

 The minimax values (\[MinimaxRed\ ]) or (\[MinimaxReg\ ]) may be infinite because there is no worst case of data rather than no intuitively good code. frequently there exists an intuitively good code but to single it out with the minimax standard, we take to change the score (\[Redundancy\ ] )
 thf data.

A large body of littrature has been bwvoted to sthdying cudes that are minimax optimap qith eespect to (\[Redundancy\]), dxactly [@Tlpsoe79; @Shrarkic87en2; @Hausslxd97] or asniptoflcallv [@UarronRissanenYo98; @Grunwald07]. Lat us notice tvag nf the minimax expected Shannon redugdancy $$\nehin{aligned}
  \lafel{MpnymaxDvd}
  \min_{C} \sup_{P\in\mathcal{M}} {\textbf{E}\, _}{x\sim P} {\left[ |C(x)|+\log P(c) \right]}\end{aligned}$$ or the mlnimwx regret $$\begin{alihned}
  \label{NiniiqxReg}
  \min_{C} \rup_{P\in\mathbcl{M}} \max_{x} {\lert[ |C(x)|+\log P(x) \right]}\end{aligned}$$ are finice, plausibli goktded in terns of the data lekbth, ang achiefed by a unique cove $C$ then the correspondiig minimax propertief appear d pmausible rationalw ro arcue xor zide $C$’s o'tijality ageinst data fypical of q class of distributoogw $\mathcal{M}$.

Thihgs chwnde when (\[MinimaxRed\]) or (\[MinimaxReg\]) are inxinjte since then every coee is a minimizer. Infijite or ugbounded minimax values appear in fact in many stdtistmcxl noqdos: (i) There are no universal redundancy rates fow suatponary ergodic prjcesses [@Shirlfs93]. (yi) Even in thg paramzfrjc iid case, like Plisson jr geimetric, oge ogten has to restrict the paeameter rangv to a compact subset tl have a rearonanle monimax code [@Grunwald07 Thzorem 7.1 and Sectiojs 11.1.1–2]. In a shfprising contrasg, tme sedundancy for computable [arameterw cau be verh smsll, whych is knoan as superefficient estlmatiln/wompressioj [@LeCam53; @Vovk91en2; @BarronHengartner98].

Vie minimax vakuas (\[KinimaxRzd\]) or (\[KinimaxReg\]) mar be infinite yecause chere ks no worsn case of data rathew than no intgltively good code. Ofeen rherw existr an intuitivelu good cobt but to sungle it out with bhe mkhimax criterion, we have to modify thd ssoge (\[Cedunqdncy\])
the data. A large body of literature devoted studying codes are minimax optimal [@Topsoe79; @Haussler97] or asymptotically @Grunwald07]. Let us that if the minimax expected Shannon $$\begin{aligned} \label{MinimaxRed} \min_{C} \sup_{P\in\mathcal{M}} {\textbf{E}\, _}{x\sim P} {\left[ |C(x)|+\log P(x) \right]}\end{aligned}$$ or the regret $$\begin{aligned} \label{MinimaxReg} \min_{C} \sup_{P\in\mathcal{M}} \max_{x} {\left[ |C(x)|+\log P(x) \right]}\end{aligned}$$ are finite, plausibly in of data and achieved by a unique code $C$ then the corresponding minimax properties appear a plausible rationale argue for code $C$’s optimality against data typical a class of distributions Things change when (\[MinimaxRed\]) or are since then code a Infinite or unbounded values appear in fact in many statistical models: (i) There are no universal redundancy rates for stationary processes [@Shields93]. in the iid like or geometric, one to restrict the parameter range to to have a reasonable minimax code [@Grunwald07 Theorem and Sections In a surprising contrast, the redundancy computable parameters can be very small, which is as superefficient estimation/compression [@LeCam53; @Vovk91en2; @BarronHengartner98]. The minimax values (\[MinimaxRed\]) or (\[MinimaxReg\]) may be infinite is no worst case data rather than intuitively code. there an intuitively code but to single it out with the minimax criterion, we to modify the score (\[Redundancy\])
 the data.

A large body of literaTure has beeN devoTed To sTuDyinG codEs that are minimAX optImal with respect to (\[RedunDancy\]), ExACtly [@tOpSoe79; @ShTarkov87eN2; @haUSSleR97] oR aSymPtOTiCally [@barRonRissAnenYu98; @GrunWalD07]. LEt us notice thAT iF the minimaX exPected ShannoN reDundanCy $$\BegIN{aligNed}
  \Label{minimaXred}
  \min_{c} \sup_{P\in\maThCAl{M}} {\texTBf{E}\, _}{x\sim p} {\LEfT[ |C(x)|+\lOg P(x) \right]}\end{alignED}$$ oR The minimax regrEt $$\begiN{aLIgNED}
  \laBel{minimaxReg}
  \MiN_{C} \sup_{p\In\mathcAL{M}} \MAX_{X} {\leFT[ |C(x)|+\log P(x) \right]}\End{aligned}$$ aRE fiNite, plAuSibLY boundEd in tErMS of The data lengTh, anD achieved By a uniQUe code $C$ THen the cOrrespOndIng MiniMAx PrOpeRtIEs aPPeAr a PLauSible ratIoNaLe to aRgue FOR COde $C$’S opTimaLity aGainst data typIcaL of a CLasS of diStribUtioNs $\MathcAl{M}$.

ThiNgs chAnGe when (\[MinimaxReD\]) or (\[MInimaxReg\]) Are InFinItE sincE Then evEry CodE is a minImizer. INFinItE OR UnBounded minimax valuEs APPeAr in fact In many STaTiSTical modElS: (i) THere ARE no unIverSAl RedundanCy rateS FoR sTationaRy ErgodiC pRocEssEs [@ShiELds93]. (iI) Even iN the paraMetriC Iid case, like PoiSSon or geometriC, OnE OFtEN has To rEstrict the pAramETer rAnge TO a ComPAct suBset tO hAVe A Reasonable minimax coDe [@grunwaLd07 TheOrem 7.1 and SectioNs 11.1.1–2]. In a surprISINg contraSt, thE ReDUndancy for compUtablE parameterS Can be verY smalL, which is Known as suPERefficieNt eStiMatIon/COMpRession [@LeCam53; @VOVK91en2; @BArRonHengArtNer98].

The mIniMax ValUes (\[miNimaxRed\]) oR (\[MinimaxreG\]) mAy Be InfInite BEcause thErE is No WorSt casE Of data RatheR thaN nO iNTuiTively gOOd CODe. OfTeN tHere ExiStS an inTuitIVelY good coDe but to siNglE It ouT wItH the minImax criterion, We Have to modiFy The Score (\[REDUndancy\])
 the data.

A large body o f literatu re ha s b een  d evot ed t o studying cod e s th at are minimax optimal  with  r e spec t  t o (\[ Redunda n cy \ ] ),ex ac tly  [ @ To psoe7 9;@Shtark ov87en2; @ Hau ss ler97] or as y mp totically[@B arronRissane nYu 98; @G ru nwa l d07].  Le t usnotice that i f the min im a x expe c ted Sha n n on  red undancy $$\begin{ a li g ned}
  \label{ Minima xR e d}  \m in_ {C} \sup_{ P\ in\ma t hcal{M} }  { \ t e xtb f {E}\, _}{x\si m P} {\left [  |C (x)|+\ lo g P ( x) \ri ght]} \e n d{a ligned}$$ o r th e minimax  regre t  $$\beg i n{align ed}
 \la bel {Min i ma xR eg} 
\mi n _{ C}\ sup _{P\in\m at hc al{M} } \m a x _ { x} { \le ft[|C(x) |+\log P(x) \ rig ht]} \ end {alig ned}$ $ ar efinit e, pla usibl ybounded in term s of  the data  le ng th,  a nd ac h ievedbya u nique c ode $C$ the nt h e  c orresponding minim ax p ro pertiesappear apl a usible r at ion alet o  argu e fo r  c ode $C$’ s opti m al it y again st  dataty pic alof ac lass  of di stributi ons $ \ mathcal{M}$.
T hings changew he n (\ [ Mini max Red\]) or ( \[Mi n imax Reg\ ] )are infin ite s in c et hen every code is a  m inimiz er. I nfinite or un bounded mi n i m ax value s ap p ea r  in fact in ma ny st atisticalm odels: ( i) Th ere areno univer s a l redund anc y r ate s f o r  s tationary erg o d ic p ro cesses[@S hields9 3].  (i i)Eve nin the pa rametric  i id  c as e,likeP oisson o rgeo me tri c, on e  often  hasto r es tr i ctthe par a me t e r ra ng eto a  co mp act s ubse t  to  have a  reasonab lem inim ax  c ode [@G runwald07 The or em 7.1 and  S ect ions 1 1 . 1.1–2].In a surprising contras t , the r edu ndanc y fo r computa ble  param ete r s canbe ver y sma ll , w h i ch is k no wnas  supereffi c i ent  esti ma tion /compre ssion [@LeCam53; @ V ovk 91en2; @Barro nHe ngar t n er 98] . 
T hemi n ima x values (\[Minim axRed\]) o r( \[ MinimaxReg \ ])ma y be in finitebecau s e there  is no wo rst caseof  dat a rat her than n o intuit ively goo d  code .  O ftenthe re exi st s a n int uitive l y g ood c ode bu tto sin gle i tout with  the minimax criterion,  we ha ve to  mo dify thesco r e ( \[Redunda ncy\ ])
 the_data.

A large body_of literature has been_devoted to_studying_codes that_are_minimax optimal with_respect to (\[Redundancy\]),_exactly [@Topsoe79; @Shtarkov87en2; @Haussler97]_or asymptotically [@BarronRissanenYu98;_@Grunwald07]._Let us notice that if the minimax expected Shannon redundancy $$\begin{aligned}
  \label{MinimaxRed}
 _\min_{C}_\sup_{P\in\mathcal{M}} {\textbf{E}\,__}{x\sim_P}_{\left[ |C(x)|+\log P(x) \right]}\end{aligned}$$ or_the minimax regret $$\begin{aligned}
 _\label{MinimaxReg}
 _\min_{C} \sup_{P\in\mathcal{M}} \max_{x} {\left[ |C(x)|+\log P(x) \right]}\end{aligned}$$ are_finite,_plausibly bounded in_terms of the data length, and achieved by a unique_code $C$ then the corresponding minimax_properties appear a plausible_rationale_to_argue for code $C$’s_optimality against data typical of a class_of distributions $\mathcal{M}$.

Things change when (\[MinimaxRed\])_or (\[MinimaxReg\]) are infinite since then every_code is a minimizer. Infinite or unbounded_minimax values appear in fact_in many_statistical models: (i) There are_no universal redundancy_rates for_stationary ergodic processes_[@Shields93]. (ii) Even in the parametric_iid case, like_Poisson or geometric, one often has_to_restrict the parameter_range_to_a compact subset_to have a reasonable_minimax_code [@Grunwald07_Theorem_7.1 and Sections 11.1.1–2]. In a surprising_contrast,_the redundancy for computable parameters can be_very small, which is_known_as superefficient estimation/compression [@LeCam53;_@Vovk91en2; @BarronHengartner98].

The minimax values (\[MinimaxRed\])_or (\[MinimaxReg\]) may be infinite because_there is_no worst_case of data rather than no intuitively good code. Often there_exists an intuitively good code but_to single it out_with the_minimax_criterion, we have_to_modify the_score (\[Redundancy\])