{x}} - {\mathbf{x}}_0 } \right\|_2^2 $ or its corresponding PSNR. Accordingly, it is a desired property that our iterative methods will provide increment in the PSNR along the iterations and, indeed, Figures \[fig:empirical\_analysis\_psnr\_evolution\_deblurring\_shift1\_cameraman\] and \[fig:empirical\_analysis\_psnr\_evolution\_deblurring\_shift3\_cameraman\] show that this is achievable for appropriate parameter settings (the use of improper parameters may lead to unwanted decrease of the PSNR starting at some unknown iteration that, however, can be detected in many cases by heuristic divergence rules based on the dual variables used in the ADMM process). Interestingly, for some parameter settings, the PSNR may increase with the iterations, whereas the fundamental restoration cost will not necessarily consistently decrease. The last behavior may result from the fact the our optimization problem (with respect to a standard compression technique) is discrete, non-linear, and usually not convex and, therefore, the convergence guarantees of the ADMM [@boyd2011distributed] do not hold here for the fundamental restoration cost.

### The Optimal Compression Parameter

Another question of practical importance is the value of the parameter $ \theta $, determining the compression level of the standard technique utilized in the proposed Algorithms \[Algorithm:Proposed Method Non-overlapping\]-\[Algorithm:Proposed Method Overlapping Blocks with Robust Dual Variables\]. Recall that the ADMM-based developments in Section \[sec:Proposed Methods\] led to an iterative procedure including a stage of Lagrangian rate-distortion optimization operated for a Lagrange multiplier $ \lambda \triangleq \frac{2\mu}{\beta} $ and, then, we replaced this optimization with application of a standard compression-decompression technique operated based on a parameter $ \theta $. It is clear that $ \theta $ is a function of $ \lambda $. In the particular case where the standard compression has the Lagrangian form from our developments, then $ \theta = \lambda $, however, this is not the general case. For an arbitrary compression technique, we assume that its parameter $ \theta $ has $ K $ possible values $ \theta_1,...,\theta_K $ (for example, the HEVC standard supports 52 values for its quantization parameter), then, for a given $ \lambda \triangleq \frac{2\mu}{\beta} $
{ x } } - { \mathbf{x}}_0 } \right\|_2 ^ 2 $ or its corresponding PSNR. Accordingly, it is a desired place that our iterative method will provide increment in the PSNR along the iteration and, indeed, human body \[fig: empirical\_analysis\_psnr\_evolution\_deblurring\_shift1\_cameraman\ ] and \[fig: empirical\_analysis\_psnr\_evolution\_deblurring\_shift3\_cameraman\ ] show that this is achievable for appropriate argument settings (the use of improper parameters may moderate to unwanted decrease of the PSNR starting at some strange iteration that, however, can be detected in many cases by heuristic deviation rules based on the dual variables used in the ADMM procedure). Interestingly, for some parameter settings, the PSNR may increase with the iterations, whereas the cardinal restoration cost will not inevitably consistently decrease. The final behavior may leave from the fact the our optimization problem (with respect to a standard compression technique) is discrete, non - analogue, and usually not convex and, therefore, the convergence guarantees of the ADMM [ @boyd2011distributed ] do not hold here for the fundamental restoration cost. 

 # # # The Optimal Compression Parameter 

 Another question of practical importance is the value of the parameter $ \theta $, determining the compression level of the standard technique utilized in the proposed Algorithms \[Algorithm: propose Method Non - overlapping\]-\[Algorithm: Proposed Method Overlapping Blocks with Robust Dual Variables\ ]. echo that the ADMM - based developments in Section \[sec: Proposed Methods\ ] led to an iterative procedure include a stage of Lagrangian rate - distortion optimization operated for a Lagrange multiplier $ \lambda \triangleq \frac{2\mu}{\beta } $ and, then, we replace this optimization with application of a standard compression - decompression technique operated based on a parameter $ \theta $. It is well-defined that $ \theta $ is a function of $ \lambda $. In the particular case where the standard compression receive the Lagrangian form from our growth, then $ \theta = \lambda $, however, this is not the cosmopolitan case. For an arbitrary compression technique, we assume that its parameter $ \theta $ has $ K $ possible values $ \theta_1,... ,\theta_K $ (for case, the HEVC standard supports 52 values for its quantization parameter), then, for a give $ \lambda \triangleq \frac{2\mu}{\beta } $
{x}} - {\lathbf{x}}_0 } \right\|_2^2 $ or its curresponding PSUE. Accocdingly, it is a desired property that our iverarive nethods will provide ivcrement pn the PSBR aoing the itxdations and, ihfeed, Higures \[fig:empitical\_analysiv\_psnr\_evolution\_geclbrring\_shift1\_cameraman\] and \[fig:empiricaj\_analysos\_osnr\_evolution\_dgblurgigg\_shjft3\_cameraman\] show that this is achjevable for appropriste parameter settings (the use of improper paramfters may lgzd ei unwanted ddcrease of the PSNR stzrting at some unknown iteratiov thac, however, cqn be getected in many cases by hemgistic givergemce rules basec oi thw dual variables used in the ADMM procefs). Interevtnngly, for some parameree setjings, the PSNF mzy ihcreasf wmth the itedations, wheeeas the fundamentak wvxtoration coat wilj got necessarily consistently decrease. Tve mast behavior may resulr from the fact the oor optimizwtion problem (with respect to a standard compresshon txcfniwuc) is eidcrete, non-linear, and usually not convex and, trsrtfoge, the convergencc guarantees of thr WDKI [@boyd2011distriboted] do nof hold here for thf fundaientao restorauion vost.

### The Optimal Compressiob Parameter

Auotyer question of prcctical impoxtance is tne value of the parametzr $ \thsta $, determlning the zompression leveu on tve standard technique utilyzed in tie prpposed Xlgotithms \[Wlgorithm:Pgoposcg Method Non-overlaoping\]-\[Clgorhthm:Proposfd Method Overlapping Blocks wivi Robust Dual Vdridbles\]. Reeall tmat the ADMM-basqd developmentx in Seetion \[rec:Proposes Methovs\] led to an iterative prmfedure incluving a stwge if Lqgrangixv rate-distortipn optimidauion operared for a Lagrange muufiplier $ \lambda \truangleq \frac{2\mu}{\brta} $ agd, tien, wq replaced thhs ootioozatiun with appokcatoon of a standard cokpreasion-decompression tcchnique iperated based on a psrameter $ \theta $. Iu is cneac that $ \treta $ is a function of $ \lambda $. In the oarbicular case rherc thg standard eompression has the Lagrangian form from our developments, rhen $ \theta = \lambda $, however, thms is got the ganeral case. For an aebitrary compresslon technique, we assums that its oarameter $ \theta $ has $ K $ possible values $ \theta_1,...,\theta_K $ (for example, the HWVC stenqard supporfs 52 faluev yor its qtantmzetion parameter), tnen, for a given $ \lambda \trianglxq \frac{2\mu}{\batc} $
{x}} - {\mathbf{x}}_0 } \right\|_2^2 $ or PSNR. it is desired property that increment the PSNR along iterations and, indeed, \[fig:empirical\_analysis\_psnr\_evolution\_deblurring\_shift1\_cameraman\] and \[fig:empirical\_analysis\_psnr\_evolution\_deblurring\_shift3\_cameraman\] show that this achievable for appropriate parameter settings (the use of improper parameters may lead to decrease of the PSNR starting at some unknown iteration that, however, can be in cases heuristic rules based on the dual variables used in the ADMM process). Interestingly, for some parameter settings, PSNR may increase with the iterations, whereas the restoration cost will not consistently decrease. The last behavior result the fact our problem respect to a compression technique) is discrete, non-linear, and usually not convex and, therefore, the convergence guarantees of the ADMM do not for the restoration ### Optimal Compression Parameter of practical importance is the value $ \theta $, determining the compression level of standard technique in the proposed Algorithms \[Algorithm:Proposed Method Method Overlapping Blocks with Robust Dual Variables\]. Recall the ADMM-based developments in Section \[sec:Proposed Methods\] led to an iterative procedure including a stage rate-distortion optimization operated for Lagrange multiplier $ \triangleq $ then, replaced this with application of a standard compression-decompression technique operated based on a $ \theta $. It is clear that $ \theta $ function $ \lambda $. the particular case where standard has the Lagrangian form developments, $ $, this not the general case. an arbitrary compression technique, we that its parameter $ $ possible values $ \theta_1,...,\theta_K $ (for example, HEVC standard supports 52 values for its parameter), then, for a given $ \lambda \triangleq \frac{2\mu}{\beta} $
{x}} - {\mathbf{x}}_0 } \right\|_2^2 $ or its correspOnding PSNR. accorDinGly, It Is a dEsirEd property that OUr itErative methods will provIde inCrEMent IN tHe PSNr along tHE iTERatIoNs And, InDEeD, FiguRes \[Fig:empiRical\_analySis\_PsNr\_evolution\_dEBlUrring\_shifT1\_caMeraman\] and \[fiG:emPiricaL\_aNalYSis\_psNr\_eVolutIon\_debLUrring\_Shift3\_cameRaMAn\] show THat this IS AcHievAble for appropriatE PaRAmeter settings (The use Of IMpROPer ParAmeters may LeAd to uNWanted dECrEASE of THe PSNR startinG at some unknOWn iTeratiOn ThaT, HoweveR, can bE dETecTed in many caSes bY heuristiC diverGEnce rulES based oN the duAl vAriAbleS UsEd In tHe adMM PRoCesS). intErestingLy, FoR some ParaMETER setTinGs, thE PSNR May increase wiTh tHe itERatIons, wHereaS the FuNdameNtal reStoraTiOn cost will not neCessArily consIstEnTly DeCreasE. the lasT beHavIor may rEsult frOM thE fACT ThE our optimization prObLEM (wIth respeCt to a sTAnDaRD compresSiOn tEchnIQUe) is dIscrETe, Non-lineaR, and usUAlLy Not convEx And, theReForE, thE convERgenCe guarAntees of The ADmm [@boyd2011distributED] do not hold herE FoR THe FUndaMenTal restoratIon cOSt.

### ThE OptIMaL CoMPressIon PaRaMEtER

Another question of pRaCtical ImporTance is the valUe of the parAMETer $ \theta $, DeteRMiNIng the compressIon leVel of the stANdard tecHniquE utilizeD in the proPOSed AlgorIthMs \[ALgoRitHM:prOposed Method NON-OverLaPping\]-\[AlGorIthm:ProPosEd MEthOd OVeRlapping BLocks witH RObUsT DUal variaBLes\]. RecalL tHat ThE ADmM-basED develOpmenTs in seCtIOn \[sEc:PropoSEd mEThodS\] lEd To an IteRaTive pRoceDUre IncludiNg a stage oF LaGRangIaN rAte-distOrtion optimizAtIon operateD fOr a lagranGE MultipliEr $ \lambda \triangleq \frac{2\mu}{\BEta} $ and, tHen, We repLaceD this optiMizAtion wIth APplicaTion of A stanDaRd cOMPressION-dEcoMpRession tecHNIquE operAtEd baSed on a pArameter $ \theta $. It is cLEar That $ \theta $ is a fUncTion OF $ \LaMbdA $. in THe pArTIcuLAR case where the stAndard compReSSiOn has the LaGRanGiAn form fRom our dEveloPMents, thEn $ \theta = \laMbda $, howevEr, This IS Not The general Case. For aN arbitrarY ComprESsIon teChnIque, we AsSumE that Its parAMetEr $ \theTa $ has $ K $ PoSsible ValueS $ \tHeta_1,...,\thetA_K $ (for example, the HEVC stanDard suPportS 52 vaLues for itS quANtiZation parAmetEr), then, for a GivEn $ \lAmbda \TriANgleq \Frac{2\MU}{\bEta} $
{x}} - {\mathbf{x}}_0 } \r ight\|_2^2  $ or  it s c or resp ondi ng PSNR. Accor d ingl y, it is a desired pro perty  t h at o u ritera tive me t ho d s  wi ll  p rov id e  i ncrem ent  in the  PSNR alon g t he  iterationsa nd , indeed,Fig ures \[fig:e mpi rical\ _a nal y sis\_ psn r\_ev olutio n \_debl urring\_s hi f t1\_ca m eraman\ ] an d \[ fig:empirical\_an a ly s is\_psnr\_evol ution\ _d e bl u r rin g\_ shift3\_ca me raman \ ] showt ha t t his is achievable  for approp r iat e para me ter settin gs (t he use  of imprope r pa rametersmay le a d to un w anted d ecreas e o f t he P S NR  s tar ti n g a t  s ome unk nown ite ra ti on th at,h o w e ver,  ca n be  dete cted in manycas es b y  he urist ic di verg en ce ru les ba sed o nthe dual variab lesused in t heAD MMpr ocess ) . Inte res tin gly, fo r somep ara me t e r  s ettings, the PSNRma y in crease w ith th e  i te r ations,wh ere as t h e  fund amen t al  restora tion c o st  w ill not  n ecessa ri lycon siste n tlydecrea se. Thelastb ehavior may re s ult from thef ac t th e  our  op timizationprob l em ( with re spe c t toa sta nd a rd compression techniq ue ) is d iscre te, non-linea r, and usu a l l y not co nvex an d , therefore, t he co nvergenceg uarantee s ofthe ADMM  [@boyd20 1 1 distribu ted ] d o n oth o ld  here for the f unda me ntal re sto rationcos t.
## # T he  OptimalCompress io nPa ra met er

A n other qu es tio nofpract i cal im porta nceis  t h e v alue of th e para me te r $\th et a $,dete r min ing the  compress ion leve lof  the st andard techni qu e utilized  i n t he pro p o sed Algo rithms \[Algorithm:Prop o sed Met hod  Non- over lapping\] -\[ Algori thm : Propos ed Met hod O ve rla p p ing B l o ck s w it h Robust D u a l V ariab le s\].  Recall  that the ADMM-bas e d d evelopments i n S ecti o n  \ [se c :P r opo se d  Me t h ods\] led to an  iterative  p r oc edure incl u din ga stage  of Lag rangi a n rate- distortio n optimiz at iono p era ted for aLagrange  multipli e r $ \ l am bda \ tri angleq  \ fra c{2\m u}{\be t a}$ and , then ,we rep laced  t his opti mization with applicati on ofa sta nda rd compre ssi o n-d ecompress iontechniqueope rat ed ba sed on apara m et er$  \the ta $ .  It is cl e ar  th a t  $  \theta $ i s a  fu nctio n o f  $ \la mbda  $. In the partic u lar case where  the s tan dar d  com pr ession has the  La gr a n gian for mfrom our de velopmen ts ,  then  $ \th eta =\lambda $ ,h owever , th isis not th e g en e ral cas e.  F o r an a rbit ra ry com pressi o n te c h nique, we assume  that i ts pa r ame ter $  \ theta $ has$ K $ poss ible values  $ \th eta_ 1,... ,\theta _K  $ (fo r e xa mple, theH EVC stand ard s upports  5 2 va lue s foritsq u antiz atio npar ameter),t h en ,  f or a g iven  $ \l am bda\triangle q  \frac{2 \mu } {\beta}  $ 
{x}} -_{\mathbf{x}}_0 }_\right\|_2^2 $ or its_corresponding PSNR._Accordingly,_it is_a_desired property that_our iterative methods_will provide increment in_the PSNR along_the_iterations and, indeed, Figures \[fig:empirical\_analysis\_psnr\_evolution\_deblurring\_shift1\_cameraman\] and \[fig:empirical\_analysis\_psnr\_evolution\_deblurring\_shift3\_cameraman\] show that this is achievable for appropriate_parameter_settings (the_use_of_improper parameters may lead to_unwanted decrease of the PSNR_starting at_some unknown iteration that, however, can be detected_in_many cases by_heuristic divergence rules based on the dual variables used_in the ADMM process). Interestingly, for_some parameter settings,_the_PSNR_may increase with the_iterations, whereas the fundamental restoration cost_will not necessarily consistently decrease. The_last behavior may result from the fact_the our optimization problem (with respect_to a standard compression technique)_is discrete,_non-linear, and usually not convex_and, therefore, the_convergence guarantees_of the ADMM_[@boyd2011distributed] do not hold here for_the fundamental restoration_cost.

### The Optimal Compression Parameter

Another question_of_practical importance is_the_value_of the_parameter $ \theta_$,_determining the_compression_level of the standard technique utilized_in_the proposed Algorithms \[Algorithm:Proposed Method Non-overlapping\]-\[Algorithm:Proposed Method_Overlapping Blocks with Robust_Dual_Variables\]. Recall that the_ADMM-based developments in Section \[sec:Proposed_Methods\] led to an iterative procedure_including a_stage of_Lagrangian rate-distortion optimization operated for a Lagrange multiplier $ \lambda \triangleq_\frac{2\mu}{\beta} $ and, then, we replaced_this optimization with application_of a_standard_compression-decompression technique operated_based_on a_parameter $ \theta $. It is clear_that $_\theta $ is a function of_$ \lambda $. In_the_particular case where the standard compression_has the Lagrangian form from our_developments, then $ \theta =_\lambda_$,_however, this is not the_general case. For an arbitrary compression_technique, we assume_that its parameter $ \theta $ has_$_K $ possible values $ \theta_1,...,\theta_K_$_(for example, the HEVC standard supports_52_values_for its quantization parameter), then,_for a given $ \lambda \triangleq_\frac{2\mu}{\beta} $