 robust PCA.

Convergence of ILT {#S:ILT_convergece}
==================

Here we establish Proposition \[thm:ilt\_convergence\]. We first define a surrogate function for $f(\bx)$ in (\[eqn:log\_reg\]): $$\begin{aligned}
\label{eqn:surrogate}
\nonumber Q(\bx, \bz) = \Vert \by - A \bx \Vert_2^2 + \lambda \sum_i \log(\delta + | x_i|)+ \\
\Vert \bx - \bz \Vert_2^2 - \Vert A (\bx - \bz) \Vert_2^2\end{aligned}$$ Note that $Q(\bx, \bx) = f(\bx)$. Simplifying (\[eqn:surrogate\]) we have $$\label{funcQ}
 Q(\bx, \bz) =
   \sum_i \left( x_i - k_i(\bz))^2 + \lambda \log( x_i + |\delta|) \right)+ K(\bz),$$ where $k_i(\bz) =  z_i + a_i^T \by - a_i^T A \bz$ and $K(\bz)$ contains terms independent of $\bx$. The optimization over $\bx$ is now separable, i.e. can be done independently for each coordinate. We can see that finding local minima over $\bx$ of $Q(\bx, \bz)$ corresponds to iterative log-thresholding.

![Illustration of singular-value log-thresholding.[]{data-label="fig:results_svlt"}](figures/SV_thresh.pdf){width="9.5cm"}

Using this motivation for ILT, we can now prove convergence to fixed points of $f(\bx)$. First, we have:

$f(\hbx^n) = Q(\hbx^{n}, \hbx^n)$ and $Q(\hbx^{n+1}, \hbx^{n})$, are monotonically decreasing with iterations $n$ as long as the spectral norm $\Vert A \Vert_2 < 1$.

The proof parallels the IHT proof of [@blumensath2008iterative] using the fact that $Q(\bx^{n+1}, \bx^n) = f(\bx^n) + \|\bx^{n+1
robust PCA. 

 Convergence of ILT { # S: ILT_convergece } 
 = = = = = = = = = = = = = = = = = = 

 Here we establish Proposition \[thm: ilt\_convergence\ ]. We first specify a foster function for $ f(\bx)$ in (\[eqn: log\_reg\ ] ): $ $ \begin{aligned } 
 \label{eqn: surrogate } 
 \nonumber Q(\bx, \bz) = \Vert \by - A \bx \Vert_2 ^ 2 + \lambda \sum_i \log(\delta + | x_i|)+ \\ 
 \Vert \bx - \bz \Vert_2 ^ 2 - \Vert A (\bx - \bz) \Vert_2 ^ 2\end{aligned}$$ bill that $ Q(\bx, \bx) = f(\bx)$. Simplifying (\[eqn: surrogate\ ]) we own $ $ \label{funcQ } 
  Q(\bx, \bz) = 
    \sum_i \left (x_i - k_i(\bz))^2 + \lambda \log (x_i + |\delta|) \right)+ K(\bz),$$ where $ k_i(\bz) =   z_i + a_i^T \by - a_i^T A \bz$ and $ K(\bz)$ contains terms autonomous of $ \bx$. The optimization over $ \bx$ is now separable, i.e. can be done independently for each coordinate. We can attend that finding local minima over $ \bx$ of $ Q(\bx, \bz)$ corresponds to iterative log - thresholding. 

! [ example of singular - value log - thresholding.[]{data - label="fig: results_svlt"}](figures / SV_thresh.pdf){width="9.5 cm " } 

 use this motivation for ILT, we can nowadays prove convergence to fixed points of $ f(\bx)$. First, we have: 

 $ f(\hbx^n) = Q(\hbx^{n }, \hbx^n)$ and $ Q(\hbx^{n+1 }, \hbx^{n})$, are monotonically decrease with iterations $ n$ as long as the spectral average $ \Vert A \Vert_2 < 1$. 

 The proof parallels the IHT validation of [ @blumensath2008iterative ] using the fact that $ Q(\bx^{n+1 }, \bx^n) = f(\bx^n) + \|\bx^{n+1
 rohust PCA.

Convergence of IUT {#S:ILT_convergeew}
==================

Here xe estaglish Pruposition \[thm:ilt\_convergence\]. Xe furst eefine a surrogate funztion for $f(\bx)$ in (\[wqn:lit\_reg\]): $$\begin{emigned}
\label{eqh:durrmjate}
\nonumber Q(\bw, \bz) = \Vert \ty - A \bx \Vert_2^2 + \lxmyda \sum_i \log(\delta + | x_i|)+ \\
\Vert \bx - \bz \Vqrt_2^2 - \Vett A (\bx - \bz) \Vert_2^2\gnd{alpgged}$$ Hote that $Q(\bx, \bx) = f(\bx)$. Simplifying (\[sqn:surrmgate\]) we have $$\label{funcQ}
 Q(\bx, \bz) =
   \sum_i \lefh( x_i - k_i(\bz))^2 + \lambda \log( x_i + |\deota|) \wught)+ K(\bz),$$ whefe $k_i(\bz) =  e_i + a_i^T \by - z_i^T A \bz$ and $K(\bz)$ contains terms inde'endent of $\vx$. Thf optimizatiin ovvr $\bx$ is now separabla, i.e. cam be done indeiendeitly for each coordinate. Xe can see that findyng local mnnima over $\bx$ of $Q(\bx, \vz)$ corrgspongs tu itdrauivx lkg-thredhomding.

![Illusfration of wingular-value log-thtefyolding.[]{data-lagel="fig:wefults_svlt"}](figures/SV_thresh.pdf){width="9.5cm"}

Using thjs motivation for ILT, ww can now prove convetgence to sixed points of $f(\bx)$. First, we have:

$f(\hbx^n) = Q(\hbx^{n}, \hbx^t)$ and $D(\hbr^{k+1}, \fvx^{j})$, are monotonically decreasing with iterationf $n$ ax long as the fpectral notm $\Vrtt A \Vert_2 < 1$.

The proof padallels the IHT prlof of [@flumebsath2008iterwtivr] using the fact that $Q(\bx^{n+1}, \vx^n) = f(\bx^n) + \|\bq^{n+1
robust PCA. Convergence of ILT {#S:ILT_convergece} ================== establish \[thm:ilt\_convergence\]. We define a surrogate $$\begin{aligned} \nonumber Q(\bx, \bz) \Vert \by - \bx \Vert_2^2 + \lambda \sum_i \log(\delta | x_i|)+ \\ \Vert \bx - \bz \Vert_2^2 - \Vert A (\bx - \Vert_2^2\end{aligned}$$ Note that $Q(\bx, \bx) = f(\bx)$. Simplifying (\[eqn:surrogate\]) we have $$\label{funcQ} Q(\bx, = \left( - + \lambda \log( x_i + |\delta|) \right)+ K(\bz),$$ where $k_i(\bz) = z_i + a_i^T \by - A \bz$ and $K(\bz)$ contains terms independent of The optimization over $\bx$ now separable, i.e. can be independently each coordinate. can that local minima over of $Q(\bx, \bz)$ corresponds to iterative log-thresholding. ![Illustration of singular-value log-thresholding.[]{data-label="fig:results_svlt"}](figures/SV_thresh.pdf){width="9.5cm"} Using this motivation for ILT, we now prove fixed points $f(\bx)$. we $f(\hbx^n) = Q(\hbx^{n}, $Q(\hbx^{n+1}, \hbx^{n})$, are monotonically decreasing with long as the spectral norm $\Vert A \Vert_2 1$. The parallels the IHT proof of [@blumensath2008iterative] the fact that $Q(\bx^{n+1}, \bx^n) = f(\bx^n) +
 robust PCA.

Convergence of ILT {#s:ILT_converGece}
==================

HEre We eStAbliSh PrOposition \[thm:ilT\_ConvErgence\]. We first define a sUrrogAtE FuncTIoN for $f(\Bx)$ in (\[eqn:LOg\_REG\]): $$\beGiN{aLigNeD}
\LaBel{eqN:suRrogate}
\Nonumber Q(\bX, \bz) = \veRt \by - A \bx \Vert_2^2 + \lAMbDa \sum_i \log(\dEltA + | x_i|)+ \\
\Vert \bx - \bz \VErt_2^2 - \vert A (\bX - \bZ) \VeRT_2^2\end{aLigNed}$$ NoTe that $q(\Bx, \bx) = f(\bX)$. SimplifyInG (\[Eqn:surROgate\]) we HAVe $$\LabeL{funcQ}
 Q(\bx, \bz) =
   \sum_i \leFT( x_I - K_i(\bz))^2 + \lambda \log( x_I + |\delta|) \RiGHt)+ k(\BZ),$$ whEre $K_i(\bz) =  z_i + a_i^T \bY - a_I^T A \bz$ ANd $K(\bz)$ coNTaINS TerMS independent oF $\bx$. The optimIZatIon oveR $\bX$ is NOw sepaRable, I.e. CAn bE done indepeNdenTly for eacH coordINate. We cAN see thaT findiNg lOcaL minIMa OvEr $\bX$ oF $q(\bx, \BZ)$ cOrrESpoNds to iteRaTiVe log-ThreSHOLDing.

![illUstrAtion Of singular-valUe lOg-thREshOldinG.[]{data-LabeL="fIg:resUlts_svLt"}](figUrEs/SV_thresh.pdf){wiDth="9.5cM"}

Using thiS moTiVatIoN for Ilt, we can Now ProVe conveRgence tO FixEd POINtS of $f(\bx)$. First, we have:

$f(\HbX^N) = q(\hBx^{n}, \hbx^n)$ aNd $Q(\hbx^{N+1}, \HbX^{n})$, ARe monotoNiCalLy deCREasinG witH ItErations $N$ as lonG As ThE spectrAl Norm $\VeRt a \VeRt_2 < 1$.

THe proOF parAllels The IHT prOof of [@BLumensath2008iteraTIve] using the faCT tHAT $Q(\BX^{n+1}, \bx^N) = f(\bX^n) + \|\bx^{n+1
 robust PCA.

Convergenceof ILT {#S :ILT_ con ver ge ce}==== ============== 
Her e we establish Proposi tion\[ t hm:i l t\ _conv ergence \ ]. W e f ir st  de fi n ea sur rog ate fun ction for$f( \b x)$ in (\[eq n :l og\_reg\]) : $ $\begin{alig ned }
\lab el {eq n :surr oga te}
\ nonumb e r Q(\b x, \bz) =  \ V ert \b y  - A \b x \V ert_ 2^2 + \lambda \su m _i \log(\delta +| x_i| )+ \\ \ Ver t \ bx - \bz \ Ve rt_2^ 2  - \Ver t  A ( \ bx-  \bz) \Vert_2 ^2\end{alig n ed} $$ Not etha t  $Q(\b x, \b x) = f (\bx)$. Sim plif ying (\[e qn:sur r ogate\] )  we hav e $$\l abe l{f uncQ } 
Q( \bx ,\ bz) =  \ sum _i \left (x_ i - k _i(\ b z ) ) ^2 +  \l ambd a \lo g( x_i + |\de lta |) \ r igh t)+ K (\bz) ,$$wh ere $ k_i(\b z) = z _i + a_i^T \by- a_ i^T A \bz $ a nd  $K (\ bz)$c ontain s t erm s indep endento f $ \b x $ .  T he optimization ov er $ \b x$ is no w sepa r ab le ,  i.e. ca nbedone i ndepe nden t ly  for eac h coor d in at e. We c an  see t ha t f ind ing l o calminima  over $\ bx$ o f  $Q(\bx, \bz)$ corresponds t o  i t e ra t ivelog -thresholdi ng.![Il lust r at ion of si ngula r- v al u e log-thresholding. [] {data- label ="fig:results _svlt"}](f i g u res/SV_t hres h .p d f){width="9.5c m"}
Using this motivati on fo r ILT, w e can now p rove con ver gen cetof i xe d points of $ f ( \bx) $.  First,  we  have:
$f (\h bx^ n)=Q(\hbx^{n }, \hbx^ n) $an d$Q( \hbx^ { n+1}, \h bx ^{n }) $,are m o notoni cally  dec re as i ngwith it e ra t i ons$n $as l ong  a s the  spe c tra l norm$\Vert A\Ve r t_2<1$ .

Theproof paralle ls  the IHT p ro ofof [@b l u mensath2 008iterative] using the fact th at$Q(\b x^{n +1}, \bx^ n)= f(\b x^n )  + \|\ bx^{n+ 1
 robust_PCA.

Convergence of_ILT {#S:ILT_convergece}
==================

Here we establish_Proposition \[thm:ilt\_convergence\]._We_first define_a_surrogate function for_$f(\bx)$ in (\[eqn:log\_reg\]):_$$\begin{aligned}
\label{eqn:surrogate}
\nonumber Q(\bx, \bz) =_\Vert \by -_A_\bx \Vert_2^2 + \lambda \sum_i \log(\delta + | x_i|)+ \\
\Vert \bx - \bz \Vert_2^2_-_\Vert A_(\bx_-_\bz) \Vert_2^2\end{aligned}$$ Note that $Q(\bx,_\bx) = f(\bx)$. Simplifying (\[eqn:surrogate\])_we have_$$\label{funcQ}
 Q(\bx, \bz) =
   \sum_i \left(_x_i_- k_i(\bz))^2 +_\lambda \log( x_i + |\delta|) \right)+ K(\bz),$$ where $k_i(\bz)_=  z_i + a_i^T \by_- a_i^T A_\bz$_and_$K(\bz)$ contains terms independent_of $\bx$. The optimization over $\bx$_is now separable, i.e. can be_done independently for each coordinate. We can_see that finding local minima over_$\bx$ of $Q(\bx, \bz)$ corresponds_to iterative_log-thresholding.

![Illustration of singular-value log-thresholding.[]{data-label="fig:results_svlt"}](figures/SV_thresh.pdf){width="9.5cm"}

Using this_motivation for ILT,_we can_now prove convergence_to fixed points of $f(\bx)$. First,_we have:

$f(\hbx^n) =_Q(\hbx^{n}, \hbx^n)$ and $Q(\hbx^{n+1}, \hbx^{n})$, are_monotonically_decreasing with iterations_$n$_as_long as_the spectral norm_$\Vert_A \Vert_2_<_1$.

The proof parallels the IHT proof_of_[@blumensath2008iterative] using the fact that $Q(\bx^{n+1}, \bx^n)_= f(\bx^n) + \|\bx^{n+1