Signal Denoising Using the Minimum-Probability-of-Error Criterion
Jishnu Sadasivan Subhadip Mukherjee Chandra Sekhar Seelamantula





Signal Denoising Using the Minimum-Probability-of-Error Criterion
    Jishnu Sadasivan, Subhadip Mukherjee, and Chandra Sekhar Seelamantula, Senior member, IEEE 


 J. Sadasivan  is with the Department of Electrical Communication Engineering, Indian Institute of Science, Bangalore, India.  Phone: +91 80 2293 2276. Fax: +91 80 2360 0563. E-mail: jishnus@ece.iisc.ernet.in.

 S. Mukherjee and C. S. Seelamantula are with the Department of Electrical Engineering, Indian Institute of Science, Bangalore, India.  Phone: +91 80 2293 2695. Fax: +91 80 2360 0444. E-mails: {subhadip, chandra.sekhar}@ee.iisc.ernet.in.



IEEE Transactions on Signal Processing
    Received: date / Accepted: date
============================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================




We address the problem of signal denoising via transform-domain shrinkage based on a novel risk criterion called  the minimum probability of error (MPE), which measures  the probability that the estimated parameter lies outside an ϵ-neighborhood of the actual value. However, the MPE,  similar to the mean-squared error (MSE), depends on the ground-truth parameter, and has to be estimated from the noisy observations. We consider linear shrinkage-based denoising functions, wherein the optimum shrinkage parameter is obtained by minimizing an estimate of the MPE. When the probability of error is integrated over  ϵ, it leads to  the expected ℓ_1 distortion. The proposed MPE and ℓ_1 distortion formulations are applicable to various noise distributions by invoking a Gaussian mixture model approximation. Within the realm of MPE, we also develop an extension of the transform-domain shrinkage by grouping transform coefficients, resulting in subband shrinkage. The denoising performance obtained within the proposed framework is shown to be better than that obtained using the minimum MSE-based approaches formulated within Stein's unbiased risk estimation (SURE) framework, especially in the low measurement signal-to-noise ratio (SNR) regime. Performance comparison with three state-of-the-art denoising algorithms, carried out on electrocardiogram signals and two test signals taken from the Wavelab toolbox, exhibits that the MPE  framework results in consistent SNR gains for input SNRs below 5 dB.


Minimum probability of error, shrinkage estimator, risk estimation, transform-domain shrinkage, subband shrinkage, expected ℓ_1 distortion, Gaussian mixture model.





§ INTRODUCTION

Signal denoising  algorithms are often developed with the objective of minimizing the mean-squared error (MSE) between an estimate and the ground-truth, which may be deterministic or stochastic with a known prior. The latter formalism leads to Bayesian estimators. Within the deterministic signal estimation paradigm, which is also the formalism considered in this paper, one typically desires that the estimator has minimum variance and is unbiased (MVU) <cit.>. An MVU estimator may not always exist, and if it does, it can be obtained using the theory of sufficient statistics. Eldar and Kay <cit.> showed that, when it comes to minimizing the MSE, biased estimates may outperform the MVU estimate. For example, one could shrink the MVU estimate and optimize for the shrinkage parameter so that the MSE is minimum.

In this paper, we consider the problem of estimating a deterministic signal corrupted by additive white noise. The noise distribution is assumed to be known, but not restricted to be Gaussian. We propose a new distortion metric based on the probability of error and develop estimators using a transform-domain shrinkage approach. Before proceeding with the developments, we review some important literature related to the problem at hand. 


 §.§ Prior Art
 
The MSE is by far the most widely used metric for obtaining the optimum shrinkage parameter. Since the MSE is a function of the parameter to be estimated, direct minimization might result in an unrealizable estimate, in the sense that it might depend on the unknown parameter. However, in some  cases, it is possible to find the optimum shrinkage parameter, for example, using a min-max approach <cit.>, where the parameter is constrained to a known set.  An optimum shrinkage estimator, when the  variance of the unbiased  estimate (or MVU) is a  scaled version of the square of the parameter, with a known scaling, is proposed in <cit.>.

 Optimum shrinkage estimators have also been computed based on risk estimation, where an unbiased estimate of the MSE that depends only on the noisy observations is obtained and subsequently minimized over the shrinkage parameter.  Under the assumption of Gaussian noise, an unbiased estimate of the MSE, namely Stein's unbiased risk estimator (SURE), was developed based on Stein's lemma <cit.>, and has been successfully employed in numerous denoising applications. In his seminal work <cit.>, Stein  proved that the  shrinkage estimator of the mean of a multivariate Gaussian distribution, obtained from its independent and identically distributed (i.i.d.) samples by minimizing SURE, dominates the classical least-squares estimate when the number of samples exceeds three <cit.>.

A risk minimization approach for denoising using a linear expansion of elementary thresholding functions has been addressed in <cit.>, wherein the combining weights are chosen optimally to minimize the SURE objective. SURE-optimized wavelet-domain thresholding techniques have been developed in <cit.>. Atto et al. <cit.> have investigated the problem of signal denoising based on optimally selecting the parameters of a wavelet-domain smooth sigmoidal shrinkage function by minimizing the SURE criterion. The use of SURE objective is not restricted to denoising; it has found applications in image deconvolution as well <cit.>.

Ramani et al. <cit.> developed a Monte-Carlo technique to select the parameters of a generic denoising operator based on SURE. An image denoising algorithm based on non-local means (NLM) is proposed in <cit.>, where parameters of NLM are optimized using SURE. Notable denoising algorithms that aim to optimize the SURE objective include wavelet-domain multivariate shrinkage <cit.>, local affine transform for image denoising <cit.>, optimal basis selection for denoising <cit.>, raised-cosine-based fast bilateral filtering for image denoising <cit.>, SURE-optimized Savitzky-Golay filter <cit.>, etc..
  
The original formulation of SURE, which assumed independent Gaussian noise was extended to certain distributions in continuous and discrete exponential families in <cit.> and <cit.>, respectively, with the assumption of independence left unchanged. Eldar generalized SURE (GSURE) for distributions  belonging to the  non-i.i.d. multivariate exponential family  <cit.>. Giryes et al. <cit.> used a projected version of GSURE for selecting parameters in the context of solving inverse problems. An unbiased estimate of the  Itakura-Saito (IS) distortion  and corresponding pointwise shrinkage was developed  in <cit.> and <cit.>, and successfully applied to speech denoising. A detailed discussion of Gaussian parameter estimation using shrinkage estimators, together with a performance comparison of SURE with the maximum-likelihood (ML) and soft-thresholding-based estimators, can be found in <cit.> (Chapter 2). 
It is shown in <cit.> that the soft-thresholding-based estimator dominates the James-Stein shrinkage estimator in terms of MSE if the parameter vector to be estimated is  sparse. On the other hand, shrinkage estimator dominates if all coordinates of the parameter to be estimated are nearly equal.



 §.§ This Paper

We address the problem of signal denoising based on the minimum probability of error (MPE), which
we first considered in <cit.>. The MPE quantifies the probability of the estimate lying outside an ϵ-neighborhood of the true value. Since the MPE risk depends on the ground truth, we consider a surrogate, which may be  biased, and optimize it to obtain the shrinkage parameter (Section <ref>). The optimization is carried out in the discrete cosine transform (DCT) domain, either in a pointwise fashion or on a subband basis. We derive the MPE risk for Gaussian, Laplacian, and Student's-t noise distributions (Sections <ref> and <ref>). In practical applications, where the noise distribution may be multimodal and not known explicitly, we propose to use a Gaussian mixture model (GMM) approximation <cit.> (Section <ref>).  We show the performance of the MPE-based denoising technique on the Piece-Regular signals  taken from the Wavelab toolbox in Gaussian, Student's-t, and Laplacian noise contaminations (Section <ref>). Proceeding further,  we also consider the probability of error accumulated over 0 < ϵ < ∞ (Section <ref>), which results in the expected ℓ_1 distortion between the parameter and its estimate.  The estimators for the expected ℓ_1 distortion are also derived by invoking the GMM approximation (Section <ref>). We also assess the denoising performance of the shrinkage estimator obtained by minimizing the ℓ_1 distortion for different input SNRs and for different number of noisy realizations (Section <ref>). 

To further boost the denoising performance of the ℓ_1 distortion-based estimator, we develop an iterative algorithm to successively refine the cost function and the resulting estimate, starting with the noisy signal as the initialization (Section <ref>). The iterations lead to an improvement of 2–3 dB in output signal-to-noise ratio (SNR) (Section <ref>).

Performance comparison of the MPE and ℓ_1 distortion-based estimators is carried out on the Piece-Regular and the HeaviSine signals from the Wavelab toolbox <cit.>, and electrocardiogram (ECG) signals from the PhysioBank database <cit.>, with three benchmark techniques: (i) wavelet-domain soft-thresholding <cit.>, (ii) SURE-based orthonormal wavelet thresholding using a linear expansion of thresholds (SURE-LET) <cit.>, and (iii) SURE-based smooth sigmoid shrinkage (SS) in wavelet domain<cit.>; all assuming Gaussian noise contamination (Section <ref>) for fair comparison. 
 


§ THE MPE RISK


Consider  the observation model x=s+w in ℝ^n, where x and s denote the noisy and clean signals, respectively. The noise vector w is assumed to have i.i.d. entries with zero mean and variance σ^2. The goal is to estimate  s  from x by minimizing a suitable risk function. The signal model is  considered in an appropriate transform domain, where the signal admits a parsimonious representation, but noise does not. We consider two types of shrinkage estimators: (i) pointwise, where a shrinkage factor a_i ∈ [0,1] is applied to x_i to obtain an estimate s_i = a_i x_i; and (ii) subband-based, wherein a single shrinkage factor a_J is applied to a group of coefficients {x_i, i∈ J} in subband J⊂{1,2,⋯,n}. Shrinkage estimators may also be interpreted as premultiplication of x with a diagonal matrix.


 §.§ MPE Risk for Pointwise Shrinkage


Assuming that the estimate of s_i does not depend on x_j, for j ≠ i, we drop the index i for brevity of notation. The MPE risk is defined as 

    ℛ   =   ℙ( | s-s |>ϵ),

where ϵ>0 is a predefined tolerance parameter. The risk ℛ quantifies the estimation error using the probability measure and takes into account the noise distribution in its entirety. On the contrary, the MSE relies only on the first- and second-order statistics of noise for linear shrinkage estimators. Substituting s=ax=a(s+w), the risk ℛ evaluates to

    ℛ(s;a)    =   ℙ( | a(s+w) -s | > ϵ)
       =   1-F  (  ϵ-(a-1)s/a) + F( - ϵ+(a-1)s/a),

where F(·) is the cumulative distribution function (c.d.f.) of the additive noise. Since ℛ depends on s, which is the parameter to be estimated, it is impractical to optimize it directly over a. To circumvent the problem, we minimize an estimate of ℛ, which is obtained by  replacing s with an estimate s̃, which, for example, may be obtained using any baseline denoising algorithm, or can even be taken as s̃=x (which is also the ML estimate of s). In the first instance, the proposed technique becomes an add-on to an existing denoising algorithm, and in the second, it is a denoising scheme in itself. Such an estimate ℛ=ℛ(s̃;a) takes the form

    ℛ=1-F  (  ϵ-(a-1)s̃/a) + F( - ϵ+(a-1)s̃/a),

and correspondingly, the optimal shrinkage parameter is obtained as a_opt = arg 0 ≤ a ≤ 1min ℛ. A grid search is performed to optimize ℛ over a ∈ [0,1], and the clean signal is obtained as s=a_optx. We next derive explicit formulae for the risk function for Gaussian, Laplacian, and Student's-t noise distributions.
 
(i) Gaussian distribution: In this case, the noisy observation x also follows a Gaussian distribution, and therefore, s-s=ax-s is distributed as 𝒩((a-1)s,a^2σ^2 ). The MPE risk estimate is given as

    ℛ   =    Q (  ϵ-(a-1)s̃/aσ) + Q (  ϵ+(a-1)s̃/aσ),

where Q(u)=1/√(2 π)∫_u^∞e^-t^2/2dt.

(ii) Student's-t distribution: Consider the case where the noise follow a Student's-t distribution with parameter λ>2 and the probability density function (p.d.f.) of noise is given by

    f(w)    =   Γ( λ+1/2) /√(λπ)  Γ( λ/2) ( 1+w^2/λ)^-λ+1/2.

The variance of w is  σ^2=λ/λ-2. The expression for ℛ is the one given in (<ref>) with 

    F(w)    =   1/2+ wΓ( λ+1/2) G_1 ( 1/2,λ+1/2;3/2;-w^2/λ)/√(λπ)  Γ( λ/2) ,

where G_1 is the hypergeometric function defined as

    G_1 ( a,b;c;z )    =   ∑_k=0^∞(a)_k (b)_k/(c)_kz^k/k!,

and (q)_k denotes the Pochhammer symbol:

    (q)_k    Δ=   1   for    k=0,   
    
     q(q+1)(q+2)⋯(q+k-1),   k>0.

(iii) Laplacian distribution: Considering the noise to be  i.i.d. Laplacian with zero-mean and parameter b (variance σ^2=2b^2), with the p.d.f. f(w)=1/2bexp(- |w|/b), the MPE risk can be obtained by using the following expression for F(w) in (<ref>): 

    F(w)    =   1/2 + 1/2sgn(w) ( 1-exp(  -  |w|/b) ).



  §.§.§ Closeness of ℛ to ℛ

To measure the closeness of ℛ to ℛ, consider the example of estimating a scalar s=4 from a noisy observation x. The MPE risk estimate ℛ is obtained by setting s̃=x. In Figures <ref>(a), <ref>(b), and <ref>(c), we show the variation of the  actual risk ℛ and its estimate ℛ with a, averaged over 100 independent trials, for Gaussian, Student's-t, and Laplacian noise distributions, respectively. The noise has zero mean, and the variance is taken as σ^2=1 for Gaussian and Laplacian models, whereas for Student's-t model, the variance is σ^2=2. The value of ϵ is set equal to σ while computing the MPE risk. We observe that ℛ is a good approximation to ℛ, particularly in the vicinity of the minima. The deviation of the shrinkage parameter a_opt(x), obtained by minimizing ℛ, with respect to its true value a_opt(s) resulted from the minimization of ℛ, is shown in Figure <ref>(d) for three noise models under consideration. The central red lines in Figure <ref>(d) indicate the medians, whereas the black lines on the top and bottom denote the 25 and the 75 percentile points, respectively. We observe that a_opt(x) is well concentrated around a_opt(s), especially for Gaussian and Laplacian noise, barring a small number of outliers. 





  §.§.§ Perturbation Probability of the location of minimum

The location of the minimum of the MPE risk determines the shrinkage parameter. Therefore, one must ensure that it does not deviate too much from its actual value, with high probability, when s is replaced by x in the original risk ℛ. Let a_opt(s)=0≤ a ≤ 1minℛ(s;a) denote the argument that minimizes the true risk ℛ.  Consider the probability of deviation, given by

    P_e^MPE=ℙ( |  a_opt(s) - a_opt(x)  | ≥δ),

for some δ>0. Using a first-order Taylor series approximation of a_opt(x) about s, and substituting x=s+w, we obtain a_opt(x)≈ a_opt(s)+w a'_opt(s),
where ' denotes the derivative. 
The deviation probability P_e^MPE in (<ref>) simplifies to P_e^MPE=ℙ( |  w  | ≥δ/| a'_opt(s) |).



For additive Gaussian noise w with zero mean and variance σ^2, placing the Chernoff bound on P_e^MPE leads to

    P_e^MPE≤ 2exp( -δ^2/2σ^2| a'_opt(s) |^2).

To ensure that P_e^MPE is less than α, for a given α∈ (0,1), it suffices to have

    | a'_opt(s) |^2    ≤   δ^2/2σ^2 log(2/α),
  
which translates to a lower-bound on the input SNR. Since there is no closed-form expression available for a'_opt(s) in the context of MPE risk, we empirically obtain the range of input SNR values s^2/σ^2, for which (<ref>) is satisfied.

Analogously, to satisfy an upper bound on the deviation probability P_e^SURE of the minimum in the case of  SURE, for a given deviation δ>0, one must ensure that

    s^6/8σ^6( δ - σ^4/(  s^2+σ^2 )s^2)^2  ≥log(2/α).

The proof of (<ref>) is given in Appendix <ref>.

The minimum input SNR required to ensure P_e≤α for both SURE- and MPE-based shrinkage estimators is shown in Figure <ref>, for different values of α and δ. The MPE-risk estimate is obtained by replacing s with x and setting ϵ=σ. We observe that reducing the amount of deviation δ for a given probability α, or vice versa, leads to a higher input SNR requirement for both SURE and MPE. We also observe from Figure <ref> that, for given δ and α, SURE requires a higher input SNR than MPE to keep the δ-deviation probability under α. Also, for a given input SNR, the δ-deviation probability of the estimated shrinkage parameter a_opt(x) from the optimum a_opt(s) is smaller for MPE than SURE, thereby indicating that the MPE-based shrinkage is comparatively more reliable than the SURE-based one at lower input SNRs.




  §.§.§ Unknown noise distributions


In practical applications, the distribution of noise may not be known in a parametric form and may also be multimodal. At best, one would have access to realizations of the noise, from which the distribution has to be estimated. In such cases, approximation of the noise p.d.f. using a GMM is a viable alternative <cit.>, wherein  one can  estimate the parameters of GMM using the expectation-maximization algorithm <cit.>. Gaussian mixture modeling is attractive as it comes with certain guarantees. For example, it is known  that a p.d.f. with a finite number of finite discontinuities can be approximated by a GMM to a desired accuracy except at the points of discontinuity <cit.>. The GMM approximation can be used even for non-Gaussian, unimodal distributions. For the GMM-based noise p.d.f.

    f(w)=∑_m=1^Mα_m/σ_m √(2π)exp( -( w-θ_m )^2/2σ_m^2),

the MPE risk turns out to be

    ℛ= ∑_m=1^Mα_m [ Q (  ϵ-(a-1)s̃-θ_m/aσ_m) +.
    . Q (  ϵ+(a-1)s̃+θ_m/aσ_m)],

using (<ref>). For illustration, consider the estimation of a scalar s=4 in the transform domain from its noisy observation x. The additive noise is Laplacian distributed with zero mean and variance σ^2=1. The noise distribution is modeled using a GMM with M=4 components and the corresponding MPE risk estimate is obtained using (<ref>) by setting s̃=x. In Figure <ref>(a), we show a Laplacian p.d.f. and  its GMM approximation. Figure <ref>(b) shows the GMM approximation to a multimodal distribution. Figure <ref>(a) shows the MPE risk based on the original Laplacian distribution as well as  the GMM approximation, as a function of the shrinkage parameter a. The close match between the two indicates that the GMM is a viable alternative when the noise distribution is unknown or follows a complicated model. In Figure <ref>(b),  we plot the GMM-based MPE risk and its estimate averaged over 100 independent trials. We observe that the locations of minima of the actual risk and its estimate match closely, thereby justifying the minimization of ℛ. The MPE risk and its estimate are shown in  Figure <ref>(c)  for the multimodal p.d.f. of Figure <ref>(b). 




 §.§ MPE Risk for Subband Shrinkage


Let a_J be the shrinkage factor applied to the set of coefficients {x_i,  i∈ J} in subband J. The estimate s_J of the clean signal is obtained by s_J= a_J  x_J, where x_J ∈ℝ^|J| and a_J ∈[0,1]. For notational brevity, we drop the subscript J, as we did for pointwise shrinkage, and express the estimator as 𝐬 = a 𝐱, where boldface letters indicate vectors.

Analogous to pointwise shrinkage, the MPE risk for subband shrinkage is defined as ℛ = ℙ( 𝐬-𝐬_2>ϵ), which, for 𝐬 = a 𝐱, becomes ℛ = ℙ(  a 𝐰 + (a-1)𝐬_2>ϵ). For 𝐰∼𝒩(0,σ^2 I), 

    ℛ   =    1-F(θ|k,λ),

where k=|J|, λ = ∑_j=1^k(1-a)^2 s_j^2/a^2σ^2, θ=(ϵ/a σ)^2, and F(θ|k,λ) is the c.d.f. of the non-central χ^2 distribution, given by

    F(θ|k,λ)=∑_m=0^∞λ^m   e^-λ/2/2^m m!ℙ[   χ^2_k+2m≤θ],

wherein χ^2_v denotes the central χ^2 random variable having v degrees of freedom.

Similar to pointwise shrinkage, we propose to obtain an estimate ℛ of ℛ for subband shrinkage estimators either by replacing s_j with x_j, or by an  estimate s̃_j produced by any standard denoising algorithm. The optimum subband shrinkage factor is obtained by minimizing ℛ

Figure <ref> shows the subband MPE risk and its estimate versus a, where the underlying clean signal 𝐬∈ℝ^|J| is corrupted by Gaussian noise and the subband size is chosen to be |J|=k=8. The clean signal 𝐬 is generated by drawing samples from  𝒩(2× 1_k, I_k), where 1_k and I_k denote a k-length vector of all ones and a k× k identity matrix, respectively. The observation 𝐱 is obtained by adding zero-mean i.i.d. Gaussian noise  to 𝐬, with an input SNR of 5 dB, where the input SNR is defined as SNR_in= 10 log_10(  1/kσ^2∑_n=1^ks_n^2)  dB.  The MPE risk estimate is obtained by replacing 𝐬 with 𝐱 in (<ref>), which does not significantly shift the location of the minimum (cf. Figure <ref>).




§ EXPERIMENTAL RESULTS FOR MPE-BASED DENOISING


The performance of the MPE-based pointwise and subband shrinkage estimator is validated on a synthesized harmonic signal (of length N=2048) in Gaussian noise and the Piece-Regular signal (of length N=4096) in Gaussian, Student's-t, and Laplacian noise. The Piece-Regular signal has both smooth and rapidly-varying regions, making it a suitable candidate for the assessment of denoising performance.  


 §.§ Performance of Pointwise-Shrinkage Estimator



  §.§.§ Harmonic signal denoising

Consider the signal 

    s_n = cos( 5π n/2048)+ 2 sin( 10π n/2048), 0≤ n ≤ 2047,

in additive white Gaussian noise, with zero mean and variance σ^2. Since the denoising is carried out in the DCT <cit.> domain, the Gaussian noise statistics remain unaltered. For the purpose of illustration, we assume that σ^2 is known. In practice, σ^2 may not be known a priori and could be replaced by the robust median estimate <cit.> or the trimmed estimate <cit.>. The clean signal is estimated using inverse DCT after applying the optimum shrinkage. The denoising performance of the MPE and SURE-based approaches is compared in Table <ref>. In case of the Wiener filter, the power spectrum of the clean signal is estimated using the standard spectral subtraction technique <cit.>. We observe that MPE-based shrinkage with ϵ = 3.5 σ is superior to SURE and Wiener filter (WF) by 8–12 dB. The comparison also shows that the performance of the MPE depends critically on ϵ. 




  §.§.§ Piece-Regular signal denoising


We consider noisy copies of the Piece-Regular signal, taken from the Wavelab toolbox <cit.>, under Gaussian, Student's-t, and Laplacian contaminations. The noise variance is assumed to be known. Notably, the Gaussian, GMM, and Student's-t distributions of noise are preserved by an orthonormal transform <cit.>, unlike the Laplacian statistics. Therefore, the MPE estimate for Laplacian noise is computed based on a four-component GMM approximation in the DCT domain. The denoised output signal corresponding to Laplacian noise is shown in Figure <ref> for illustration. The MPE estimates are better than SURE estimates. The SNR plots in Figure <ref> indicate that the MPE outperforms SURE for the noise statistics under consideration and that the gains are particularly  high in the input SNR range of -5 to 20 dB, and tend to reduce beyond 20 dB.





  §.§.§ Effect of ϵ on the denoising performance of MPE


Obtaining a closed-form expression for the ϵ  that maximizes the output SNR is not straightforward. We determine the optimum ϵ empirically by measuring the SNR gain as a function of ϵ (cf. Figure <ref>), for i.i.d. Gaussian noise. We observe that the output SNR exhibits a peak approximately at β = ϵ/σ=3.5 for the harmonic signal in (<ref>) and at β=3 for the Piece-Regular signal. As a rule of thumb, we recommend to choose ϵ=3σ for pointwise shrinkage estimators.





 §.§ Performance of Subband MPE Shrinkage

To validate the performance of the MPE-based subband shrinkage estimator (cf. Section <ref>), we consider denoising of  the Piece-Regular signal in additive Gaussian noise. The clean signal and its noisy measurement are shown in Figure <ref>(a). Denoising is carried out by grouping k adjacent DCT coefficients to form a subband. The denoised signals obtained using SURE and MPE are shown in Figures <ref>(b) and <ref>(c), respectively.  The subband size k is chosen to be 16 and the parameter ϵ is set equal to 1.75√(k)σ, a value that was determined experimentally and found to be nearly optimal. We observe that the MPE gives 1 dB improvement in SNR than the SURE approach.

Variation of the output SNR is also studied as a function of k (cf. Figure <ref>). We experimented with  ϵ=3σ, ϵ=1.75√(k)σ, and ϵ=1.25√(k)σ corresponding to subband sizes k=1, k∈[2,16], and k>16, respectively. For both SURE and MPE, as k increases, the output SNR also increases and eventually saturates for k ≥ 40. For input SNR below 15 dB, MPE gives a comparatively higher SNR than SURE, and the margin diminishes with increase in input SNR or the subband size k. The degradation in performance of SURE for low SNRs is due to the large error in estimating the MSE at such SNRs. The SURE-based estimate of MSE becomes increasingly reliable as k increases, thereby leading to superior performance. 







§ ACCUMULATED PROBABILITY OF ERROR: MPE MEETS  THE EXPECTED ℓ_1 DISTORTION


The MPE is parametrized by ϵ, which has to be appropriately chosen in order to achieve optimal denoising performance. To suppress the direct dependence on ϵ, we consider the accumulated probability of error, namely ∫_0^∞ℙ( | s-s |>ϵ)dϵ as the risk to be minimized.
For a nonnegative random variable Y, we know that ℰ{Y}=∫_0^∞ℙ( Y>ϵ)dϵ. Therefore, the accumulated probability of error is the expected ℓ_1 distortion:

    ℰ{| s-s |}  =  ∫_0^∞ℙ( | s-s |>ϵ)dϵ.

For Gaussian noise distribution,

    ℛ_ℓ_1(a,s )=ℰ{| s-s |} = ∫_0^∞ Q (  ϵ-(a-1)s/aσ)dϵ
     +∫_0^∞ Q (  ϵ+(a-1)s/aσ)dϵ.

Denoting u=ϵ-(a-1)s/aσ and μ=-(a-1)s/aσ, the first integral in (<ref>) is evaluated as

∫_0^∞Q (ϵ-(a-1)s/aσ)dϵ=aσ∫_μ^∞ Q (  u )du

    = aσ(    ∫_0^∞ Q (  u )du -  ∫_0^μ Q (  u )du   )
           = aσ(    1/√(2π)- μ Q (  μ)-  1/√(2π)( 1-e^-μ^2/2)    )
           = aσ( e^-μ^2/2/√(2π) - μ Q (  μ) ).

The second term in (<ref>) can be evaluated by replacing μ with -μ in (<ref>). Combining both integrals, we obtain the expression for the expected ℓ_1 distortion:

ℛ_ℓ_1(a,s ) = aσ[ √(2/π)e^-μ^2/2 - μ Q ( μ) + μ Q ( - μ) ]

    = aσ[ √(2/π)exp(-(a-1)^2 s^2/2a^2 σ^2)+ 2(a-1)s/aσ.
          . Q ( -(a-1)s/aσ)  -(a-1)s/aσ].


An estimate of the  expected ℓ_1 distortion is calculated by replacing s with an estimate s̃, which could also be x, to begin with. In Figure <ref>(a), we show the variation of the original ℓ_1 distortion and its estimate obtained by setting s̃=x, as functions of a, averaged over 100 independent realizations of 𝒩(0,1) noise. The actual parameter value is s=4. The figure shows that the minimum of the expected ℓ_1 risk is close to that of its estimate.



In principle, one could iteratively minimize the ℓ_1 distortion by starting with s=x and successively refining it. Such an approach is given in Algorithm 1. An illustration of the denoising performance of the iterative algorithm is deferred to Section <ref>.



 §.§ Expected ℓ_1 risk Using GMM Approximation


For the GMM p.d.f. in (<ref>), the expected ℓ_1 distortion evaluates to (cf. Appendix  <ref> for the derivation)

    ℛ_ℓ_1=∑_m=1^M a α_m  σ_m( √(2/π)e^-μ_m^2/2 -2 μ_m Q (  μ_m) + μ_m),

where μ_m=-(a-1)s+θ_m/aσ_m. The expected ℓ_1 risk and its estimate for a multimodal (cf. Figure <ref>(b)) and Laplacian noise p.d.f.s are shown in Figures <ref>(b) and <ref>(c), respectively. We observe that,  in both cases, the locations of the minima of the true risk and its estimate are in good agreement.


 §.§  Optimum Shrinkage a_opt Versus Posterior SNR


We next study the behavior of a_opt for different input SNRs to compare the denoising capabilities of the MPE and the expected ℓ_1-distortion-based shrinkage estimators.  The optimum pointwise shrinkage parameter a_opt  for Gaussian noise statistics, obtained by minimizing SURE, MPE risk estimate, and the estimated ℓ_1 risk, for different values of the a posteriori SNR  x^2/σ^2 is plotted in Figure <ref>(a). To illustrate the effect of ϵ, the variation of a_opt versus a posteriori SNR for MPE corresponding to Gaussian noise is shown in Figure <ref>(b), for different ϵ. We observe that the shrinkage profiles are characteristic of a reasonable denoising algorithm, as Figures <ref>(a) and <ref>(b) exhibit that  the  shrinkage parameters increase as the a posteriori SNR increases. Whereas in case of the MPE, the choice of ϵ is crucial, the expected ℓ_1 distortion does not require tuning such a parameter. Moreover, the MPE attenuation profile for larger values of ϵ is reminiscent of a hard-thresholding function, whereas the expected ℓ_1 distortion has an attenuation profile that resembles a soft-threshold.



§ PERFORMANCE OF THE EXPECTED ℓ_1 DISTORTION-BASED POINTWISE SHRINKAGE ESTIMATOR
 
In a practical denoising application, we have only one noisy realization from which the clean signal has to be estimated. However, it is instructive to consider the case of multiple realizations as it throws some light on the performance comparisons vis-à-vis other estimators such as the ML estimator. Consider the observation model x^(m)= s+  w^(m) in ℝ^n,  1≤ m ≤ M, where one has access to M noisy copies of the signal s, and the noise vectors w^(m) are drawn independently from the 𝒩( 0,σ^2 I_n) distribution. The ML estimator of the i^th signal coefficient s_i is given by ŝ_ML,i=1/M∑_m=1^M x_i^(m), where  x_i^(m) is the i^th component of x^(m). Dropping the subscript i, as each  coefficient is treated independently of the others, the shrinkage estimator takes the form s=a_optŝ_ML. To study the behavior of the estimate with respect to M, we consider two variants: (i) where a_opt is obtained by minimizing ℛ_ℓ_1(s,a), referred to as the oracle-ℓ_1; and (ii) where a_opt is chosen to minimize ℛ_ℓ_1(ŝ_ML,a), referred to as ML-ℓ_1. The output SNR as a function of M for the Piece-Regular signal, corresponding to an input SNR of 5 dB, is shown in Figure <ref>(a). For all three estimators, namely, oracle-ℓ_1, ML-ℓ_1, and the ML estimate, the output SNR increases with M. However, for the oracle-ℓ_1 and the ML-ℓ_1 estimators, the output SNR stagnates as M increases beyond 40. For M≤ 60, the oracle-ℓ_1 and the ML-ℓ_1 shrinkage estimators exhibit  better performance  compared with the  ML estimator. As one would expect, the performance of the ML-ℓ_1 estimator matches with that obtained using the oracle-ℓ_1 as M becomes large, because the ML estimate converges in probability to the true parameter. For M=1, which is often the case in practice, the ML-ℓ_1 estimate significantly dominates the ML estimator as seen in Figure <ref>(a). The SNR gain over the ML estimator could be further improved by using the iterative minimization algorithm introduced  in Section <ref> (cf. Algorithm <ref>). The performance of the ML-ℓ_1 and the ML estimators, for different values of M and input SNR is shown in Figure <ref>(b). The figures show that for small values of SNR and M, the ML-ℓ_1 estimate outperforms the ML estimator. This is of significant importance in a practical setting where we have only one noisy realization (M = 1).
 


 §.§ Iterative Minimization of the Expected ℓ_1-Risk
 

When M=1, the ML-ℓ_1 estimator is obtained by minimizing ℛ_ℓ_1(x,a), where x is the noisy version of s. We refer to this estimate as the non-iterative ℓ_1-based shrinkage estimator. Following Algorithm 1, one could iteratively refine the estimate, starting from x. We compare the non-iterative ℓ_1-based estimator with its iterative counterpart, and present the results in Figures <ref>, <ref>, and <ref>, corresponding to Gaussian, multimodal (c.f. Figure <ref>(b)), and a GMM approximation to the Laplacian noise, respectively. The output SNR obtained using the oracle-ℓ_1 estimator, calculated by minimizing ℛ_ℓ_1(s,a), is also shown for benchmarking the performance.

We make the following observations from the Figures <ref>, <ref>, and <ref>: (i) the output SNR increases with iterations, albeit marginally after about 10 iterations; (ii) the iterative method consistently dominates the non-iterative one, with an overall SNR improvement of about 2 to 3 dB, for input SNR in the range -5 dB to 20 dB; and (iii) the SNR gain of the iterative technique also reduces for higher input SNR, similar to other denoising algorithms.







§ PERFORMANCE ASSESSMENT OF MPE AND ℓ_1-RISK MINIMIZATION ALGORITHMS VERSUS STATE-OF-THE-ART DENOISING ALGORITHMS


We compare the MPE and the ℓ_1-based shrinkage estimators with three state-of-the-art denoising algorithms:  (i) wavelet soft-thresholding[A Matlab implementation is included in the Wavelab toolbox available at: 
<http://statweb.stanford.edu/ wavelab/>.] <cit.>; (ii) the SURE-LET  denoising algorithm[A MATLAB implementation of the SURE-LET algorithm is available at: 
 <http://bigwww.epfl.ch/demo/suredenoising>.] <cit.>; and (iii) smooth sigmoid shrinkage (SS) <cit.> in the wavelet domain [Pastor et al. kindly provided the MATLAB implementation of their denoising technique <cit.>, which facilitated the comparisons reported in this paper.]. In <cit.>, a wavelet-based soft-thresholding scheme is used for denoising, with the threshold selected as τ=σ√(2 log(N)) for an N length signal. The SURE-LET technique employs a linear expansion of thresholds (LET), which is a linear combination of elementary denoising functions and optimizes for the coefficients by minimizing the SURE criterion. In <cit.>, a smooth sigmoid shrinkage is applied on the wavelet coefficients to achieve denoising, and the parameters of the sigmoid, which control the degree of attenuation, are obtained by minimizing the SURE objective. We consider ECG signals taken from the PhysioBank database, and the HeaviSine and Piece-Regular signals taken from Wavelab toolbox for performance evaluation.

The  noise is assumed  to follow a  Gaussian distribution and the output SNR values are averaged over 100 independent realizations. The noise variance is estimated using a median-based estimator <cit.>, which is also used by Luisier et al.first_foot and Donohosecond_foot. In  SURE-LET, SS,  and wavelet thresholding techniques, denoising is performed using Symmlet-4, with three levels of decomposition, as these settings were found to be the best for the ECG signal (following <cit.>). In case of MPE and ℓ_1-based shrinkage estimators, denoising is performed in the DCT domain. We use the shorthand notations MPE and MPE-subband to denote the pointwise and subband shrinkage estimators, respectively. The corresponding SURE-based subband shrinkage estimator is denoted as  SURE-subband. We set k=16 and ϵ=1.75√(k)σ for computing the subband shrinkage parameters. These parameters have not been specifically optimized; however, they were found to work well in practice. The output SNR as a function of the input SNR, obtained using various algorithms, is shown in Figure <ref>.

From the ECG signal denoising performance shown in Figure <ref>(a),  we observe that the MPE  estimate consistently dominates the soft-thresholding-based denoising  for   input SNRs ranging from -5 dB to 20 dB. The iterative ℓ_1-distortion-based shrinkage estimator (20 iterations) yields lower output SNR compared with the MPE-based estimate for input SNR values in the range -5 to 17.5 dB, but surpasses it for relatively higher values of input SNR (17.5 to 20 dB). The SURE-LET and the SS algorithms dominate both MPE and the ℓ_1-based shrinkage estimators, because they use more sophisticated denoising functions in the transform domain, thereby offering greater flexibility. For  input SNR range of 0 dB to 20 dB, the expected  ℓ_1-distortion-based shrinkage estimator consistently outperforms the soft-thresholding-based techniques.

We have also found that it is possible to boost the denoising performance of an algorithm in the low-SNR regime by adding the MPE denoiser in tandem, that is, by replacing s̃ in the expression for the MPE risk estimate in (<ref>) with the estimate obtained using a denoising technique, for example, the SURE-LET. We refer to this tandem approach as MPE-SURE-LET in Figure <ref>. This approach results in 1 to 2 dB gain in output SNR over SURE-LET for low and medium values of input SNR. We observe in Figure <ref> that the MPE-subband estimator outperforms the competing algorithms (except for MPE-SURE-LET in Figure <ref>(a)),  at low input SNR. 

 



§ CONCLUSIONS


We have proposed a new framework for signal denoising based on a novel criterion, namely the probability of error. Our framework is applicable to scenarios where the noise samples are independent and  additively  distort the signal. Denoising is performed by transform-domain shrinkage and  the optimum shrinkage parameter is obtained by minimizing an estimate of the MPE risk. We have considered both pointwise and subband shrinkage estimators within the MPE paradigm. The performance of the proposed MPE estimators depends on the choice of the error-tolerance parameter ϵ. In pointwise shrinkage, to deal with the issue of selecting an appropriate ϵ, we have proposed two approaches. In the first one, we experimentally determined an ϵ value that results in maximum SNR gain for a particular signal by evaluating the output SNR for different ϵ. In the second approach, we  computed the accumulated probability of error, which is the expected ℓ_1 distortion, and developed an iterative algorithm for minimization. We demonstrated that  the shrinkage estimator obtained using the expected ℓ_1 risk outperforms the classical ML estimator, when the number of observations is small or the input SNR is low. We  also showed that the  shrinkage estimator obtained by iteratively minimizing the ℓ_1 risk dominates the non-iterative approach in terms of the output SNR.

Extensive performance comparison of the proposed MPE and the ℓ_1 distortion-based approaches with state-of-the-art denoising algorithms is carried out on real ECG signals and Wavelab signals. Experimental results demonstrate that the  shrinkage estimator based on the  MPE-risk estimate outperforms the SURE-based  estimator in terms of  SNR gain,  particularly in the regime of low SNR and smaller subband size. The proposed MPE-framework could be used as an add-on over an  existing denoising technique, leading to an estimator that has a higher  output SNR, particularly in the low input SNR regime.

For deriving the expression and validating the performance of the MPE-based subband shrinkage estimator, we considered denoising of signals corrupted with Gaussian noise. Experimentally, we have found that increase in the subband size leads to an increase in output SNR, and saturates beyond a point. We have also observed that, when the subband size or the input SNR is low, the MPE-based estimate has superior performance compared with the SURE-based estimator.

We demonstrated that the optimum shrinkage parameter obtained by minimizing  estimates of the MPE/ℓ_1 distortions  increases monotonically with the  increase in a posteriori SNR. Such behavior of the shrinkage parameter is essential for denoising.  A theoretical characterization of this behavior is needed and may lead to interesting inferences, which could potentially lead to a rigorous convergence proof for the iterative expected ℓ_1 distortion minimization technique. Another important observation is that, for lower input SNRs, the proposed denoising framework yields a higher output SNR compared with the MSE-based techniques. The improvement in performance in terms of SNR of the denoised output may be attributed to the fact that the MPE framework incorporates knowledge of the distribution of the observations, which goes beyond the second-order statistics considered in de facto MSE-based optimization. We also believe that this is the first attempt at demonstrating competitive denoising performance with probability of error chosen as the distortion metric, in a non-Bayesian estimation framework.




§ PERTURBATION OF SURE-BASED POINTWISE SHRINKAGE


To analyze the perturbation in the location of the minimum of the SURE cost function, in comparison with the true MSE, one needs to evaluate  

    P_e^SURE=ℙ{|  a_opt(s) - a_opt(x)  | ≥δ},

where a_opt(s)=s^2/s^2+σ^2 and  a_opt(x)=1-σ^2/x^2. Let 

    h(x)=a_opt(s) - a_opt(x)=(s^2/s^2+σ^2- 1+σ^2/x^2).

 The Taylor-series expansion of h(x) about  s yields 
 
    h(x)=σ^4/s^2(s^2+σ^2)-2 w σ^2/s^3+∑_n=2^∞d^(n)(s) /n! w^n,

where h^(n) is the n^th derivative h. Using the first-order Taylor series approximation h(x)≈ h(s)+w h^(1)(s), we obtain

    h(x)≈σ^4/s^2(s^2+σ^2)-2 w σ^2/s^3,

which, in turn, leads to an approximation of the perturbation probability P_e^SURE:

    P_e^SURE=ℙ{|  h(x)  | ≥δ}
    ≈ℙ{|  σ^4/s^2(s^2+σ^2)-2 w σ^2/s^3| ≥δ}.

Invoking w∼𝒩(0,σ^2), and using  the Chernoff bound<cit.>, we obtain

    P_e^SURE≤ 2exp( -s^6/8σ^6( δ - σ^4/(  s^2+σ^2 )s^2)^2    ).

Consequently, to satisfy an upper bound on the deviation probability of the form P_e^SURE≤α, for a given δ>0, one must ensure that

    s^6/8σ^6( δ - σ^4/(  s^2+σ^2 )s^2)^2  ≥log(2/α).

The condition in (<ref>) translates to an equivalent condition on the minimum required  SNR  s^2/σ^2 to achieve a certain P_e^SURE. 



§  EXPECTED ℓ_1 RISK FOR GMM


For  additive noise with the p.d.f. given in  (<ref>), we have

    ℰ{| s-s |}   =   ∑_m=1^Mα_m (  ∫_0^∞ Q (  ϵ-(a-1)s-θ_m/aσ_m) . dϵ
       +   ∫_0^∞. Q (  ϵ+(a-1)s+θ_m/aσ_m)dϵ),

using (<ref>) and (<ref>). Letting  μ_m=-(a-1)s+θ_m/aσ_m and  u_m=ϵ -(a-1)s-θ_m/aσ_m, we get

    ∫_0^∞ Q ( ϵ -(a-1)s-θ_m/aσ_m)dϵ
    =  aσ_m ( e^-μ_m^2/2/√(2π) - μ_m Q (  μ_m) ).

Subsequently, substituting (<ref>) in (<ref>) yields

    ℰ{| s-s |}=∑_m=1^M a α_m  σ_m( √(2/π)e^-μ_m^2/2 -2 μ_m Q (  μ_m) + μ_m),

which is the expression for the expected ℓ_1 distortion for noise following a GMM distribution.





99

poor
H. V. Poor, An Introduction to Signal Detection and Estimation, Springer, 1994.

kay
S. Kay and Y. C. Eldar, “Rethinking  biased estimation," IEEE Signal Process. Mag., vol. 25, no. 3, pp. 133–136, May 2008.

Stein
C. M. Stein, “Estimation of the mean of a multivariate normal distribution,” Ann. Stat., vol. 9, no. 6, pp. 1135–1151, Nov. 1981.

Jamesproof
W. James and C. Stein, “Estimation with quadratic loss,” Berkeley Symp. Math. Statist. and Prob., Berkeley, CA, vol.1, pp. 361–379, 1961.

Luisier
F. Luisier, T. Blu, and M. Unser, “A new SURE approach to image denoising: Interscale orthonormal wavelet thresholding,” IEEE Trans. Image Process., vol. 16, no. 3, pp. 593–606, Mar. 2007.

Bluv
F. Luisier, T. Blu,  and M. Unser, “SURE-LET for orthonormal wavelet-domain video denoising,” IEEE Trans.  Circ.  Syst.   Video Tech., vol. 20, no. 6, pp. 913–919, Jun. 2010.

Blu3
F. Luisier and T. Blu, “The SURE-LET multichannel  image denoising: Interscale orthonormal wavelet thresholding,” IEEE Trans. Image Process., vol. 17, no. 4, pp. 482-492, Apr. 2008.

Blu
T. Blu and F. Luisier, “The SURE-LET approach to image denoising,” IEEE Trans. Image Process., vol. 16, no. 11, pp. 2778–2786, Nov. 2007.

Zheng
N. Zheng, X. Li, T. Blu, and T. Lee, “SURE-MSE speech enhancement for robust speech recognition,” in Proc. 7^th Int. Symp. Chinese Spoken Language Process., pp. 271–274, Nov. 2010.

Donoho
D. L. Donoho and I. M. Johnstone, “Adapting to unknown smoothness via wavelet shrinkage,” J. Amer. Stat. Assoc., vol. 90, no. 432, pp.1200–1224, Dec. 1995.

Benazza
A. Benazza-Benyahia and J. C. Pesquet, “Building robust wavelet estimators for multicomponent images using Stein's principle,”  IEEE Trans. Image Process, vol. 14, no. 11, pp. 1814–1830, Nov. 2005.

Zhang
X. Zhang and M. D. Desai, “Adapting denoising based on SURE risk,” IEEE  Signal Process. Lett., vol. 5, no. 10, pp. 265-267, Oct. 1998.

Atto1
A. M. Atto, D. Pastor, and G. Mercier, “Smooth adaptation by sigmoid shrinkage,” EURASIP J. Image and Video Process., article ID 532312,  2009.

Atto2
A. M. Atto, D. Pastor, and G. Mercier, “Optimal SURE parameter for sigmoidal wavelet shrinkage,” in Proc. Eur. Signal Process. Conf., pp. 40-43, Aug. 2009.

Pesquet
J. C. Pesquet, A. Benazza-Benyahia, and C. Chaux, “A SURE approach for digital signal/image deconvolution problems,” IEEE Trans. Signal Process., vol. 57, no. 12, pp. 4616–4632, Dec. 2009.

Ramani
S. Ramani,  T. Blu, and M. Unser, “Monte Carlo SURE: A black-box optimization of regularization parameters for general denoising algorithms,” IEEE Trans. Image Process, vol. 17, no. 9, pp. 1540-1544, Sep. 2008.

Dimitri2
D. V. De  Ville and M.  Kocher, “Nonlocal means with dimensionality reduction and SURE-based parameter selection,” IEEE  Trans. Image Process., vol. 20, no. 9, pp. 2683-2690, Sep. 2011.

Hsung
T. Hsung, D. Lun, and  K. C. Ho, “Optimizing the multiwavelet shrinkage denoising,” IEEE  Trans. Signal Process., vol. 53, no. 1, pp. 240-251, Jan. 2005.

Qiu
T. Qiu, A. Wang, N. Yu, and A. Song, “LLSURE: Local linear SURE-based edge-preserving image filtering,” IEEE Trans. Image Process., vol. 22, no. 1, pp. 80–90, Jan. 2013.




Krim
H. Krim, D. Tucker, S. Mallat, and D. Donoho, “On denoising and best signal representation,” IEEE Trans. Info. Theory, vol. 45, no. 7, pp. 2225-2238, Nov. 1999.

Harini
H. Kishan and C. S. Seelamantula, “SURE-fast bilateral filters,” in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process.,  pp. 1129–1132, Mar. 2012.

Skrishnan
S. R. Krishnan and C. S. Seelamantula, “On the selection of optimum Savitzky-Golay filters,” IEEE Trans. Signal Process., vol. 61, no. 2, pp. 380–391, Jan. 2013.

Hudson
H. M. Hudson, “A natural identity for exponential families with applications in multi-parameter estimation,” Ann. Stat., vol. 6, no. 3, pp. 473–484, 1978.

Hwang
J. T. Hwang, “Improving upon standard estimators in discrete exponential families with applications to Poisson and negative binomial cases," Ann. Stat., vol. 10, no. 3, pp. 857–867, 1982.

Eldar
Y. C. Eldar, “Generalized SURE for exponential families: Applications to regularization,” IEEE Trans. Signal Process., vol. 57, no. 2, pp. 471–481, Feb. 2009.

Giryes
R. Giryes, M. Elad, and Y. C. Eldar, “The projected GSURE for automatic parameter tuning in iterative shrinkage methods,” Appl. Comput. Harmon. Anal., vol. 30, no. 3, pp. 407–422, May 2011.


Nag
N. R. Muraka and C. S. Seelamantula, “A risk-estimation-based comparison of mean-square error and Itakura-Saito distortion measures for speech enhancement,” in Proc. Interspeech, pp. 349–352, Aug. 2011.

Krishnan
S. R. Krishnan and C. S. Seelamantula, “A generalized Stein's estimation approach for speech enhancement based on perceptual criteria,” in Proc. Workshop on Statistical and Perceptual Audition (SAPA)–Speech Communication with Adaptive Learning (SCALE), Sep. 2012.

JohnstoneI. M. Johnstone, “Gaussian estimation: Sequence and wavelet models,” Jun. 2013, Available at : http://statweb.stanford.edu/ imj/GE06-11-13.pdf. 

icassp2014
J. Sadasivan, S. Mukherjee, and C. S. Seelamantula, “An optimum shrinkage estimator based on minimum-probability-of-error criterion and application to signal denoising,” in Proc. IEEE Intl. Conf. on Acoust. Speech and Signal Process., pp. 4249–4253, 2014.

Plataniotis
K. N. Plataniotis and D. Hatzinakos,  Gaussian Mixtures and Their Applications to Signal Processing, CRC Press, Dec. 2000. 

Sorenson
H. W. Sorenson and D. L. Alspach, “Recursive Bayesian estimation using Gaussian sums,” Automatica, Vol.  7, pp. 465–479, 1971.

wavelab
WAVELAB toolbox  [Online].  Available: <http://statweb.stanford.edu/ wavelab/ >

donoho
D. L. Donoho, “Denoising by soft thresholding,” IEEE Trans. Info. Theory., vol. 41, no. 3, pp. 613–627, May 1995.

Redner
R. Redner and H. Walker. “Mixture densities, maximum likelihood and the em algorithm,” SIAM Review, Vol.  26, no. 2, pp. 195–239,  Apr. 1984.

roth 
B. M. G. Kibria and A. H. Joarder, “A short review of multivariate t-distribution," J.   Stat. Res., vol. 40, no. 1, pp. 59–72, 2006.

mallat
S. Mallat, A Wavelet Tour of Signal Processing, 3rd edition, Academic Press, 2009.

Socheleau
D. Pastor  and F. Socheleau, “Robust estimation of noise standard deviation in presence of signals with unknown distributions and occurrences,” IEEE Trans. Signal Process., vol. 60, no. 4, pp. 1545–1555,  Apr.  2012. 

KRRao
 K. R. Rao and P. Yip, Discrete Cosine Transform: Algorithms, Advantages, Applications. Boston, MA: Academic, 1990


Boll
S. Boll, “Suppression of acoustic noise in speech using spectral subtraction,”  IEEE Trans. Acoust. Speech, Signal Process., vol. 27, no. 2, pp. 113–120, Apr. 1979. 
PLoizou
P. Loizou, Speech Enhancement — Theory and Practice, CRC Press, 2007.


database
PhysioBank database [Online]. Available: http://www.physionet.org/ physiobank/database/aami-ec13/

Mitzenmacher
M. Mitzenmacher and E. Upfal,  Probability and Computing:  Randomized Algorithms and Probabilistic Analysis, Cambridge University Press, 2005.



