A novel normalized sign algorithm for system
identification under impulsive noise interference
Lu Lu • Haiquan Zhao • Kan Li • Badong Chen

Abstract. To overcome the performance degradation of adaptive filtering algorithms in the presence of impulsive noise, a novel normalized sign algorithm
(NSA) based on a convex combination strategy, called NSA-NSA, is proposed
in this paper. The proposed algorithm is capable of solving the conflicting
requirement of fast convergence rate and low steady-state error for an
individual NSA filter. To further improve the robustness to impulsive noises, a
mixing parameter updating formula based on a sign cost function is derived.
Moreover, a tracking weight transfer scheme of coefficients from a fast NSA
filter to a slow NSA filter is proposed to speed up the convergence rate. The
convergence behavior and performance of the new algorithm are verified by
theoretical analysis and simulation studies.
Keywords. Adaptive filtering • Convex combination • Normalized sign
algorithm • System identification • Impulsive noise

1

Introduction

In general, the performance of an adaptive filtering algorithm degrades when signals
are contaminated by impulsive or heavy-tailed noise. To overcome this limitation,
many algorithms were proposed, such as the sign algorithm (SA) [26], the signed
regressor algorithm (SRA) [7] and the sign-sign algorithm (SSA) [9]. Although the
SA has been successfully applied to system identification under impulsive noise, its
convergence rate is slow [26]. As a variant of SA, the convergence behavior of the
SRA is heavily dependent on the inputs, and there may exist some inputs for which
the SRA is unstable while the least mean square (LMS) algorithm is stable [7].
Among the family of SA algorithms, the SSA has the lowest computational complexiL. Lu • H. Zhao()
Key Laboratory of Magnetic Suspension Technology and Maglev Vehicle, Ministry of Education, and
School of Electrical Engineering, Southwest Jiaotong University, Chengdu, China
e-mail: lulu@my.swjtu.edu.cn, hqzhao@home.swjtu.edu.cn
Kan Li
Computational Neuro-Engineering Laboratory, University of Florida, Gainesville, FL 32611, USA
email: likan@uﬂ.edu.
B. Chen
School of Electronic and Information Engineering, Xi’an Jiaotong University, Xi’an, China.
e-mail: chenbd@mail.xjtu.edu.cn

ty and the most similar characteristic to SA [9]. In addition, the degradations of two
algorithms depend significantly on the initial weights. Similar to the normalized least
mean square (NLMS), the normalized versions of these sign algorithms can be easily
derived, including the normalized SA (NSA) [10], the normalized SRA (NSRA) [11]
and the normalized SSA (NSSA) [12]. The NSA can improve the robustness of the
filter against impulsive noises. However, its convergence performance is still not
good in general. Several variants have been proposed aiming at improving the convergence [5-6,8,13-14,16,27,30,32,]. Particularly, in [14], a dual SA (DSA) with a
variable step-size (VSS) was proposed, but it has a local divergence problem especially when a large disparity occurs between two successive step sizes. In [8], attempt
was made to obtain better stability and convergence performance by inserting another
step-size. Note that the above-mentioned efforts have all been made for a single adaptive filtering architecture.
On the other hand, to cope with impulsive noise, the family of mixed-norm algorithms were developed to combine the benefits of stochastic gradient adaptive filter
algorithms [3-4,17,24,31]. Chambers et al. introduced a robust mixed-norm (RMN)
algorithm, where the cost function is a combination of the error norms that underlie
the LMS and SA [4]. Later, Papoulis et al. [17,23] proposed a novel VSS RMN
(NRMN) algorithm, which circumvents the drawback of slow convergence for RMN
to some extent, by using time-varying learning rate..
The convex combination approach is another way to effectively balance the convergence rate and steady-state error. An adaptive approach using combination LMS
(CLMS) was proposed in [1], utilizing two LMS filters with different step sizes to
obtain fast convergence and small misadjustment. Nevertheless, when the signals are
corrupted by impulsive noise, the algorithms in [1] and [15] usually fail to converge.
To improve performance, an NLMS-NSA algorithm was developed where a combination scheme was used to switch between the NLMS and NSA algorithms [2]. Regrettably, in the initial stage of adaptation, the NLMS algorithm may cause large misadjustment especially when the noise becomes severe. Moreover, the adaptation rule of
the mixing parameter of NLMS-NSA is unsuitable for impulsive noise, such that the
algorithm fails to perform at a desirable level.
In this work, to address the above-mentioned problems, a NSA-NSA algorithm is
proposed by using the convex combination approach. This novel algorithm achieves
robust performance in impulsive noise environments by leveraging two independent
NSA filters with a large and a small step-sizes, respectively. To further enhance the
robustness against impulsive noise, the mixing parameter is adjusted using a sign cost
function. In addition, a tracking weight transfer of coefficients is proposed in order to
obtain fast convergence speed during a transition period. Our main contributions are
listed as follows: 1) propose a NSA-NSA that is well-suited for system identification
problems under impulsive noise; 2) modify an existing update scheme of the mixing
parameter, and analyze its behavior; 3) propose a novel weight transfer scheme that is
computationally simple yet can significantly improve the convergence rate.
The rest of this paper is organized as follows. In Section 2, we propose the NSANSA and develop a novel weight transfer scheme. In Section 3, simulation results in

different impulsive noise environments are presented. Lastly, Section 4 concludes the
paper.

2

Adaptive combination of NSA algorithms

2.1

The proposed algorithm

v (n)

w0

unknown system
e1 (n)

d (n)



w1 ( n) y ( n )
1

x(n)

w 2 ( n)

e( n )

 ( n)

y2 ( n )

e2 (n)





y (n)

1   ( n)



Fig. 1. Diagram of the proposed algorithm.

The diagram of adaptive combination scheme of two NSA filters is illustrated in Fig.
1, where x(n) and y (n) are the filter input and output signals respectively, d (n) is
the desired signal, y1 (n) and y2 (n) are symbols of the two component filters defined
by weight vectors w1 and w 2 , respectively, v(n) is the impulsive noise, and w 0 is
the weight vector of the unknown system. The overall error of the combined filter is
given by e(n)  d (n)  y (n) . To improve performance, both filters are combined with
a scalar mixing parameter  (n) :
(1)
y (n)   (n) y1 (n)  [1   (n)] y2 (n)
(2)
e(n)   (n)e1 (n)  [1   (n)]e2 (n)
where  (n)  [0,1] is defined by a sigmoidal activation function with auxiliary parameter a (n)

 (n)  1 (1  e  a ( n ) ) .

(3)

A gradient descent adaptation of a (n) is given as

a (n  1)  a(n) 

 a e2 (n)

2 a(n)
 a(n)   a e(n)[ y1 (n)  y2 (n)] (n)[1   (n)].

(4)

Note that  a is the step-size of the auxiliary parameter a (n) . This adaptation rule is
derived by the cost function J (n)  e(n) 2 [1]. To improve the robustness against
impulsive noise, the new cost function is defined as J s (n) | e(n) | based on the classical sign-error LMS algorithm [26]. Therefore, the updated scheme of a (n) is derived by minimizing the cost function J s (n) as follows:

a (n  1)  a(n) 

a J s (n)
2 a (n)

(5)

where a is the step-size.
Using the chain rule, the gradient adaptation of J s (n) can be calculated as follows:

a J s (n)  (n)
2  (n) a (n)
 a(n)   a sign{e(n)}[ y1 (n)  y2 (n)] (n)[1   (n)]

a (n  1)  a(n) 

(6)

where  a is a positive constant, and the sign function sign() can be expressed as

if x  0
1,
x

sign( x) 
 0, if x  0 .
(7)
|| x ||2 
1, if x  0
At each iteration cycle, the weight update of NSA-NSA takes the form [10]
x(n) sign{ei (n)}
w i (n  1)  w i (n)  i
(i  1, 2)
(8)
 i  || x(n) ||22
where w i (n) is the weight vectors with length M, i is the constant step-size,  i  0
is a regularization constant close to zero, and ||  ||2 represents the Euclidian-norm. As
a result, the combined filter is obtained by using the following convex combination
scheme
(9)
w (n)   (n)w1 (n)  [1   (n)]w 2 (n) .
2.2

Proposed weight transfer scheme

Inspired by the instantaneous transfer scheme from [22], a tracking weight transfer
scheme is proposed, as shown in Table 1. By using a sliding window approach, the
proposed scheme involves few parameters and retains the robustness against impulsive noise with low-cost. Like the instantaneous transfer scheme in [22], the parameter of proposed weight transfer scheme is not sensitive to the choice. This scheme can
speed up the convergence property of the overall filter, especially during the period of
convergence transition. Define N 0 as the window length. If n−1 mod N 0 is equal to
zero, then implement the following operations. It is well known that the standard convex combination scheme needs to check if a(n  1)  a  , so the only additional operation is the n mod N 0 operation. The judgment condition a (n  1)  a  represents the
condition when the fast filter (filter with large step-size) switches to the slow filter

(filter with small step size) at the transient stage. The operations  (n  1)  0 and

 (n  1)  1 are the limitations for a (n  1)  a  and a(n  1)  a  , respectively. The
operation w 2 (n  1)  w1 (n  1) denotes the transfer of coefficients, which is only
applied in the transient stage. By applying the weight transfer, the adaptation of
w 2 (n  1) is similar to that of the fast filter, which speeds up the convergence rate of

2 NSA filter. Moreover, the cost of the proposed weight transfer scheme is smaller
than that of the original combination, because only one filter is adapted.
15

Fast filter
Slow filter(weight transfer)
Slow filter(no transfer)
Comb.,no transfer
Comb.,weight transfer

10
5

EMSE(dB)

0
-5
-10
-15
-20
-25
-30
-35

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2
4

n

x 10

Fig. 2. Comparison of EMSE of NSA-NSA for Gaussian input in example 1.

15

Fast filter
Slow filter(weight transfer)
Slow filter(no transfer)
Comb.,no transfer
Comb.,weight transfer

10

EMSE(dB)

5
0
-5
-10
-15
-20
-25
-30

0

0.2

0.4

0.6

0.8

1

n

1.2

1.4

1.6

1.8

2
4

x 10

Fig. 3. Comparison of EMSE of NSA-NSA for Gaussian input in example 2.

Figs. 2 and 3 display the comparison of excess means-square error (EMSE) obtained from NSA-NSA with the tracking weight transfer scheme and no transfer
scheme (the mixing parameter is adjusted according to (6)). The same step-size is

chosen for this comparison. As can be seen, the overall performance of the filter bank
is improved by the transfer scheme. It shows from these figures that the proposed
weight transfer scheme exhibit faster convergence than no transfer scheme. The proposed algorithm is summarized in Table 1.

Table 1. The Proposed algorithm


Initialize N 0 , a ,  a ,  i , i , w i (0)  0, a (0)  0,  (0)  0.5
Loop n=1→end do
yi (n)  wTi (n)x(n) (i  1,2)

ei (n)  d (n)  yi (n)
y (n)   (n) y1 (n)  [1   (n)] y2 (n)
w1 ( n  1)  w1 ( n)  1

sign{e1 ( n)}x( n)
 1  || x( n) ||22

w 2 ( n  1)  w 2 ( n)   2

sign{e2 ( n)}x( n)
 2  || x( n) ||22

a( n  1)  a (n)   a sign{e(n)}[( y1 ( n)  y2 (n)] (n)[1   (n)]

 (n  1)  1 / (1  e  a ( n 1) )
% Tracking weight transfer scheme (the proposed method)
if (mod( n  1, N 0 )equal to zero)
if a (n  1)  a 

a (n  1)  a 
 (n  1)  0
endif
if a (n  1)  a 
a(n  1)  a 
 (n  1)  1
w 2 (n  1)  w1 (n  1)
endif
endif
Let n=n+1
end

2.3

Computational complexity

The computational complexity of the basic CLMS [1], NLMS-NSA [2] and NSANSA algorithms is listed in Table 2. Since the basic CLMS combines two LMS algorithms, it requires 4M+2 multiplications for the adaptation of the component filters.
The NLMS-NSA algorithm provides additional insensitivity to the input signal level

by combining the NLMS and NSA, it requires 6M+1 multiplications for the adaptation of the component filters. In contrast with the CLMS, the proposed algorithm uses
NSA as the fast filter to replace the NLMS filter, which reduces the computational
burden and the negative effect of impulsive noise. From (1) and (4), the basic-CLMS
and NLMS-NSA algorithms need 6 multiplications to compute the filter output and to
update a (n) . However, the proposed algorithm requires 5 multiplications to update

a (n) (see (1) and (6), respectively). According to (9), all the algorithms demand 2M
multiplications to calculate the explicit weight vector. Moreover, due to using the
slide window of tracking weight transfer scheme, the NSA-NSA can further reduce
the computation operations. Consequently, these would lead to significant computational efficiency.
Table 2. Summary of the computational complexity.
Algorithms
Basic-CLMS
[1]
NLMS-NSA
[2]
NSA-NSA

2.4

Component filter
adaptation
4M+2

Basic combination
6

Explicit weight
calculation
2M

Weight
transfer
2M

6M+1

6

2M

M+3

2M

No

6M

5(using (6))

The analysis of the mixing parameter

In this section, the convergence behavior of the mixing parameter is analyzed, and the
range of  a will be discussed. When the error term e(n) is expanded with a Taylor
series [18-20], we have
e(n)
1  2 e( n ) 2
e(n  1)  e(n) 
a (n) 
a (n)  h.o.t
(10)
a(n)
2  2 a ( n)
where h.o.t. represents the higher order terms of the remainder of the Taylor series
e(n)
expansion. According to e(n)  d (n)  y (n) and (3),
can be obtained as fola(n)
lows:
e(n)
  (n)[1   (n)][ y2 (n)  y1 (n)] .
(11)
a(n)
The mixing parameter correction a(n) can be calculated from (6)

a(n)   a sign(e(n))[( y1 (n)  y2 (n))]  (n) [1   (n)] .
Combining (10), (11) and (12), we can express (10) as
 ( y (n)  y2 (n))2
e(n  1)  e(n)[1  a 1
 (n)2 (1   (n))2 ] .
| e( n ) |
The NSA-NSA can converge if

(12)

(13)

| e(n  1) || e(n) ||1 

a ( y1 (n)  y2 (n))2
| e( n ) |

 (n)2 (1   (n))2 | . (14)

Hence

|1 

a ( y1 (n)  y2 (n)) 2

| e( n ) |
Solving the inequality with respect to  a gives
0  a 

2.5

 (n) 2 (1   (n))2 | 1 .

2 | e(n) |
.
( y1 (n)  y2 (n))2  (n)2 (1   (n))]2

(15)

(16)

Steady-state performance of the proposed algorithm

To measure the steady-state performance, the EMSEs of the filters are expressed as
[1]
(17)
J ex ,i ( )  lim E{ea2,i ( n )}, i  1, 2
n 

J ex ( )  lim E{ea2 ( n )}
n 

J ex,12 ()  lim E{ea ,1 (n)ea ,2 (n)}
n 

(18)
(19)

 denotes the expectation, J ex ,i () represents the individual EMSE of the
where E{}
ith filter, J ex () is the cross-EMSE of the combined filters, J ex ,12 ( ) is the steadystate correlation between the a priori errors of the elements of the combination,
ea ,i (n) and ea ( n) are a priori error, respectively, defined by

ea ,i (n)  [w 0  wi (n)]T x(n)   iT (n)x(n)
T

T

ea (n)  [w 0 (n)  w (n)] x(n)   (n)x(n)

(20)
(21)

where  i ( n) is the weight error vector of the individual filter, and  (n) is the weight
error vector of the overall filter.
Additionally, for the modified combination (2), J ex,u () is defined as
J ex ,u ( )  lim E{u2 ( n )ea2,1 ( n )  [1  u ( n )]2 ea2,2 ( n )  2u ( n )[1  u ( n)]ea ,1 ( n )ea ,2 ( n )}
n 

(22)
where

1

u (n)   (n)

0

a ( n)  a   
a     a ( n)   a   

(23)



a ( n)   a  

and  is a small positive constant.
Taking expectations of both sides of (6) and using y1 (n)  y2 (n)  ea ,2 (n)  ea ,1 (n)
yields：
E{a(n  1)}  E{a(n)}   a E{sign(e(n))[ea ,2 (n)  ea ,1 (n)] (n)[1   (n)]} .
According to the Price theorem [21,25], we have

(24)

E{sign[e( n)] ( n)} 

2 1

  e, n

E{e( n) ( n)}

(25)

where  e , n is the standard deviation of the error e(n) , i.e.,  e2, n  E{e2 (n)} , and

 (n) can be defined as  (n)  [ea ,2 (n)  ea ,1 (n)] (n)[1   (n)] . Therefore, (24) becomes

E{a (n  1)}  E{a(n)}  a E{e(n)[ea ,2 (n)  ea ,1 (n)] (n)[1   (n)])
where a   a

2 1

  e, n

(26)

. Then, (26) can be rewritten as：

E{a (n  1)}  [ E{a (n)}  a E{[ea2,2 (n)  ea ,1 (n)ea ,2 (n)] (n)[1   (n)]2 )


 a E{[ea ,1 ( n)ea ,2 (n)  ea2,1 (n)] 2 (n)[1   (n)])]a a 

.

(27)

Assume  (n) is independent of a prior error ea ,i (n) in the steady state, under this
assumption, E{a(n  1)} is governed by

E{a (n  1)}  [ E{a(n)}  a E{ (n)[1   (n)]2 J 2 )


 a E{ 2 (n)[1   (n)]J1 ]a a

(28)

where J i  J ex ,i ()  J ex,12 (), i  1, 2 . Suppose the NSA-NSA convergences, the
optimal mean combination weights under convex constraint are given by [1], which is
discussed in the three situations as follows:
1) If J ex,1 ()  J ex ,12 ()  J ex ,2 () , we have J1  0 and J 2  0 . Since a (n) and

 (n) are limited in the effective range, an assumption can be expressed as


E{a(n  1)}  [ E{a (n)}  C ]a a as n  

(29)

where C   (1- )(J 2 -J1 ) is a positive constant. In this case, we can conclude
that
 J ex ( )  J ex ,1 ( )
.
(30)

 J ex ,u ()  J ex ,1 ()
Therefore, (30) shows that the NSA-NSA algorithm performs as well as the component filters.
2) If J ex,1 ()  J ex ,12 ()  J ex ,2 () , we have J1  0 and J 2  0 . Then, (28) can
+

+ 2

be rewritten as


E{a(n  1)}  [ E{a (n)}  C ]a a  as n  

(31)

for a positive constant C   (1- )(J1 -J 2 ) and
+

+ 2

 J ex ( )  J ex ,2 ( )
.
(32)

 J ex ,u ()  J ex ,2 ()
From (32), the overall filter performs approximately equal to the better component
filter.
3) If J ex,12 ()  J ex ,i ( ), i  1, 2 , we have J1  0 and J 2  0 .

Assume  (n)  0 when n   , we obtain

[1   ()]J 2   ()J1

(33)

where  () is given by

 ()  [


J 2
]  .
J1  J 2 1 

(34)

Consequently, it can be concluded from (34) that: if J ex,1 ()  J ex ,2 () , then


   ()  0.5 ; if J ex,1 ()  J ex ,2 () , so 0.5   ()  1    .
Consider the following formulas
J ex ()   2 () J ex ,1 ()  [1   ()]2 J ex ,2 ()  2 ()[1   ()]J ex ,12 ()
2

(35)

2

J ex,u ()   () J ex ,1 ()  [1   ()] J ex ,2 ()  2 ()[1   ()]J ex ,12 () (36)
and rearranging (35), we have
J ex ()   (){J ex ,1 ()  [1   ()]J ex ,12 ()}
 [1   ()]{[1   ()]J ex ,2 ()   () J ex ,12 ()}

(37)

  (){J ex ,12 ()   ()[ J ex ,1 ()  J ex ,12 ()]}
 [1   ()]{J ex ,12 ()  [1   ()][ J ex ,2 ()  J ex ,12 ()]}.

Then, we can rewrite (37) using (34) as
J ex ()   ()[ J ex,12 ()   ()J1 ]

 [1   ()]{J ex ,12 ()  [1   ()]J 2 }

.

(38)

Since  ()  J 2 / (J1  J 2 ) and 1   ()  J1 / (J1  J 2 ) , yielding

J ex ()   ()[ J ex ,12 () 

J1J 2
J1J 2
]  [1   ()][ J ex,12 () 
] . (39)
J1  J 2
J1  J 2

Hence, we obtain

J ex ()  J ex ,u ()  J ex ,12 ( ) 

J1J 2
.
J1  J 2

(40)

According to  ()  (1    ,   ) , the following bounds hold:

J ex ()  J ex ,u ()  J ex ,12 ()   ()J1  J ex ,1 ()

(41)

J ex ()  J ex ,u ()  J ex ,12 ()   ()J 2  J ex ,2 () .

(42)

That is
 J ex ()  min{ J ex ,1 (), J ex ,2 ()}
.
(43)

 J ex ,u ( )  min{ J ex ,1 ( ), J ex ,2 ()}
From the above three situations, it is clear that the proposed NSA-NSA filter performs equally or outperforms the best component filter.

3

Simulation results

To evaluate the performance of the proposed algorithm, three examples of system
(channel) identification were carried out. The results presented here were obtained
from 200 independent Monte Carlo trials. The software of Matlab 8.1 version (2013a)
was used to simulate the proposed algorithm under the computer environment of
AMD (R) A-10 CPU 2.10 GHz and 8Gb memory. To measure the performance of the
algorithms, EMSE using logarithmic scale(dB) was used, defined as:
(44)
EMSE  10 log10 {| ea2 (n) |} .
The unknown system was a ten-tap FIR filter given by random. White Gaussian
noise (WGN) with zero mean and unit variance was used as input. The system was
corrupted by additive WGN and an impulsive noise sequence. The impulsive noise
v(n) was generated from the Bernoulli-Gaussian (BG) distribution [4,17,23,29]

v (n)  A(n) I (n)
(45)
where A(n) is a binary independent and identically distributed (i.i.d.) Bernoulli
process with p{ A(n)  1}  c and p{ A(n)  0}  1  c , and c is the probability of
occurrence for the impulsive interference I(n). The mean value of v(n) is zero, and its
variance is given by
(46)
var{v (n)}  c I2
where  I2  var{I (n)} , and the parameters c is set as c  0.01 [4,17,23].
3.1

Example 1

For the first example, the parameter  I2 in (46) was fixed at  I2  10 4 / 12 , and the
10dB SNR WGN [4,17,23]. The unknown system changes abruptly at n=10000.
Figs. 4 and 5 show the performances of the proposed algorithm with different sets
of N 0 and  a . The filter values of the NSA were 1  0.05 , 2  0.005 (which satisfies the stability condition), 1   2  0.0001 , and a   4 . Consider the stability of
evolution of the mixing parameter and the convergence rate, the best choice is
N 0  2 . In addition, we can observe from Fig. 5 that the best choice is  a  10 .
Figs. 6 and 7 display the evolution of the mixing parameters  (n) and a (n) in
NSA-NSA. Run 1 used the no transfer scheme [1], Run 2 and Run 3 represent the
mixing parameters based on the tracking weight transfer scheme, according to (4) and
(6), respectively. Results demonstrate that the proposed transfer scheme achieves
faster convergence rate and improve the filter robustness in the presence of impulsive
noise. Moreover, Figs. 6 and 7 shows that adjusting the mixing parameter a (n) using
(6) (Run3) results in better stability than other methods.

15
N0=6

10

EMSE(dB)

N0=5
5

N0=4

0

N0=3

-5

N0=2

-10
-15
-20
-25
-30

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2
4

n

x 10

Fig. 4. The choice of parameter N 0 in example 1.

4
ρα=5

3

ρα=8
ρα=10

Magnitude

2

ρα=15

1
0
-1
-2
-3
-4

0

0.2

0.4

0.6

0.8

1

n

1.2

1.4

1.6

1.8

2
4

x 10

Fig. 5. The choice of parameter a in example 1.(the mixing parameter a (n) ).

To further show the performance advantage of the proposed method, Fig. 8 depicts
the learning curves of the NLMS-NSA and the NSA-NSA algorithms. This figure
verifies that the performance of the proposed algorithm is at least as good as the better
component in the combination. Both algorithms have the same misadjustment, since
the step size of the slow filters are the same. However, the fast filter of the NLMSNSA is the NLMS, which results in large misadjustment in high background noise
environments. Consequently, the NLMS-NSA suffers from higher misadjustment in
the initial convergence stage. Fig. 9 plots a comparison of NRMN [23], NSA [10],
VSS-NSA [27], VSS-APSA [29], and the proposed algorithm. Clearly, the NSA has a
tradeoff between fast convergence rate and low EMSE, while the proposed algorithm
shows a good balance between the steady-state error and convergence rate.

1

Run1
Run2
Run3

0.9
0.8

Magnitude

0.7
0.6
0.5
0.4
0.3
0.2
0.1
0

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2
4

n

x 10

Fig. 6. Evolution of the mixing parameter λ(n) of NSA-NSA.

8

Run1
Run2
Run3

6

Magnitude

4
2
0
-2
-4
-6
0

0.2

0.4

0.6

0.8

1

n

1.2

1.4

1.6

1.8

2
4

x 10

Fig. 7. Evolution of the mixing parameter a (n) of NSA-NSA.

15

5

Slow filter
Fast filter
NLMS-NSA(μ 1=0.05, μ 2=0.005)

0

NSA-NSA(μ 1 =0.05,μ 2=0.005)

EMSE(dB)

10

-5
-10
-15
-20
-25
-30
-35

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2
4

n

x 10

Fig. 8. Comparison of EMSE of NLMS-NSA algorithm and NSA-NSA for Gaussian input
when 1% impulsive noises are added.

15

VSS-NSA(λ=0.976)
VSS-APSA(K =8,α=0.9)
NRMN(A=0.0005, K=2)
NSA(μ =0.007)
NSA-NSA(μ 1 =0.05,μ 2 =0.005)

10
5

EMSE(dB)

0
-5
-10
-15
-20
-25
-30
-35

0

0.2

0.4

0.6

0.8

1

n

1.2

1.4

1.6

1.8

2
x 10

4

Fig. 9. Comparison of EMSE of NRMN, NSA, VSS-NSA, VSS-APSA algorithms and NSANSA for Gaussian input when 1% impulsive noises are added.

3.2

Example 2

Next, we consider the case of  I2  10 4 / 20 and SNR=5dB, which corresponds to
case with the slightly impulsive case and highly Gaussian noises. The abrupt change
appeared in the system at the n=10000.
In this example, the step size of NSA-NSA filter was selected as 1  0.05 ,

2  0.008 , and 1   2  0.0001 . This selection of the parameters ensures good
performance of the algorithm in terms of the convergence rate and steady-state misad-

justment. Fig. 10 displays the choice of N 0 in example 2. We can see that the proposed method is not sensitive to this selection, with the optimal value at N 0  2 . Fig.
11 shows the EMSE of NSA-NSA for different  a . The mixing parameter  a  10
for the proposed algorithm was selected to guarantee the stability.
Figs. 12 and 13 show the time evolution of the mixing coefficients, where Run 1
represents the no weight transfer scheme [1], and Run 2 and Run 3 represent the mixing parameters based on the tracking weight transfer scheme given by (4) and (6),
respectively. Clearly, it can be observed from these figures that the best selection is
Run 3. The robust performance in the presence of impulsive noise is also improved by
using (9).
15
N0=6

10

EMSE(dB)

N0=5
5

N0=4

0

N0=3

-5

N0=2

-10
-15
-20
-25
-30

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2
4

n

x 10

Fig. 10. The choice of parameter N 0 in example 2.

4
ρα=5

3

ρα=8
ρα=10

Magnitude

2

ρα=15

1
0
-1
-2
-3
-4

0

0.2

0.4

0.6

0.8

1

n

1.2

1.4

1.6

1.8

2
4

x 10

Fig. 11. The choice of parameter a in example 2. (the mixing parameter a (n) ).

1

Run1
Run2
Run3

0.9
0.8

Magnitude

0.7
0.6
0.5
0.4
0.3
0.2
0.1
0

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2
4

n

x 10

Fig. 12. Evolution of the mixing parameter λ(n) of NSA-NSA.

12

Run1
Run2
Run3

10
8

Magnitude

6
4
2
0
-2
-4
-6
-8

0

0.2

0.4

0.6

0.8

1

n

1.2

1.4

1.6

1.8

2
4

x 10

Fig. 13. Evolution of the mixing parameter a (n) of NSA-NSA.

15

Slow filter
Fast filter
NLMS-NSA( μ1 =0.05, μ2=0.008)

10
5

EMSE(dB)

NSA-NSA( μ1=0.05, μ2 =0.008)
0
-5
-10
-15
-20
-25
-30

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2
4

n

x 10

Fig. 14. Comparison of EMSE of NLMS-NSA algorithm and NSA-NSA for Gaussian input
when 1% impulsive noises are added.

15

VSS-NSA(λ =0.976)
VSS-APSA(K =8, α=0.9)
NRMN(A=0.0005,K =2)
NSA( μ=0.007)
NSA-NSA( μ 1 =0.05, μ 2 =0.008)

10
5

EMSE(dB)

0
-5
-10
-15
-20
-25
-30

0

0.2

0.4

0.6

0.8

1

n

1.2

1.4

1.6

1.8

2
4

x 10

Fig. 15. Comparison of EMSE of NRMN, NSA, VSS-NSA, VSS-APSA algorithms and NSANSA for Gaussian input when 1% impulsive noises are added.

Fig. 14 plots a comparison of NLMS-NSA and the proposed algorithms. Again, we
see that the EMSE of NSA-NSA is consistent with the theoretical analysis. Both algorithms achieve quite similar steady-state error, but the proposed algorithm has the
smaller misadjustment in the initial stage of convergence. This is due to the fact that
the NLMS algorithm is not well-suited for impulsive noise environment. Fig. 15
shows a comparison of the learning curves from NRMN [23], NSA [10], VSS-NSA
[27], VSS-APSA [29] and NSA-NSA for high Gaussian noise and low impulsive
noise environments. It is observed that the proposed algorithm achieves an improved
performance in the presence of impulsive noise.

3.3

Intersymbol Interference (ISI) channel identification under impulsive
noise environment
300
impulsive noise
200

Magnitude

100

0

-100

-200

-300

-400

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2
4

n

x 10

Fig. 16. Impulsive noise in ISI channel.

In this section, we consider a real intersymbol interference (ISI) channel corrupted by
impulsive noise, which occurs quite often in communication systems. Here, we model
the ISI channel as
T
(47)
w 0  [0.04,
0.05, 0.07, 0.21, 0.5,
0.72, 0.36,0, 0.21, 0.03, 0.07]

.
eleven coefficients

In practice, the channel information is unknown. To deal with such problem, the
length of our filter was set to M=13. Quadrature phase shift keyin (QPSK) was used
as the input signal. A segment of 10000 samples was used as the training data and
another 10000 as the test data. The ISI channel was corrupted by impulsive noise, as
shown in Fig. 16. The performance of the proposed NSA-NSA1 is demonstrated, in
comparison with the NLMS-NSA2.
Fig. 17 shows the learning curves of the two algorithms in impulsive noise. Clearly, with impulsive noise, the performance of NSA-NSA is barely affected by large
disturbances, while the performance of NLMS-NSA deteriorates significantly due to
NLMS’s sensitivity to outliers.

1

With

QPSK

input,

the

adaptation

of

a(n)

of

NSA-NSA

is

given

as

a(n  1)  a(n)   a conj{sign{e(n)}}[ y1 (n)  y2 (n)] (n)[1   (n)] , where conj{} denotes conjugate
2

operation.
The derivation of VSS-NSA, VSS-APSA, and NRMN are different from the original literatures, when input signal is the complex number. For paper length optimization, and in order
to focus on the simplicity of the proposed approach, we have decided to only compare to
NLMS-NSA algorithm.

25

15

Slow filter
Fast filter
NLMS-NSA(μ 1 =0.1,μ 2=0.03)

10

NSA-NSA(μ 1=0.1,μ 2 =0.03)

EMSE(dB)

20

5
0
-5
-10
-15
-20
-25

0

1000

2000

3000

4000

5000

6000

7000

8000

9000 10000

n
Fig. 17. Learning curves of NLMS-NSA and NSA-NSA in ISI channel identification (Testing
stage).

4

Conclusions

A novel NSA-NSA was proposed to improve the performance of NSA for system
identification under impulsive noise. The proposed adaptive convex scheme which
combines a fast and a slow NSA filter to achieve both fast convergence speed and low
steady-state error. Moreover, a sign cost function scheme to adjust the mixing parameter was introduced to improve the robustness of the algorithm under impulsive
noise. To further accelerate the initial convergence rate, a tracking weight transfer
scheme was applied in the NSA-NSA. Simulation results demonstrated that the proposed algorithm has better performance than the existing algorithms in terms of convergence rate and steady-state error.

Acknowledgments
The authors want to express their deep thanks to the anonymous reviewers for many
valuable comments which greatly helped to improve the quality of this work.
This work was supported in part by National Natural Science Foundation of China
(Grants: 61271340, 61571374, 61134002, 61433011, U1234203), the Sichuan Provincial Youth Science and Technology Fund (Grant: 2012JQ0046), and the Fundamental Research Funds for the Central Universities (Grant: SWJTU12CX026).

Reference
1. J. Arenas-García, A. R. Figueiras-Vidal, A. H. Sayed, Mean-square performance of a convex combination of two adaptive filters. IEEE Trans. Signal Process. 54(3), 1078–1090
(2006). doi: 10.1109/TSP.2005.863126

2. J. Arenas-Garcia, A. R. Figueiras-Vidal, Adaptive combination of normalised filters for
robust system identification. Electronics Lett. 41(15), 874–875 (2005). doi:
10.1049/el:20051936
3. J. A. Chambers, O. Tanrikulu, A. G. Constantinides, Least mean mixed-norm adaptive filtering. Electronics Lett. 30(19), 1574–1575 (1994). doi: 10.1049/el:19941060
4. J. Chambers, A. Avlonitis, A robust mixed-norm adaptive ﬁlter algorithm. IEEE Signal
Process. Lett. 4(2), 46–48 (1997). doi: 10.1109/97.554469
5. S. C. Douglas, A family of normalized LMS algorithms. IEEE Signal Process. Lett. 1(3),
49–51 (1994). doi: 10.1109/97.295321
6. S. C. Douglas, Analysis and implementation of the max-NLMS adaptive filter. In PRoc.
29th Asilomar Conf. on Signals, Systems, and Computers, 659–663 (1995).
7. E. Eweda, Analysis and design of signed regressor LMS algorithm for stationary and nonstationary adaptive filtering with correlated Gaussian data. IEEE Trans. Circuits Syst.
37(11), 1367–1374 (1990). doi: 10.1109/31.62411
8. S. B. Jebara, H. Besbes, Variable step size filtered sign algorithm for acoustic echo cancellation. Electronics Lett. 39(12), (2003). 936–938. doi: 10.1049/el:20030583
9. B. E. Jun, D. J. Park, Y. W. Kim, Convergence analysis of sign-sign LMS algorithm for
adaptive filters with correlated Gaussian data. In: IEEE International Conference on
Acoustics, Speech, and Signal Processing, 1380–1383 (1995).
10. S. Koike, Variable step size normalized sign algorithm for fast convergent adaptive filters
with robustness against impulsive noise. NEC research & development 41(3), 278–288
(2000).
11. S. Koike, Analysis of adaptive filters using normalized sign regressor LMS algorithm.
IEEE Trans. Signal Process. 47(10), 2710–2723 (1999). doi: 10.1109/78.790653
12. S. Koike, Convergence analysis of adaptive filters using normalized sign-sign algorithm,
IEICE Trans. Fundamentals of Electronics Communications and Computer Sciences, E88A(11), 3218–3224 (2006).
13. R. H. Kwong, E. W. Johnston, A variable step size LMS algorithm. IEEE Trans. Signal
Process. 40(7), 1633–1642(1992). doi: 10.1109/78.143435
14. C. P. Kwong, Dual sign algorithm for adaptive filtering. IEEE Trans. Commun. 34(12),
1272–1275 (1986). doi: 10.1109/TCOM.1986.1096490
15. L. Lu, H. Zhao, A novel convex combination of LMS adaptive filter for system identification. 2014 12th International Conference on Signal Processing (ICSP), Hangzhou 225–229
(2014).
16. V. J. Mathews, Z. Xie, A stochastic gradient adaptive filter with gradient adaptive step
size. IEEE Trans. Signal Process. 41(6), 2075–2087 (1993). doi: 10.1109/78.218137
17. D. P. Mandic, E. V. Papoulis, C. G. Boukis, A normalized mixed-norm adaptive ﬁltering
algorithm robust under impulsive noise interference. In: IEEE International Conference on
Acoustics, Speech, and Signal Processing, 333–336 (2003).
18. D. P. Mandic, NNGD algorithm for neural adaptive filters. Electronics Lett. 36(9), 845–
846 (2000). doi: 10.1049/el:20000631
19. D. P. Mandic, J. A. Chambers, Toward the optimal learning rate for backpropagation.
Neural Processing Letters, 11(1), 1–5 (2000). doi: 10.1023/A:1009686825582
20. D. P. Mandic, A. I. Hanna, M. Razaz, A normalized gradient descent algorithm for nonlinear adaptive filters using a gradient adaptive step size. IEEE Signal Process. Lett. 8(11),
295–297 (2001). doi: 10.1109/97.969448
21. V. J. Mathews, S. H. Cho, Improved convergence analysis of stochastic gradient adaptive
filters using the sign algorithm. IEEE Trans. Acoust., Speech, Signal Process. 35(4), 450–
454 (1987). doi: 10.1109/TASSP.1987.1165167

22. V. H. Nascimento, R. C. de Lamare, A low-complexity strategy for speeding up the convergence of convex combinations of adaptive filters. In: IEEE International Conference on
Acoustics, Speech and Signal Processing, 3553–3556 (2012).
23. E. V. Papoulis, T. Stathaki, A normalized robust mixed-norm adaptive algorithm for system identification. IEEE Signal Process. Lett. 11(1), 56–59 (2004). doi:
10.1109/LSP.2003.819353
24. D. I. Pazaitis, A. G. Constantinides, LMS+F algorithm. Electronics Lett. 31(17), 1423–
1424 (1995). doi: 10.1049/el:19951026
25. R. Price, A useful theorem for nonlinear devices having Gaussian inputs. IRE Transactions
on Information Theory, 4(2), 69–72 (1958). doi: 10.1109/TIT.1958.1057444
26. A. H. Sayed, Fundamentals of Adaptive Filtering. (Wiley IEEE Press, 2003).
27. T. Shao, Y. R. Zheng, J. Benesty, A variable step-size normalized sign algorithm for
acoustic echo cancelation. In: IEEE International Conference on Acoustics, Speech, and
Signal Processing, 333–336 (2010).
28. T. Shao, Y. R. Zheng, J. Benesty, An affine projection sign algorithm robust against impulsive interferences. IEEE Signal Process. Lett. 17(4), 327–330 (2010). doi:
10.1109/LSP.2010.2040203
29. J. Shin, J. Yoo, P. Park, Variable step-size affine projection sign algorithm. Electronics
Lett. 48(9), 483–485 (2012). doi: 10.1049/el.2012.0751
30. J. Soo, K. K. Pang, A multi step size (MSS) frequency domain adaptive filter. IEEE Trans.
Signal Process. 39(1), 115–121 (1991). doi: 10.1109/78.80770
31. O. Tanrikulu, J. A. Chambers, Convergence and steady-state properties of the least-mean
mixed-norm (LMMN) adaptive algorithm. IEE Proceedings-Vision, Image and Signal
Processing, 143, 137–142 (1996).
32. P. Yuvapoositanon, J. Chambers, An adaptive step-size code-constrained minimum output
energy receiver for nonstationary CDMA channels. In: IEEE International Conference on
Acoustics, Speech, and Signal Processing, 465–468 (2003).

