Enhanced q-Least Mean Square
Shujaat Khana , Alishba Sadiqb , Imran Naseemb,c,∗, Roberto Togneric ,
Mohammed Bennamound
a Department

arXiv:1801.00410v1 [math.OC] 1 Jan 2018

of Bio and Brain Engineering, Korea Advanced Institute of Science and
Technology (KAIST), Daejeon, Republic of Korea.
b College of Engineering, Karachi Institute of Economics and Technology, Korangi Creek,
Karachi 75190, Pakistan.
c School of Electrical, Electronic and Computer Engineering, The University of Western
Australia, 35 Stirling Highway, Crawley, Western Australia 6009, Australia.
d School of Computer Science and Software Engineering, The University of Western
Australia, 35 Stirling Highway, Crawley, Western Australia 6009, Australia.

Abstract
In this work, a new class of stochastic gradient algorithm is developed based on
q-calculus. Unlike the existing q-LMS algorithm, the proposed approach fully
utilizes the concept of q-calculus by incorporating time-varying q parameter.
The proposed enhanced q-LMS (Eq-LMS) algorithm utilizes a novel, parameterless concept of error-correlation energy and normalization of signal to ensure
high convergence, stability and low steady-state error. The proposed algorithm
automatically adapts the learning rate with respect to the error. For the evaluation purpose the system identification problem is considered. Extensive experiments show better performance of the proposed Eq-LMS algorithm compared
to the standard q-LMS approach.
Keywords: Adaptive algorithms, Least Mean Squares Algorithm, q-calculus,
Jackson derivative, system identification, q-LMS.

1. Introduction

5

Least square method is considered to be widely used optimization technique.
It has been applied in diversified applications such as plant identification [1],
detection of elastic inclusions [2], noise cancellation [3], echo cancellation [4],
ECG signal analysis [5], elasticity imaging [6], and time series prediction [7], etc.
The least mean square (LMS) is one of the most popular least square algorithms
for adaptive filtering due to its low computational complexity, however, it has
∗ Corresponding

author
Email addresses: shujaat@kaist.ac.kr (Shujaat Khan),
alishba.sadiq@pafkiet.edu.pk (Alishba Sadiq), imran.naseem@uwa.edu.au (Imran
Naseem), roberto.togneri@uwa.edu.au (Roberto Togneri),
mohammed.bennamoun@uwa.edu.au (Mohammed Bennamoun)

Preprint submitted to Signal Processing - Journal - Elsevier

January 3, 2018

10

15

20

25

30

35

40

45

50

a slow convergence rate due to the dependency on the eigenvalue-spread of
the input correlation matrix [8]. Extensive research has been done towards the
optimization of the LMS algorithm [3, 9, 10, 11, 12]. One of the disadvantages of
the LMS is that it is sensitive to the scaling of its input. In [8, 11], the normalized
LMS (NLMS) and its variants were proposed to solve this problem through
normalization. To improve the convergence rate and steady state performance,
variable step size frameworks were devised in [13, 14]. In [15, 16, 17, 18], different
solutions for complex signal processing were proposed. Similarly, to deal with
non-linear signal processing problem, the concept of kernel function-based LMS
algorithms was proposed in [19, 20, 21].
Beside these variants, various definitions of gradient have also been used
to derive improved LMS algorithms; for instance in [22], a fractional order
calculus (FOC) based least mean square algorithm, named as robust variable
step size fractional least mean square (RVSS-FLMS), is proposed. The algorithm
is derived using Riemann-Liouville fractional derivative for high convergence
performance. In [23, 24], some adaptive schemes were proposed for maintaining
stability through adaptive variable fractional power. The FOC variants are,
however, not stable and diverge if the weights are negative or the input signal
is complex [25].
Recently, the q-LMS algorithm is proposed which utilizes q-gradient from the
Jackson’s derivative so that the secant of the cost function is computed instead of
the tangent [12]. The algorithm takes larger steps towards the optimum solution
and therefore, achieves a higher convergence rate. The q-LMS algorithm has
also been used for various applications, such as adaptive noise cancellation [26],
system identification, and designing of whitening filter [27]. In [28] q-normalized
LMS algorithm is proposed and its convergence performance is analyzed. In
[29], using the same definition of q-calculus, variants of steady state least mean
algorithms are derived.
All the aforementioned variants of the q-LMS algorithm enhance convergence
speed at the cost of increased computational complexity and steady-state error.
In order to improve convergence rate without compromising the steady-state
performance, a time-varying q-LMS is proposed in [30]. However, it requires
the tuning of two additional parameters (β and γ), and the performance of
the time-varying q-LMS [30] is very sensitive to the selection of the tuning
parameters. In this paper, we propose a new variant of the q-LMS by making the
q-parameter time-varying. The proposed enhanced q-LMS (Eq-LMS) utilizes a
novel, parameterless concept of error-correlation energy and normalization to
ensure rapid convergence without compromising stability and low steady-state
error. The proposed algorithm automatically adapts the learning rate with
respect to error. It takes larger steps in case of larger error and reduces the
learning rate with decreased error. Unlike the contemporary methods [12, 13,
14], the proposed method is a parameterless technique and does not require
manual tuning of any parameter. The proposed algorithm is evaluated for the
system identification problem and the results are demonstrated for both the
steady-state performance and the convergence rate. Extensive experiments are
performed to show the superiority of the proposed Eq-LMS algorithm over a
2

55

variety of contemporary methods.
The rest of the paper is structured as follows. An overview of the q-calculus
and q-least mean square algorithm is provided in Section 2. The details of
proposed algorithm are discussed in Section 3, followed by the experimental
results in Section 4. The paper is finally concluded in Section 5.
2. Overview of q -Least Mean Square Algorithm

60

The conventional LMS algorithm is derived using the concept of steepest
descent with the weight- update rule
w(i + 1) = w(i) −

µ
∇w J(w),
2

(1)

where J(w) is the cost function for the LMS algorithm and is defined as
J(w) = E[e2 (i)],

(2)

where E[·] is the expectation operator and e(i) is the estimation error between
the desired response d(i) and the output signal at the ith instant, i.e.,
e(i) = d(i) − x| (i)w(i),

(3)

Here, x(i) is the input signal vector defined as
x(i) = [x1 (i), x2 (i), . . . xM (i)]| ,

(4)

and w(i) is the weight vector defined as:
w(i) = [w1 (i), w2 (i), . . . wM (i)]

(5)

where M is the length of the filter.
65

70

2.1. Overview of q-Calculus
The Quantum calculus or q-calculus is sometimes referred to as the calculus
without a limit [31]. It has been successfully used in various areas such as
number theory, combinatorics, orthogonal polynomials, basic hyper-geometric
functions and other sciences quantum theory, operational theory, mechanics,
and the theory of relativity [32, 33, 34, 35].
In q-calculus, the differential of a function is defined as (See, [36])
dq (f (x)) = f (qx) − f (x).

(6)

The derivative therefore takes the form
Dq (f (x)) =

f (qx) − f (x)
dq (f (x))
=
.
dq (x)
(q − 1)x

3

(7)

When q → 1, the expression becomes the derivative in the classical sense. The
q-derivative of a function of the form xn is
 n
 q − 1 xn−1 , q 6= 1,
n
q−1
Dq,x x =
(8)
nxn−1 ,
q = 1.
For a function f (x) of n number of variables, x = [x1 , x2 , ....xn ]| , the q-gradient
is defined as
∇q,w f (x) , [Dq1,x1 f (x), Dq2,x2 f (x), ...Dqn,xn f (x)]| ,

(9)

where q = [q1 , q2 , . . . qN ]| .
2.2. q-Least Mean Square ( q-LMS) Algorithm

75

80

The performance of the LMS algorithm depends on the eigenvalue spread of
the input correlation matrix. The LMS is therefore regarded as an inherently
slowly converging approach [14]. In order to resolve this issue the q-LMS has
been proposed in [12]. Instead of the conventional gradient, the q-LMS is derived
using the q-calculus and utilizes the Jackson derivative method [12], it takes
larger steps (for q > 1) in the search direction as it evaluates the secant of
the cost function rather than the tangent [12]. By replacing the conventional
gradient in (1) with the q-gradient, we get
w(i + 1) = w(i) −

µ
∇q,w J(w).
2

(10)

The q-gradient of the cost function J(w) for the k th weight is defined as
∇q,wk J(w) =

∂qk
∂q
∂qk
J(w) k e(i)
y(i).
∂qk e
∂qk y
∂qk wk (i)

(11)

Solving partial derivatives in (11) using the Jackson derivative defined in Section
(2) gives
∂q
q2 − 1
∂q
J(w) =
(E[e2 (i)]) = E[ k
e(i)] = E[(qk + 1)e(i)],
∂q e
∂q e
qk − 1

85

(12)

where J(w) = E[e2 (i)], E[·] is the expectation operator and e(i) = d(i) − y(i).
Similarly
∂qk
y(i) = xk (i),
∂qk wk (i)

(13)

∂qk
e(i) = −1,
∂qk y

(14)

and

4

Substituting equations (12), (13), and (14) in (11) gives
∇q,wk (i)J(w) = −E[(qk + 1)e(i)xk (i)].

(15)

Similarly, for k = 1, 2, . . . , M ,
∇q,w J(w) = −E[(q1 +1)e(i)x1 (i), (q2 +1)e(i)x2 (i), . . . (qM +1)e(i)xM (i)]. (16)
Consequently, Eq. (16) can be written as
∇q,w J(w) = −2E[Gx(i)e(i)],

(17)

where µ is the learning rate (step-size) and G is a diagonal matrix
diag(G) = [(
90

qM + 1 |
q1 + 1 q2 + 1
), (
), .....(
)] .
2
2
2

(18)

Dropping the expectation from the q-gradient in (17) results in
∇q,w J(w) ≈ −2Gx(i)e(i).

(19)

Substituting (19) in (1) renders the weight update rule of the q-LMS algorithm
by
w(i + 1) = w(i) + µGx(i)e(i).

95

(20)

2.2.1. q-LMS as a whitening filter-q-Normalized LMS
The q-normalized least mean square (q-NLMS) algorithm is defined (See
[28])
w(i + 1) = w(i) + µ

Gx(i)e(i)
,
ζ + ||x(i)||2G

(21)

where ζ is a small value added in the denominator to avoid the indeterminate
form, and ||x(i)||2G is the weighted norm of the input vector. By selecting the q
parameter in (20) as q = 1/λ max, we can design a whitening filter and hence
it can remove the dependency on the input correlation [28].
100

2.2.2. Time-varying q-LMS
The time-varying q-LMS algorithm is based on the variable step size (VSS)
method [14] and is given (See [30])
Ψ (i + 1) = βΨ (i) + γe(i)2 ,
(0 < β < 1, γ > 0),


qupper , Ψ (i + 1) > qupper ,
q(i + 1) = 1,
Ψ (i + 1) < 1,


Ψ
otherwise.

5

(22)

(23)

where qupper is so chosen to satisfy the stability bound, (See [12])
qupper =

2
.
µλmax

(24)

The q(i + 1) is updated according to the estimation of the square of the
estimation error. When the estimation error is large, q(i) will approach its
upper bound denoted by qupper , while for smaller values q(i) goes to unity for a
lower steady-state error.
105

3. The Proposed Enhanced q -Least Mean Square (Eq-LMS) Algorithm
The q-LMS algorithm has an extra degree of freedom to control the performance via the diagonal matrix G, which comprises of the q-dependent entries.
The weight-update rule of the q-LMS algorithm can be written as
w(i + 1) = w(i) + µx̂(i)e(i),

110

115

120

125

(25)

where x̂(i) = Gx(i). For the special case of G=I (identity matrix), the q-LMS
algorithm will be transformed into the conventional LMS. Based on the above
discussion, we make the following important observations.
• We argue that the q-gradient with q > 1 enhances the speed of convergence
as it takes the secant of function rather than the tangent [12]. The larger
the value of the q parameter, the faster the convergence of the algorithm.
But this improvement in the rate of convergence comes at the cost of a
degradation in the steady-state performance.
• The time varying q-LMS technique [30] is based on variable step-size
method [14], which requires the tuning of additional parameters such as
β and γ.
• By selecting the q parameter in (20) as q = 1/λ max, we can design a
whitening filter and hence it can remove the dependency on the input
correlation [28]. However, with a large step-size the q-NLMS converges
rapidly with a compromised steady state performance. Similarly, a smaller
step-size results in better steady state performance but with slow convergence. As such, the two important performance parameters cannot be
optimized simultaneously.
3.1. Proposed Improvements

130

To overcome the aforementioned issues, we propose the Eq-LMS algorithm
with the following improvements.

6

135

• To achieve higher convergence rate with lower steady-state error, we propose to incorporate the instantaneous error energy to adapt the q-parameter.
The proposed algorithm automatically takes large steps when the error is
large and reduces the step-size with the decreasing error. Note that, unlike the time-varying q-LMS [30], no additional tuning parameters are
introduced and the proposed approach is completely automatic.
• The whitening factor (q = 1/λ max) is also utilized to set the limits of
adaptive q-vector, this allows the algorithm to operate at a higher convergence rate without worrying about the divergence issues.

140

• To update each q-parameter, the proposed Eq-LMS utilize, a responsible
error for each tap of the filter. With this improvement, the q-variable for
each tap will be updated accordingly, hence, both the steady-state error
and convergence performance can be improved significantly.
3.2. Formulation of the Eq-LMS Algorithm
By replacing the fixed G in (20) with its time-varying form G(i), the weight
update rule of the proposed Eq-LMS is given as
w(i + 1) = w(i) + µe(i)x(i)G(i),

(26)
th

where µ is the learning rate, and e(i) is the error at the i

instant defined by

e(i) = d(i) − y(i),

(27)

where d(i) and y(i) are the desired and estimated output at the ith instant, respectively. Here, G(i) is a diagonal matrix with time-varying diagonal elements,
and is defined as


q1 (i)


q2 (i)


(28)
G(i) = 
.
..


.
qM (i)
It can also be written as
G(i) = diag(q1 (i), . . . , qM (i)) = diag(q(i)),

(29)

q(i) = {q1 (i), q2 (i), . . . qM (i)}.

(30)

where,
145

We propose an update rule for vector q defined in the following steps.
• Step1: Initialize vector q with any positive random values.
• Step2: Use instantaneous error to update first entry q1 of q vector, which
is associated with weight of the instant input tap, i.e.,
M
X
1
q1 (i + 1) =
{|e(i)| +
qj (i)},
M +1
j=1

where M is the length of the filter.
7

(31)

• Step3: To avoid divergence while maintaining the higher convergence rate,
the following conditions will be evaluated:

1
1
if |q1 (i + 1)| > λmax
,
λmax
(32)
q(i + 1) =
1
,
q1 (i + 1) if |q1 (i + 1)| < λmax
where λmax is the maximum eigenvalue of the input auto-correlation matrix.
• Step4: Update all entries of vector α except for the first entry, simply by
shifting:
qk+1 (i + 1) = qk (i),
(33)
where 1 < k < M − 1

150

• Step5: For next iterations, repeat steps 2 to 5.
Finally, the weight-update equation of the proposed Eq-LMS can be written
as:
w(i + 1) = x(i) + µe(i)x(i)
where

q(i),

(34)

indicates the element wise multiplication.

4. Experiments
For the evaluation of the proposed algorithm the problem of system identification is used. Adaptive learning methods have been successfully used to
identify the unknown system, with numerous applications, for example, in control engineering, communication systems [11, 37, 38, 39, 40, 41]. Channel estimation, for instance, is a widely used method in communication systems to
estimate the characteristics of an unknown channel. Consider a linear channel
shown in Fig. 1.

Figure 1: Channel estimation using adaptive learning algorithm.

8

y(t) = h1 x(t) + h2 x(t − 1) + h3 x(t − 2) + h4 x(t − 3) + h5 x(t − 4).

155

160

165

Equation (35) shows the mathematical model of the system, where x(t) and y(t)
are the input and output of the system, respectively and d(t) is the disturbance
which is taken to be white Gaussian noise in this case. For this experiment, x(t)
is chosen to be consisting of 1 × 106 randomly generated samples obtained from
Gaussian distribution of mean zero and variance of 1. In (35), the system is
defined by its impulse response h(t) while ŷ(t), ĥ(t), and e(t) are the estimated
output, estimated impulse response, and the error of estimation, respectively.
The simulation parameters selected are as follows: coefficient values of h1 = −2,
h2 = −1, h3 = 0, h4 = 1 and h5 = 2 are selected for the channel, the experiments
are performed on three noise levels with the SNR values of 10dB, 20dB and 30dB.
The weights are initialized to zero for all algorithms. Specifically, the objective
of these simulations is to compare the performance of the proposed enhanced
q−LMS (Eq-LMS) algorithm with the contemporary counterparts, i.e., Least
Mean Square (LMS)/ q-LMS [12] at q = 1, q-LMS [12] at q = 2, time-varying
q-LMS [30] and the normalized LMS (NLMS), for given convergence rate and
given steady state error in three different scenarios.
For the performance evaluation, the normalized weight deviation (NWD) in
the actual and the obtained weights is compared. Specifically, we define
NWD =

170

175

(35)

kh − wk
,
khk

(36)

where h is the actual impulse response of the channel and w is the estimated
weight-vector. The simulations are repeated for 1000 independent runs and
mean results are reported. The simulations are performed primarily to evaluate
the steady-state and convergence performances of the proposed algorithm for
various learning rates. Accordingly, three Evaluation protocols are designed:
1. Evaluation protocol 1: learning rate=1 × 10−1 , SNR={10,20,30} dB.
2. Evaluation protocol 2: learning rate=1 × 10−2 , SNR={10,20,30} dB.
3. Evaluation protocol 3: learning rate=1 × 10−3 , SNR={10,20,30} dB.
4.1. Experiments to Evaluate the Steady-State Performance

180

4.1.1. Evaluation Protocol 1: Fast Convergence
For the comparison of the steady-state performance, all algorithms were
setup for equal convergence and after 10000 iterations, the steady-state value
of the NWD is examined. The learning rate (step size) configurations for equal
convergence rate (Evaluation Protocol 1) is shown in Table 1.

9

Table 1: Evaluation protocol 1: Configuration of learning rates of different approaches for an
equal convergence rate of 1 × 10−1 .

Algorithm
LMS/q-LMS
Time-varying q-LMS
Normalized LMS
Proposed Eq-LMS

185

190

195

10 dB SNR
4.4 × 10−2
3.5 × 10−1
2.45 × 10−1
1 × 10−1

Learning Rate µ
20 dB SNR 30 dB SNR
2.45 × 10−2
2.7 × 10−5
−1
1.3 × 10
3.2 × 10−5
−1
1.2 × 10
8.7 × 10−5
1 × 10−1
1 × 10−1

The relevant normalized weight difference (NWD) curves with three different
SNR values are depicted in Fig. 2. From the Fig. 2 (a),(b), and (c), it can be
seen that the proposed Eq-LMS produced the best performance under all three
conditions: (1) for the SNR value of 10dB, it outperformed the LMS/q-LMS
at q = 1, the q-LMS at q = 2, the time-varying q-LMS, and the NLMS by the
NWD value of −1.03 dB, −2.95 dB, −8.06 dB, and −2.14 dB, respectively, (2)
for the SNR value of 20 dB, it surpassed the listed algorithms by the NWD
value of −2.36 dB, −4.05 dB, −6.96 dB, and −3.29 dB, respectively, and (3)
for the SNR value of 30 dB, the above mentioned algorithms were outperformed
by a margin of −2.29 dB, −3.86 dB, −7.18, and −3.47 dB, respectively. Note
that the proposed Eq-LMS algorithm showed the lowest steady state error in
all conditions while the LMS, NLMS and the q-LMS at q = 2 show faster
convergence than the time-varying q-LMS but with greater steady state error
than the proposed Eq-LMS. With the above discussed settings, results for the
channel estimation problem are summarized in Table 2.
Table 2: Evaluation protocol 1: Results of various approaches for an equal convergence rate.

Algorithm
LMS/q-LMS (q = 1)
q-LMS (q = 2)
Time-varying q-LMS
Normalized LMS
Proposed Eq-LMS

200

Steady-state NWD (dB)
10 dB SNR 20 dB SNR 30 dB SNR
-14.63
-21.06
-28.77
-12.71
-19.37
-27.02
-7.60
-16.46
-23.88
-13.52
-20.13
-27.59
-15.66
-23.42
-31.06

4.1.2. Evaluation protocol 2: Medium Convergence
The learning rate (step size) configurations for an equal convergence rate
(Evaluation protocol 2) is shown in Table 3.

10

Table 3: Evaluation protocol 2: Configuration of learning rates of different approaches for an
equal convergence rate of 1 × 10−2 .

Algorithm
LMS/q-LMS
Time-varying q-LMS
Normalized LMS
Proposed Eq-LMS

205

210

215

10 dB SNR
4 × 10−3
2 × 10−2
2 × 10−2
1 × 10−2

Learning Rate µ
20 dB SNR 30 dB SNR
1.5 × 10−3
5.2 × 10−4
−3
1 × 10
1.8 × 10−3
−3
9 × 10
2.5 × 10−3
1 × 10−2
1 × 10−2

The relevant normalized weight difference (NWD) curves with three different
SNR values are depicted in Fig. 2. From the Fig. 2 (d),(e), and (f), it can be
seen that the proposed Eq-LMS produced the best performance under all three
conditions: (1) for the SNR value of 10 dB, it outperformed the LMS/q-LMS
at q = 1, q-LMS at q = 2, time-varying q-LMS and the NLMS by the NWD
value of −0.68 dB, −2.23 dB, −4.08 dB, and −1.74 dB, respectively, (2) for the
SNR value of 20 dB, it surpassed the above-mentioned algorithms by the NWD
value of −1.22 dB, −2.74 dB, −5.34 dB, and −2.45 dB, respectively, and (3)
for the SNR value of 30 dB, the above mentioned algorithms were outperformed
by a margin of −1.44 dB, −2.94 dB, −4.07 and −2.47 dB, respectively. Note
that the proposed Eq-LMS algorithm showed the lowest steady state error in
all conditions while the LMS, NLMS and the q-LMS at q = 2 show faster
convergence than time-varying q-LMS but with a greater steady state error
than the proposed Eq-LMS. With the above discussed settings, results for the
channel estimation problem are summarized in Table 4.
Table 4: Evaluation protocol 2: Results of various approaches for an equal convergence rate.

Algorithm
LMS/qLMS (q=1)
qLMS (q=2)
Time-varying qLMS
Normalized LMS
Proposed Eq-LMS

Steady-state NWD (dB)
10 dB SNR 20 dB SNR 30 dB SNR
-25.42
-32.70
-40.00
-23.62
-30.86
-38.19
-22.40
-29.40
-37.45
-24.18
-31.45
-38.80
-25.85
-33.60
-41.13

4.1.3. Evaluation protocol 3: Slow Convergence
The learning rate (step size) configurations for equal convergence rate (Evaluation protocol 3) is shown in Table 5.

11

Table 5: Evaluation protocol 3: Configuration of learning rates of different approaches for the
fix convergence rate of 1 × 10−3 .

Algorithm
LMS/qLMS
Time-varying qLMS
Normalized LMS
Proposed Eq-LMS
220

225

230

10 dB SNR
3.6 × 10−4
1.6 × 10−3
1.9 × 10−3
1 × 10−3

Learning Rate µ
20 dB SNR 30 dB SNR
1.3 × 10−4
4.5 × 10−3
−4
6 × 10
1.5 × 10−4
−4
7 × 10
2.3 × 10−4
1 × 10−3
1 × 10−3

The relevant NWD curves with three different SNR values are delineated in
Fig. 2. From the Fig. 2 (g), (h), and (i), it can be seen that the proposed
Eq-LMS produced the best performance under all three conditions: (1) for the
SNR value of 10 dB, it outperformed the LMS/q-LMS at q = 1, q-LMS at
q = 2, Time-varying q-LMS, and the NLMS by the NWD value of −0.43 dB,
−1.92 dB, −3.45 dB, and −1.67 dB, respectively, (2) for the SNR value of 20
dB, it surpassed the above-mentioned algorithms by the NWD value of −0.9
dB, −2.43 dB, −4.2 dB, and −2.15 dB, respectively, and (3) for the SNR value
of 30 dB, the above mentioned algorithms were outperformed by a margin of
−1.13 dB, −2.66 dB, −3.68, and −2.33 dB, respectively. Note that the proposed
Eq-LMS algorithm showed the lowest steady state error in all conditions while
the q-LMS at q = 2 showed faster convergence with greater steady state error
than the proposed Eq-LMS. With the above discussed settings, results for the
channel estimation problem are summarized in Table 6.
Table 6: Evaluation protocol 3: Results of various approaches for an equal convergence rate.

Algorithm
LMS/q-LMS (q = 1)
q-LMS (q = 2)
Time-varying q-LMS
Normalized LMS
Proposed Eq-LMS

Steady-state NWD (dB)
10 dB SNR 20 dB SNR 30 dB SNR
-20.16
-27.34
-34.67
-18.92
-26.13
-33.45
-16.76
-23.22
-32.04
-19.10
-26.11
-33.64
-20.84
-28.56
-36.11

4.2. Experiments to Evaluate the Convergence Performance
235

4.2.1. Evaluation Protocol 1: Fast Convergence
For the comparison of convergence performances, all algorithms were setup
for equal steady-state error. The learning rate (step size) configurations for
equal steady-state (Evaluation Protocol 1) is shown in Table (7). Learning rate
for proposed Eq-LMS has been set according to three evaluation protocols.

12

Normalized weight difference (dB)

-1

0

200

400

600

800

10-2

10-3

1000

0

1000

Iterations

10 (dB) SNR and 1e -2 Learning rate
Normalized weight difference (dB)

Normalized weight difference (dB)

10-2

2000

4000

6000

8000

10-1

10-3

10000

0

1

Normalized weight difference (dB)

Normalized weight difference (dB)

-3

4

Iterations

g

3

4

6

8

10
104

10

10-3

0

2

-4

1

2

Iterations

h

4

6

8

10
104

Iterations

3

4

30 (dB) SNR and 1e -3 Learning rate

100

10-3

0

10000

f

10-2

10

8000

10-2

10-4

5

LMS / qLMS at q=1
qLMS at q=2
NLMS
Time varing qLMS
EqLMS (Proposed)

-1

6000

LMS / qLMS at q=1
qLMS at q=2
NLMS
Time varing qLMS
EqLMS (Proposed)

10-1

104

20 (dB) SNR and 1e -3 Learning rate

100

10-2

2

4000

30 (dB) SNR and 1e -2 Learning rate

100

e

LMS / qLMS at q=1
qLMS at q=2
NLMS
Time varing qLMS
EqLMS (Proposed)

10-1

2

Iterations

10 (dB) SNR and 1e -3 Learning rate

0

2000

c

10-2

d

10

0

Iterations

LMS / qLMS at q=1
qLMS at q=2
NLMS
Time varing qLMS
EqLMS (Proposed)

Iterations

100

10-3

10-4

5000

20 (dB) SNR and 1e -2 Learning rate

100

LMS / qLMS at q=1
qLMS at q=2
NLMS
Time varing qLMS
EqLMS (Proposed)

0

4000

10-2

b

10-1

10-3

3000

LMS / qLMS at q=1
qLMS at q=2
NLMS
Time varing qLMS
EqLMS (Proposed)

10-1

Iterations

a
100

2000

Normalized weight difference (dB)

10-2

10-1

30 (dB) SNR and 1e -1 Learning rate

100

LMS / qLMS at q=1
qLMS at q=2
NLMS
Time varing qLMS
EqLMS (Proposed)

Normalized weight difference (dB)

Normalized weight difference (dB)

10

20 (dB) SNR and 1e -1 Learning rate

100

LMS / qLMS at q=1
qLMS at q=2
NLMS
Time varing qLMS
EqLMS (Proposed)

Normalized weight difference (dB)

10 (dB) SNR and 1e -1 Learning rate

100

5
105

LMS / qLMS at q=1
qLMS at q=2
NLMS
Time varing qLMS
EqLMS (Proposed)

10-1
10-2
10-3
10-4
10-5

0

2

4

Iterations

6

8

10
105

i

Figure 2: NWD curves for the LMS/q-LMS at q = 1, q-LMS at q = 2, time-varying q-LMS,
NLMS, and the Eq-LMS. normalized weight deviation with learning rate and SNR of (a) 1e−1 ,
10 dB, (b) 1e−1 , 20 dB,(c) 1e−1 , 30 dB, (d) 1e−2 , 10 dB, (e) 1e−2 , 20 dB, (f) 1e−2 , 30 dB,
(g) 1e−3 , 10 dB, (h) 1e−3 , 20 dB, and (i) 1e−3 , 30 dB.

13

Table 7: Evaluation protocol 1: Configuration of learning rates of different approaches for an
equal steady-state error.

Algorithm
LMS/q-LMS
Time-varying q-LMS
Normalized LMS
Proposed Eq-LMS
240

245

250

255

10 dB SNR
3 × 10−2
3.3 × 10−2
1 × 10−1
1 × 10−1

Learning Rate µ
20 dB SNR 30 dB SNR
8.9 × 10−3
2.7 × 10−3
−3
8.8 × 10
3.1 × 10−3
−2
2.8 × 10
8.5 × 10−3
1 × 10−1
1 × 10−1

The relevant normalized weight difference (NWD) curves with three different
SNR values are depicted in Fig. 3. From the Fig. 3 (a), (b), and (c), it can be
seen that the proposed Eq-LMS algorithm produced the best results under all
three conditions: (1) for the SNR value of 10 dB, algorithms are run for 1000
iterations. The convergence point of the proposed Eq-LMS is reached at 120th
iteration, q-LMS at (q = 2) converged on the 80th iteration but its steady state
error is much larger compared to the proposed Eq-LMS, (2) for the SNR value
of 20 dB, algorithms are run for 5000 iterations. Note that the proposed EqLMS algorithm outperformed all competing approaches by converging in only
400 iterations. The q-LMS (q = 2) was unable to reach the given error-floor and
took 400 iterations to reach a much higher error, and (3) for the SNR value of
30 dB, algorithms are run for 10000 iterations, the proposed Eq-LMS algorithm
took the least number of iterations by converging at the 1600th iteration. The
proposed EqLMS shows best performance in terms of steady state error and
convergence rate. Thus, showing the best overall performance. With the above
discussed settings, results for the channel estimation problem are summarized
in Table 8.
Table 8: Evaluation protocol 1: Results of various approaches for an equal steady-state error.

Algorithm
LMS/q-LMS at (q = 1)
q-LMS at (q = 2)
Time-varying q-LMS
Normalized LMS
Proposed Eq-LMS

260

Convergence point (number of Iterations ×1000))
10 dB SNR 20 dB SNR
30 dB SNR
0.20
0.70
2.70
0.08
0.40
1.55
0.64
2.80
7.20
0.23
1.10
4.40
0.12
0.40
1.60

4.2.2. Evaluation protocol 2: Medium Convergence
The learning rate (step size) configuration for equal steady-state (Evaluation
Protocol 2) is shown in configuration Table (9). Learning rate for proposed EqLMS has been set according to three evaluation protocols.

14

Table 9: Evaluation protocol 2: Configuration of learning rates of different approaches for an
equal steady-state error.

Algorithm
LMS/q-LMS
Time-varying q-LMS
Normalized LMS
Proposed Eq-LMS

265

270

10 dB SNR
3 × 10−3
3.3 × 10−3
9 × 10−2
1 × 10−2

Learning Rate µ
20 dB SNR 30 dB SNR
8.8 × 10−4
2.7 × 10−4
−4
9.3 × 10
3.1 × 10−4
−3
2.72 × 10
8.5 × 10−4
1 × 10−2
1 × 10−2

The relevant NWD curves with three different SNR values are depicted in
Fig. 3. From the Fig. 3 (d), (e), and (f), it can be seen that the proposed
Eq-LMS algorithm produced the best results under all three conditions: (1) for
the SNR value of 10 dB, algorithms are run for 10000 iterations, the convergence
point of the proposed Eq-LMS is reached at 1500th iteration, the q-LMS at (q =
2) converged on the 1100th iteration but its steady state error is much larger than
the proposed Eq-LMS, (2) for the SNR value of 20 dB, algorithms are run for
50000 iterations, the proposed Eq-LMS algorithm outperformed all competing
approaches in terms of convergence point with least steady state error, and
(3) for the SNR value of 30 dB, algorithms are run for 100000 iterations, the
proposed Eq-LMS algorithm converged on 19000th iteration, it showed best
performance in terms of steady state error and convergence rate. Thus, showing
the best overall performance. With the above discussed settings, results for the
channel estimation problem are summarized in Table 10.
Table 10: Evaluation protocol 2: Results of various approaches for an equal steady-state error.

Algorithm
LMS/q-LMS at (q = 1)
q-LMS at (q = 2)
Time-varying q-LMS
Normalized LMS
Proposed Eq-LMS
275

Convergence point (number of Iterations ×1000))
10 dB SNR 20 dB SNR
30 dB SNR
2
9
32
1.1
5.9
19
6.2
28
80
3.1
14
52
1.5
6
19

4.2.3. Evaluation protocol 3: Slow Convergence
The learning rate (step size) configuration for equal steady-state (Evaluation
protocol 3) is shown in configuration Table (11). Learning rate for the proposed
Eq-LMS has been set according to three evaluation protocols.

15

Table 11: Evaluation protocol 3: Configuration of learning rates of different approaches for
an equal steady-state error.

Algorithm
LMS/q-LMS
Time-varying q-LMS
Normalized LMS
Proposed Eq-LMS

280

285

290

10 dB SNR
3 × 10−4
3.3 × 10−4
1 × 10−3
1 × 10−3

Learning Rate µ
20 dB SNR 30 dB SNR
8.9 × 10−4
2.7 × 10−5
−5
9 × 10
3.2 × 10−5
−4
2.8 × 10
8.5 × 10−5
1 × 10−3
1 × 10−3

The relevant NWD curves with three different SNR values are depicted in
Fig. 3. From the Fig. 3 (g), (h), and (i), it can be seen that the proposed EqLMS algorithm produced the best results under all three conditions: (1) for the
SNR value of 10 dB, algorithms are run for 100000 iterations, the convergence
point of the proposed Eq-LMS is reached at 21000th iteration, the q-LMS at
(q = 2), converged at the 12000th iteration but its steady state error is much
larger than the proposed Eq-LMS, (2) for the SNR value of 20 dB, the algorithms
are run for 500000 iterations, the proposed Eq-LMS algorithm outperformed all
competing approaches in terms of convergence point with least steady state
error, and (3) for the SNR value of 30 dB, the algorithms are run for 1000000
iterations, the proposed Eq-LMS algorithm converged on 200000th iteration.
The proposed Eq-LMS showed the best performance in terms of steady state
error and convergence rate. With the above discussed settings, results for the
channel estimation problem are summarized in Table 12.
Table 12: Evaluation protocol 3: Results of various approaches for an equal steady-state error.

Algorithm
LMS/qLMS at (q=1)
qLMS at (q=2)
Time-varying qLMS
Normalized LMS
Proposed Eq-LMS

Convergence point (number of Iterations ×1000))
10 dB SNR 20 dB SNR
30 dB SNR
23
100
370
12
60
200
72
28
840
36
320
600
21
70
240

5. Conclusion

295

300

In this work, we proposed a quantum calculus-based steepest descent algorithm called enhanced q-least mean square algorithm (Eq-LMS) using a novel
concept of error correlation energy. The proposed algorithm is a parameterless
method and unlike the contemporary time varyying q-LMS, it does not require
additional tuning. The proposed Eq-LMS was compared with the LMS, q-LMS,
time varying q-LMS, and the NLMS algorithms for a problem of linear channel
estimation. Extensive simulation tests were conducted to analyze the convergence and the steady-state performance at three different SNR levels. For all
16

Normalized weight difference (dB)

10-1

0

200

400

600

800

10-1

10-2

10-3

1000

0

1000

Iterations

10 (dB) SNR and 1e -2 Learning rate
Normalized weight difference (dB)

Normalized weight difference (dB)

10-2

2000

4000

6000

8000

10-1

10-3

10000

0

1

Normalized weight difference (dB)

Normalized weight difference (dB)

10-2

-3

2

4

Iterations

g

3

4

6

8

10
104

10

LMS / qLMS at q=1
qLMS at q=2
NLMS
Time varing qLMS
EqLMS (Proposed)

10-3

0

2

-4

1

2

Iterations

h

4

6

8

10
104

Iterations

3

4

30 (dB) SNR and 1e -3 Learning rate

100

10-3

0

10000

f

10-2

10

8000

10-2

10-4

5

LMS / qLMS at q=1
qLMS at q=2
NLMS
Time varing qLMS
EqLMS (Proposed)

-1

6000

10-1

104

20 (dB) SNR and 1e -3 Learning rate

100

4000

30 (dB) SNR and 1e -2 Learning rate

100

e

LMS / qLMS at q=1
qLMS at q=2
NLMS
Time varing qLMS
EqLMS (Proposed)

10-1

2

Iterations

10 (dB) SNR and 1e -3 Learning rate

0

2000

c

10-2

d

10

0

Iterations

LMS / qLMS at q=1
qLMS at q=2
NLMS
Time varing qLMS
EqLMS (Proposed)

Iterations

100

10-3

10-4

5000

20 (dB) SNR and 1e -2 Learning rate

100

LMS / qLMS at q=1
qLMS at q=2
NLMS
Time varing qLMS
EqLMS (Proposed)

0

4000

10-2

b

10-1

10-3

3000

LMS / qLMS at q=1
qLMS at q=2
NLMS
Time varing qLMS
EqLMS (Proposed)

10-1

Iterations

a
100

2000

Normalized weight difference (dB)

10-2

100

30 (dB) SNR and 1e -1 Learning rate

100

LMS / qLMS at q=1
qLMS at q=2
NLMS
Time varing qLMS
EqLMS (Proposed)

Normalized weight difference (dB)

Normalized weight difference (dB)

100

20 (dB) SNR and 1e -1 Learning rate

101

LMS / qLMS at q=1
qLMS at q=2
NLMS
Time varing qLMS
EqLMS (Proposed)

Normalized weight difference (dB)

10 (dB) SNR and 1e -1 Learning rate

101

5
105

LMS / qLMS at q=1
qLMS at q=2
NLMS
Time varing qLMS
EqLMS (Proposed)

10-1
10-2
10-3
10-4
10-5

0

2

4

Iterations

6

8

10
105

i

Figure 3: NWD curves for the LMS/q-LMS at q = 1, q-LMS at q = 2, time-varying q-LMS,
NLMS, and the Eq-LMS. normalized weight deviation with learning rate and SNR of (a) 1e−1 ,
10 dB, (b) 1e−1 , 20 dB,(c) 1e−1 , 30 dB, (d) 1e−2 , 10 dB, (e) 1e−2 , 20 dB, (f) 1e−2 , 30 dB,
(g) 1e−3 , 10 dB, (h) 1e−3 , 20 dB, and (i) 1e−3 , 30 dB.

17

scenarios, the proposed Eq-LMS algorithm comprehensively outperformed the
contemporary approaches achieving the best performance in terms of steadystate error and convergence.
305

References
[1] Y. Chen, Y. Gu, A. O. Hero, Sparse LMS for system identification, in: IEEE
International Conference on Acoustics, Speech and Signal Processing, 2009.
ICASSP 2009., IEEE, 2009, pp. 3125–3128.

310

315

[2] T. Abbas, S. Khan, M. Sajid, A. Wahab, J. C. Ye, Topological sensitivity based far-field detection of elastic inclusions, Results
in Physics, Volume 8, March 2018, Pages 442-460, ISSN 2211-3797,
https://doi.org/10.1016/j.rinp.2017.12.041.
[3] J. M. Górriz, J. Ramı́rez, S. Cruces-Alvarez, C. G. Puntonet, E. W. Lang,
D. Erdogmus, A novel LMS algorithm applied to adaptive noise cancellation, IEEE Signal Processing Letters 16 (1) (2009) 34–37.
[4] J. Benesty, S. L. Gay, An improved PNLMS algorithm, in: IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP),
2002, Vol. 2, IEEE, 2002, pp. II–1881.

320

[5] N. V. Thakor, Y.-S. Zhu, Applications of adaptive filtering to ECG analysis: noise cancellation and arrhythmia detection, IEEE Transactions on
Biomedical Engineering 38 (8) (1991) 785–794.
[6] H. Ammari, E. Bretin, J. Garnier, H. Kang, H. Lee, A. Wahab, Mathematical methods in elasticity imaging, Princeton University Press, 2015.

325

[7] Y. Zheng, S. Wang, J. Feng, C. K. Tse, A modified quantized kernel least
mean square algorithm for prediction of chaotic time series, Digital Signal
Processing 48 (Supplement C) (2016) 130 – 136. doi:https://doi.org/
10.1016/j.dsp.2015.09.015.
[8] S. C. Douglas, A family of normalized LMS algorithms, IEEE Signal Processing Letters 1 (3) (1994) 49–51.

330

335

[9] W. Wang, H. Zhao, Boxed-constraint least mean square algorithm and its
performance analysis, Signal Processing 144 (Supplement C) (2018) 201 –
213. doi:https://doi.org/10.1016/j.sigpro.2017.10.006.
[10] S. Cheng, Y. Wei, Y. Chen, Y. Li, Y. Wang, An innovative fractional order
LMS based on variable initial value and gradient order, Signal Processing
133 (2017) 260–269.
[11] S. Ciochin, C. Paleologu, J. Benesty, An optimized NLMS algorithm for
system identification, Signal Processing 118 (2016) 115 – 121.

18

340

[12] U. M. Al-Saggaf, M. Moinuddin, M. Arif, A. Zerguine, The q-Least Mean
Squares algorithm, Signal Processing 111 (Supplement C) (2015) 50 – 60.
doi:https://doi.org/10.1016/j.sigpro.2014.11.016.
[13] T. Aboulnasr, K. Mayyas, A robust variable step-size LMS-type algorithm:
analysis and simulations, IEEE Transactions on Signal Processing 45 (3)
(1997) 631–639.

345

[14] R. H. Kwong, E. W. Johnston, A variable step size LMS algorithm, IEEE
Transactions on Signal Processing 40 (7) (1992) 1633–1642. doi:10.1109/
78.143435.
[15] B. Widrow, J. McCool, M. Ball, The complex LMS algorithm, Proceedings
of the IEEE 63 (4) (1975) 719–720. doi:10.1109/PROC.1975.9807.

350

[16] A. Khalili, A. Rastegarnia, W. M. Bazzi, Z. Yang, Derivation and analysis
of incremental augmented complex least mean square algorithm, IET Signal
Processing 9 (4) (2015) 312–319.
[17] A. Khalili, A. Rastegarnia, S. Sanei, Quantized augmented complex leastmean square algorithm: Derivation and performance analysis, Signal Processing 121 (2016) 54–59.

355

360

[18] J. Ahmad, S. Khan, M. Usman, I. Naseem, M. Moinuddin, FCLMS: Fractional Complex LMS Algorithm for Complex System Identification, in: 13th
IEEE Colloquium on Signal Processing and its Applications (CSPA 2017),
IEEE, 2017.
[19] W. Liu, P. P. Pokharel, J. C. Principe, The kernel least-mean-square algorithm, IEEE Transactions on Signal Processing 56 (2) (2008) 543–554.
[20] P. P. Pokharel, W. Liu, J. C. Principe, Kernel least mean square algorithm
with constrained growth, Signal Processing 89 (3) (2009) 257 – 265. doi:
https://doi.org/10.1016/j.sigpro.2008.08.009.

365

370

[21] B. Chen, S. Zhao, P. Zhu, J. C. Principe, Quantized kernel least mean
square algorithm, IEEE Transactions on Neural Networks and Learning
Systems 23 (1) (2012) 22–32.
[22] S. Khan, M. Usman, I. Naseem, R. Togneri, M. Bennamoun, A Robust Variable Step Size Fractional Least Mean Square (RVSS-FLMS) Algorithm, in:
13th IEEE Colloquium on Signal Processing and its Applications (CSPA
2017), IEEE, 2017.
[23] S. Khan, M. Usman, I. Naseem, R. Togneri, M. Bennamoun, VP-FLMS: a
Novel Variable Power Fractional LMS Algorithm, in: 2017 Ninth International Conference on Ubiquitous and Future Networks (ICUFN) (ICUFN
2017), Milan, Italy, 2017.

19

375

380

[24] J. Ahmad, M. Usman, S. Khan, I. Naseem, H. J. Syed, RVP-FLMS : A
Robust Variable Power Fractional LMS Algorithm, in: 2016 IEEE International Conference on Control System, Computing and Engineering (ICCSCE), IEEE, 2016.
[25] N. J. Bershad, F. Wen, H. C. So, Comments on “Fractional LMS algorithm”, Signal Processing 133 (2017) 219–226.
[26] M. Arif, I. Naseem, M. Moinuddin, S. S. Khan, M. M. Ammar, Adaptive noise cancellation using q-LMS, in: 2017 International Conference
on Innovations in Electrical Engineering and Computational Technologies
(ICIEECT), 2017, pp. 1–4. doi:10.1109/ICIEECT.2017.7916527.

385

390

395

[27] U. M. Al-Saggaf, M. Moinuddin, A. Zerguine, An efficient least mean
squares algorithm based on q-gradient, in: 2014 48th Asilomar Conference on Signals, Systems and Computers, 2014, pp. 891–894. doi:
10.1109/ACSSC.2014.7094580.
[28] A. U. Al-Saggaf, M. Arif, U. M. Al-Saggaf, M. Moinuddin, The qnormalized least mean square algorithm, in: 2016 6th International Conference on Intelligent and Advanced Systems (ICIAS), 2016, pp. 1–6.
doi:10.1109/ICIAS.2016.7824098.
[29] A. Ahmed, M. Moinuddin, U. M. Al-Saggaf, q-State Space Least Mean
Family of Algorithms, Circuits, Systems, and Signal Processing. doi:10.
1007/s00034-017-0569-7.
[30] M. Arif, I. Naseem, M. Moinuddin, U. M. Al-Saggaf, Design of an Intelligent
q-LMS Algorithm for Tracking a Non-stationary Channel, Arabian Journal
for Science and Engineering. doi:10.1007/s13369-017-2883-6.

400

[31] T. Ernst, A Comprehensive Treatment of q-Calculus, 1st Edition,
Springer Science & Business Media, Basel, 2012. doi:10.1007/
978-3-0348-0431-8.
[32] G. Bangerezako, Variational q-calculus, Journal of Mathematical Analysis
and Applications 289 (2) (2004) 650 – 665. doi:https://doi.org/10.
1016/j.jmaa.2003.09.004.

405

410

[33] J. Tariboon, S. K. Ntouyas, P. Agarwal, New concepts of fractional quantum calculus and applications to impulsive fractional q-difference equations, Advances in Difference Equations 2015 (1) (2015) 18. doi:10.1186/
s13662-014-0348-8.
[34] J. Tariboon, S. K. Ntouyas, Quantum calculus on finite intervals and applications to impulsive difference equations, Advances in Difference Equations
2013 (1) (2013) 282. doi:10.1186/1687-1847-2013-282.
[35] A. R. A. L. Ali, V. Gupta, R. P. Agarwal, A. Aral, V. Gupta, Applications
of q-Calculus in Operator Theory, Springer New York, 2013.
20

[36] V. Kac, P. Cheung, Quantum Calculus, Springer New York, 2012.
415

420

425

[37] S. Khan, N. Ahmed, M. A. Malik, I. Naseem, R. Togneri, M. Bennamoun,
FLMF: fractional least mean fourth algorithm for channel estimation in
non-gaussian environment, in: International Conference on Information
and Communications Technology Convergence 2017 (ICTC 2017), Jeju Island, Korea, 2017.
[38] P. A. C. Lopes, J. A. B. Gerald, A close to optimal adaptive filter for sudden
system changes, IEEE Signal Processing Letters 24 (11) (2017) 1734–1738.
doi:10.1109/LSP.2017.2757147.
[39] S. Khan, I. Naseem, R. Togneri, M. Bennamoun, A novel adaptive kernel
for the rbf neural networks, Circuits, Systems, and Signal Processing (2016)
1–15.
[40] S. Khan, J. Ahmad, I. Naseem, M. Moinuddin, A novel fractional gradientbased learning algorithm for recurrent neural networks, Circuits, Systems,
and Signal Processing (2017) 1–20.

430

[41] A. I. Sulyman, A. Zerguine, Convergence and steady-state analysis of a
variable step-size nlms algorithm, Signal Processing 83 (6) (2003) 1255–
1273.

21

