A class of diffusion algorithms with logarithmic cost over adaptive sparse
Volterra networkI
Lu Lua , Haiquan Zhaoa∗
a)School of Electrical Engineering, Southwest Jiaotong University, Chengdu, China.

Abstract
In this paper, we present a novel class of diffusion algorithms that can be used to estimate the coefficients of

arXiv:1606.08541v5 [cs.SY] 2 May 2017

sparse Volterra network (SVN). The development of the algorithms is based on the logarithmic cost and l0 norm constraint. Simulations for Gaussian and impulsive scenarios are conducted to demonstrate the superior
performance of the proposed algorithms as compared with the existing algorithms.
Keywords: Distributed adaptation, Volterra filter, Sparse, Logarithmic cost.

1. Introduction
The Volterra filter has been widely used as a nonlinear system modelling tool with considerable success
[1, 2, 3]. However, such a filter becomes very computationally expensive when a large number of coefficients
are required. A second order Volterra (SOV) filter was developed to cope with the enormous amount of
5

computations needed to obtain acceptable errors [1, 2, 3, 4].
Diffusion algorithms are the method for estimating parameters over adaptive networks, whose nodes can
collect noisy observations related to a certain parameter of interest [5]. Recently, some diffusion algorithms
have been proposed [5, 6]. These algorithms aimed at enhancing the linear estimation performance have
been presented in the literature, but few algorithms aimed at enhancing the nonlinear estimation capability of

10

diffusion algorithms have been investigated. Particularly, in [7], an interesting trial was attempted to nonlinear
adaptive learning by employing the kernel adaptive filter. Unfortunately, the structure of this method grows
linearly with the number of processed patterns, which prohibits its practical applications.
Motivated by these considerations, in this paper, we proposed a new diffusion algorithm for adaptive sparse
Volterra network (SVN). The parameters of SVN at every node are sparse, i.e., only a small portion of the

15

coefficients (called active coefficients) have large magnitude while the rest of the coefficients (called inactive
coefficients) are close or equal to zero. The development of the algorithm is based on an innovative approach:
the algorithms are introduced based on the minimization of cost functions with logarithmic dependence on
the adaptation error, instead of minimizing the pth power error. Moreover, these algorithms with l0 -norm
constraint are proposed to achieve improved performance for SVN identification.
I * Corresponding author at: School of Electrical Engineering, Southwest Jiaotong University, Chengdu, Sichuan, 610031, PR
China.
E-mail addresses: lulu@my.swjtu.edu.cn (L. Lu), hqzhao swjtu@126.com (H. Zhao).

Preprint submitted to Elsevier

January 16, 2018

20

2. Problem formulation
Consider the problem of estimating Volterra coefficients from a network with N sensor nodes. In each
iteration i, each sensor node k ∈ {1, 2, ..., N } has access to the realization of some zero-mean random process
{dk (i), uk.i }, where {·} represents a set, uk,i is an regression vector with length M , and dk (i) is the desired
signal. Suppose the measurements arising from the model
dk (i) = uk,i wo + vk (i)

(1)

where wo is the coefficients of the Volterra series model, and vk (i) is the measurement noise. Assume that uk,i
and vk (i) are spatially independent and independent identically distributed (i.i.d.), and vk (i) is independent
of uk,i . The output of the SOV network can be expressed as
yk (i) =wT uk,i
=

P
−1
X

hk,1 (m1 )xk (i − m1 )

(2)

m1 =0

+

P
−1
X

P
−1
X

hk,2 (m1 , m2 )xk (i − m2 )x(i − m1 )

m1 =0 m2 =m1

where (·)T denotes the transposition, xk (i) is the input data at node k, hr is the rth order Volterra kernel at
node k, and P is the length of Volterra system, P + P (P + 1)/2 = M . The expanded input vector uk,i and
the expanded coefficients vector w of SOV system are expressed by

T
uk,i = xk (i), ..., xk (i − P + 1), x2k (i), xk (i)xk (i − 1), ..., x2k (i − P + 1)

(3)

T

w = [hk,1 (0), ..., hk,1 (P − 1), hk,2 (0, 0), hk,2 (0, 1), ..., hk,2 (P − 1, ..., P − 1)]

(4)

3. Proposed algorithm
To obtain an improved performance, we adopted the local cost function that has logarithmic dependence
on the error [8], i.e.,
∆

Jkloc1 (w) =

X
l∈Nk



1
al,k E F (el,i ) − ln {δF (el,i )}
δ

(5)

where E(·) denotes the expectation, δ > 0, Nk is the set of nodes with which node k shares information
(including k itself), and the weighting coefficients {al,k } are real, non-negative. The function F (el,i ) denotes a
function of the error signal. Using the steepest descent algorithm, we hold the steepest descent adaptation as

X
∂ F (el,i ) − 1δ ln (δF (el,i ))
|wk,i−1
(6)
wk,i = wk,i−1 − µ
al,k
∂w
l∈Nk

where µ is the step size. Under the linear combination assumption [5], let us define the linear combination wk,i
P
at node k as wk,i−1 =
al,k ϕl,i−1 , where ϕk,i is the local estimates at node k. Moreover, in adaptation step,
l∈Nk

ϕk,i−1 is replaced by linear combination wk,i−1 . Such substitution is reasonable, since the linear combination
2

25

contains more data information from neighbour nodes than ϕk,i−1 [6]. Then, we can obtain the iterative
expressions for the proposed algorithms.
The diffusion least mean logarithmic square (dLMLS) algorithm
For F (el,i ) = E(e2l,i ) this simplifies into the dLMLS algorithm:
ϕk,i = wk,i−1 + µ

X

cl,k

l∈Nk

δuTl,i e3l,i
1 + δe2l,i

(7)

where {cl,k } is the non-negative weighting coefficients, satisfying the condition cl,k = al,k = 0 if l ∈
/ Nk .
The diffusion least logarithmic absolute difference (dLLAD) algorithm
The dLLAD algorithm is derived by substituting F (el,i ) = E(|el,i |) in the general formula given by (6). Its
update is given by
X

ϕk,i = wk,i−1 + µ

l∈Nk
30

cl,k

δuTl,i el,i
1 + δ|el,i |

(8)

where | · | denotes absolute value of a scalar.
The diffusion logarithmic least mean p-power (dLLMP) algorithm
For F (el,i ) = E(|el,i |pl,i ), we obtain
X

ϕk,i = wk,i−1 + µ

cl,k

l∈Nk

δuTl,i |el,i |2pl,i −1 sign(el,i )
1 + δ|el,i |pl,i

(9)

Remark 3.1: The proposed dLMLS algorithm is based on fourth-statistics of the error, and can therefore
achieve a smaller steady-state kernel error. The dLLAD algorithm intrinsically combines the l2 -norm and
l1 -norm. In impulsive noise environments, it may be expected to converge faster than the dLMS algorithm
35

does.
Remark 3.2: The dLLMP algorithm resembles the algorithm that we proposed in [4].

Because the

logarithmic-order class includes α-stable noise process, it can be used estimate the coefficient with reduced
negative effects of outliers. When pl,i = 1, it reduces to the dLLAD algorithm. When pl,i = 2, it becomes the
dLMLS algorithm.
40

Remark 3.3: For the Eq. (7-9) above, the diffusion algorithms obtain solutions via Adapt-then-Combine
(ATC) step. The Combine-then-Adapt (CTA) [5] algorithms can be easily derived by exchanging the order of
these two steps.
l0 -norm-based algorithms: In diffusion adaptation, one often encounters many systems with sparsity property, i.e., there are only a small number of nonzero entries in the impulse response at every node. To introduce
sparsity, we can minimize the following penalized local cost function with l0 -norm:


X
1
∆
loc2
Jk (w) =
al,k E F (el,i ) − ln (δF (el,i )) + λ||w||0
δ

(10)

l∈Nk

where || · ||0 denotes the l0 -norm, λ is a controller factor to balance the new penalty and the estimation error.
Similarly, the adaptation of logarithmic cost with l0 -norm can be given as

P
δF (ek,i )


cl,k ∇F (ek,i ) 1+δF (e
 ϕk,i = wk,i−1 − µ
) − ρ∇||wk,i−1 ||0
k,i

l∈Nk



 wk,i =

P

al,k ϕl,i

l∈Nk

3

(11)

where ρ = λµ. Since the minimization of the l0 -norm is a Non-Polynomial (NP) hard problem, ∇||wk,i−1 ||0 is
usually approximated by a first-order Taylor series expansion, i.e.,



−β 2 wm − β,
−1/β ≤ wm < 0


2
∇||wm ||0 =
−β wm + β,
0 < wm ≤ 1/β



 0,
elsewhere

(12)

where β ∈ [5, 20] is the positive constant. It should be noted that only w within the neighborhood of zero, i.e.,
[−1/β, 1/β] which is named attraction region, are attracted. Besides, in this region, the closer w is to zero, the
45

greater the attraction intensity is. If w is not in this region, no attraction will be performed. We can conclude
that based on the zero-attraction function in (12), the l0 -based algorithms can enhance the performance of
diffusion algorithm for SVN, because in such network the near-zero coefficients are dominant. Moreover, a
larger β results in a stronger intensity but a narrower attraction region, which affects the performances of the
algorithms.

50

4. Simulation results
We present the simulation results to verify the effectiveness of the proposed algorithms in comparison with
the diffusion LMS (dLMS) [5], and the diffusion LMP (dLMP) algorithm [9]. We consider a SVN composed
of 20 nodes. The unknown plant which is given in [9] is modelled as a second-order nonlinear system with
M =14 (P =4). The Gaussian signal with zero mean and unit variance is employed as the exciting input. In
the simulation study, the effectiveness is assessed in terms of network mean-square deviation (NMSD), which
is defined as
∆

NMSD =

N
1 X
||wk − wo ||22
N

(13)

k=1

∆

The noise is modelled by standard symmetric α-stable (SαS) distribution η(t) = exp (−|t|α ) [4], where α is a
characteristic exponent. In all our simulations, the curves are drawn from the average of 25 independent runs.
In first example, the noise signal is generated with α = 2, which corresponds to the Gaussian distribution.
Fig. 1 illustrates the NMSD curves for algorithms. It can be observed that the proposed algorithms outperform
55

the dLMS algorithm. The only sacrifice to make is a slow convergence rate at the initial stage.
In second example, α of all nodes are ranging from 1.2 to 1.8. The learning curves for the algorithms are
plotted in Fig. 2, where p in dLLMP is chosen by random. We can clearly see that the dLLMP l0 method
is very robust to the impulsive noise and achieves small kernel estimation error. The other algorithms, in
contrast, have large fluctuations during the adaptation.

60

5. Conclusion
An adaptive SVN using a class of logarithmic cost algorithms has been proposed and investigated in
detail through simulations. In Gaussian noise environment, the proposed algorithms enjoy smaller kernel
misadjustment as compared with the dLMS algorithm. If the network is corrupted by -stable noise, the

4

Figure 1: NMSD in SVN for α = 2.

Figure 2: NMSD in SVN for SαS noise.

proposed dLLMP algorithm will provide overwhelmingly better stability in comparison with the existing
65

algorithms.

Acknowledgments
This work was partially supported by National Science Foundation of P.R. China (Grant: 61571374,
61271340, 61433011). The first author would also like to acknowledge the China Scholarship Council (CSC)
for providing him with financial support to study abroad (No. 201607000050).

5

70

References
[1] C. Contan, B. S. Kirei, et al., Modified NLMF adaptation of Volterra filters used for nonlinear acoustic
echo cancellation, Signal Process. 93 (5) (2013) 1152–1161.
[2] H. Zhao, L. Lu, Z. He, B. Chen, Adaptive recursive algorithm with logarithmic transformation for nonlinear
system identification in α-stable noise, Digit.l Signal Process. 46 (2015) 120–132.

75

[3] L. Lu, H. Zhao, Adaptive volterra filter with continuous lp -norm using a logarithmic cost for nonlinear
active noise control, J. Sound Vib. 364 (2016) 14–29.
[4] L. Lu, H. Zhao, B. Chen, Improved-variable-forgetting-factor recursive algorithm based on the logarithmic
cost for Volterra system identification, IEEE Trans. Circuits Syst. II 63 (6) (2016) 588–592.
[5] A. H. Sayed, Adaptive networks, Proc. IEEE 102 (4) (2014) 460–497.

80

[6] R. Abdolee, B. Champagne, A. H. Sayed, Estimation of space-time varying parameters using a diffusion
LMS algorithm, IEEE Trans. Signal Process. 62 (2) (2014) 403–418.
[7] S. Chouvardas, M. Draief, A diffusion kernel LMS algorithm for nonlinear adaptive networks, in: International Conference onAcoustics, Speech and Signal Processing (ICASSP), IEEE, 2016, pp. 4164–4168.
[8] M. O. Sayin, N. D. Vanli, S. S. Kozat, A novel family of adaptive filtering algorithms based on the

85

logarithmic cost, IEEE Trans. Signal Process. 62 (17) (2014) 4411–4424.
[9] F. Wen, Diffusion least-mean p-power algorithms for distributed estimation in alpha-stable noise environments, Electron. Lett. 49 (21) (2013) 1355–1356.

6

