DFE/THP DUALITY FOR FBMC WITH HIGHLY FREQUENCY SELECTIVE CHANNELS
Hela Jedda, Leonardo G. Baltar, Oliver De Candido, Amine Mezghani, Josef A. Nossek
Institute for Circuit Theory and Signal Processing
Technische Universität München
80290 Munich, Germany
Email: {hela.jedda, leo.baltar}@tum.de

arXiv:1706.08718v1 [] 27 Jun 2017

ABSTRACT
Filter bank based multicarrier with Offset-QAM systems
(FBMC/OQAM) are strong candidates for the waveform of
future 5-th generation (5G) wireless standards. These systems can achieve maximum spectral efficiency compared to
other multicarrier schemes, particularly in highly frequency
selective propagation conditions. In this case a multi-tap,
fractionally spaced equalizer or precoder needs to be inserted in each subcarrier at the receiver or transmitter side to
compensate inter-symbol interference (ISI) and inter-carrier
interference (ICI). In this paper we propose a new TomlinsonHarashima precoder (THP) design for FBMC/OQAM based
on the mean squared error (MSE) duality from a minimum
MSE (MMSE) designed decision feedback equalizer (DFE).
Index Terms— Filter Bank Multicarrier, Offset-QAM,
Decision Feedback, Tomlinson-Harashima, MSE duality
1. INTRODUCTION
In recent years multi-carrier systems have been at the forefront of communication systems due to their attractive properties at high data rates. Orthogonal frequency division multiplexing with a cyclic prefix (CP-OFDM) is a widely implemented solution for multi-carrier systems in standards such as
IEEE 802.11, LTE or VDSL. Its popularity is partly due to the
simple equalization enabled by the CP and the efficient implementation using Fast Fourier Transform (FFT). However, this
comes at the price of a loss in spectral efficiency due to the CP,
which is extremely long in the presence of highly frequency
selective channels. CP-OFDM additionally suffers from high
out-of-band emissions and the necessity of perfect synchronization.
An alternative solution to CP-OFDM are FBMC/OQAM
systems which are a strong contender for 5G mobile communication systems [1]. FBMC/OQAM systems have improved
spectral efficiency due to the Synthesis and Analysis Filter
Banks (SFB and AFB) at the transmitter and receiver [2],
The authors acknowledge the financial support by the EU FP7-ICT
project EMPhAtiC (http://www.ictemphatic.eu) under grant agreement no.
318362.

which guarantees higher selectivity in the frequency domain
and a much lower out-of-band radiation compared with CPOFDM [3]. This form of pulse shaping limits the ICI, whilst
simultaneously attributing to more ISI within each individual sub-carrier. Furthermore, FBMC/OQAM systems are extremely efficient in the presence of highly frequency selective
channels. These advantages over CP-OFDM come at the cost
of slightly higher computational complexity, however, this is
not problematic [4].
In [5] an MMSE-based, multi-tap, per sub-carrier, linear
equalizer design for a Single Input Single Output (SISO)
setting was introduced to compensate the ISI and ICI in the
presence of highly frequency selective channels. In [6] a
DFE extension of [5] was proposed. In situations where the
channel impulse response is known at the transmitter side, it
is usually preferred to employ a precoder in order to avoid
noise coloring and noise power amplification at the receiver.
Additionally, if there is a symmetry in the computational
power, e.g. cellular networks, where most of the processing
can be concentrated at the base station, i.e. precoding in the
downlink (DL) and equalization in the uplink (UL). In [7]
the authors proposed MIMO-FBMC precoders based on the
signal-to-leakage plus noise ratio (SLNR) and on the signalto-interference plus noise ratio (SINR). In [8] three precoders
are inserted in each subcarrier to combat ICI. In [9] a THP
is proposed for frequency flat channels in each subcarrier.
In [10] linear precoders based on the MSE duality with the
linear equalizer was extended for the Multi-User Multiple
Input Single Output (MU-MISO) setting. In this contribution
we propose a THP design for SISO FBMC/OQAM systems
using the MSE duality and the DFE design from [6].
This paper is organized as follows: in Section 2 we introduce the FBMC system model. In Section 3 we calculate the
MSE for the DFE and show its MMSE solution as in [6]. In
Section 4 we explain the THP and define its MSE. We explore
two different methods to transform the DFE into a dual THP
in Section 5. In Sections 6 and 7 we interpret the simulation
results and summarize this work.
Notation: Bold letters indicate vectors and matrices, nonbold letters express scalars. The operators (.)∗ , (.)T , (.)H and
E [.] stand for complex conjugation, the transposition, Hermi-

tian transposition and the expectation, respectively. The n× n
identity matrix is denoted by In while the zeros (ones) matrix
with n rows and m columns is defined as 0n,m (1n,m ). We
(R)
(I)
define (•)
= ℜ{•}, (•) = ℑ{•}. The operator x̄ concatenates vertically the real- and imaginary valued parts of the
vector/matrix x. The vector el represents a zero vector with 1
in the l-th position.

the frequency response of the filter hk [r] only significantly
overlaps with the two adjacent filters.
For a simple total transmission notation we define the
following filtering and downsampling operation, hl,k [n] =
(hl ∗hch ∗hk ) [r] |r=n M . This represents the total impulse
2
response from the sub-carrier l at the transmitter into the
sub-carrier k at the receiver, with
l l ∈ {k − 1,
m k, k + 1}. The
2(L −1)+L

2. FBMC SYSTEM MODEL
In a SISO FBMC system, the SFB combines the M complex
valued QAM input signals dk [m], k = 1, ..., M , generated at
a rate of 1/Ts, into a single complex valued signal t[r] at a
higher rate of M/Ts . The signal is transmitted across a highly
frequency selective additive white Gaussian noise channel to
the receiver. In our system, M corresponds to the total number of sub-carriers and Mu the number of sub-carriers we
transmit across. The AFB separates the received signal back
into its Mu components at the lower rate 1/Ts per sub-carrier.
The first operation in the SFB is the O-QAM staggering
of the input dk [m] and the output sequence xk [n] reads as
h
iT
 αk [m] jβk [m] αk [m − 1] · · · , k + n is odd,
i
xk [n] = h
 jβ [m] α [m] jβ [m − 1] · · · T , k + n is even.
k
k
k
(R)

The input symbol dk [m] is split into its real dk [m] = αk [m]
(I)
and imaginary jdk [m] = jβk [m] parts, up-sampled by a factor of 2, then depending on which sub-carrier we observe,
either αk [m] or jβk [m] symbol is delayed by exactly Ts /2
and finally these components are added together. Due to this
characteristic of the O-QAM symbols, there is a phase change
of π/2 between immediately adjacent sub-carriers, ensuring
orthogonality between them. At the receiver, the AFB applies O-QAM de-staggering to reconstruct the complex QAM
dˆk [m] symbols from the equalized x̂k [n] symbols.
Since an implementation as described above is not very
efficient due to the extremely high data rate, an implementation as a Modified DFT (MDFT) filter bank is much more
efficient. An MDFT filter bank takes advantage of exponentially modulated, pulse shaping filters given by
hk [r] = hp [r] exp



j

2π
k
M



r−

Lp − 1
2



, r = 0, . . . , Lp − 1,

where hp [r] is a lowpass prototype filter with length Lp =
KM + 1, with K representing the overlapping factor to indicate the number of symbols which overlap in time. K should
be kept as small as possible not only to limit the complexity
but also to reduce the time-domain spreading of the symbols
and the transmission latency. Furthermore, MDFT takes advantage of the polyphase decomposition of hp [r] so that the
filtering can be performed at a rate of only 2/Ts .
To minimize the complexity in the calculations of the
equalizer and precoder vectors, we set K = 4 and the roll-off
factor of our root raised cosine filter equal to one. Thus,

p
ch
resulting filter has length N =
.
M/2
In the following sections we work with a purely real notation and therefore define a purely real input sequence as x′k [n]
where the relation xk [n] = Jk,n x′k [n] holds with

h
i
diag 1 j 1 j · · · , k + n is odd,
h
i
Jk,n =
diag j 1 j 1 · · · , k + n is even.

This extracts the imaginary j’s from the input signal. We then
multiply the transposed convolution matrix of hl,k [n] by Jk,n
and are left with H′l,k = Hl,k Jk,n .
3. DECISION FEEDBACK EQUALIZER

We are interested in designing a multi-tap decision feedback
UL
equalizer per sub-carrier with feed-forward (FF) filter f2,k
of
UL
length Lf and feedback (FB) filter bk of length Lb and a
UL
single-tap precoder f1,k
∈ R+ per sub-carrier. We assume
there is no channel state information at the transmitter and,
UL
therefore, set the scalar precoder to one, i.e. f1,k
= 1. The
received and equalized signal in sub-carrier k, when n + k is
odd, i.e. for α̂k [m], is defined as
x̂k [n] = wkT (Ak x1,k + Bk x2,k + Ξk η̄) ,

(1)

where we define


UL
f̄2,k
(2Lf +Lb )
wk =
,
(R) ∈ R
)
(bUL
k


H̄′k,k
02Lf ×Lb
Ak =
∈ R(2Lf +Lb )×(N +Lf +Lb −1) ,
0Lb ×(N +Lf −1) −ILb


H̄′k−1,k
H̄′k+1,k
∈ R(2Lf +Lb )×2(N +Lf −1) ,
Bk =
0Lb ×2(N +Lf −1)


Γk
,
Ξk =
0Lb ×2(Lf +Lp −1)


(Hk )(R) −(Hk )(I)
Γk =
∈ R2Lf ×2(Lf +Lp−1) ,
(Hk )(I) (Hk )(R)


x′k [n]
x1,k = ′
∈ R(N +Lf +Lb −1) ,
xk [n − (ν + 1)]
 ′

x
[n]
x2,k = k−1
∈ R2(N +Lf −1) ,
(2)
x′k+1 [n]
where Hk is the M/2 downsampled, transposed convolution
matrix of hk [r] which filters the noise η and ν is the equalization latency in our system. For the following derivations

we assume the input signals to be independent and identically distributed (i.i.d.) and Gaussian distributed.
The co
variance matrix of x′k [n] is defined as E x′k [n]x′T
k [n] =
σx2 I. Furthermore, we assume the additive
noise
is
Gaussian

distributed with η[n] ∼ NC 0, ση2 I .
The MMSE equalizer is given by
wk = arg min ǫUL
k
wk

= arg min E kx̂k [n] − xk [n − ν]k22
2
T
= Ak ΨAT
k + σx Bk Bk +

2

Ξk ΞT
k

xl [n]
+ +
−
al [n]

vl [n]

bDL
l [n]

T
2

DL
f1,l
[n]

tl [n]

+ hl,k [n]

DL
f2,k

tl+1 [n]

x̂k [n]
+
−ãk [n]

Fig. 1. FBMC with THP subcarrier model

wk

ση2

tl−1 [n]

!−1

Ak Ψeν+1 ,

where




Υ
2 IN +Lf −1
2
=
σ
Ψ = E kx1,k xT
k
x
1,k 2
ΥT
ILb
h
iT

0Lb ×(ν+1) ILb 0Lb ×(L−Lb ) , if L > Lb



iT

h
, if L = Lb
0
I
L
b
Υ = " Lb ×(ν+1)
#



0

 (ν+1)×Lb
, if L < Lb ,

IL
0L×(Lb −L)

(3)

where L = Lf + N − ν − 2. The MSE can be expressed by
!
k+1
X
ση2
UL,T
T
UL
UL
2
′
′T
Γk Γk f̄2,k
ǫk = f̄2,k
σx
H̄l,k H̄l,k +
2
l=k−1



UL,T ′
T
2
H̄k,k r ,
+ σx r r − 2 f̄2,k


T (R)
with rT = 01×ν 1 ((bUL
01×(L−Lb ) . (4)
k ) )

It can be shown, that calculating the precoder or equalizer filters with either the real part α̂k [m] (n + k is odd) or
imaginary part β̂k [m] (n + k is even) result in the same solution [5, 11].
4. THOMLINSON HARASHIMA PRECODER
DL
We define a per sub-carrier, multi-tap THP with FF filter f1,k
and FB filter bDL
k and a per sub-carrier single-tap equalizer
DL
f2,k
∈ R+ at the receiver. Since we have a feedback loop in
the THP, we are at risk of stability problems. Therefore, we
additionally introduce the modulo operator M (.) at the transmitter and the receiver in the DL scenario, that upper bounds
the output signals. The modulo operator that is adapted to the
O-QAM structure is expressed as

 (R)

x
[n]

(R)

if l + n is odd
xl [n] − l τ + 21 τ,
 (I)

M (xl [n]) =
x [n]

(I)

jxl [n] − j l τ + 21 τ, if l + n is even,

where τ ∈ R+ is the modulo constant and depends on the

modulation alphabet [12]. The modulo operators can be replaced by the summations of the signals al [n] and −ãk [n],
which are multiples of τ , as shown in Fig. 1. The output
of the modulo operator at the transmitter in sub-carrier l is
denoted by vl [n] and we define vl [n] = Jl,n vl′ [n] with co

2
variance matrix E vl′ [n]vl′T [n] = σv2 I and σv2 = τ12 [13].
The MSE of the DL scenario can be expressed by
n
o
2
ǫDL
k = E kx̂k [n] − xk [n − ν]k2
!
k+1
2
X DL,T
σ
η
DL
DL 2
f̄1,l H̄′l,k H̄′T
) σv2
= (f2,k
khp k22
l,k f̄1,l +
2
l=k−1



2
T
DL DL,T ′
+ σv s s − 2 f2,k
f̄1,k H̄k,k s


T (R)
with sT = 01×ν 1 ((bDL
(5)
01×(L−Lb ) .
k ) )
5. DFE/THP MSE DUALITY TRANSFORMATION
In this Section we investigate two different methods of transforming the DFE into an equivalent THP using the duality
principle as introduced in [14]. The basic idea behind an MSE
duality transformation is to switch the roles of the UL and DL
filters, i.e. we interchange each receiver filter in the UL scenario with the respective transmitter filter in the DL scenario.
As the dual DL scenario has purely transmitter processing,
we must ensure that the transmit power is subsequently limited, thus we weigh every transmitter filter with a strictly real
constant and multiply the receiver with its inverse.
These two duality transformation methods are summarized as follows:
• In Subsection 5.1 we aim at conserving the Sum-MSE.
This is the simplest form of duality since the Sum-MSE
of all sub-carriers is kept equal when transforming the UL
to a DL scenario. To this end, a single scaling factor is required which leads to relatively low computational complexity.
• In Subsection 5.2 we aim at conserving the Sub-Carrier
MSE. In this method the MSE per sub-carrier is preserved
when transforming the UL to a DL scenario resulting in
an individual scaling factor for each sub-carrier. We get a
linear system of equations for Mu scaling factors, which
results in a higher computational complexity than a SumMSE transformation.

In both of these MSE duality transformations the total trans2
PMu
DL
mit power is preserved, i.e. k=1
f1,k
≤ Mu .

We end up with a system of linear equations to solve for Mu
scaling factors γk

2

5.1. Sum-MSE
First, we define a relation between the UL and DL filters with
a real-valued scaling factor for all sub-carriers such that
DL
UL
DL
UL
UL
f̄1,k
= γ f̄2,k
; f2,k
= γ −1 f1,k
= γ −1 ; bDL
k = bk .

with γ ∈ R+ and recalling that the UL precoder scalar is set
UL
such that f1,k
= 1, ∀k. To perform the Sum-MSE duality, the
Sum-MSE is set equal between the UL and the DL scenario,
i.e. we sum over all sub-carriers and set them equal
Mu
X

!
ǫDL
k =

k=1

Mu
X

ǫUL
k .

(6)

k=1

We get the following expression of the scaling factor γ
2

γ =
δ=

Mu

ση2
2
2 khp k2

δ

Mu
X

UL,T
f̄2,k

k=1

−
+

k+1
X

σv2

UL,T ′
UL
f̄2,l
H̄l,k H̄′T
l,k f̄2,l

l=k−1

(σx2

−

σv2 )

ση2
Γk ΓT
+
k
2
!

H̄′l,k H̄′T
l,k

l=k−1
k+1
X

!

UL
f̄2,k

!


UL,T ′
r r − 2 f̄2,k H̄k,k r
.
T

(7)

This form of duality transformation guarantees equality in
the Sum-MSE between the UL and the DL scenario. Therefore, this method can be interpreted as allocating an equal
amount of transmit power whilst spreading this transmit
power across the sub-carriers as required. The disadvantage of this method arises if the MSE of certain sub-carriers
is disproportionately large. This leads to these sub-carriers
obtaining a greater amount of transmit power.
5.2. Sub-Carrier MSE (SC-MSE)
Second, we define a relation between the UL and DL filters
with a real-valued scaling factor per sub-carrier such that
UL
DL
DL
; f2,k
f̄1,k
= γk f̄2,k

UL
UL
= γk−1 f1,k
= γk−1 ; bDL
k = bk ,

with γk ∈ R+ and recalling that the UL precoder is set such
UL
that f1,k
= 1, ∀k. We then set sub-carrier MSE equal between the UL and the DL scenario, i.e. we set the individual
MSE expressions per sub-carrier equal such that
!

2
· · · γM
u

T

=

ση2
2
khp k2 1Mu , where
2


  UL,T ′
UL T

H̄k,k H̄′T
σx2 − σv2 f̄2,k

k,k f̄2,k r r





UL,T
UL,T ′

2

σx
−2f̄2,k H̄k,k r + f̄2,k



2

P
σ

k+1
UL


H̄′ H̄′T + 2η Γk ΓT

k f̄2,k
 l=k−1,l6=k l,k l,k
T̃k,j =

UL,T

UL

H̄′k−1,k H̄′T
−σv2 f̄2,k−1

k−1,k f̄2,k−1








UL
′T
′
2 UL,T

−σv f̄2,k+1 H̄k+1,k H̄k+1,k f̄2,k+1


0

(9)

, if j = k
, if j = k − 1
, if j = k + 1
else.

This duality transformation guarantees that each SC-MSE
remains equal for both UL and DL scenario. Therefore, we
cannot spread the transmit power amongst the sub-carriers but
instead we normalize the filter in each sub-carrier individually.
6. SIMULATION RESULTS

, with

σx2


T̃ γ12

UL
ǫDL
k = ǫk ,

∀k.

(8)

Throughout our simulations we transmitted data across
Mu = 210 of the available M = 256 sub-carriers. The used
multipath fading channel is based on bad urban area model
(BU) with 6 taps. We used a sampling rate of fs = 15.36MHz
and a channel impulse response of duration Lch = 110 samples. Thus, we had a subcarrier distance of 60kHz. We
used randomly generated 16-QAM symbols and took a block
length of 1000 symbols per sub-carrier. For the used modulation scheme,
we have following modulo constant value
√
τ = 8/ 10. With the chosen system configurations, especially due to Lch = 110 and the highly frequency selective
channel, a CP-OFDM system would have required a CP with
a minimum length of 109. This would have limited the datathroughput of the CP-OFDM to almost 50%, therefore we
have not included a direct comparison in the simulation results. Throughout the simulations we took the quantity of
Eb /N0 to be a pseudo-signal-to-noise ratio. We took the
Bit Error Rate (BER) and MSE as an average over all subcarriers. We took an average over 200 randomly generated
channel realizations.
We investigated six equalizers and precoders designs: a
linear equalizer of length Llin = 9 taps, its two dual precoders
(SC-MSE and Sum-MSE), DFE with FF filter with Lf = 7
and FB filter with Lb = 4 taps and its two dual THPs. Note
that the complexity of the FB filter is proportional to half of
its length Lb , since its input and output are either purely real
or purely imaginary.
As can be concluded from Fig. 2 and Fig. 3, the non-linear
processing (DFE/THP) outperform the linear processing significantly in medium to high Eb /N0 regime for the same complexity. Second, the Sum-MSE duality seems to be more beneficial than the SC-MSE duality. Third, the Sum-MSE dual

REFERENCES

0

10

THP, SC−MSE
THP, Sum−MSE
Linear Prec., SC−MSE
Linear Prec., Sum−MSE
Linear Eq.
DFE

−1

BER

10

[1] P. Banelli et al., “Modulation formats and waveforms
for 5G Networks: Who will be the heir of OFDM?,”
Signal Proc. Mag., IEEE, vol. 31, no. 6, Nov 2014.
[2] P. Siohan, C. Siclet, and N. Lacaille, “Analysis and
design of OFDM/OQAM systems based on filterbank
theory,” IEEE Trans. on Signal Proc., vol. 50, no. 5, pp.
1170–1183, 2002.

−2

10

[3] L. G. Baltar, D.S. Waldhauser, and J. A. Nossek, “Outof-band radiation in multicarrier systems: A comparison,” in 6th Int. Workshop on Multi-Carrier SpreadSpectrum, May 2007.

−3

10

−4

10

−5

0

5

10

15
E /N (dB)
b

20

25

30

35

0

Fig. 2. BER performance comparison

[4] L. G. Baltar, F. Schaich, M. Renfors, and J. A. Nossek,
“Computational complexity analysis of advanced physical layers based on multicarrier modulation,” in Future
Networks & Mobile Summit, Jun. 2011.
[5] D. S. Waldhauser, L. G. Baltar, and J. A. Nossek,
“MMSE subcarrier equalization for filter bank based
multicarrier systems,” in IEEE 9th Workshop on Signal
Proc. Adv. in Wireless Comm., Jul. 2008, pp. 525–529.

0

10

−1

MSE

10

[6] L.G. Baltar, D. S. Waldhauser, and J. A. Nossek,
“MMSE subchannel decision feedback equalization for
filter bank based multicarrier systems,” in IEEE Int.
Symp. on Circuits and Syst., Mai. 2009, pp. 2802–2805.

−2

10

−3

10

−4

10

−5

measured (THP, SC−MSE)
measured (THP, Sum−MSE)
analytical (THP, SC−MSE)
analytical (THP, Sum−MSE)
measured (Linear Prec., SC−MSE)
measured (Linear Prec., Sum−MSE)
analytical (Linear Eq.)
measured (Linear Eq.)
measured (DFE)
analytical(DFE)
0

5

10

15
E /N (dB)
b

[7] M. Caus and A.I. Perez-Neira, “Transmitter-receiver
designs for highly frequency selective channels in
MIMO FBMC systems,” IEEE Trans. on Signal Proc.,
vol. 60, no. 12, pp. 6519–6532, Dec. 2012.
20

25

30

35

0

Fig. 3. MSE performance comparison

THP compared to DFE shows an improved BER performance
and similar behavior in terms of MSE.
7. CONCLUSIONS
In this work, we proposed a new method to design a dual THP
scheme from the DFE of a SISO FBMC system. To this end,
we made use of the MSE duality transformation between an
UL and a DL scenario, where either the Sum-MSE or the SCMSE is conserved. Throughout our simulations we observed
first that the non-linear processing (DFE/THP) outperform the
linear processing in medium to high Eb /N0 region for the
same complexity. Second, we have seen that the Sum-MSE
duality performed the best in terms of BER over the whole
Eb /N0 regime, which could be explained by its equivalence
to an inverse water-filling technique, i.e. the subcarriers with
poorer channel response are allocated more power.

[8] F. Horlin et al., “Interference-free SDMA for FBMCOQAM,” EURASIP J. on Adv. in Signal Proc., vol. 46,
Mar. 2013.
[9] M. Caus and A. I. Perez-Neira, “SDMA for filterbank
with Tomlinson Harashima precoding,” in Proc. IEEE
Int. Comm. Conf., Jun. 2013.
[10] O. De Candido, L. G. Baltar, A. Mezghani, and J. A.
Nossek, “SIMO/MISO MSE-Duality for Multi-User
FBMC for Highly Frequency Selective Channels,” in
Proc. Int. ITG Workshop on Smart Antennas, Mar. 2015.
[11] M. Newinger et al., “MISO broadcasting FBMC system
for highly frequency selective channels,” in Proc. Int.
ITG Workshop on Smart Antennas, Mar. 2014.
[12] R. F. H. Fischer, Precoding and Signal shaping for Digital Transmission, John Willey & Sons, 2002.
[13] M. Joham, Optimization of Linear and Nonlinear
Transmit Signal Processing, Ph.D. thesis, Technische
Universität München, 2004.
[14] A. Mezghani et al., “Iterative THP transceiver optimization for multi-user MIMO systems based on weighted
SUM-MSE minimization,” in IEEE 7th Workshop on
Signal Proc. Adv. in Wireless Comm., Jul. 2006.

