Design of Capacity Approaching Ensembles of LDPC Codes for Correlated Sources using EXIT Charts
Mohamad Khas Mohamadi Hamid Saeedi Reza Asvadi





Design of Capacity Approaching Ensembles of LDPC Codes for Correlated Sources using EXIT Charts
    Mohamad Khas, Hamid Saeedi, Member, IEEE, and Reza Asvadi, Member, IEEE 
	M. Khas and H. Saeedi are with the Department of Electrical
		Engineering, Tarbiat Modares University, Tehran, Iran (e-mail:
		hsaeedi@ieee.org).
	R. Asvadi is with the Faculty of Electrical Engineering, Shahid Beheshti University (SBU),
		Tehran, Iran. (e-mail: r_asvadi@sbu.ac.ir). 
    December 30, 2023
==========================================================================================================================================================================================================================================================================================================================================================================




This paper is concerned with the design of capacity approaching ensembles of Low-Densiy Parity-Check (LDPC) codes for correlated sources. We consider correlated binary sources where the data is encoded independently at each source through a systematic LDPC encoder and sent over two independent channels. At the receiver, a iterative joint decoder consisting of two component LDPC decoders is considered where the encoded bits at the output of each component decoder are used at the other decoder as the a priori information.  We first provide asymptotic performance analysis using the concept of extrinsic information transfer (EXIT) charts. Compared to the conventional EXIT charts devised to analyze LDPC codes for point to point communication, the proposed EXIT charts have been completely modified to able to accommodate the systematic nature of the codes as well as the iterative behavior between the two component decoders. Then the developed modified EXIT charts are deployed to design ensembles for different levels of correlation. Our results show that as the average degree of the designed ensembles grow, the thresholds corresponding to the designed ensembles approach the capacity. In particular, for ensembles with average degree of around 9, the gap to capacity is reduced to about 0.2dB. Finite block length performance evaluation is also provided for the designed ensembles to verify the asymptotic results.

 








§ INTRODUCTION



Source and channel encoding/decoding of correlated sources has been the subject of several studies <cit.>. Perhaps the most immediate example of correlated sources is in sensor networks in which each sensor measures the data, encode it to bits and transmits it to a central node for decoding <cit.>. The correlation of the encoded bits comes from the fact that in many cases, several sensors measure the same phenomenon. The closer the sensors are, the larger the degree of  correlation will become in most cases.
 On the other hand, due to energy limitation in sensors which is enforced to increase the sensor lifetime, it is essential that the data is transmitted with the lowest possible energy while maintaining the required bit error rate. Consequently, channel coding is usually deployed at each sensor prior to transmission. At the central node, the channel decoder should be applied to each block of data received from each sensor. Now if the received bit streams are correlated, it is natural to consider joint channel decoding to take advantage of such a correlation.

Low-density Parity-check (LDPC) codes <cit.> have been widely suggested to be deployed in sensor networks due to their remarkable performance and reasonable decoding complexity <cit.>. For point to point communications, sequences of capacity approaching ensembles over memoryless Gaussian channels have been proposed in <cit.> where for a given code rate, the threshold of the ensemble is numerically shown to approach the Shannon capacity as the average check node degree increases. For the binary erasure channels, capacity achieving sequences of ensembles have been designed and their thresholds have been analytically shown to achieve the capacity <cit.>.

For point to point LDPC codes, different tools and techniques have been deployed for ensemble design. The most well-known analysis tool is density evolution (DE) <cit.>. To reduce the design complexity, an important alternative tool known as Extrinsic Information Transfer (EXIT) chart was proposed in <cit.>, <cit.> based on the assumption that the exchanged messages between variable nodes (VN) and check nodes (CN) of the corresponding Tanner graph can be approximated by consistent[For a consistent Gaussian random, the variance of the distribution is twice its mean.] Gaussian random variables.




In this paper, we consider the problem of joint channel decoding of LDPC-encoded correlated binary sources. We consider a simplified model where two sources generate
correlated binary bit streams. Then the streams are fed to an LDPC encoder blockwise and sent through two independent additive white Gaussian noise (AWGN) channels as shown in Fig. <ref>. Then the streams of data are received blockwise by the central node and are fed-back to the joint LDPC decoder proposed in <cit.> to obtain the original bit streams. In this decoder, two types of iterations, namely, inner and outer iterations are deployed such that at each outer iteration, the output of one decoder is used as the a priori information of the other decoder while the inner iterations are performed similar to a conventional LDPC message passing decoder. 

Our aim in this paper is to design capacity approaching ensembles of LDPC codes in which we show that the decoding threshold of the joint decoding of the designed ensembles tends to the capacity limit obtained in <cit.> as the average check node degree of the designed ensembles grow. We in fact obtain tables of degree distributions (similar to those of <cit.> proposed for the point to point scenario) for different levels of correlation in the source bits for the first time. The claimed capacity approaching thresholds are then verified by finite block length simulations.

To design the ensembles, we use the concept of EXIT charts. There are, however, important obstacles to apply the original EXIT chart scheme to our case that are addressed in this paper. First, as there are two decoders in place, the EXIT curve corresponding to the variable node of each decoder should be generated taking into account the a priori information from the other decoder. Second, as we consider systematic codes, the corresponding Tanner graph in our case has a two edge-type structure <cit.>, one corresponding to the message bits and one corresponding to the  parity bits. Therefore, two types of variable node EXIT curve have to be considered and the corresponding degree distributions have different structure than the conventional non-systematic LDPC codes. We show that with reasonable average check node degree, we can get as close as 0.2dB of the Shannon limit for different amount of correlations.



The organization of the paper is as follows. In Section <ref>, basic concepts and notations related to the source and correlation model and the Shannon limit are given. In Section <ref>, we describe the algorithm for iterative joint channel decoding of correlated sources and propose the two edge-type structure for this model. In Section <ref>, we propose the modified EXIT charts for joint iterative LDPC decoding of correlated sources and analyse the performance of regular and irregular LDPC codes. The code design procedure is presented in Section <ref>. The simulation results and numerical examples are summarized in Section <ref>. Finally, Section <ref> draws the conclusion.



§ PRELIMINARIES



 §.§ Source and Correlation Model 


Consider the two binary memoryless sources (U_1,U_2) that generate binary sequences segmented in blocks of length K and denoted by u_1={u_1,1,u_1,2,…,u_1,K} and u_2={u_2,1,u_2,2,…,u_2,K}.

The bits within each sequence are assumed to be i.i.d with equal probability of being zero and one. <cit.>.

Let z=u_1⊕u_2 be the component-wise addition modulo-2 of the two sources output. The vector z=(z_1,…,z_k) implies correlation between the two sources and it is called correlation vector. We define the empirical correlation between these two sources as p =γ /K where γ is the number of zeros in z. Obviously, sequences with the empirical correlation values of p and 1-p have the same entropy value <cit.>. This correlation can be generated by simply passing tone of the sequences through a binary systematic channel (BSC) with transition probability 1-p to generate the sequence for the other source. 

A joint channel decoding for the correlated sources is considered, where the sources are independently encoded by identical LDPC encoders, i.e., encoders have no communication. The encoders map a k bit vector corresponding to u_1 (u_2) to n_1 (n_2) bit vector of x_1 (x_2). The code rate of each encoder is then equal to RR_c_1=K/n_1.
 and R_c_2=k/n_2. In what follows, we assume that the code rates are the same (symmetric system), i.e, R_c_1=R_c_2=R_c , or equivalently, n_1=n_2=n
<cit.>. Each source is encoded according to a systematic (n,K) LDPC code. Hence, the generated codeword c is the augmentation  of information bit  u and parity bit p vectors. The binary phase-shift keying (BPSK) modulation is used before sending a codeword c over the AWGN channel. 



 §.§ Theoretical limit

To be able to evaluate the performance of the joint-decoding, the Shannon-SW limit is considered <cit.>.
In our simulations, we employ the energy per generated source, denoted by E_so, which is related to the energy per information bit, denoted by E_b, and the energy per transmitted symbol, denoted by E_s, as follows <cit.>:

    2E_so=H(U_1,U_2)E_b=(1/R_c_1+1/R_c_2)E_s,

where H(U_1,U_2) represents the joint entropy between the two correlated sources and E_s equals 1 in BPSK modulation. Since U_1 and U_2 are  uniformly distributed binary sources, the binary entropy of each source is equal 1, i.e, H(U_1)=H(U_2)=1. Then, H(U_1|U_2)=H(U_2|U_1)=h_2(p), where h_2(p) is the entropy of the empirical correlation p and H(U_1,U_2)=H(U_1)+h_2(p). In the considered symmetric system, reliable transmission over a channel pair is possible as long as E_so/N_0 satisfies the Shannon-SW condition <cit.>

    E_so/N_0>1/R_c(2^H(u_1,u_2)R_c-1),

where N_0 is the noise power spectral density.





§ SYSTEM MODEL




 §.§ Two edge-Type LDPC Codes 

In this section, a two edge-type LDPC code and its associated graph are presented. Consider the parity check matrix H_m× n of an LDPC code represented by a Tanner graph <cit.>, denoted by 𝒢=(𝒱,𝒞,ℰ), where 𝒱 and 𝒞 denote a set of n VN and m CN, respectively, and ℰ is a set of edges of the graph. According to 𝒢, we have an edge e={v_i,c_j}∈ℰ, where a VN v_i ∈𝒱 is connected to a CN c_j ∈𝒞 in 𝒢, if and only if, h_i,j=1 in H=[h_i,j].

Conventional LDPC codes are described asymptotically by coefficient pairs (λ,ρ) or degree distribution polynomials of the VN and CN as follows:

    λ (x)=∑_i=2^D_vλ_i x^i-1, ‎  ‎ ρ (x)=∑_j=2^D_cρ_j x^j-1,

where D_v and D_c are the maximum VN and CN degrees. The coefficient λ_i (resp. ρ_i) is the fraction of edges that are connected to VNs (resp. CNs) of degree-i.

LDPC codes can be used in a systematic form, and hence its codeword comprises two disjoint source and parity parts. Accordingly, the VNs are divided into two sets: source nodes and parity nodes. Then, the edges connecting to source or parity nodes are called source edges, denoted by ℰ^s, and parity edges, denoted by ℰ^p, respectively. In this paper, we use a family of LDPC codes whose CNs are connected to source nodes via at least one edge. The LDPC codes with such a structure are called fully-source-involved LDPC (FSI-LDPC) codes.

Since a joint decoder employs two compound LDPC decoders, the associated Tanner graph of the joint decoder consists of three types of nodes. They are called source nodes denoted by 𝒱^s={v^s_1,v^s_2,…,v^s_n-m}, parity nodes denoted by 𝒱^p={v^p_1,v^p_2,…,v^p_m}, and CNs denoted by 𝒞 for each of LDPC decoders. Moreover, there are state nodes denoted by 𝒮 which connect the two iterative decoders to exchange extrinsic information. A schematic of the associated Tanner graph of the joint decoder is presented in Figure <ref>. In this figure, information of source nodes of each decoder are passed to the source nodes of the other decoder via state nodes. Furthermore, we use 𝒢=(𝒱^s,𝒱^p,𝒞,𝒮,ℰ^s, ℰ^p) to denote a two edge-type graph of a joint LDPC decoder.



We follow the notations defined in <cit.> in this paper. Let n^s_i and n^p_i denote the number of source and parity nodes of degree-i, respectively. The total number of source and parity nodes are also given by n^s=∑_i=2^D_vn^s_i and n^p=∑_i=2^D_vn^p_i, respectively. Let n_i=n^s_i+n^p_i denote the number of VNs of degree-i. Hence, the number of VNs n equals ∑_i=2^D_vn_i. Let m_j,k be the number of CNs of degree-j, k edges of which are connected to the source nodes and (j-k) edges of which are connected to the parity nodes. Thus, the number of CNs of degree-j, denoted by m_j, is equal to ∑_k=1^j-1m_j,k. Similarly, the total number of CNs m is determined by ∑_j=2^D_c m_j, and the total number of edges on a two edge-type graph is determined by

    E=E^s+E^p=n∑_i=2^D_vλ_i/i=m∑_j=2^D_cρ_j/j.


In addition to (λ,ρ), we need to introduce additional coefficient pairs (α,β) to asymptotically describe a two edge-type graph where α_i=n^s_i/n_i is in fact the fraction source nodes of degree-i out of all degree-i VNs. Similarly, β_j,k=m_j,k/m_j, where ∑_k=1^j-1β_j,k=1. Now, the source and parity variable degree distribution polynomials are, respectively, defined as follows:


    λ^s (x)=∑_i=2^D_vλ_i^s x^i-1, ‎  ‎ λ^p (y)=∑_i=2^D_vλ_i^p y^i-1,

where

    λ_i^s=n_i^s· iE^s=n_i^sn_i n_i· iE EE^s = α_i λ_i/∑_j=2^D_vα_j λ_j,



    λ_i^p=n_i^p· iE^p=n_i-n_i^sn_i n_i· iE EE^p=(1-α_i) λ_i/∑_j=2^D_v (1 - α_j) λ_j.

The source and parity side CN degree distribution polynomials are, respectively, defined as follows:

    ρ^s (x,y)=∑_j=2^D_c∑_k=1^j-1ρ_j,k^s  x^k-1  y^j-k,



    ρ^p (x,y)=∑_j=2^D_c∑_k=1^j-1ρ_j,k^p  x^k  y^j-k-1,

where


    ρ_j,k^s = m_j,k· kE^s = ρ_jj·β_j,k k/∑_i=2^D_vα_i λ_i,



    ρ_j,k^p = m_j,k· (j-k)E^p = ρ_jj·β_j,k (j-k)/∑_i=2^D_v (1-α_i) λ_i.



There are two variables denoted by “x" and “y" in ρ^s (x,y) and ρ^p (x,y), which indicate two types of edges incident to each CN. The inner summation of ρ_j,k^s and ρ_j,k^p over k, indicate the fraction of j-th degree CNs which are connected to the the source and parity VNs, respectively.
The code rate must be the same in two edge-type and its corresponding single edge-type graph. Furthermore, the number of edges emerging from each type of variable and CNs must be the same too. Hence,the following conditions <cit.> must be satisfied:

    ∑_i=2^D_vα_i λ_i=∑_j=2^D_cρ_jj∑_k=1^j-1β_j,kk,


    ∑_i=2^D_vα_i λ_ii=R ∑_i=2^D_vλ_ii.


Note that a symmetric two edge-type graph corresponding to a joint decoder can be described by (λ, ρ, α, β).
Moreover, an ensemble corresponding to (λ, ρ, α, β) can be obtained from a single edge-type ensemble (λ, ρ) by partitioning the VNs and their associated edges connected to the CNs according to (α,β) distribution. 



 §.§ Iterative Joint LDPC Decoding 

We assume that the data block  corresponding to sources U_1 and U_2, i.e, u_1 and u_2 are encoded  through the systematic LDPC encoders respectively into x_1 and x_2. The encoded bits are transmitted over two independent AWGN channels. At the receiver, we receive vectors r_1=x_1+n_1 and r_2=x_2+n_2,
where n_i ∼𝒩(0, σ_n^2), for i=1,2, 





Let û_i, i=1,2, denote decoded bits of i-th decoder. The joint receiver employs the empirical estimate of the correlation parameter to benefit from the intra-sources correlation. The estimate of correlation vector, denoted by ẑ, is calculated by ẑ=û_1⊕û_2.





The joint decoder is composed of two parallel LDPC decoders working based on the sum-product (SP) algorithm <cit.> where it also accepts an estimate of the transmitted bits from the other decoder as a priori information. The structure of iterative joint decoder is shown in Fig. <ref>.
There are tow types of iterations, which are called global and local iterations indicated by superscripts g and l, respectively. The estimate of correlation is updated during each global iteration. Then, this update estimation is passed on to the both decoders for being used as a side information. Next, each decoder performs the SP algorithm with a specified maximum number of local iterations. Note that log-likelihood ratio (LLR) of the side information are only added to the systematic bit nodes of each decoder, while these LLRs are first set to zero for the both decoders.
Consider the g-th global and l-th local iterations of the joint decoder. In each local iteration, the encoded bits coming from the channel are transformed to a posteriori LLRs, denoted by L_ch(r), and fed into the VNs as follows:


    L_ch(r_i,v)=log((x_i,v=1|r_i,v)(x_i,j=0|r_i,v))=2σ^2r_i,v,

where i ∈{1,2} indicates the i-th channel, and v ∈{1,2,…,n} is the number of VN. For simplicity, we drop the channels index in the sequel unless an ambiguity arises. Let L_v,c^(l) and L_c,v^(l) denote the LLR of messages emanating from a VN v to a CN c and, vice versa, from the CN to the VN at local iteration l, respectively. Since the side information are only added to the information bit nodes, LLR update equations from these nodes to the CNs are given by:


    L_v,c^(l)=L_ch(r_v)+∑_c'≠ cL_c',v^(l-1) + L_s^(g-1)(û),

where v∈𝒱^s, and L_s^(g)(û) denotes LLR of the side information associated to the estimated correlation at the global iteration g. It is worth noting that the summation is only applied on all connected CNs to the VN v, except the CN c. The messages forwarded from parity VNs to the CNs are calculated in a same way as for the standard SP decoder <cit.>


    L_v,c^(l)=L_ch(r_v)+∑_c'≠ cL_c',v^(l-1),

where v∈𝒱^p. Furthermore, the messages L_c,v^(l) are updated in each local iteration in the same equation as represented in the standard SP decoder <cit.>.

In the g-th global iteration, the correlation vector ẑ is estimated by ẑ^(g)=û_1^(g)⊕û_2^(g), where û_1^(g) and û_2^(g) are, respectively, the hard estimates of the source bits u_1 and u_2 in the terminated local iteration l_t in each associated decoder. The LLR L^(g)(ẑ) at each global iteration is obtained using the proposed technique in <cit.>, as follows:


    L^(g)(ẑ_v)=(1-2ẑ_v^(g))log_2 (k-W_HW_H),

where k and W_H are, respectively, the source block size and Hamming weight of the correlation vector ẑ^(g)=(ẑ_1^(g),…,ẑ_k^(g)).

Finally, the LLR of side information of the source bits that are input to the first and the second decoders, at the next global iteration, are calculated by <cit.>:

    L_s^(g)(û_1,v)=(L^(g)(ẑ_v))·(L^(g)(û_1,v))
    · 2 (tanh(|L^(g)(ẑ_v)|/2)tanh(|L^(g)(û_2,v)|/2)),

where v∈𝒱^s, and similarly

    L_s^(g)(û_2,v)=(L^(g)(ẑ_v))·(L^(g)(û_2,v))
    · 2 (tanh(|L^(g)(ẑ_v)|/2)tanh(|L^(g)(û_1,v)|/2)),

where

    L^(g)(û_1,v)=L_ch(r_1,v)+∑_c'L_c',v^(l_t-1),
        L^(g)(û_2,v)=L_ch(r_2,v)+∑_c'L_c',v^(l_t-1),

and v ∈𝒱^s. According to the Turbo principle, side information, also called extrinsic information, added to each SP decoder should not include the same decoder's information. Therefore, side information from L(û_1) and L(û_2) are removed from the two above equations, but they have to be added when information source bits are estimated at the end of the local iteration to calculate a posteriori information.



§ EXIT CHART ANALYSIS

In EXIT chart analysis tool firstly proposed in <cit.> to design ensembles of conventional LDPC codes, evolution of the Mutual Information (MI) between a given transmitted binary bit at source and several LLR's within the decoder is traced. In a point to point (P2P) LDPC decoder, variable node EXIT curve displays the MI between the transmitted bit and the LLR at the output of the VN (I_EV) versus the MI between the transmitted bit and the LLR at the input of the VN (I_AV). Similarly, check node EXIT curve displays the MI between the transmitted bit and the LLR at the output of the CN (I_EC) versus the MI between the transmitted bit and the LLR at the input of the CN (I_AC).

To obtain these curves, it is usually assumed that the density at the input of each decoding unit (VN and CN in this case) are consistent Gaussian random variables. To track the iterative exchange of messages between VN and CN, we set I_AV at iteration l equal to I_EC of iteration l-1.  Similarly, we set I_AC at iteration l equal to I_EV of iteration l. Alternatively, one can plot the VN curve and inverse of the CN curve ad following the trajectories. This is referred to as EXIT chart. As far as the EXIT chart analysis of the proposed system model is concerned, we have to deal with a modified chart that can incorporate both inner and outer iterations as well as the fact the considered graph is two edge-type in contrast to the conventional case.




 §.§ Mutual Information between the Transmitted Bit and the LLR of the Received Data

Let X_1 be a BPSK modulated transmitted bit in the first source and X_2 be the BPSK modulated transmitted bit at the second source on the same time instant. Since the sources are correlated, we have P(X_1=X_2)=p. Now let A_1 and A_2 be the LLRs corresponding to the received information at the input of the first and second decoders, respectively. It has been already established that A_i are consistent Gaussian random variables with variance σ^2_A=4/σ^2. It has been shown that <cit.> in this case we have I(X_c,A_d)=J(σ^2_A), c=d, c=1, 2 where

    J(σ_A)=
    
    1-1/√(2π)σ_A∫_-∞^∞log_2(1+e^-l) exp(-(l-σ_A^2/2)^2/2σ_A^2)dl.

Moreover, for c ≠ d, we have I(A_c;X_d) =J̃(σ_A,p) <cit.> where

    J̃(σ_A,p)=1-
    1/√(2π)σ_A∫_-∞^∞log_2(1+e^-l/p+p̅e^-l)
    (pe^ -(l-σ_A^2/2)^2/2σ_A^2 + p̅e^ -(l+σ_A^2/2)^2/2σ_A^2 ) dl,

and p̅=1-p.

If the correlation parameter is assumed to 100%, i.e., p = 1, Eq. (<ref>) is reduced to the well-known equation (<ref>).
 Likely, if the correlation parameter is set to be 50%, i.e., p=0.5, then MI of the extrinsic messages passed to the other decoder would be equal to zero which is intuitively reasonable.
Eq. (<ref>) will be used in the next subsection to incorporate the correlation between X_1 and X_2 in the corresponding EXIT chart.


 §.§ Modified EXIT Chart


Consider one of the decoders at the receiver. At outer iterating g, the variable node EXIT curve of degree i belonging to source (systematic) bits at inner iteration l, I_EV^s(l)(i), can be obtained  based on the data from channel, extrinsic information from the check node at iteration l-1 and, extrinsic information coming from the other decoder at outer iteration g-1. The latter term is referred to as the helping information and denoted by I_h^g. I_h^g is in fact a function of I_EV^s(l_t) of the other decoder at outer iteration g-1 where l_t is the last inner iteration. At first outer iteration, this is set to zero.

Moreover, at outer iterating g, the variable node EXIT curve belonging to parity bits at inner iteration l, I_EV^p(l)(i), can be obtained  based on the data from channel and extrinsic information from the check node at iteration l-1. Similar statements can be made for the check node exits curves.
Given the above explanations and using (<ref>), we obtain I_EV^s(l)(i) as

    I   _EV^s(l)(i)  = 
        J ( √(σ_ch^2 + (i-1)[J^-1(I_EC^s(l-1))]^2 + [J^-1(I_h^(g-1))]^2 )),

where I_EC^s(l-1) denotes the mutual information from the CNs to the source nodes at g-th global iteration.
So for an irregular variable node the EXIT curve is obtained as:

    I_EV^s(l) = ∑_i=2^D_vλ_i^s I_EV^s(l)(i) ,

To obtain I_h^(g-1) we act as follows:
We first obtain I_h^(g-1)(i), the I_h^(g-1) corresponding to degree-i VNs. Note that it is in fact equal to I(X_c,A_d) for c≠ d.  Therefore we have

    I_h^(g)(i) = J̃(σ_h,p),  where σ_h = √(σ_ch+i[J^-1(I_EC^s(l_t))]^2) ,

where I_EC^s(l_t) is the extrinsic information of at the output of the check nodes of the other decoder at last iteration.
We finally obtain:

    I_h^(g)=∑_i=2^D_vλ_i^s  I_h^g(i) ,


To obtain I_EV^p(l)(i), using (<ref>) we write

    I_EV^p(l)   (i)  =  J ( √(σ_ch^2 + (i-1)[J^-1(I_EC^p(l-1))]^2 )),

where I_EC^p(l-1) denote the mutual information from the CNs to the parity nodes at g-th global iteration.
So for an irregular variable node the EXIT curve is obtained as:

    I_EV^p(l) = ∑_i=2^D_vλ_i^p I_EV^p(l)(i) ,

where I_EC^s(l), and I_EC^p(l) denote MI from the CNs to the source and the parity nodes at g-th global iteration, respectively. Let γ_s and γ_p  be defined as ratio of source edges to total edges and parity edges to total edges respectively that can be written as

    γ_s=E^s/E =∑_i=2^D_vλ_i α_i,     γ_p= E^p/E=∑_i=2^D_vλ_i (1-α_i).

Using (<ref>) and (<ref>), the VN EXIT curve at outer iteration g is finally obtained as

    I_EV =  γ_s I_EV^s + γ_p I_EV^p.



To obtain EXIT curve of the CNs, 
we first obtain I_EC^s(l) and I_EC^p(l) as follows. Consider degree j CNs which are connected through k edges the source nodes and (j-k) edges to the parity nodes. The corresponding MI to source and parity nodes are, respectively, given by:


    I_EC^s(l)(j,k)=1 - 
        J (√( (k-1) [J^-1(1-I_EV^s(l))]^2 + (j-k) [J^-1(1-I_EV^r(l))]^2 )),

and

    I_EC^p(l)(j,k)=1 - 
        J ( √( k [J^-1(1-I_EV^s(l))]^2 + (j-k-1) [J^-1(1-I_EV^r(l))]^2 )),

Consequently we have:

    I_EC^s(l)=∑_j=2^D_c∑_k=1^j-1ρ_j,k^s   I_EC^s(l)(j,k) ,

and

    I_EC^p(l)=∑_j=2^D_c∑_k=1^j-1ρ_j,k^p   I_EC^p(l)(j,k) .

Therefore, the overall EXIT curve of the CNs at is obtained as follows:

    I_EC=γ_s  I_EC^s + γ_p  I_EC^p.

From (<ref>) and (<ref>), we observe that similar to the P2P case, the CN curve is only concerned with the degree of CNs and dose not change with SNR of the channel. Moreover, evaluation of (<ref>) shows that the CN curve independent from g which also makes sense intuitively.
























 §.§ Modified EXIT chart example

To get a better insight out of the above ugly equations, the modified EXIT chart corresponding to a joint decoder with a (3,6)-regular LDPC code is depicted in Fig. <ref> for the correlated sources when the correlation parameter p is set to 0.95 where the maximum inner iterations are set to be 50. Inner and outer iterations have also been plotted. As can be seen, in the absence of the helping information at the first outer iteration, each decoder performs inner iterations but it is stuck in the intersection of the check node curve and the blue variable node curve. Then, at the second outer iteration, with the help of extrinsic formation of the other decoder, the inner iterations continue and the MI continues to grow until it is stuck again at the intersection of check node curve and the red variable node curve. In the 3rd outer iteration, the MI jumps to continue its path toward 1 on the yellow variable node curve.








§ CODE DESIGN FOR THE JOINT DECODER

In this section, our aim is designing VN degree distribution λ(x) for an ensemble of LDPC codes with a given CN degree distribution ρ(x) so as to minimize the gap between code rate R and the Shannon-SW limit corresponding to the channel parameter σ^2. In general, the design of irregular LDPC codes using EXIT chart analysis is based on a curve-fitting method including EXIT curves of the VNs and the CNs, see, e.g., <cit.>, <cit.>. To make the design procedure simpler, it is often assumed that the ensemble codes have regular degree distribution of CNs <cit.>, <cit.>.

EXIT function of a CN output (or a VN input in the previous local iteration) and EXIT function of the VN output (or the CN input in the same local iteration),  denoted by I_EC^(l)(I_EV^(l)) and I_EV^(l)(I_EC^(l-1),σ_ch,p) can be found according to our discussion in Section <ref>.

In order to utilize EXIT chart in designing of LDPC codes for correlated sources, it should be found two edge-type coefficients α and β in addition to λ(x).

By applying a linear programming method, we first design a degree distribution pair (λ, ρ) and its corresponding coefficient pair (α, β). Thus, the initial value of (α, β) is realized and then the {λ_d_v, d_v∈ (2,…,D_v) } is designed again for the check-regular ensemble where the maximum VN degree D_v is predefined. For an ensemble given {λ_d_v}, EXIT function of the VN output can be represented by

    I_EV=∑_d_v=2^D_vλ_d_v I_EV(d_v),

where I_cv(d_v) is EXIT curve of degree d_v VN that is obtained as described in Section <ref>.

As formerly mentioned, EXIT curve of a CN is the same as that in the P2P case, as follows

    I_E,C(I_A,C,d_c)=1 - J( √((d_c-1)[J^-1(1-I_A,C)]^2)),

where I_A,C is easily obtained associated to I_E,C according to the inverse of J(.) function.

The code rate optimization problem is formulated as follows:

    maximize:         R=1-∑_d_c=2^D_cρ_d_c/d_c/∑_d_v=2^D_vλ_d_v/d_v ,
      subject to:        1. ∑_d_v=2^D_vλ_d_v = 1 ,
         2. λ_d_v≥ 0 ,
         3. I_EV > I_A,C(I_E,C) ,
       for I_E,C∈ [0  1] .

The third constraint is the zero-error constraint or is equivalent to that the output MI of the VN in each local iteration greater than the previous local iteration (i.e, I_EV^(l)>I_EV^(l-1)). Also maximizing R is equivalent to maximize ∑_d_v=2^D_vλ_d_v/d_v. When I_EV(d_v) is given for each degree so I_EV is linear in {λ_d_v, d_v∈ (2,…,d_v,max(D_v)) }.

Therefore after finding λ(x) and the given CN degree distribution ρ(x), we can obtain the optimum (α,β) by using Conditions (<ref>), (<ref>) and a stability condition that is given under Gaussian assumption and using MI evolution as <cit.>

    α_2 λ_2e^(-M^2/8) + (1-α_2) λ_2 < e^1/2σ_n^2/∑_j=2^D_cρ_j (j-1) ,

where M = J^-1( J̃(σ_max,p) ) and σ_max=J^-1(1). Note that the stability condition depends on the channel parameters and J̃(.) function.

The result of our search for the BIAWGN channel is summarized in Table <ref> and Table <ref>.  Table <ref> and Table <ref> contains those degree distribution pairs of rate one-half with coefficient pairs (α,β) and various correlation parameter. We also consider an upper bound from (<ref>) which is useful to measure the performance of our designed ensembles.







§ FINITE-LENGTH RESULTS

In this section, we illustrate the performance of finite-length LDPC codes constructed from optimized irregular degree distributions for iterative joint channel decoder at different rates. The finite-length construction is performed with a modified progressive edge growth (PEG) method that has very low error-floor performance <cit.>.

We simulate the proposed scheme for different values of the correlation parameters p and assume that both of transmit nodes use the same degree distribution of LDPC codes and also each of the channel parameters are the same. For the given p and R, the theoretical bound of the E_so/N_0 can be calculated from (<ref>) for error-floor recovery.

In the following, we represent sample simulation results associated with several designed LDPC codes  for different values of the correlation parameters and the code rates in section <ref>. We show the performance of the designed codes in section <ref> are bater than the obtained results in <cit.> by using of the simulation results with finite-length. We consider the ensemble codes with rate R=0.5 for several the correlation parameter p. The blocks length are selected to be 20000 and the maximum number of local and global iteration are set to 100 and 10 respectively. We provide our simulation results by using the irregular LDPC codes according to Table <ref> and <ref>.
 

Example 1: From Table <ref>, we consider the designed code for R=0.5 and p=0.9 with following VN and CN degree distribution and its corresponding the pair (α,β)

    λ(x)=     0.23559  x + 0.39783  x^3 + 0.14198  x^6 + 
        0.00148  x^13 + 0.22312  x^14,


    ρ(x)= x^6.


 

Example 1: From Table <ref>, we consider the designed code for R=0.5 and p=0.95 with following VN and CN degree distribution and its corresponding the pair (α,β)

    λ(x)=     0.2518  x + 0.38081  x^3 + 0.10944  x^6 + 
        0.00121  x^13 + 0.25674  x^14,


    ρ(x)= x^6.


For the correlated sources, the BER values of three codes as a function of E_so/N_0 (SNR) and p are reported in Fig. <ref>. Note that the curves labeled “glob.it.0” show the performance of the LDPC without using the correlation information that are equal to the performance of point to point.





Table <ref> shows, for various values of the correlation parameter p with the constant rate R=0.5, the corresponding joint entropy H(u_1,u_2) of the correlated sources, the theoretical limit for E_so/N_0 in (<ref>), [E_so/N_0]_lim, the obtained threshold from EXIT chart, [E_so/N_0]_th, and the value of E_so/N_0 for which the proposed iterative joint decoder (at target
BER=10^-5).




§ CONCLUSIONS





IEEEtran



