{'fraction_non_alphanumeric': 0.03875719217491369, 'fraction_numerical': 0.007825086306098964, 'mean_word_length': 4.468411779511704, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Deep Joint Source-Channel Coding Over Cooperative Relay Networks

This paper presents a novel deep joint source-channel coding (DeepJSCC) scheme for image transmission over a half-duplex cooperative relay channel. Specifically, we apply DeepJSCC to two basic modes of cooperative communications, namely amplify-and-forward (AF) and decode-and-forward (DF). In DeepJSCC-AF, the relay simply amplifies and forwards its received signal. In DeepJSCC-DF, on the other hand, the relay first reconstructs the transmitted image and then re-encodes it before forwarding. Considering the excessive computation overhead of DeepJSCC-DF for recovering the image at the relay, we propose an alternative scheme, called DeepJSCC-PF, in which the relay processes and forwards its received signal without necessarily recovering the image. Simulation results show that the proposed DeepJSCC-AF, DF, and PF schemes are superior to the digital baselines with BPG compression with polar codes and provides a graceful performance degradation with deteriorating channel quality. Further investigation shows that the PSNR gain of DeepJSCC-DF/PF over DeepJSCC-AF improves as the channel condition between the source and relay improves. Moreover, DeepJSCC-PF scheme achieves a similar performance to DeepJSCC-DF with lower computational complexity.


I. INTRODUCTION
Cooperative communications allow the nodes in a communication network to leverage the resources of their neighboring nodes to boost their spectral efficiency and to enable robustness against channel fading [1]. One of the simplest cooperation models is the relay channel, which consists of three terminals, the source, the relay and the destination (see Fig. 1 for an illustration). The source broadcasts its message to both the relay and the destination, and the relay processes its received signal and forwards to the destination, while the destination performs joint decoding to recover the original message. There are three classical relaying protocols: amplifyand-forward (AF), decode-and-forward (DF) and compressand-forward (CF) [2]- [4], and their variations [5], [6]. In AF, the relay simply scales its received signal and forwards to the destination. The main limitation of AF is the inherent noise forwarding. In DF, the relay decodes the received signal into the original bit sequence, and then re-encodes. While DF mitigates the noise forwarding problem, its performance becomes limited when the source-to-relay channel is poor. In CF, the relay forwards its received signal to the destination, but unlike in AF, it compresses the received signal using Wyner-Ziv source coding treating the signal received at the destination as side information.  It is shown in [7] that Shannon's separation theorem applies to the relay channel when there is no correlated side information at the relay; that is, separate compression followed by cooperative channel coding is theoretically optimal in the infinitely long source and channel blocks. This holds even though we do not know the capacity of the relay channel. Although joint source-channel coding (JSCC) is known to outperform separation in practical finite block length regimes, there are limited studies on JSCC over relay channels. JSCC over cooperative relay networks is studied in [7] from an information theoretic perspective. JSCC for cooperative transmission of multimedia sources have been considered in [8], [9]; however, these papers mainly consider separate codes for compression and error correction, whose parameters are optimized jointly to create robustness against channel variations.
Recently, deep neural networks (DNNs) have been successfully applied to a wide range of communication problems, from image compression [10] to channel coding [11], [12] and signal processing [13]. Due to its strong capability to extract important features, which are also robust to noise, DNNs have also been successfully applied to the JSCC problems, and the resultant scheme is shown to outperform digital alternatives employing BPG and capacity-achieving channel codes, as well as providing graceful degradation with weakening channel quality [14]. It is further extended to more flexible frameworks, which enable layered transmissions [15], channel feedback [16], and compatibility with hardwares adopting hard-coded standard protocols [17]. Recent works show that DeepJSCC can also adapt to the more challenging multipath fading channels [18] and to different sources, such as texts and videos [19], [20]. To be best of our knowledge, there is no prior work that studies DeepJSCC in the cooperative setting, which, as we will show in this paper, is a non-trivial extension.
In this paper, we propose three different DeepJSCC schemes, namely DeepJSCC-AF, DeepJSCC-DF, and DeepJSCC-PF for image transmission in cooperative relay networks. DeepJSCC-AF and DeepJSCC-DF schemes resemble the classical AF and DF protocols in relay networks, while the DeepJSCC-PF is a more efficient variation of DeepJSCC-DF with a simplified neural network at the relay. To be more specific, in DeepJSCC-AF, the relay amplifies its received signal while the destination node applies the maximum ratio combining (MRC) rule to combine the two received signals. The source and destination then apply DeepJSCC treating the outcome of MRC as the channel output. In DeepJSCC-DF, the DNN at the relay reconstructs the original image, and re-encodes it to generate additional 'parity' symbols to be forwarded to the destination, which concatenates the signals from the source (termed as 'systematic' symbols) with those from the relay to decode the image. DeepJSCC-PF is similar to DeepJSCC-DF, except that it does not require the relay to explicitly reconstruct the image, which reduces both the computation load and the delay at the relay. Our experiment results show that all the proposed schemes outperform the digital baseline with BPG compression followed by polar coded channel transmission while providing a graceful performance degradation with respect to channel quality. Moreover, the proposed DeepJSCC-PF achieves comparable performance with DeepJSCC-DF despite reduced complexity. Finally, we reveal the robustness of DeepJSCC-DF/PF against poor source to relay channel condition and their superior performance over DeepJSCC-AF with higher source to relay SNRs.
II. SYSTEM MODEL As illustrated in Fig. 1, we consider a classical relay channel model with a source node S, a destination node D, and a relay node R. The goal is to deliver an image S ∈ R C×H×W from S to D with the help of relay R, where C, H, W denote the number of color channels, the height and width of the image, respectively.
We assume a half-duplex relay node that cannot receive and transmit at the same time. Therefore, the transmission is divided into two periods [2]: the relay-receive period and the relay-transmit period. In the former, S encodes the image S into a channel codeword x s ∈ C k using an encoder function The source input is subject to a power constraint: In the context of JSCC, the 'bandwidth ratio' quantifies the available channel uses per pixel (CPP), defined as ρ k C * H * W .
The received signal at the relay R and destination D can be written as where α sr , α sd are real constants governed by the transmission distances of the S − R and S − D links, respectively; n r , n d are complex additive white Gaussian noise (AWGN) with zero mean and variances N r and N d , respectively. The SNRs of the S − R and S − D links are defined as Upon receiving y sr , the relay processes it by a function f r (·) and forwards the processed signal to the destination in the relay-transmit period. Denoting the signal transmitted from the relay by z r ∈ C k , we have where z r is also subject to a power constraint: 1 k E(z H r z r ) ≤ 1. Note that the form of f r (·) will vary according to different relaying protocols, where the details will be expounded in Section III.
The signal received at the destination in the relay-transmit period can be written as: where α rd is determined by the distance between R and D; Given the received signal from the source and relay in the two periods, the destination aims to reconstruct the image using a decoding function g = C k × C k → R C×H×W . The reconstructed image is denoted by S ∈ R C×H×W , where S = g(y sd , y rd ).
We remark here that we do not allow the source to continue transmission during the relay-transmit period. Also, we divide the available channel bandwidth equally between the relayreceive and transmit periods More general schemes that go beyond these limitations will be considered in future work.

III. DEEPJSCC FOR RELAY-AIDED COMMUNICATIONS
We parameterize the encoder f s , decoder g, and the transformation f r at the relay (when using DeepJSCC-DF/PF) by DNNs, the architectures of which are presented in Fig. 2. Specifically, ResNet in Fig. 2 refers to a block of 2D convolutional neural networks with residual connections. Inside ResNet, we also adopt the generalized divisive normalization (GDN) layers for modeling densities of image sources [21]. We emphasize that residual connections are essential in our problem for learning good encoding and decoding structures especially at the relay. The up-sampling blocks in g are realized by pixel shuffling. The parameter C f eat denotes the number of filters used and C out is determined by the CPP defined in Section II.
Inspired by [17], [22], we introduce the channel attention (CA) module in the DNNs to improve its robustness to different channel conditions. These CA modules take both  the features and channel conditions as inputs, where the channel conditions refer to a collection of SNRs defined as SN R = {SN R sr , SN R sd , SN R rd }, allowing DNNs to assign different weights to different input features according to the experienced channel conditions. Based on the DNNs defined above, we next explain the processing at the relay and destination nodes, and the loss functions used in the training, when operating under DeepJSCC-AF, DF and PF protocols, respectively.

A. DeepJSCC-AF
As shown in Fig. 3, f r (·) is simply a linear function when operating in the DeepJSCC-AF mode. The transmitted signal from the relay is a scaled version of the relay's received signal in the relay-receive period; that is, where β 1 α 2 sr +Nr ensures that the power constraint at the relay is satisfied. Substituting z r into (6) yields where n is AWGN with zero mean and variance N d + Nrα 2 rd α 2 sr +Nr . Given (3) and (10), the received signal at the destination from two transmission periods can be combined by MRC to generate an estimate of z, denoted by z, expressed as: .
Then we convert z into a real vector, reshape it and feed it to the DeepJSCC-AF decoder along with the SNR values of the three links. The whole system is trained in an end-to-end fashion and we choose the mean square error (MSE) as the loss function:

B. DeepJSCC-DF
In the conventional (digital) DF protocol, the relay decodes the original information bits, re-encodes the bits to create additional parity symbols, and forwards these parity symbols to the destination. We can apply a similar technique in conjunction with DeepJSCC: the relay tries to reconstruct the source image from y sr , then performs another DeepJSCC encoding on the recovered image into z r for transmission. Specifically, we first feed y sr to the relay decoder network g r (along with the SNRs), yielding an estimated image S r = g r (y sr ) followed by the relay encoder f r (·) to generate z r : Note that z r will be power normalized before transmission. As shown in Fig. 2, we use f DF = f r (g r (·)) to represent the operations at the relay in the DF mode. At the destination, we concatenate the received signal y sd and y rd , and feed it into the decoder to reconstruct the transmitted imageS. The loss function of the proposed DeepJSCC-DF is comprised of two parts, representing the image reconstruction quality at the relay and the destination, respectively. To balance the reconstruction performance at R and D, we introduce a hyper-parameter λ ≥ 0 and the overall loss function is given by Intuitively, with a smaller λ, the loss function emphasizes more on the image reconstruction at the destination. When λ = 0, only the reconstruction performance at the destination is taken into account.

C. DeepJSCC-PF
DeepJSCC-DF requires the reconstruction of the image at the relay, leading to non-negligible computing-resource consumption and higher delays compared to the DeepJSCC-AF scheme. In this context, we introduce an alternative relaying protocol called DeepJSCC-PF that achieves comparable reconstruction results with DeepJSCC-DF. In particular, DeepJSCC-PF does not require explicit image reconstruction at the relay, and hence, it employs only a lightweight neural network for the relay's processing.
With DeepJSCC-PF, the 'parity' generation process can be expressed as: where z r will be power normalized before transmission. The processing of DeepJSCC-PF at the destination is identical to that of DeepJSCC-DF: the received signals are concatenated, reshaped, and fed into the decoder for reconstruction. The training loss L P F is the same as that of DeepJSCC-AF. The whole relaying and decoding processes for DeepJSCC-AF, DF, and PF are illustrated in Fig. 3.

A. Parameter Settings and Training Details
Throughout this section, we assume that the S−D and R−D links have the same quality, while the S − R link has better quality, i.e., SN R sr > SN R sd = SN R rd γ.
We evaluate the proposed DeepJSCC-AF, DF, and PF architectures 1 considering the transmission of images from the CIFAR-10 dataset, which consists of 50, 000 training and 10, 000 test RGB images with 32 × 32 resolution. The Adam optimizer is adopted to train the model with a varying learning rate. Specifically, the learning rate is initialized to 10 −4 and will be dropped by a factor of 0.8 if the validation loss does not improve in 4 consecutive training epochs. Early stopping is used with a patience of 12 epochs, and the maximum number of epochs is set to 400. Throughout this section, we fix a CPP of ρ = 0.125, and hence, C out = 12.

B. Performance evaluation
To start with, we perform an ablation study to evaluate the robustness of the DeepJSCC architecture with the CA module to test-SNR variations. The performance of the proposed DeepJSCC-PF scheme trained with varying SNR values γ ∈ [2,8] dB is shown in Fig. 4. Note that DeepJSCC-AF and DF with CA modules are also robust to test-SNRs, but we provide the comparison only for PF due to page limit. Throughout the experiments, during both training and testing, we set SN R sr = 12 dB. We then consider training the networks for fixed values of SN R sd = SN R rd = γ ∈ {0, 3, 6}dB without the CA module. In the test phase, these models are evaluated with the same SN R sr and γ ∈ [0, 8] dB.
As can be seen from Fig. 4, with the CA module (i.e., the curves labeled with 'adaptive with CA module'), the welltrained DeepJSCC model is robust to SNR variations in the test phase. Under a given test SNR, the PSNR performance achieved by the model with the CA module is comparable (and sometimes even better) than the one trained under the given test SNR. We adopt CA modules in all the following simulations.
Next, we compare the performances of DeepJSCC-AF and DeepJSCC-DF, benchmarked against non-cooperative transmission, which refers to transmission from S to D in the absence of R [14]. The non-cooperative scheme uses the same neural network structure and the source utilizes k symbols to transmit the image. The PSNR comparisons of these schemes are shown in Fig. 5, where we train and test different DeepJSCC-AF and DF pairs with SN R sr ∈ {0, 12, 24, ∞}dB. Note that DeepJSCC-DF curves shown in this simulation are obtained with the optimal λ for each SN R sr value, which will be detailed later.
In traditional digital communications, the condition of the S−R link determines the relative performances of AF, DF, and non-cooperative transmission [3]. In particular, when SN R sr is low (e.g., 0 dB), it is no longer possible to decode the source bit stream at the relay; and hence, DF becomes worse than AF and non-cooperative transmission in this regime. Our results in Fig. 5, however, tell a different story: the PSNR of our DeepJSCC-DF scheme is strictly better than that of the non-cooperative transmission, even when SN R sr is as low as 0 dB. This is because, even though we require the relay to decode the image, we allow lossy reconstruction in DeepJSCC. Therefore, in DeepJSCC-DF, the reconstruction quality at the destination is not limited by that of the relay, unlike in the digital DF scheme.
Compared with DeepJSCC-AF, the performance gain of DeepJSCC-DF increases with SN R sr . In particular, when SN R sr = ∞ dB, i.e., the transmission between the source and relay is error-free, DeepJSCC-DF outperforms DeepJSCC-AF by approximately 1 dB. This can be understood as a coding gain: when SN R sr = ∞ dB, DeepJSCC-AF is essentially a repetition code. Notice that, we have set SN R sd = SN R rd , as a result, the PSNR performance of DeepJSCC-AF is 3 dB better than that of the non-cooperative transmission. For DeepJSCC-DF, on the other hand, the relay first reconstructs the image S r from y sr , then re-encodes S r into 'parity' symbols z r . The destination reconstructs the image using both the 'systematic' and 'parity' symbols, providing an additional coding gain compared with DeepJSCC-AF. However, under lower SN R sr , the noise in y sr will hinder the parity generation process and the coding gain vanishes.
We then study the effect of the hyper-parameter λ introduced in Sec. III and compare the DeepJSCC-PF and DeepJSCC-DF schemes in Table I. In the training phase, different DeepJSCC-DF, PF models are trained with SN R sr ∈ {0, 12, 24, ∞}dB while γ is chosen uniformly from interval [2,8] dB, i.e., γ ∈ U(2, 8) dB. We consider λ ∈ {0, 0.5, 1, 2} for the training of DeepJSCC-DF. Note that γ = 8 dB for all the simulations in the table 2 . As can be seen, when SN R sr = 0 or SN R sr = ∞ dB, λ = 0 is preferable as trying to reconstruct the image at the relay requires f s to generate features that work for very good (∞ dB)/bad (0 dB) S−R link quality as well as the modest S − D link (8 dB), which would result in performance degradation. When SN R sr = 12, 24 dB, the conditions of the S − R link and the S − D link are closer; and thus, having non-zero λ would ensure a good reconstruction at the relay providing better results compared to λ = 0. We note that, even in this case, increasing λ beyond 1 harms the final performance as the system is mainly focusing on the reconstruction at the relay node. From Table I, we find the gain obtained by tuning λ is rather limited. To be specific, DeepJSCC-DF with λ = 0 is close to optimal making the regulation term in (14) unnecessary. This has motivated the introduction of the DeepJSCC-PF scheme, which is equivalent to DeepJSCC-DF with λ = 0, with a simplified DNN at the relay. We observed that DeepJSCC-PF can achieve comparable or even better performance than DeepJSCC-DF with λ = 0 with simpler structure.
Finally, we compare the proposed DeepJSCC-PF with the digital baselines, i.e., the separate source and channel coding scheme in Fig. 6. In this simulation, we keep SN R sr = 12 dB and vary γ from 0 dB to 8 dB. For the baseline, we use the state-of-the-art BPG compression algorithm with the 3GPP polar code. To exploit the full potential of the baseline, we assume that the relay can always decode the y sr perfectly. The processing of the digital baseline is as follows: the source node first compresses the image via BPG and encodes it using a rate R, R ∈ (0, 1) polar code followed by a given modulation scheme. The relay perfectly decodes the information bits and generates and forwards parity sequence to the destination. The received signal at the destination is a noisy version of the rate R/2 polar code with the same modulation. We also provide an upper bound on the performance of the separation scheme by calculating the capacity of the system described above, given by log 2 (1 + 10 γ/10 ). Thus, the maximum number of bits that can be generated by the BPG encoder and reliably transmitted to the destination is 2k log 2 (1 + 10 γ/10 ). It can be seen that the proposed DeepJSCC-PF outperforms the separation-based approach at all noise levels in terms of both PSNR and SSIM. We note that DeepJSCC-PF has comparable performance even with the ideal separation-based scheme with capacityachieving channel codes, which also assumed ideal S−R link. DeepJSCC schemes can also provide graceful degradation with error in channel estimation (i.e., if the channel becomes worse than what is assumed by the nodes), unlike in separation-based schemes, which completely fail below a certain SNR threshold.

V. CONCLUSION
We have investigated the cooperative image transmission problem in a relay channel. We have proposed three Deep-JSCC schemes, namely DeepJSCC-AF, DeepJSCC-DF, and DeepJSCC-PF, all employing DNNs for encoding and decoding, but each with a different requirement and computational complexity at the relay. All three schemes are trained in an end-to-end fashion and employ channel-attention modules so that a single trained network can be deployed in different channel conditions. It is shown that the proposed schemes outperform digital, separation-based protocols, while avoiding the cliff effect. Moreover, the proposed DeepJSCC-DF and PF schemes are robust against poor source-to-relay channel condition, and both schemes exhibit superior performance over DeepJSCC-AF with higher source-to-relay SNRs.