
    \end{align}
     
 
 
   {\bf Uplink transmission:} Mobile device $k$ for $k\in [K]$ sends one-bit quantized local gradient information ${\bf \hat g}_k^t$ to the server over a faded and noisy communication channel. We mainly consider the orthogonalized massive access channel, in which $K$ local devices send the one-bit gradient parameter to the server simultaneously by using orthogonal subchannels. We focus on the real part of the complex base-band equivalent model for ease of exposition, which can readily be extended using quadrature amplitude modulation signaling to orthogonal BPSK symbols. 
   
   
    
    Let $h_k^t\in \mathbb{R}$ be the fading channel coefficient of the $k$th subchannel during communication round $t$. This fading channel is assume to be a {\it constant} during the uplink transmission, and change independently every communication round $t$ while remaining constant during the communication round, i.e., a block fading process. Then, the received signal of the server from the $k$th subchannel of the $t$th communication round is
    \begin{align}
        {\bf  y}_{k}^t=h_k^t {\sf sign}\left({\bf g}_{k}^t\right) +{\bf  n}_{k}^t, 
        \label{eq:channel}
    \end{align}
    where ${\bf n}_{k}^t$ is the Gaussian noise of the $k$th subchannel with zero-mean and variance $\sigma^2$, i.e., ${\bf n}_{k}^t\sim \mathcal{N}\left({\bf 0},\sigma^2_k{\bf I}\right)$. From \eqref{eq:channel}, the likelihood of ${\bf y}_{k}^t$ given ${\bf g}_{k}^t$ and assuming fixed $h_k^t$ is
    \begin{align}
        P\left({\bf y}_{k}^t|{\bf g}_{k}^t \right)=\prod_{m=1}^M\frac{1}{\sqrt{2\pi \sigma^2}}\exp\left(-\frac{\left|y_{k,m}^t-h_k^t{\sf sign}\left({g}_{k,m}^t\right)\right|^2}{2\sigma^2_k}\right). \label{eq:likelihood}
    \end{align}
 
 
  {\bf Majority-voting for aggregation and model update:} The server performs maximum-likelihood detection (MLD) to estimate one-bit local gradients per subchannel. Specifically, using the received signal of the $k$th subchannel, one-bit local gradient information is estimated as
  \begin{align}
  	 {\bf \hat g}_k = {\sf sign}\left( \frac{{\bf y}_k^t}{h_k^t}\right),
  \end{align} 
for $k\in [K]$. Then, to attain a global gradient estimate, the server applies a majority-vote based aggregation method, which takes the sign of the sum of the estimated one-bit local gradients:
  \begin{align}
  	 {\bf \hat g}_{\sf MV} = {\sf sign}\left(\sum_{k=1}^K{\bf \hat g}_k \right)={\sf sign}\left(\sum_{k=1}^K{\sf sign}\left( \frac{{\bf y}_k^t}{h_k^t}\right)\right).
  \end{align} 
  After computing ${\bf \hat g}_{\sf MV}$, the server updates the model parameter as    \begin{align}
        {\bf w}^{t+1}&= {\bf w}^t -\gamma^t {\bf \hat g}_{\sf MV}^t,
    \end{align}
    where $\gamma^t\in(0,1)$ is a learning rate of the gradient descent algorithm. The updated model parameter is then sent to the mobile devices and the learning procedure moves to the next round.  To speed up convergence, the server may update the model parameter by taking a weighted average between the currently estimated gradient and the previously estimated gradients ${\bf m}^{t}=\delta^t {\bf m}^{t-1}+{\bf \hat g}^{t}_{\sf MV}$ as
     \begin{align}
     {\bf w}^{t+1}= {\bf w}^{t} -\gamma^t {\bf m}^{t},
    \end{align}
     where $\delta^t\in(0,1)$ is a weight parameter and ${\bf m}^{0}$ is the initial value of the moment.   
     
    

\section{ Bayesian Federated Learning}
In this section, we present a framework for BFL.  The motivation of BFL is to find an optimal aggregation method of local gradients from a Bayesian viewpoint.  The majority-voting based aggregation is effective in detecting the sign of the sum of the one-bit local gradients when the SNRs and  fadings are identical across mobile devices. In heterogenous wireless networks, however, this majority-voting based aggregation cannot be optimal in detecting the sign of the local gradients' sum due to the distinct reliabilities of the communication links. In addition, to enhance the FL performance, the server may require to accurately know the sum of local gradients rather than the sum of the local gradients' signs.  Motivated by these facts, we aim at finding an optimal aggregation method of the local gradients in the sense of minimizing MSE. The key idea of BFL is to aggregate local gradients per iteration by jointly harnessing the knowledge of 1) the prior distributions of local gradients, 2) the gradient quantizer function,  and 3) the likelihood functions of the communication channels.   






 \subsection{Algorithm}

 
 
 
        

    
    {\bf Computation of local gradient prior:} Unlike conventional federated learning, BFL exploits the local gradient's prior information. Let $P({\bf g}_{k}^t)$ be the prior distribution of  ${\bf g}_{k}^t$ computed by mobile device $k$ at communication round $t$. Characterizing the prior distribution is very challenging because it not only depends on the loss function $f_k({\bf w}^t)$ but also on the underlying distribution of data samples; the exact characterization of the prior distribution is impossible in general. To overcome this difficulty, we model the distribution of ${\bf g}_k^{t}$ as a Gaussian prior distribution with a proper moment matching technique. Specifically, we assume the prior is a multivariate normal distribution with mean vector ${\bm \mu}_k^t=\mathbb{E}\left[ {\bf g}_{k}^t\right]   $ and covariance matrix ${\bm \Sigma}_k^t =\mathbb{E}\left[ \left({\bf g}_{k}^t-{\bm \mu}_k^t \right)\left({\bf g}_{k}^t-{\bm \mu}_k^t\right)^{\top}\right]$ as
    \begin{align}
        P\left( {\bf g}_{k}^t \right)=\frac{1}{(2\pi)^{\frac{M}{2}} \det({\bm \Sigma}_k^t)} e^{-\frac{1}{2}\left({\bf g}_k^t -{\bm \mu}_k^t\right)^{\top}\left({\bm \Sigma}_k^t\right)^{-1} \left({\bf g}_k^t -{\bm \mu}_k^t\right)}.
        \label{eq:prior}
    \end{align}
   The prior distribution can change over communication rounds according to the underlying distribution of data samples and the local loss function.
  
    
  Notice that local gradients ${\bf g}_k^t=\nabla f_k\left({\bf w}^{t}\right)$ and ${\bf g}_{\ell}^t=\nabla f_{\ell}\left({\bf w}^{t}\right)$ for $k\neq \ell$ are statistically correlated because they are evaluated at a common model parameter ${\bf w}^t$. To illustrate this, consider a linear regression loss function 
  \begin{align}
  	f_k({\bf w}^t)=\|{\bf X}_k^{\top}{\bf w}^t-{\bf y}_k\|_2^2,
  \end{align}
   where ${\bf X}_k=\left[ {\bf x}^{1}_k, {\bf x}^{2}_k,\ldots, {\bf x}^{N_k}_k \right]\in \mathbb{R}^{M\times N_k}$ and ${\bf z}_k=\left[z_k^1, \ldots, z_k^{N_k}\right]^{\top}\in \mathbb{R}^{N_k\times 1}$ for $k\in [K]$. Furthermore, we assume all local data distributions $P({\bf X}_k, {\bf z}_k)$ for $({\bf X}_k,{\bf z}_k)$ are statistically independent, each with $\mathbb{E}\left[{\bf X}_k^{\top}{\bf X}_k\right]={\bf R}_k$ and $\mathbb{E}\left[ {\bf X}_k^{\top}{\bf z}_k\right]={\bf 0}$. In this case, the local gradient at mobile device $k\in[K]$ computed using model parameter ${\bf w}^t$ is ${\bf g}_k^t = {\bf X}_k^{\top