L selects a portion of the total devices to participate the global update. Initially the PS sets the model as 00 and sends it to all the users. Afterwards each user iteratively performs the local training and updates the gradient gk = VFk(O). In the FL setting, each user can actually run multiple iterations on gradient calculations in each round. More specifically, in round t, user k calculates 0k=0k-nVFk(0) multiple times. The participating users then send their locally trained gradients to the PS for aggregation. The PS further calculates - sends Ot+1 to all the users for the next round update. A brief illustration of the FL model update is shown in Figure 9.1a. Furthermore, each scheduled device needs to adjust their update size in order to fit that into the data rate supported by the dynamic fading channel. If user k's total update size exceeds the maximally allowable data rate Mk, model compression needs be applied. In the following we briefly introduce two lossy compression algorithms and their rationales. K users K users NOMA Round 1 NOMA Round 2 NOMA update protocol FL update model Figure 9.1 An illustration of the proposed scheme. (a) A general FL model update. (b) NOMA update protocol in each round. Shaded area is for uplink and blank is for downlink. 9 Wireless Systems for Distributed Machine Learning 9.2.2 Gradient Quantization It is well-known that quantization can help compress the size of a large data. Standard algorithms in ML typically use 32-bit floating-point to represent each model parameter. Cost to store, transmit, and manipulate those data tends to be high. Alternatively, a simple implementation uses less bits for such a representation. Even though quantization creates "rounding errors," existing works show that this approach demonstrates a good model convergence [106] at the cost of more communication rounds. In this work, we adopt DoReFa scheme [235] suitable for quantizing gradients within [-1, 1], which is true for most ML models. Specifically, mapping can be established with the functi