ced q-tables for all UEs. Although the reduced q-table of all UEs cannot construct the original big Q-table, the following propo- sition gives a very good property of the reduced q-table, which makes it possible to take actions in a distributed manner. Proposition 9.1 The value of q(6) (S, aj) in the reduced qi-table is the minimum value in the original Q-table defined in (9.20) when the action of UE i is a, i.e. q(6) (S.a)) = (9.22) where a(i) denotes the i-th element in action vector A. 9.2 Smart Handoff Policy Design for RAN Slicing Proof: Similar to that in [17], we can easily obtain our Proposition 9.1 by replacing all the notation "max" to "min." Proposition 9.1 states that by using the q-value update method in (9.21), we can obtain the minimum value Qt(S,A), which makes it possible to design an optimal NS-BS selection policy for UEs in a distributed manner. In the following, we illustrate how to use the q-value to obtain the optimal target BS-NS selection policy. 9.2.3.2 Optimal Action Policy For traditional Q-learning, we know that once we get a proper converged Q-table, the policy that we choose the action with the smallest Q-value can guarantee the optimality [16]. However, for the reduced qi-table, if we choose the action with the smallest value q.(s,a) for each UE, it cannot guarantee the optimal policy. In other words, choosing action a* with the smallest value q(6) (a*, S) for each UE may not provide the optimal action-vector A* of all UEs with the smallest Q-value Qt(A*, S) [17], i.e. we cannot guarantee (9.23) To overcome this issue, we use the following policy to choose actions. The main idea is to update and store an action policy in parallel with q(6) (S, aj) update. Once the value of decreases, we update our action policy as we can find a better action, and then we store the better one as the current optimal action. When the q-table converges, implying that the value of minq (5) (S, aj) stays unchanged, our action policy is stable, and the current stored policy is optimal. The update rule of s