Title: Implications of Decentralized Q-learning Resource Allocation in Wireless Networks,
Abstract: Reinforcement Learning is gaining attention by the wireless networking
community due to its potential to learn good-performing configurations only
from the observed results. In this work we propose a stateless variation of
Q-learning, which we apply to exploit spatial reuse in a wireless network. In
particular, we allow networks to modify both their transmission power and the
channel used solely based on the experienced throughput. We concentrate in a
completely decentralized scenario in which no information about neighbouring
nodes is available to the learners. Our results show that although the
algorithm is able to find the best-performing actions to enhance aggregate
throughput, there is high variability in the throughput experienced by the
individual networks. We identify the cause of this variability as the
adversarial setting of our setup, in which the most played actions provide
intermittent good/poor performance depending on the neighbouring decisions. We
also evaluate the effect of the intrinsic learning parameters of the algorithm
on this variability.