proposed scheme improved network scalability by creating larger clusters and improved cluster stability by reducing the number of re-clusterings (the number of cluster splits) and clustering overhead while reducing interference between licensed and unlicensed users in CRNs. Using Q-value to evaluate the channel quality in Cluster-based CR Ad-Hoc Networks (CRAHN), a Q-learning-based cluster formation mechanism was studied in [74]. Channel quality, residual energy, and network conditions were jointly considered to form a dis- tributed cluster network. All the nodes built their neighboring topology by exchanging the channel status and neighbor list information. Each node then selected the optimal cluster head candidate. Distributed cluster head selections, optimum common active data channel decisions, and gateway node selection procedures were presented. It was shown that the proposed approach could extend the network lifetime and enhance reachability. 10.2.1.3 Spectrum Access One important question in CRNs spectrum access is how to assign limited resources, such as available spectrum channels and transmit powers, to maximize the system throughput and efficiency. Numerous related works based on RL [94, 147, 241], DL [113, 115], and Deep RL (DRL) [101, 161] have been carried out. An RL-based resource allocation approach entitled Q-Learning and State-Action-Reward- State-Action (SARSA) was proposed in [94] that mitigated interference without the require- ments of the network model information. Users in this method act as multiple agents and cooperate in a decentralized manner. A stochastic dynamic algorithm was formed to 10 Secure Spectrum Sharing with Machine Learning: An Overview determine the best resource allocation strategy. It was shown that the energy efficiency could be significantly improved by the proposed approach without sacrificing user QoS. An energy harvesting-enabled CRN was investigated in [147]. To achieve higher throughput, a harvest-or-transmit policy for SUs transmit power optimization was pro- po