How to use and update a shared/global policy between Reinforcement Learning Agents
This sounds like distributed RL, and most of the work goes in building the distributed system; the actual RL part is just a DQN (+some tricks from Rainbow DQN).
NB: multi-agent RL arises when the agents interact with each other in the same environment (like Hanabi the card game), while in this case we have multiple agents that collect experiences in parallel.
Here's a possible design from the Ape-X paper: