Flatland-RL : Multi-Agent Reinforcement Learning on Trains
  Efficient automated scheduling of trains remains a major challenge for modern
railway systems. The underlying vehicle rescheduling problem (VRSP) has been a
major focus of Operations Research (OR) since decades. Traditional approaches
use complex simulators to study VRSP, where experimenting with a broad range of
novel ideas is time consuming and has a huge computational overhead. In this
paper, we introduce a two-dimensional simplified grid environment called
"Flatland" that allows for faster experimentation. Flatland does not only
reduce the complexity of the full physical simulation, but also provides an
easy-to-use interface to test novel approaches for the VRSP, such as
Reinforcement Learning (RL) and Imitation Learning (IL). In order to probe the
potential of Machine Learning (ML) research on Flatland, we (1) ran a first
series of RL and IL experiments and (2) design and executed a public Benchmark
at NeurIPS 2020 to engage a large community of researchers to work on this
problem. Our own experimental results, on the one hand, demonstrate that ML has
potential in solving the VRSP on Flatland. On the other hand, we identify key
topics that need further research. Overall, the Flatland environment has proven
to be a robust and valuable framework to investigate the VRSP for railway
networks. Our experiments provide a good starting point for further research
and for the participants of the NeurIPS 2020 Flatland Benchmark. All of these
efforts together have the potential to have a substantial impact on shaping the
mobility of the future.
