Is Windows a bad choice for DRL? I'm looking into using PPO implementations like OpenAi's SpinningUp and Baselines. However, I fear that these implementations require packages which are not available for Windows. So I'm wondering if in general Linux should be used for working with DRL algorithms. Also, I'm not planing on using OpenAI's gym environments. Environments, created with the ml-agents toolkit from Unity and its gym wrapper are going to be used.

What is your opinion on this? Should I rather work on Linux or Windows? And what do you think about alternatives such as: VM on Windows with Ubuntu or Subsystem on Windows with Ubuntu (WSL)?
TLDR Use whatever environment you are most comfortable with. 

Today the choice of operating system is of much less importance than few years ago. Generally speaking, you will have more application choices under Linux, however, some of these choices will be less than spectacular :-) Windows works better with some hardware. Hence, under Linux, it is a very good idea to check the forums before buying or installing. 

I have played with OpenAI gym under Linux for some time and it worked as advertised. 

Early versions of many important pieces of deep learning software get released for Linux only. For instance, PyTorch and Tensorflow were available for Linux at least one year before the release of Windows ports.