When to use model.train() vs model.eval() in Pytoch?
I can see arguments to have the self-play phase use both .train() and .eval(), so I had a look at the implementation of facebook's ELF OpenGo and saw that they have the model in eval mode during self-play (see selfplay.py). I would do as they do because their software seems to work.