RL: What should be the output of the NN for an agent trying to learn how to play a game?
It depends on whether the action is part of the input or output of a neural network estimating the Q-value(state, action).

The network on the left has the state as input and outputs one scalar value for each of the categorical actions. It has the advantage of being easy to setup and only needs one network evaluation to predict the Q-value for all actions. If the action space is categorical and single-dimensional I would use it.

The network on the right has both the state and a representation of the action as input and outputs one single scalar value. This architecture also allows to compute the Q-value for multi-dimensional and continuous action spaces. 

The action space of tic-tac-toe can be easily represented by a vector of length 9, thus I would recommend the left NN-architecture. However, if your game has continuous-valued variables in the action space (e.g. the position of your mouse pointer), you should use the NN-architecture on the right.  

The approach to prevent illegal moves is only partially dependent on the choice of the Q-function architecture and covered by another question: How to forbid actions