What is the most biologically plausible representation for the actor and critic? Which representation is most biologically plausible for actor nodes? For example, actions represented across several output nodes which may be either


mutually exclusive with each other (e.g., go north, go south, etc),
achieved by winner-takes-all. 
NOT mutually exclusive with each other (e.g. left leg forward, right leg forward); these actions may occur concurrently. To go north, the correct combination of nodes must be active.


Similarly which representation is most plausible for critic output nodes?


A single output node that outputs a real number representing the
reward. 
A set of output nodes each representing a separate value, achieved by winner-takes-all.


Or do other representations better align with real brains ?
For the actor, I'd say the 'not mutually exclusive' option is more biologically plausible in the context of muscle systems, where the actions can be seen as simultaneous muscle activations. Maybe at a higher level, an agent thinks of the action as 'go north' or 'go south', but the final outputs which have to control muscles at a lower level have to represent simultaneous muscle activations.

For the critic, I'd say the 'single output node' is more biologically plausible. Agents perceive the world in the form of high dimensional inputs, such as images. The approach where a value function is learned in a tabular fashion where you know the value for every single state doesn't really scale very well and is limited to small discrete state spaces. For biological agents, it makes sense to have a function that senses the current state of the environment and outputs a single number that represents the value, which gives the agent an idea of how things are going so far given the actions it took in the past.