Does Profile Manager support nested device groups?
Yes.

The problem was that a process on the server or the client did not like one of the settings payloads in the "outer" group (Directory in my case).