1. An apparatus for generating a number of output audio channels, the apparatus comprising: a receiver for receiving a data stream comprising a number of input audio channels, the number being one or greater than one, and parametric audio data describing spatial properties; the data stream further comprising decoder tree structure data for a matrix decoder structure, the decoder tree structure data comprising at least one data value from which matrix multiplication coefficients of the matrix decoder structure are generatable, the matrix decoder structure comprising matrix multiplications and intermediate decorrelation units; a structure generator for generating the matrix decoder structure in response to the decoder tree structure data; and an output generator for generating the number of output audio channels from the data stream using the matrix decoder structure.