1. An apparatus for processing an encoded audio signal comprising a plurality of downmix signals associated with a plurality of input audio objects and object parameters E, comprising: a grouper configured to group said plurality of downmix signals into a plurality of groups of downmix signals based on information within said encoded audio signal, wherein each group of downmix signals is associated with a set of input audio objects of said plurality of input audio objects, a processor configured to perform at least one processing step individually on the object parameters E k of each set of input audio objects in order to provide group results, and a combiner configured to combine said group results in order to provide a decoded audio signal, wherein said grouper is configured to group said plurality of downmix signals into said plurality of groups of downmix signals so that each input audio object of said plurality of input audio objects belongs to just one set of input audio objects, and wherein said grouper is configured to group said plurality of downmix signals into said plurality of groups of downmix signals so that each input audio object of each set of input audio objects either is free from a relation signaled in the encoded audio signal with other input audio objects or comprises a relation signaled in the encoded audio signal only with at least one input audio object belonging to the same set of input audio objects.