How did the OpenAI 5 for Dota concatenate units?
Tl;dr max-pool
You can see in the diagram, everywhere there are a variable number of inputs (pickups, units, hero modifiers/abilities/items), a max-pool follows, though I don't know the specifics of the max-pool implementation.
From https://neuro.cs.ut.ee/the-use-of-embeddings-in-openai-five :

Notice that while the number of modifiers, abilities and items is variable, the network max-pools over each of those lists. This means that only the highest value in all those dimensions actually gets through. At first it does not seem to make sense – it might give the impression that you have an ability that is combination of all existing abilities, e.g. ranged passive heal. But it seems to work for them.


Above processing is done separately for each of the nearby units, the results from general attributes, hero modifiers, abilities and items are all concatenated together. Then different post-processing is applied depending if it was enemy non-hero, allied non-hero, neutral, allied hero or enemy hero.


Finally the results of post-processing are max-pooled over all units of that type. Again this seems to be questionable at the first sight, because different qualities of nearby units would be combined, e.g. if one of the dimensions would represent the health of a unit, then the networks sees only the maximum health over the same type of units. But, again, it seems to work fine.