Can I set the same relation in a field multiple times?
You're right, this can only be accomplished with a Matrix field.

That said, you could probably accomplish what you need via a custom field type. You'd need to create a custom plugin (obviously), but you could probably build something that simply stores an array of asset IDs:

[12,33,42,10,10,10]


I won't diver deeper into what the mechanics of that plugin &amp; field type should be... that's a bit beyond the scope of this question.