Limitations of connecting to many Arduinos via Ethernet?
With a lot of devices on the ethernet, you may want to consider a private wire that is dedicated to those devices.  That is, a second ethernet port or a VLAN off a capable switch.

As the number of devices increases, and the number of simultaneous talkers increases, you will start getting contention.  Chances are a modern switch can handle this to some extent, but you were not clear on how much traffic you are planning on sending, or how often each device talks, or if it is centrally coordinated (aka "polled") or just speaking whenever.