Is Ethernet traffic slowing OS X?
If you're connected to a switch then it should be filtering the traffic for you, other than broadcast data as you mentioned. If you're using a dumb hub then all traffic would be sent to all ports. In fact, pretty sure WiFi is like a dumb hub- it sends all traffic and all clients are listening.