Arduino-like ethernet only board? I don't think USB chip is more expensive than Ethernet chip. So, I expect there should exist a board like Arduino, but where USB is removed in favour of Ethernet.

Are there such boards?
Yes, there's the Arduino Ethernet. Available from Sparkfun for example