Smaller alternative to RJ45 Ethernet connector
In theory, you could get a custom cable to go from RJ45 to anything you want. Samtec has some great connectors. The only thing that (supposedly) matters is the pairs being twisted to prevent crosstalk between the pairs. However, if your cabling is very short, it might not matter as much, especially for 10BaseT. You can probably use an old-school 0.1 pitch connector for that.