512 channel LED strip controller over RJ45 (TCP or UDP based commands)
You need:


512 channels of high-current PWM driver;
Some means of configuring any/all of the channels on the fly;
Some means of communicating with the configuration device.


If you can find a multi-channel high-current PWM driver with a serial control interface then that would be a good start (because you could eliminate the FPGA's).  Failing that, to me, the solution is:


512 channels of open-collector/-drain driver, either as individual drivers or as multi-channel units.
An FPGA (or a collection of FPGA's) with at least 512 output pins to provide your PWM, and possibly fade logic.
A reasonably powerful MCU with an Ethernet interface and an external memory bus that you can use to map the PWM control registers in the FPGA/s directly into MCU memory for easy access.


An ARM Cortex-M3 or M4 with the necessary peripherals would be ideal, e.g. the STM32F407 in a QFP144 package, plus supporting circuitry including an Ethernet PHY and magnetics.
A collection of Xilinx XC6SLX4's in QFP144 package for the FPGA's; you'd need at least 8, plus supporting circuitry.
Drivers will cost; your current and voltage requirements are non-trivial so you will pay through the nose.  Bear in mind how much heat those drivers will generate, too; cooling will not be optional.
Last but not least, the power supply.  This is what will really hurt: 512 channels at up to 500mA each is 256 amps @ 12V.  That's over 3kW of power.  If you try to low-side the spec on the power supply (to 512*0.2A i.e. 1.2kW) then sooner or later it will let you down, so don't do it.



In short, this is a sizable design project that, particularly as a one-off, will cost quite a bit of money (at least $750 in parts, excluding the PSU and circuit boards).  You would also have to program it to do what you want.