Why are integrated circuits powered by low voltage and high current?
I am not sure why this wasn't the first thing pointed out by any of the earlier answers, but it is because as transistors are made smaller to increase speed, increase density, and reduce power consumption, the gate oxide layer is made thinner (which also increases leakage currents). 

A thin gate oxide layer can't withstand very high voltages so you end up with a device that only operates at very low voltages. Thin oxide layers also have more leakage so you don't want a high voltage anyways since that would just increase leakage current and increases static power consumption.

Your mistake is this:

Data processing, unlike power systems, isn't about power delivery; It's about data processing. So it is not that designers choose to operate at low voltages and high currents thus going against \$I^2R\$. Yes, they are concerned about power consumption and heat due to losses, but they aren't concerned with the efficient delivery of power. A power designer has to deliver X amount of power and would increase voltage so they could decrease current while delivering that same power. A digital designer would outright decrease the "power output" if they could.

Their optimizations necessitate low operating voltages which results in high leakage currents. The goal of these optimizations is to allow smaller transistors so you can pack more of them in as well as switch them faster, and when you have millions upon millions of transistors switching very frequently that results a lot of charging/discharging the gate capacitances. This dynamic current results in the high peak currents which can be  tens of amps in high-speed, high density digital logic. You can see that all this current and power is undesired and unintentional.

Ideally, we would like really no current at all because our concern is information, not energy/power. High voltages would also be nice for noise immunity but this runs directly counter to making transistors smaller.