Discharge and charge rates of a capacitor - Comparing Energy Movements
If that is correct, then let me also ask something else: The voltage is irrelevant to the measured energy expenditure right? It doesn't matter if the first capacitor starts at 100 volts and discharges to 0 volts while the second capacitor starts at 0 and charges to 10 volts. All that matters for computing the energy movement is the actual charge/amps. Right?


This part is not right. The voltage is very relevant to the energy.

The energy stored in a capacitor is:

$$E = \frac{1}{2}CV^2$$

The voltage on a capacitor is determined by the charge and the capacitance:

$$V = \frac{Q}{C}$$

If you have a smaller capacitor, a given amount of charge will create a higher voltage, and it will take significantly more energy to put it there.