Why does an ideal capacitor give rise to a rectangular cyclic voltammogram (CV)?
The kind of analysis truly required is outside the scope of chemistry and would be best handled in the careful hands of an electrical engineer. I'll give a brief attempt that exploits an easy-to-use circuit simulator.

We have to first think of an equivalent circuit that we can use as a model to determine its behavior. I suggest the following:



where my variable is R1's value for its resistance, which I shall set to 0, 10 and 100 ohm. If we were to watch this happen in time then we would see the following:




Quickly converting these to Current vs Voltage and running two other simulations at different resistances and we get:



These results are due to setting up differential equations and solving them appropriately.

You can play with the circuit I made for yourself here.