Complex impedances of capacitors/inductors in the phasor domain
The core of your question is

is it just a mathematical requirement without a conceptual reason?

Recall that the exponential notation is for the stationary case. In other words, it assumes that the voltage oscillation has been applied for an infinite amount of time prior to t=0. Practically this assumption is satisfied when any transient effects, after starting for instance with v(0)=V and I(0)=0, have died out well before t=0, and only a pure and constant sinusoidal oscillation remains at t&gt;0.
You can also cut into t=0, and start a pure and constant sinusoidal oscillation by applying the appropriate initial conditions as though the transients have settled. In your case this means starting with v(0) = V and i(0) as mathematically calculated in your question, and letting the system oscillate with v(t) after that.
This is a real physical concept. In linear systems, transients can be entirely eliminated by initializing the system with appropriate initial conditions, as you have done.