Transient Current in an LC circuit with a DC supply
simulate this circuit &ndash; Schematic created using CircuitLab

I would like to provide you the analytical approach of getting the result as mentioned by others already!

In the circuit shown, assuming the the initial conditions of the inductor current and capacitor voltages being:

$$ i_L(0^-)= 0 A \\ v_c(0^-)= 0 V $$ 
From the topology,
$$ V_s = L \frac{di_L(t)}{dt} + v_c(t)$$
$$ v_c(t) = \frac{1}{C}\int i_L(t) dt$$
$$ V_s = L \frac{di_L(t)}{dt} + \frac{1}{C}\int i_L(t) dt $$
Now writing the laplace transform for the above we end up with
$$V_s(s) = L[sI_L(s) - i_L(0^-)] + \frac{I_L(s)}{Cs} -v_c(0^-)$$
The variables in capital letters denote frequency domain representation and those in small letters denote it's time domain counterparts.
Knowing the initial conditions for the state variables respectively,
$$ i_L(0^-)= 0 A \ ,  v_c(0^-)= 0 V $$
$$V_s(s) = LsI_L(s) + \frac{I_L(s)}{Cs}$$
For a dc voltage source or step input excitation voltage of magnitude Vs,
$$V_s(s)=\frac{V_s}{s}$$
$$\frac{V_s}{s} = \frac{LCs^2+1}{Cs}I_L(s)$$
$$I_L(s) = \frac{\frac{V_s}{L}}{s^2+\frac{1}{LC}}$$
Laplace transform of sin(at) is $$\frac{a}{s^2+a^2}$$
Similarly we get, $$i_L(t) = \sqrt\frac{C}{L}V_s \sin(\frac{t}{\sqrt{LC}})$$
Substituting iL(t) in other equations , one can get the cap voltages as,
$$v_c(t) = V_s[1- \cos\frac{t}{\sqrt{LC}}]$$