Parallel RLC Step Response with Voltage Source
Assuming the context of ideal circuit theory, the voltage source current will not be finite at the instant the switch is closed; there will be an impulse of current to charge the capacitor instantly to 12V.  After the initial impulse, the current will be a ramp, increasing linearly with time (without bound) due to the inductor.

However, ideal circuit theory is not physically relevant in this case since some of the assumptions of ideal circuit theory do not hold for the second circuit.

Physically, the current will be finite and the voltage across the capacitor will not be a step.  This can be seen if the naive ideal circuit model is augmented with additional circuit elements to model the fact that, e.g.,


All physical voltage sources have finite internal resistance (finite
short circuit current)
Physical capacitors have parasitic inductance and resistance which
must be included in the ideal circuit model
A physical circuit (the wires and circuit elements that form the
closed path for current) have parasitic R, L, and C that must be
modelled
A physical circuit has radiation resistance, i.e., for large and
fast current changes, the circuit will radiate energy into space and
this must be modelled.


Regarding your perceptive question 3, replace current above with voltage, inductor with capacitor, short with open and then essentially the same argument holds.