Why can electrolytes be used as an insulator?
They generally can't.

In an electrolytic capacitor, the  electrolyte conducts pretty well (look at the ESR of the capacitor). But it electrolytically oxidises a layer on the aluminium anode, and that oxide layer is the insulator.

I'll let someone else explain about batteries.