In a simple chemical cell
The crucial point which explains this is that there are two different forms of conduction happening here.

The first kind of conduction, and the one we are most familiar with is electronic. This is a flow of electrons, and is the conduction which flows in the wire and through the load (a motor, LED etc).

The second kind is the flow which occurs in the electrolyte, and this conduction is ionic. It is a flow of charged particles (ions) within the electrolyte. Even though we often speak of the conductivity of an electrolyte, what is really meant is the electrolyte's ionic conductivity.
This is governed by the mass of the ions, their concentration, temperature and the viscosity of the solution.

Electrolytes do not really conduct electrons at all, which is why there is no 'short circuit' through them. The conductivity arises due to several mechanisms - either the ions themselves move through the solution, or electrons (or protons) hop from one ion to the next, conveying the charge.

This will only happen if there is a means of causing the flow of charge through the electrolyte. When it is open circuit, the electrolyte is overall neutral in charge and no current will flow in it. 
If an external circuit is connected, it allows electrons to flow from one electrode to the other, and then the corresponding amount of charge will flow through the electrolyte to balance it out, producing current.