How does a battery create a voltage?
Pretty close but what you describe much more accurately reflects a capacitor than a battery. In a capacitor, you add energy to pull electrons away from where they want to be thus storing potential energy, and that potential energy is released when the electron is released and allowed to pop back. You do move  electrons in a similar way in a battery, but the electron don't stay &quot;free&quot;. They got locked into chemical bonds instead.
A charged battery has a different high-energy chemical compound on each side. They are high energy and therefore unstable and want to release the energy so that the elements can re-form into lower energy compounds which are more stable. One side requires more electrons to go from high energy to low energy, while the other side has excess electrons when going from high energy to low energy.
The elements on each side of the battery can form one of two compounds: a low energy compound and a high energy compound. But the the compounds between the two sides of the battery have opposing characteristics:

On one side, the low energy compound requires more electrons to form than its corresponding high energy compound contains.
On the other side, the low energy compound requires fewer electrons to form its chemical bonds than its corresponding high energy compound actually has.

The high energy compounds are prevented from reforming into the low energy compounds if the battery is left disconnected because the elements on one side require more electrons to form the low energy compound than the high energy compound actually contains, while on the other side the high energy compound contains more electrons than the low energy compound needs and has no way to get rid of the extra electrons if the low energy compound were to form.

An uncharged battery has only low energy compounds on both sides of the battery. When you charge the battery, you are adding energy to force those electrons from one side of the battery to the other:

You rob electrons from the side with low energy compounds that need
them (yet its corresponding high energy compound does not need them),
and move those electrons to the side with low energy compounds that
does not need them (yet its corresponding high energy compound does need them).

In the presence of an electron deficiency or surplus on the correct side, this breaks up the low energy compounds and forces the high energy compounds to form in their place. This effectively stores that energy in the chemical bonds of those compounds thus charging the battery. It's the chemistry version of winding up a spring.
These compounds are high energy and therefore unstable. When you give a path for electrons to flow between the two materials formed by those chemical reactions, they have an easy way to re-enter the lowest energy state (as so many things in the universe try to do) which is more stable by undoing the chemical reaction and as a useful byproduct, charge from side to the other and releases the energy you initially stored in the chemical bonds.
Then you come full circle, where:

the side with high energy compound that needs those electrons no
longers has them while its corresponding low energy compound doesn't
need them,
while on the other side, the high energy compound has electrons but
doesn't need them while its corresponding low energy compound does
need them.

So the high energy compounds on both sides get broken up and recombine to form the low energy compound, producing a dead battery.

What makes a battery rechargeable is that these chemical reactions are reversible. If the chemical reaction is not reversible then you have a non-rechargeable battery where the materials used to produce the battery must be prepared outside of the battery in their &quot;charged&quot; form.
I hope that gives you a clearer picture of what is going on inside. You need to take a basic chemistry class to see how the moving pieces actually work inside the battery (with why certain elements are used and with the ions and electrons moving back and forth).