Battery voltage changing drastically after use
You're probably sucking too much current out of the battery pack too quickly.  If you can dig up a data sheet for the battery there will likely be a rating for continuous draw and one for peak impulse draw.  Find those numbers, and try to stay under the continuous draw figure.  you might need more batteries in parallel.