If use the weights from previous iteration of a k-fold cross validation to seed a neural network classifier would I be overfitting?
To answer directly your question: estimating a model's performance on data previously used for fitting will overestimate performance.

When your dataset is "small" you are faced with a bias-variance dilemma with regards to how much data should be used for train and test sets:


Too much train data, you end up with little test samples and your
performance estimate has high variance.
Too much test data, your train samples do not represent well the
population you are trying to model/target, so your average
performance will be much lower than what could be achieved.


K-fold CV is a compromise in the evaluation of performance of certain procedure. Once you have settled on a certain model or hyperparameters, when moving to production you can:

Choose to take one of the k models you have trained, perhaps using the "one std variation rule of thumb".


Re-train on all of your data and you can expect the resulting model
to be at least as good as you have estimated.
Use all k models to form an ensemble and you can expect the resulting
model to be at least as good as you have estimated.