Bagging with Neural Networks Best practices
I think you have the right general idea.


Divide the dataset into training and test.
Divide the training dataset into k folds, call this M. 


For each iteration of hyperparameter tuning (whatever you choose to tune);

For each fold i in M;


Set the validation set to be equal to fold i, and the (inner) training set to be equal to all folds that are not i.
Using the inner training set, generate N bootstrap resamples. Fit a single neural network model to each bootstrap resample, generating N models.
Predict the validation set using each of the N neural network models.
Count the votes of each neural network model. Set the final prediction for an observation in the validation set to whatever has the most votes as predicted by your N models. You should have a matrix of m observations by N predictions. Since you are actually classifying objects, try to keep N odd so that there are no ties. Alternatively, don't classify objects, and instead generate probabilities from your N models for each observation in the validation set. Then, calculate the mean predicted probability for each observation in your validation set, and choose a threshold for classification based off the costs of false positives/false negatives (probably superior, but I won't discuss this).
Calculate your error using your final predictions from step 6 and the actual ground truth labels from the validation set. Save these error scores.


end


Calculate the average error from all of the folds. If this average error is lower than a previous set of hyperparameters error, then save these hyperparameters from this iteration of tuning.


end


Generate N bootstrap resamples of the entire training set (the training set created in step 1) and fit a single neural network model to each of the N bootstrap resamples, using the optimal set of hyperparameters that you found above for each neural network model.
Generate predictions on the test set (created in step 1) using each of your N models. Set final predictions for the test set using the exact same methodology you used in step 6.
Calculate the error using your final predictions from step 10 and the ground truth labels found in the test set. This is your final, unbiased estimate of model performance.


In general, the more bootstrap resamples you can generate, the better. However, since you are fitting neural networks you might need to be conservative with how many models you decide to bag due to computational costs. I recommend setting the number of bootstrap samples, N, to be as large as possible while being mindful of your own computational/time constraints. 

EDIT: In response to the comments in which we require a faster method, one can replace step 2 (and delete step 3) and instead divide the training dataset into an inner training set and a validation one. We then validate our model to a single validation set rather than k sets, which has the potential to increase the likelihood of overfitting to a specific validation set (though since we are presumably dealing with a large dataset, it probably will be fine).

All other steps should remain the same, though there is clearly no loop over folds anymore. In step 8, there will also be no need to calculate the average error since we will only have one estimate of model error from the single validation set.