How to fill NaNs in Cross-Validation?
I would do the exact same thing as you are describing! One of the main reasons that you would want to do cross-validation is to prevent that your model is unable to generalize later. Therefore, you take out a random small subset which will be your new small validation set and do all the 'operations' on that which you are also doing on your training set. This way, you can check that these 'operations' are thus also generalizable (if they work for all different cross-validations).