How to handle preprocessing (StandardScaler, LabelEncoder) when using data generator to train?
It is a correct approach to standardize on your training features. In that way, you ensure not to give any information from the testing set to the training set.
About features scaling, if you have too many samples to fit your scaler at once, you could use the partial_fit (see here) method of StandardScaler in sklearn. Load sequentially your training features and do the partial_fit. Once finished, your scaler is ready to be used on your training/testing batches.
About label encoding, either you already have an array containing all the labels, so you can fit your LabelEncoder, or you would have to load sequentially all your data to get all the different labels before fitting (LabelEncoder does not have partial_fit method).