Opinions on an LSTM hyper-parameter tuning process I am using I am training an LSTM to predict a price chart. I am using Bayesian optimization to speed things slightly since I have a large number of hyperparameters and only my CPU as a resource.

Making 100 iterations from the hyperparameter space and 100 epochs for each when training is still taking too much time to find a decent set of hyperparameters. 

My idea is this. If I only train for one epoch during the Bayesian optimization, is that still a good enough indicator of the best loss overall? This will speed up the hyperparameter optimization quite a bit and later I can afford to re-train the best 2 or 3 hyperparameter sets with 100 epochs. Is this a good approach?

The other option is to leave 100 epochs for each training but decrease the no. of iterations. i.e. decrease the number of training with different hyperparameters. 

Any opinions and/or tips on the above two solutions?

( I am using keras for the training and hyperopt for the Bayesian optimisation)
First of all you might want to know there is a "new" Keras tuner, which includes BayesianOptimization, so building an LSTM with keras and optimizing its hyperparams is completely a plug-in task with keras tuner :) You can find a recent answer I posted about tuning an LSTM for time series with keras tuner here

So, 2 points I would consider:


I would not loop only once over your dataset, it does not sound like enough times to find the right weights.
I would rather control the number of possible hyperparams configurations as you said, which is something you can indicate in keras tuner via max_trials param
About using keras tuner with Bayesian tuner, you can find some code below as an example for tuning the units (nodes) in the hidden layers and the learning rate:

from tensorflow import keras
from kerastuner.tuners import BayesianOptimization

n_input = 6
def build_model(hp):
    model = Sequential()
    model.add(LSTM(units=hp.Int('units',min_value=32,
                                    max_value=512,
                                    step=32), 
               activation='relu', input_shape=(n_input, 1)))
    model.add(Dense(units=hp.Int('units',min_value=32,
                                    max_value=512,
                                    step=32), activation='relu'))
    model.add(Dense(1))
    model.compile(loss='mse', metrics=['mse'], optimizer=keras.optimizers.Adam(
        hp.Choice('learning_rate',
                  values=[1e-2, 1e-3, 1e-4])))

return model

bayesian_opt_tuner = BayesianOptimization(
    build_model,
    objective='mse',
    max_trials=3,
    executions_per_trial=1,
    directory=os.path.normpath('C:/keras_tuning'),
    project_name='kerastuner_bayesian_poc',
    overwrite=True)

bayesian_opt_tuner.search(train_x, train_y,epochs=n_epochs,
     #validation_data=(X_test, y_test)
     validation_split=0.2,verbose=1)


bayes_opt_model_best_model = bayesian_opt_tuner.get_best_models(num_models=1)
model = bayes_opt_model_best_model[0]



You would get something like this, informing you about the searched configurations and evaluation metrics: