Opinions on an LSTM hyper-parameter tuning process I am using
First of all you might want to know there is a "new" Keras tuner, which includes BayesianOptimization, so building an LSTM with keras and optimizing its hyperparams is completely a plug-in task with keras tuner :) You can find a recent answer I posted about tuning an LSTM for time series with keras tuner here

So, 2 points I would consider:


I would not loop only once over your dataset, it does not sound like enough times to find the right weights.
I would rather control the number of possible hyperparams configurations as you said, which is something you can indicate in keras tuner via max_trials param
About using keras tuner with Bayesian tuner, you can find some code below as an example for tuning the units (nodes) in the hidden layers and the learning rate:

from tensorflow import keras
from kerastuner.tuners import BayesianOptimization

n_input = 6
def build_model(hp):
    model = Sequential()
    model.add(LSTM(units=hp.Int('units',min_value=32,
                                    max_value=512,
                                    step=32), 
               activation='relu', input_shape=(n_input, 1)))
    model.add(Dense(units=hp.Int('units',min_value=32,
                                    max_value=512,
                                    step=32), activation='relu'))
    model.add(Dense(1))
    model.compile(loss='mse', metrics=['mse'], optimizer=keras.optimizers.Adam(
        hp.Choice('learning_rate',
                  values=[1e-2, 1e-3, 1e-4])))

return model

bayesian_opt_tuner = BayesianOptimization(
    build_model,
    objective='mse',
    max_trials=3,
    executions_per_trial=1,
    directory=os.path.normpath('C:/keras_tuning'),
    project_name='kerastuner_bayesian_poc',
    overwrite=True)

bayesian_opt_tuner.search(train_x, train_y,epochs=n_epochs,
     #validation_data=(X_test, y_test)
     validation_split=0.2,verbose=1)


bayes_opt_model_best_model = bayesian_opt_tuner.get_best_models(num_models=1)
model = bayes_opt_model_best_model[0]



You would get something like this, informing you about the searched configurations and evaluation metrics: