From Keras RNN Tutorial: RNNs are tricky. Choice of batch size is important, choice of loss and optimizer is critical, etc. Some configurations won't converge.So this is more a general question about tuning the hyperparameters of a LSTM-RNN on Keras. I would like to know about an approach to finding the best parameters for your RNN. I began with the IMDB example on Keras' Github.the main model looks like this:(X_train, y_train), (X_test, y_test) = imdb.load_data(nb_words=max_features,                                                      test_split=0.2)max_features = 20000maxlen = 100  # cut texts after this number of words (among top max_features most common words)batch_size = 32model = Sequential()model.add(Embedding(max_features, 128, input_length=maxlen))model.add(LSTM(128))  model.add(Dropout(0.5))model.add(Dense(1))model.add(Activation('sigmoid'))# try using different optimizers and different optimizer configsmodel.compile(loss='binary_crossentropy',          optimizer='adam',          class_mode=binary)print(Train...)model.fit(X_train, y_train, batch_size=batch_size, nb_epoch=3,      validation_data=(X_test, y_test), show_accuracy=True)score, acc = model.evaluate(X_test, y_test,                        batch_size=batch_size,                        show_accuracy=True)print('Test accuracy:', acc)Test accuracy:81.5432184681.5 is a fair score and more importantly it means that the model, even though not fully optimized, it works. My data is Time Series and the task is binary prediction, the same as the example. And now my problem looks like this:#Training Datatrain = genfromtxt(os.getcwd() + /Data/trainMatrix.csv, delimiter=',', skip_header=1)validation = genfromtxt(os.getcwd() + /Data/validationMatrix.csv, delimiter=',', skip_header=1)#TargetsminiTrainTargets = [int(x) for x in genfromtxt(os.getcwd() + /Data/trainTarget.csv, delimiter=',', skip_header=1)]validationTargets = [int(x) for x in genfromtxt(os.getcwd() + /Data/validationTarget.csv, delimiter=',', skip_header=1)]#LSTMmodel = Sequential()model.add(Embedding(train.shape[0], 64, input_length=train.shape[1]))model.add(LSTM(64)) model.add(Dropout(0.5))model.add(Dense(1))model.add(Activation('sigmoid'))# try using different optimizers and different optimizer configsmodel.compile(loss='binary_crossentropy',          optimizer='adam',          class_mode=binary)model.fit(train, miniTrainTargets, batch_size=batch_size, nb_epoch=5,      validation_data=(validation, validationTargets), show_accuracy=True)valid_preds = model.predict_proba(validation, verbose=0)roc = metrics.roc_auc_score(validationTargets, valid_preds)print(ROC:, roc)ROC:0.5006526The model is basically the same as the IMDB one. Though the result means it's not learning anything. However, when I use a vanilla MLP-NN I don't have the same problem, the model learns and the score increases. I tried increasing the number of epochs and increasing-decreasing the number of LTSM units but the score won't increase.So I would like to know a standard approach to tuning the network because in theory the algorithm should perform better than a multilayer perceptron network specially for this time series data.
Hyperparameter search for LSTM-RNN using Keras (Python)
python;neural network;deep learning;keras;hyperparameter