title: Forming a Multi input LSTM in Keras
body: <p>I am trying to predict neutron widths from resonance energies, using a Neural Network (I'm quite new to Keras/NNs in general so apologies in advance).</p>
  
  <p>There is said to be a link between resonance energies and neutron widths, and the similiarities between energy increasing monotonically this can be modelled similiar to a time series problem.</p>
  
  <p>In essences I have 2 columns of <a href="https://pastebin.com/index/9qwJU3AQ" rel="nofollow noreferrer">data</a> with the first column being resonance energy and the other column containing the respective neutron width on each row.  I have decided to use an LSTM layer to help in the networks predict by utlising previous computations.</p>
  
  <p>From various <a href="https://machinelearningmastery.com/time-series-prediction-lstm-recurrent-neural-networks-python-keras/" rel="nofollow noreferrer">tutorials</a> and <a href="https://stackoverflow.com/questions/42532386/how-to-work-with-multiple-inputs-for-lstm-in-keras/51411847#51411847">other answers</a>, it seems common to use a "look_back" argument to allow the network to use previous timesteps to help predict the current timestep when creating the dataset e.g</p>
  
  <pre><code>trainX, trainY = create_dataset(train, look_back)
  </code></pre>
  
  <p>I would like to ask regarding forming the NN:</p>
  
  <p>1) Given my particular application do I need to explicitly map each resonance energy to its corresponding neutron width on the same row?</p>
  
  <p>2) Look_back indicates how many previous values the NN can use to help predict the current value, but how is it incorporated with the LSTM layer? I.e I dont quite understand how both can be used?</p>
  
  <p>3) At which point do I inverse the MinMaxScaler?</p>
  
  <p>That is the main two queries, for 1) I have assumed its okay not to, for 2) I believe it is possible but I dont really understand how.  I can't quite work out what I have done wrong in the code, ideally I would like to plot the relative deviation of predicted to reference values in the train and test data once the code works. Any advice would be much appreciated:</p>
  
  <pre><code>import numpy
  import matplotlib.pyplot as plt
  import pandas
  import math
  
  from keras.models import Sequential
  from keras.layers import Dense, LSTM, Dropout
  from sklearn.preprocessing import MinMaxScaler
  from sklearn.metrics import mean_squared_error
  
  
  # convert an array of values into a dataset matrix
  
  def create_dataset(dataset, look_back=1):
      dataX, dataY = [], []
      for i in range(len(dataset) - look_back - 1):
          a = dataset[i:(i + look_back), 0]
          dataX.append(a)
          dataY.append(dataset[i + look_back, 1])
      return numpy.array(dataX), numpy.array(dataY)
  
  # fix random seed for reproducibility
  numpy.random.seed(7)      
  # load the dataset
  dataframe = pandas.read_csv('CSVDataFe56Energyneutron.csv', engine='python') 
  dataset = dataframe.values
  print("dataset")
  print(dataset.shape)
  print(dataset)
  
  # normalize the dataset
  scaler = MinMaxScaler(feature_range=(0, 1))
  dataset = scaler.fit_transform(dataset)
  print(dataset)
  # split into train and test sets
  train_size = int(len(dataset) * 0.67) 
  test_size = len(dataset) - train_size
  train, test = dataset[0:train_size, :], dataset[train_size:len(dataset), :]
  
  # reshape into X=t and Y=t+1
  look_back = 3
  trainX, trainY = create_dataset(train, look_back)  
  testX, testY = create_dataset(test, look_back)
  # reshape input to be  [samples, time steps, features]
  trainX = numpy.reshape(trainX, (trainX.shape[0], look_back, 1))
  testX = numpy.reshape(testX, (testX.shape[0],look_back, 1))
  # # create and fit the LSTM network
  # 
  number_of_hidden_layers=16
  model = Sequential()
  model.add(LSTM(6, input_shape=(look_back,1)))
  for x in range(0, number_of_hidden_layers):
      model.add(Dense(50, activation='relu'))
  model.add(Dense(1))
  model.compile(loss='mean_squared_error', optimizer='adam')
  history= model.fit(trainX, trainY, nb_epoch=200, batch_size=32)
  trainPredict = model.predict(trainX)
  testPredict = model.predict(testX)
  print('Train Score: %.2f MSE (%.2f RMSE)' % (trainScore, math.sqrt(trainScore)))
  testScore = model.evaluate(testX, testY, verbose=0)
  print('Test Score: %.2f MSE (%.2f RMSE)' % (testScore, math.sqrt(testScore)))
  </code></pre>