How to put multiple features into RNN input vector I am trying to code a recurrent neural network (LSTM) to create music in python and was considering using multiple features instead of just the note pitch as an input into the network. Initially I had just the note pitch so it was fed into the network by one-hot encoding it. The other two features I want to add are the note duration and the offset between the notes. How should the input vector be organised so that all the data is fed through the network?

I have tried combining all of the data into a long vector with all 3 features one-hot encoded and then concatenated but this caused the output to become 'NaN'. Any help would be appreciated.

Link a gist of my code:

https://gist.github.com/treutm/1b1f49e3d0a9de53cd67c136292f329f
A common way to input several features to an LSTM (or any RNN) is, as you did, to concatenate them in a vector. I suspect your NaN are related to a different issue in the code, and I recommend you to debug it and see when and why it happens.

A different way of combining several features is using embeddings for each feature and combine them via concatenation. For example, for the note pitch, you have an embedding for each note, for the note duration, the same, and so on. The input to the RNN would be a concatenation of that.

A third option would be to have an embedding to all possible combinations of the embeddings. That would be an embedding for each triplet of note pitch, note duration and offset. This representation can be also powerful.

My recommendation is that you start with the easiest to implement, and if the result is not satisfactory in terms of model performance, think about the next one.