How can I train a many-to-one RNN with an array of 2D matrices? I have eye tracking data for every word of a novel. Features for every word is given separately. I want to take groups of 100 words to make a sample and then use each of these samples as a single training instance (instead of a single word). My samples are 2D matrices of shape (no. of words, features). How can I train my many-to-one RNN with these samples?
you can use CNN just before inputting your raw matrix into RNN. An other option, which is just made for that purpose is ConvLSTM. Several research papers show that ConvLSTM can be used to process in sequential frames of video which is similar to your input type.