How to do feature selection after using pre-trained word embeddings? I am working on a multiclass text classification problem. I want to use the top k features based on mutual information (mutual_info_classif) for training my model.

I started this project on ML models:

I used tfidf for feature extraction and then used mutual_info_classif for feature selection. 

    svc = Pipeline([('tfidf', TfidfVectorizer()),
                   ('ig', SelectKBest(mutual_info_classif,k=1000)),
                   ('clf',LinearSVC(multi_class='ovr')),
              ])
    svc.fit(X_train, y_train)
    y_pred = svc.predict(X_test)


This was pretty straight-forward.

Next, I started to work on RNN (more specifically a simple LSTM model)and that is where I am having a problem.

I have used pre-trained word embeddings from GloVe (300d) to get features from my data.
The embedding matrix I feed into the embedding layer of my RNN has the shape (4293,300), 4293 is the number of unique words found in my data and 300 is the dimension.

My questions are: Is there any way that I can use the top 1000 words (features) out of these 4293 based on mutual_info_classif? Is it even possible to do so? If yes, then should it be done before making the embedding_matrix or after?
When using a RNN, you don't feed all the data at once, you usually have a seq2seq model.
The models are created with an encoder-decoder architecture. The LSTM is used in the encoding phase.

So, let's say you have a text of 78 words. You will feed the embedding vector (size 300) of those 78 words, 1-by-1 into your LSTM and in the end you will get a hidden vector which represents your sentence.

Then, you can take this hidden vector and use it for classification (with a feedforward neural network, for example).

So, it doesn't matter that you have 4293 unique words in your data. You need to feed your LSTM a sequence of size [&lt;number of words in sentence&gt; × 300].