Keras LSTM accuracy stuck at 50% I'm trying to train an LSTM for sentiment analysis on the IMDb review dataset. 

As input to the word embedding layer, I transform each review to a list of indices (that corresponds to word index in the vocabulary set). I thought of converting the text into one-hot/count matrix, but I will end up with huge sparse matrix (should I worry about this?).

Here is how I am creating the network architecture:

model = Sequential()
model.add(Embedding(
    input_dim=vocab_size,
    output_dim=word_embed_vector_size,
    input_length=sentence_len_max)
         )
model.add(LSTM(units=1))
model.add(Dense(1, activation='softmax'))
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy', 'binary_accuracy'])
model.summary()


Here is the model summary:

_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
embedding_2 (Embedding)      (None, 1422, 4)           201764    
_________________________________________________________________
lstm_2 (LSTM)                (None, 1)                 24        
_________________________________________________________________
dense_2 (Dense)              (None, 1)                 2         
=================================================================
Total params: 201,790
Trainable params: 201,790
Non-trainable params: 0
___________________________


Now when I try to train the model I see accuracy stuck at 50%

losses = model.fit(
    x                = term_idx_train,
    y                = y_train,
    epochs           = epochs,
    batch_size       = batch_size,
    validation_split = 0.01
)


Here is the epochs output:

Epoch 1/10
25000/25000 [==============================] - 1148s 46ms/step - loss: 7.9712 - acc: 0.5000 - binary_accuracy: 0.5000
Epoch 2/10
25000/25000 [==============================] - 1156s 46ms/step - loss: 7.9712 - acc: 0.5000 - binary_accuracy: 0.5000
Epoch 3/10
25000/25000 [==============================] - 1149s 46ms/step - loss: 7.9712 - acc: 0.5000 - binary_accuracy: 0.5000
Epoch 4/10
25000/25000 [==============================] - 1110s 44ms/step - loss: 7.9712 - acc: 0.5000 - binary_accuracy: 0.5000
Epoch 5/10
16800/25000 [===================&gt;..........] - ETA: 6:10 - loss: 7.9816 - acc: 0.4993 - binary_accuracy: 0.4993


Changing the activation function to a sigmoid and the LSTM blocks to 32 didn't help mush (with 1 epoch):

Train on 24750 samples, validate on 250 samples
Epoch 1/1
24750/24750 [==============================] - 1186s 48ms/step - loss: 0.6932 - acc: 0.5022 - binary_accuracy: 0.5022 - val_loss: 0.6951 - val_acc: 0.0000e+00 - val_binary_accuracy: 0.0000e+00

Epoch 00001: val_loss improved from inf to 0.69513, saving model to sentiment_model


Looking at what the LSTM is predicting, I see:

count   25000.000000
mean    0.499023
std 0.000013
min 0.499010
25% 0.499010
50% 0.499010
75% 0.499010
max 0.499443


Any idea why it's doing this? and how I could fix the issue?
activation = ‘softmax’ should be used for multiclass classification whereas ’sigmoid’ for binary classification. 

You can refer to: http://dataaspirant.com/2017/03/07/difference-between-softmax-function-and-sigmoid-function/

If changing the activation function does not help, I will be around for alternative solutions.