LSTM not learning with extra nontemporal data added after LSTM layer - Keras I have three different inputs I would like to send into my LSTM - the sequence of "words", extra temporal information, as well as extra nontemporal information.

Following Adam Sypniewski excellent answer to this question (Adding Features To Time Series Model LSTM) I came with this architecture:

word_input = Input(shape=(sequence_length,))
temporal_metadata = Input(shape=(sequence_length, 1))
nontemporal_metadata = Input(shape=(nontemporal_num_features,))

embedding = Embedding(input_dim=vocab_size, output_dim=100)(word_input)
temporal_info = concatenate(inputs=[embedding, temporal_metadata])
rnn = LSTM(50, dropout = .5, recurrent_dropout=.5)(temporal_info)
all_info = concatenate(inputs=[rnn, nontemporal_metadata])
dense_1 = Dense(10, activation='relu')(all_info)
dropout_1 = Dropout(.5)(dense_1)
dense_2 = Dense(1, activation='sigmoid')(dropout_1)

model = Model([word_input, temporal_metadata, nontemporal_metadata], dense_2)
model.compile(loss='binary_crossentropy', optimizer='adam'
model.fit(x=[my_words, my_temporal, my_nontemporal], y = my_y, 
     epochs = 30, 
     validation =([my_words_test, my_temporal_test, my_nontemporal_test], y_test))


This does not learn, and gets an AUC of 0.5

However if I don't add the nontemporal data and change the above code to

word_input = Input(shape=(sequence_length,))
temporal_metadata = Input(shape=(sequence_length, 1))

embedding = Embedding(input_dim=vocab_size, output_dim=100)(word_input)
temporal_info = concatenate(inputs=[embedding, temporal_metadata])
rnn = LSTM(50, dropout = .5, recurrent_dropout=.5)(temporal_info)
dense_1 = Dense(10, activation='relu')(rnn)
dropout_1 = Dropout(.5)(dense_1)
dense_2 = Dense(1, activation='sigmoid')(dropout_1)

model = Model([word_input, temporal_metadata], dense_2)
model.compile(loss='binary_crossentropy', optimizer='adam'
model.fit(x=[my_words, my_temporal], y = my_y, 
     epochs = 30, 
     validation =([my_words_test, my_temporal_test], y_test))


It works great. Can anyone please help me find what is going wrong in the first code sample that would make the model unable to learn. Thanks!
if anyone stumbles upon this - it seemed like my dimension of nontemporal data was too high and sparse. After filtering only for the most common/important variables in the nontemporal data, the model learned.