How can I explain the cause of different performances for two different LSTM models and improve the performance?
LSTM especially works for time-series data. I'm assuming these features are not dependent temporally, so it's not ideal to use LSTM or any RNN variant.

Remove the LSTM layers, reduce the number of neurons, use regularizer, dropout.
Also, using ReLU in the last layer doesn't make any sense. If it's a regression problem, just use linear activation instead.

To reduce the error, do a hyperparameter tuning with a randomized search or grid search.