Does LSTM without delayed inputs work as a deep net?
LSTMs are RNN with memory cell
It can be difficult to train standard RNNs to solve problems that require learning long-term temporal dependencies.
LSTM units include a 'memory cell' that can maintain information in memory for long periods of time. A set of gates is used to control when information enters the memory, when it's output, and when it's forgotten. This architecture lets them learn longer-term dependencies.
So without the delayed inputs a LSTM is simply a RNN.

As you can see here, RNN has a recurrent connection on the hidden state. This looping constraint ensures that sequential information is captured in the input data.
Check this for more details