How to Build Simple LSTM Model for Anomally Detection
I want to create a LSTM model in PyTorch that will be used for anomally detection, but I'm having trouble understanding the details in doing so.

Note, my training-data consists of sets with 16 features in 80 time-steps. Here is what I've written for the model below:

class AutoEncoder(torch.nn.Module):
    def __init__(self, input_dim, hidden_dim, layer_dim):
        super(AutoEncoder, self).__init__()
        self.hidden_dim = hidden_dim
        self.layer_dim = layer_dim

        self.lstm = nn.LSTM(input_dim, hidden_dim, layer_dim, batch_first=True)
        self.fc1 = torch.nn.Linear(hidden_dim, hidden_dim)
        self.fc2 = torch.nn.Linear(hidden_dim, input_dim)   
    def forward(self, x):
        h0 = torch.zeros(self.layer_dim, x.size(0), self.hidden_dim).requires_grad_()
        c0 = torch.zeros(self.layer_dim, x.size(0), self.hidden_dim).requires_grad_()

        out, (hn, cn) = self.lstm(x, (h0.detach(), c0.detach()))
        out = self.fc1(out[:, -1, :]) 
        out = self.fc2(out)
        return out

input_dim = 16
hidden_dim = 8
layer_dim = 2
model = AutoEncoder(input_dim, hidden_dim, layer_dim)


I don't think I've built the model correctly. How does it know that I'm feeding it 80 time-steps of data? How will the auto-encoder reconstruct those 80 time-steps of data?

I'm having a hard time understanding the material online. What would the final layer have to be?

If you check out the PyTorch LSTM documentation, you will see that the LSTM equations are applied to each timestep in your sequence. nn.LSTM will internally obtain the seq_len dimension and optimize from there, so you do not need to provide the number of time steps.

At the moment, the line

out = self.fc1(out[:, -1, :])


is selecting the final hidden state (corresponding to time step 80) and then this is being projected onto a space of size input_dim.

To output a sequence of length 80, you should have an output for each hidden state. All hidden states are stacked in out so you can simply use

out = self.fc1(out)
out = self.fc2(out)


I would also note that if you must have two fully connected layers after encoding in your hidden state, you should use a non-linearity in between otherwise this is the equivalent of just one layer but with more parameters.
