How to train an LSTM to classify based on rare historic event?
How about a Temporal Convolutional Network? It feels like for such a long sequences having the recurrent/memory based approach is not too feasible. But, intuitively, the 1D convolutions should be able to pick out those rare features from your extremely long sequences.
There are also claims that TCNs are comparable to RNNs in performance on common tasks, so there's that.