How to process data in a data stream for a LSTM
One thing I might add is not to expect the ML model to take care of everything, that is where ML engineering comes into play. One suggestion I have, without knowing much about your data stream, is that you need to implement a real-time data ingestion/pipeline, like Apache Kafka Stream (so similar implementation), where you can decompose your stream into output that your model was trained for. You could split by dictionary, pattern or whatever you need, then push the split output into another real-time stream that can be ingested by your model. Not a lot to go for, but hopefully it helps a little.