Model-based RL for time series data
To my knowledge, there does not exist anything along the lines of model-based reinforcement learning with time-sensitive data. I think the best chance you have is to try to abstract the data that you have into a model which is not time-sensitive.
What would happen when you get past the timestamps of your original data? I am guessing that when testing this model, you will have states with time stamps past your original latest timestamp. Or are you only using your model in the time stamps of your training data? Is it going to be a useful model then?
Firstly, please note that the topics of this forum are mostly theoretical. This question leans on the boundary. If you frame your question a bit differently, ending your post with a clear question and also putting that question in the title, you will get more responses.
I feel like the question is 'What are the possibilities for model-based reinforcement learning for time-sensitive data?' But this question is inherently vague as it implies that you have a model according to the definitions of models in RL, which are not time sensitive. If you would like a better answer then please provide more information or ask your question on a more practical forum if it is more practical than what I guessed.
Disclaimer: would have just put some comments under the post as this is not a real sufficient answer, but my reputation is not high enough for that :P