How to estimate the not available observation in time series data?
One of the most common ways for this purpose is to generate artificial data for that intervals. One of the most novel and powerful algorithms for this purpose is Generative Query Network (GQN). 

In this algorithm the basic application is to create a 3D model of an object by observing a few 2D samples of it. So, the network by receiving the angle of the camera beside the 2D image, try to construct the 3D model.

But in this case for time series data, time can play the role of camera angle. So, you can generate high precision artificial data in the case of data lost.

Here is a brief description of this algorithm.

And also here is an implementation of it using Pytorch.

For more information you can read it's respective paper.