Predicting high frequency sparse time series data in python
Here is what you could try doing:

Get a feel of the shape of your data.
Split your time-series by day. Compare the time-series across multiple days.
Plot time in X-axis and the target variable (for forecast) on the Y-axis. Do the shapes look similar or different ? Can you eyeball N-Distinct shapes that the curve takes ?

Find out the &quot;daily mean&quot; of the target variable to forecast. Plot the mean over the course of the year. Do you see a trend or seasonality ? Are you able to get SARIMA or other related techniques to model the changing mean. If so that by itself may be an achievement. Repeat the same exercise for the &quot;daily variance&quot;. Forecasting the daily mean / daily variance &quot;may&quot; be a simpler problem to solve - than forecasting the entire time-series.

Normalize your daily data. Subtract the daily series by the &quot;daily mean&quot; and divide by the &quot;daily standard deviation&quot;. Run a clustering algorithm (perhaps K-Means) on your daily timeseries. Use the elbow to identify the best number of clusters.

Plot the centroids of your clusters. If you are lucky - you may be able to see distinct shapes that your forecast curve takes.

Use the Cluster Number to label each daily time series. Then use a classification model to predict the correct cluster. The features for the classification model could be &quot;day of week&quot;, &quot;is_holiday&quot;, &quot;expected_average_temperature_for_the_day&quot; etc etc.

Check if your classification model does a reasonably good job. If it does you are probably in luck. Your classification model assigns probabilities for each cluster. Combine the cluster centroids weighted by the predicted probabilities - to arrive at a predicted curve.

The predicted curve from the previous step is probably normalized - since the data-prep step (Step 3) normalized the data. You now have the task to rescale back to original data. From Step 2 - if you were able to construct a model that does a reasonable job at predicting the &quot;mean&quot; and &quot;variance&quot; for the target day - Then you could do something as simple as:
Final_Curve = Normalized_Curve * sqrt(Forecasted_Variance) + Forecasted_Mean


Let me know if this worked for you.