Choosing the right model for predicting demand
It looks to me that the data are in fact a Time Serie. Therefore, you can treat them as such.

Now, there are many approaches with each having each own strengths/weaknesses. I'd start with simple models such as ARMA or SARIMA. Possibly also GARCH or VAR. If you wish to go for something more advanced, Recurrent Neural Networks are also used in Time Series prediction.

Beside that, I am surprised that for example Random Forest have not given sufficient performance. Have you tried thorough feature selection? It may be that you have too high expectations.