How to determine the amount of layers and unit count in price prediction with Machine Learning and LSTM?
A good method is to start with a small dataset, with a few features.
Create a very simple neural model and try to train it on this set.
If your net predicts the test-set correctly, it complicates the dataset and re-train.
But , if net does not work, it increases its size, by adding nodes or layer.
When dataset is full of all features, you have a best network for this task
I am trying to come up with an experimental solution for a problem related to prediction (similar to stock price prediction).
I've been reading a lot of documentation about using Machine Learning and about stock prices prediction using LSTM (keras and tensorflow). I understand the general theory of machine learning in itself and how a LSTM layer works behind the scenes.
My main concern is that every tutorial about stock price prediction I find, it comes up with a different amount of hidden LSTM layers and each one of them has a pre-defined number of units without much explanation. In the case of the classic "Apple (AAPL) stock price prediction" scenario, I see tutorials with 3 layers and 50 units, others with only 2 layers but 128 units.
In the case I have to code something by myself, how can I determine the amount of LSTM and the amount of units for the prediction to function properly? My main guess at the moment is trying to figure it out with trials and errors. I was wondering if there is some sort of relation between the problem, the data and the solution.
Also if someone can explain the effects of having more/less LSTM layers and more/less units other than doing more calculations, it would be awesome!