How to obtain the optimum model using Keras? Deep Learning There are too many parameters while building an artificial neural network. Some of which that comes to my mind are:

Number of layers
Types of layers
Number of nodes in each level of layer
Activation functions in each layer
Ordering of the layers
Different types of optimizers
Different types of loss functions
Batch size while fitting
Epochs while fitting
A lot of other things I never heard of (I suppose)

The lone fact that the number of layers might be in the order of tens and the number of nodes might be in the order of thousands implies that we can end up with billions of different combinations. Considering all the parameters in the list above, how can one know that their model is close to the optimum ?
The cases I study on does not require me to be too precise in terms of r2_score. In other words, it would not make a huge difference if the r2_score is 0.85 or 0.87. But the question is how do I know if I am real close to the saturation point of the r2_score for the given data, i.e. the optimum model would yield an r2_score of 0.90 and I am not stuck at 0.80?
A lot of this comes down to just trying different configurations, it can almost be seen as more as an art than a science. However, there are certain rules of thumb that can be applied based on existing research and models. Examples of this are that the Adam optimizer works well enough for most cases (point 6) and convolutional layers generally work better than others (e.g. dense layers) when working with images (point 2). It is therefore always good to see if you can find existing papers that work on problems that are similar to yours and see which approaches are used and what (doesn't) work(s) well.