How to obtain the optimum model using Keras? Deep Learning
A lot of this comes down to just trying different configurations, it can almost be seen as more as an art than a science. However, there are certain rules of thumb that can be applied based on existing research and models. Examples of this are that the Adam optimizer works well enough for most cases (point 6) and convolutional layers generally work better than others (e.g. dense layers) when working with images (point 2). It is therefore always good to see if you can find existing papers that work on problems that are similar to yours and see which approaches are used and what (doesn't) work(s) well.