How to determine layers structures appropriate for a specific application? I'm playing around with Keras. I went through their example of MNIST data, cats vs dog etc.
The structure of the network they suggest is different in each case. The MNIST stucture:
# Build a simple model
inputs = keras.Input(shape=(28, 28))
x = layers.experimental.preprocessing.Rescaling(1.0 / 255)(inputs)
x = layers.Flatten()(x)
x = layers.Dense(128, activation=&quot;relu&quot;)(x)
x = layers.Dense(128, activation=&quot;relu&quot;)(x)
outputs = layers.Dense(10, activation=&quot;softmax&quot;)(x)
model = keras.Model(inputs, outputs)

So basically

preprocessing
flattening the data
2 intermediate layers
output layers.

The cats vs dogs classifier has some preprocessing as well, but then many more intermediate layers of different types. I've played around with each, adding/remove layers and changing characteristics in each. Seems to influence loss/accuracy, but then many different variations also seems to have pretty similar results.
My question is - for some lambda application, what is a good process to determine the basic architecture of the networks (how many layers, what types)? I understand that Keras has a tuner to set the parameters, however my understanding is that this is for the model parameters. I'm talking about one step back from that - how many layers? Which types? Are there good rules of thumbs for that, good resources of &quot;standard&quot; structure as a function of types of inputs?
From the structure you have outlined it looks like you're going for a dense fully-connected neural network. A general rule of thumb for practitioners on this topic is that increasing the size of a single hidden layer is more likely to yield a more performant network than adding successive layers. This is obviously not true in all cases - but as you are experimenting with architectures it is more useful to start by increasing the size of your one hidden layer before you jump to two of even three. See this for more details.
I would recommend removing one of those hidden layers and increasing the number of nodes on the remaining.
Something else to note: if you are looking at image data you might consider using convolutional layers. These are much more appropriate for problems with data that are spatially related (images etc..). If you do decide to go with this approach the previous advice does not apply as it relates only to simple dense multilayer perceptrons. Instead review this.