Is a Neural Network with 20 times the number of input neurons (on hidden layers) guaranteed to overfit? When is this not so? I'm aware of the problem of over-fitting.  One way to describe it is your Neural Network learning your training data to a high accuracy and performing poorly (generalizing) on new data.

Was wondering if there are situations where having 20 times the number of input neurons on the first hidden layer wouldn't necessarily produce overfitting always.

Here is a screen shot of my work flow.  As you can see I'm using test data.  The way I've done it is split the data myself using a "Past/Future" column with a P or an F.  The future is the last 5% of the sequential time data.

I don't use randomization since I don't think it makes much sense to unsort the sequential time datapoints.



Thanks.
Figuring out the correct size for hidden layers in NN is a bit of an art, that said I have never used hidden layers with more than twice the number of neurones as the input layer.

As an example the MNIST problem has an input size of 784, I cannot imagine how the network will not overfit if the hidden layer is of size 784*20...

So answering to your question, I think overfit will pretty much occur on those occasions, if unsure, just try it in the MNIST dataser