DeepLearning: does it make sense to have more nodes in the initial layer than inputs--for tabular data I was looking at a couple of keras tutorials and something struck me as odd. So in these two tutorials, both on housing data, the number of nodes in the first layer of the network were larger than the number of columns in the dataset itself. NOTE: I am talking about tabular data and not image data or autoencoders. I understand that autoencoders will widen during the decoding phase, and I know that segmentation models will widen during the deconvolution phase. Both of these examples seem to be different than what I am seeing in these tutorials. 

The first tutorial used data on Kings County, CA housing prices to create a simple neural network. In this case, the Kings County data only has 19 variables but the network itself has an initial dense layer of size 100 nodes. Here is the code:

def basic_model_1(x_size, y_size):
    t_model = Sequential()
    t_model.add(Dense(100, activation="tanh", input_shape=(x_size,)))
    t_model.add(Dense(50, activation="relu"))
    t_model.add(Dense(y_size))
    print(t_model.summary())
    t_model.compile(loss='mean_squared_error',
        optimizer=Adam(),
        metrics=[metrics.mae])
    return(t_model)


In the second example I was looking at Aurelion Geron's recent book on Keras and Tensorflow. In chapter 10 he uses some different California housing data to estimate a keras neural network. Again in that case he has 8 variables in the dataset, but he starts with a Dense layer of 30. 

As a caveat, I did not see any columns get converted over to one-hot encodings or some sparse formulation. 

This situation where the network starts wider than the data seemed odd to me, but perhaps that is just me coming from the statistics world. Can anyone explain this? Thanks.
This might seem a little bit confusing, especially for newcomers. But the problem here is basically agreeing on some terminology. 

Typical Neural Networks by definition have 3 types of layers, namely Input, Hidden and Output. Because the Input layer has no specific function other that passing the input values to the next layer, many times is not mentioned. This layer does not have any weights, biases or activations and in the vast majority of frameworks (such as Keras) you don't have to write it in you code. You only need to specify the size of that input layer with the argument input_shape=(x_size,) in the first hidden layer.

What you call in your example initial layer of 100 nodes is actually that first hidden layer. Which can be on any size. And your intuition is completely right: the number of nodes in the input layer of course has to match the size of the inputs. 

Hope this clarifies.