Network Size in neural network What are the limitations of having too many hidden units in the Neural Network
Does it take more memory or takes longer time to train the model
The number of hidden units is a hyperparameter which increases the model capacity. Therefore, increasing it not only increases time and memory requirements but can also lead to overfitting (also see section 11.4.1 in the Deep Learning Book).