Network Size in neural network
The number of hidden units is a hyperparameter which increases the model capacity. Therefore, increasing it not only increases time and memory requirements but can also lead to overfitting (also see section 11.4.1 in the Deep Learning Book).