ful ones--often take long time to train they also require careful preprocessing of the dataas we saw here similarly to svmsthey work best with "homogeneousdatawhere all the features have similar meanings for data that has very different kinds of featurestree-based models might work better tuning neural network parameters is also an art unto itself in our experimentswe barely scratched the surface of possible ways to adjust neural network models and how to train them estimating complexity in neural networks the most important parameters are the number of layers and the number of hidden units per layer you should start with one or two hidden layersand possibly expand from there the number of nodes per hidden layer is often similar to the number of input featuresbut rarely higher than in the low to mid-thousands helpful measure when thinking about the model complexity of neural network is the number of weights or coefficients that are learned if you have binary classification dataset with featuresand you have hidden unitsthen there are , weights between the input and the first hidden layer there are also weights between the hidden layer and the output layerfor total of around , weights if you add second hidden layer with hidden unitsthere will be another , weights from the first hidden layer to the second hidden layerresulting in total of , weights if instead you use one layer with , hidden unitsyou are learning , , weights from the input to the hidden layer and , weights from the hidden layer to the output layerfor total of , if you add second hidden layer you add , , , , weightsfor whopping total of , , -- times larger than the model with two hidden layers of size common way to adjust parameters in neural network is to first create network that is large enough to overfitmaking sure that the task can actually be learned by the network thenonce you know the training data can be learnedeither shrink the network or increase alpha to add regularizationwhich will improve generalization performance in our experimentswe focused mostly on the definition of the modelthe number of layers and nodes per layerthe regularizationand the nonlinearity these define the model we want to learn there is also the question of how to learn the modelor the algorithm that is used for learning the parameterswhich is set using the algorithm parameter there are two easy-to-use choices for algorithm the default is 'adam'which works well in most situations but is quite sensitive to the scaling of the data (so it is important to always scale your data to mean and unit variancethe other one is ' -bfgs'which is quite robust but might take long time on larger models or larger datasets there is also the more advanced 'sgdoptionwhich is what many deep learning researchers use the 'sgdoption comes with many additional param supervised learning