table fundamentals of machine learning choosing he right last -layer activation and loss function for your model problem ype last -layer activat ion loss function binary classification sigmoid binary_crossentropy multiclasssingle-label classification softmax categorical_crossentropy multiclassmultilabel classification sigmoid binary_crossentropy regression to arbitrary values none mse regression to values between and sigmoid mse or binary_crossentropy scaling updeveloping model that overfits once you've obtained model that has statistical powerthe question becomesis your model sufficiently powerfuldoes it have enough layers and parameters to properly model the problem at handfor instancea network with single hidden layer with two units would have statistical power on mnist but wouldn' be sufficient to solve the problem well remember that the universal tension in machine learning is between optimization and generalizationthe ideal model is one that stands right at the border between underfitting and overfittingbetween undercapacity and overcapacity to figure out where this border liesfirst you must cross it to figure out how big model you'll needyou must develop model that overfits this is fairly easy add layers make the layers bigger train for more epochs always monitor the training loss and validation lossas well as the training and validation values for any metrics you care about when you see that the model' performance on the validation data begins to degradeyou've achieved overfitting the next stage is to start regularizing and tuning the modelto get as close as possible to the ideal model that neither underfits nor overfits regularizing your model and tuning your hyperparameters this step will take the most timeyou'll repeatedly modify your modeltrain itevaluate on your validation data (not the test dataat this point)modify it againand repeatuntil the model is as good as it can get these are some things you should tryadd dropout try different architecturesadd or remove layers add and/or regularization licensed to