getting the most out of your models advanced deep-learning best practices trying out architectures blindly works well enough if you just need something that works okay in this sectionwe'll go beyond "works okayto "works great and wins machine-learning competitionsby offering you quick guide to set of must-know techniques for building state-of-the-art deep-learning models advanced architecture patterns we covered one important design pattern in detail in the previous sectionresidual connections there are two more design patterns you should know aboutnormalization and depthwise separable convolution these patterns are especially relevant when you're building high-performing deep convnetsbut they're commonly found in many other types of architectures as well batch normalization normalization is broad category of methods that seek to make different samples seen by machine-learning model more similar to each otherwhich helps the model learn and generalize well to new data the most common form of data normalization is one you've seen several times in this book alreadycentering the data on by subtracting the mean from the dataand giving the data unit standard deviation by dividing the data by its standard deviation in effectthis makes the assumption that the data follows normal (or gaussiandistribution and makes sure this distribution is centered and scaled to unit variancenormalized_data (data np mean(dataaxis)np std(dataaxisprevious examples normalized data before feeding it into models but data normalization should be concern after every transformation operated by the networkeven if the data entering dense or conv network has mean and unit variancethere' no reason to expect priori that this will be the case for the data coming out batch normalization is type of layer (batchnormalization in kerasintroduced in by ioffe and szegedy; it can adaptively normalize data even as the mean and variance change over time during training it works by internally maintaining an exponential moving average of the batch-wise mean and variance of the data seen during training the main effect of batch normalization is that it helps with gradient propagation--much like residual connections--and thus allows for deeper networks some very deep networks can only be trained if they include multiple batchnormalization layers for instancebatchnormalization is used liberally in many of the advanced convnet architectures that come packaged with kerassuch as resnet inception and xception sergey ioffe and christian szegedy"batch normalizationaccelerating deep network training by reducing internal covariate shift,proceedings of the nd international conference on machine learning ( )licensed to