getting the most out of your models the batchnormalization layer is typically used after convolutional or densely connected layerconv_model add(layers conv ( activation='relu')conv_model add(layers batchnormalization()dense_model add(layers dense( activation='relu')dense_model add(layers batchnormalization()after conv layer after dense layer the batchnormalization layer takes an axis argumentwhich specifies the feature axis that should be normalized this argument defaults to - the last axis in the input tensor this is the correct value when using dense layersconv layersrnn layersand conv layers with data_format set to "channels_lastbut in the niche use case of conv layers with data_format set to "channels_first"the features axis is axis the axis argument in batchnormalization should accordingly be set to batch renormalizat ion recent improvement over regular batch normalization is batch renormalizationintroduced by ioffe in it offers clears benefits over batch normalizationat no apparent cost at the time of writingit' too early to tell whether it will supplant batch normalization--but think it' likely even more recentlyklambauer et al introduced self-normalizing neural networks, which manage to keep data normalized after going through any dense layer by using specific activation function (seluand specific initializer (lecun_normalthis schemealthough highly interestingis limited to densely connected networks for nowand its usefulness hasn' yet been broadly replicated sergey ioffebatch renormalizationtowards reducing minibatch dependence in batchnormalized models( )https:arxiv orgabs gunter klambauer et al self-normalizing neural networks,conference on neural information processing systems ( )https:arxiv orgabs depthwise separable convolution what if told you that there' layer you can use as drop-in replacement for conv that will make your model lighter (fewer trainable weight parametersand faster (fewer floating-point operationsand cause it to perform few percentage points better on its taskthat is precisely what the depthwise separable convolution layer does (separableconv dthis layer performs spatial convolution on each channel of its inputindependentlybefore mixing output channels via pointwise convolution ( convolution)as shown in figure this is equivalent to separating the learning of spatial features and the learning of channel-wise featureswhich makes lot of sense if you assume that spatial locations in the input are highly correlatedbut different channels are fairly independent it requires significantly fewer parameters and involves fewer computationsthus resulting in smallerspeedier models and because it' more representationally efficient way to perform convolutionit tends to learn better representations using less dataresulting in better-performing models licensed to