If we are using batch normalization as the first layer, can we forego standard scaling of inputs?
Scaling is a bit different from what Batch normalization does. Performing scaling creates scale indifference amongst all the data points. Ex: Values 5 and 55 will have a higher magnitude of scale difference than the log(5)=0.698 and log(55)=1.740. This is the idea behind scaling. Similarly, We scale the images with 255 which helps in faster convergence

When we pass in the scaled data to our network, due to the operations that take place in every layer of the network, the distribution of data observed in let's say 5th layer of the network is different from the distribution in the 1st layer. This is a problem because we are kind of training the first part of the network in one form of data and the latter part of the network with another form of data. So to remove this effect, We do BatchNorm in later layers of the network