How should one standardize input when transfer learning
You should use the same transformation (i.e same mean and std in case of z-score normalization) on the new data while using any pretrained model.
If the pretrained model was trained without any normalization then it does not matter if you use normalization or not as long as range of input data is fixed (0-255 for the 8-bit input images). You can use pretrained models with batch normalization which takes care of internal covariance shift and thus there is no need to normalize data.