title: Pytorch: Add input normalization to model (division layer)
body: <p>I want to add the image normalization to an existing pytorch model, so that I don't have to normalize the input image anymore.</p>
  <p>Say I have an existing model</p>
  <pre><code>model = torch.hub.load('pytorch/vision:v0.6.0', 'mobilenet_v2', pretrained=True)
  model.eval()
  </code></pre>
  <p>Now I can add new layers (for example a relu) using torch.nn.Sequential:</p>
  <pre><code>new_model = nn.Sequential(
      model,
      nn.ReLU()
  )
  </code></pre>
  <p>However I couldn't find a layer to do perform just a division or subtraction as needed for the input normalization here shown in numpy:</p>
  <pre><code>import cv2
  import numpy as np
  img = cv2.imread(&quot;my_img.jpg&quot;)
  img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
  img = img.astype(np.float32)
  mean = np.array([0.485, 0.456, 0.406], dtype=np.float32)
  std = np.array([0.229, 0.224, 0.225], dtype=np.float32)
  img = img / 255.0
  img = img - mean
  img = img / std
  img = np.transpose(img, (2, 0, 1))
  img = np.expand_dims(img, axis=0)
  </code></pre>
  <p>The goal is that normalization is eventually done on GPU to save time during inference. Also I cannot use torchvision transforms as those operation are not stored inside the model itself. For example, if I want to save the model to disk (in order to convert it to tflite using onnx) the torchvision transform operations will not be saved along with the model. Is there an elegant way of doing this?</p>
  <p>(preferably without using a linear layer, which would fix my model input size, which should be flexible as my real model is fully convolutional)</p>