title: Faster performance for normalizing a numpy array?
body: <p>I am currently normalizing an numpy array in python created by splicing an image by windows with a stride which creates about 20K patches. The current normalization implementation is a big pain point in my runtime, and I'm trying to replace it with the same functionality done maybe in a C extension or something. I am looking to see what advice the community has to get this done easily and simple? 
  Current runtime is about 0.34s just for the normalization part, I'm trying to get below 0.1s or better. You can see creating patches is extremely efficient with view_as_windows, and I am looking for something similar for normalization. Note you can simply comment/uncomment the lines labeled " # ---- Normalization" to see the runtimes yourself for the different implementations.</p>
  
  <p>Here is the current implementation:</p>
  
  <pre><code>import gc
  import cv2, time
  from libraries import GCN
  from skimage.util.shape import view_as_windows
  
  
  def create_imageArray(patch_list):
      returnImageArray = numpy.zeros(shape=(len(patch_list), 1, 40, 60))
      idx = 0
      for patch, name, coords in patch_list:
          imgArray = numpy.asarray(patch[:,:], dtype=numpy.float32)
          imgArray = imgArray[numpy.newaxis, ...]
          returnImageArray[idx] = imgArray
          idx += 1
      return returnImageArray
      # print "normImgArray[0]:",normImgArray[0]
  
  
  def NormalizeData(imageArray):
      tempImageArray = imageArray
  
      # Normalize the data in batches
      batchSize = 25000
      dataSize = tempImageArray.shape[0]
      imageChannels = tempImageArray.shape[1]
      imageHeight = tempImageArray.shape[2]
      imageWidth = tempImageArray.shape[3]
  
      for i in xrange(0, dataSize, batchSize):
          stop = i + batchSize
          print("Normalizing data [{0} to {1}]...".format(i, stop))
          dataTemp = tempImageArray[i:stop]
          dataTemp = dataTemp.reshape(dataTemp.shape[0], imageChannels * imageHeight * imageWidth)
          #print("Performing GCN [{0} to {1}]...".format(i, stop))
          dataTemp = GCN(dataTemp)
          #print("Reshaping data again [{0} to {1}]...".format(i, stop))
          dataTemp = dataTemp.reshape(dataTemp.shape[0], imageChannels, imageHeight, imageWidth)
          #print("Updating data with new values [{0} to {1}]...".format(i, stop))
          tempImageArray[i:stop] = dataTemp
      del dataTemp
      gc.collect()
  
      return tempImageArray
  
  
  start_time = time.time()
  img1_path = "777628-1032-0048.jpg"
  img_list = ["images/1.jpg", "images/2.jpg", "images/3.jpg", "images/4.jpg", "images/5.jpg"]
  
  patchWidth = 60
  patchHeight = 40
  channels = 1
  stride = patchWidth/6
  multiplier = 1.31
  finalImgArray = []
  vaw_time = 0
  norm_time = 0
  array_time = 0
  
  for im_path in img_list:
      start = time.time()
      baseFileWithExt = os.path.basename(im_path)
      baseFile = os.path.splitext(baseFileWithExt)[0]
      img = cv2.imread(im_path, cv2.IMREAD_GRAYSCALE)
      nxtWidth = 800
      nxtHeight = 1200
      patchesList = []
      for i in xrange(7):
          img = cv2.resize(img, (nxtWidth, nxtHeight))
          nxtWidth = int(nxtWidth//multiplier)
          nxtHeight = int(nxtHeight//multiplier)
          patches = view_as_windows(img, (patchHeight, patchWidth), stride)
          cols = patches.shape[0]
          rows = patches.shape[1]
          patchCount = cols*rows
          print "patchCount:",patchCount, "     patches.shape:",patches.shape
          returnImageArray = numpy.zeros(shape=(patchCount, channels, patchHeight, patchWidth))
          idx = 0
  
          for col in xrange(cols):
              for row in xrange(rows):
                  patch = patches[col][row]
                  imageName = "{0}-patch{1}-{2}.jpg".format(baseFile, i, idx)
                  patchCoodrinates = (0, 1, 2, 3)  # don't need these for example
                  patchesList.append((patch, imageName, patchCoodrinates))
                  # ---- Normalization inside 7 iterations &lt;&gt; Part 1
                  # imgArray = numpy.asarray(patch[:,:], dtype=numpy.float32)
                  # imgArray = patch.astype(numpy.float32)
                  # imgArray = imgArray[numpy.newaxis, ...] # Add a new axis for channel so goes from shape [40,60] to [1,40,60]
                  # returnImageArray[idx] = imgArray
                  idx += 1
  
          # if i == 0: finalImgArray = returnImageArray
          # else: finalImgArray = numpy.concatenate((finalImgArray, returnImageArray), axis=0)
  
      vaw_time += time.time() - start
  
      # ---- Normalizaion inside 7 iterations &lt;&gt; Part 2
      # start = time.time()
      # normImageArray = NormalizeData(finalImgArray)
      # norm_time += time.time() - start
      # print "returnImageArray.shape:", finalImgArray.shape
  
      # ---- Normalization outside 7 iterations
      start = time.time()
      imgArray = create_imageArray(patchesList)
      array_time += time.time() - start
  
      start = time.time()
      normImgArray = NormalizeData( imgArray )
      norm_time += time.time() - start
      print "len(patchesList):",len(patchesList)
  
  
  total_time = (time.time() - start_time)/len(img_list)
  print "\npatches_time per img: {0:.3f} s".format(vaw_time/len(img_list))
  print "create imgArray per img: {0:.3f} s".format(array_time/len(img_list))
  print "normalization_time per img: {0:.3f} s".format(norm_time/len(img_list))
  print "total time per image: {0:.3f} s \n".format(total_time)
  </code></pre>
  
  <p>Here is the GCN code in case you need to download it to use it: <a href="http://pastebin.com/RdVMD2P3" rel="nofollow noreferrer">http://pastebin.com/RdVMD2P3</a></p>
  
  <p><strong>Details on code inside GCN</strong></p>
  
  <p>I am calling GCN using the default params.
  <a href="https://i.stack.imgur.com/8HZQ5.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/8HZQ5.png" alt="GCN Params"></a></p>
  
  <p>At high level it is taking the average of all of the pixels and dividing all the pixels by that average. So if there's an an image array that looks like this [1 2 3], then the average is 2. Therefore we divide each number by 2 and get [0.5, 1, 1.5]. That's what the normalize does. I forgot to highlight in the image above the mean = X.mean(axis=1).
  <a href="https://i.stack.imgur.com/pV7mT.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/pV7mT.png" alt="Normalization function"></a></p>
  
  <p>Notes: 
  If you are wondering why I am re-iterating and creating a new imgArray to normalize instead of doing it in the original patch creation it is to keep data transfer to a minimum. I am implementing this with the multiprocess library, and serializing data takes a LOOONG time, so trying to keep the data serialization to a minimum (meaning pass as little data back from the process). I have measured the difference between doing inside the 7 loops or outside, and notes are below so I can deal with that. However if you know of a faster implementation, do let me know.</p>
  
  <p>Runtimes for creating imageArray inside 7 loops:</p>
  
  <pre><code>patches_time per img: 0.560 s
  normalization_time per img: 0.336 s
  total time per image: 0.896 s 
  </code></pre>
  
  <p>Runtimes for creating imageArray and normalizing outside of 7 iterations:</p>
  
  <pre><code>patches_time per img: 0.040 s
  create imgArray per img: 0.146 s
  normalization_time per img: 0.339 s
  total time per image: 0.524 s 
  </code></pre>
  
  <p>I didn't see this before, but it seems creating the array is also taking some time.</p>