title: what is the Faster way to compare two list or numpy arrays
body: <p>I am doing some image processing through the traversing the folders.
  There is almost 1 million images and i want to check the file full name before processing whether is processed before or not.</p>
  
  <p>I am keeping the processed file names in a numpy array . I am taking the full file list with below code and checking the final list with my numpy list whether is processed before . </p>
  
  <p>Below code takes too long...</p>
  
  <p>How can I make it faster. ?</p>
  
  <pre class="lang-py prettyprint-override"><code>
  newFiles = []
  duplicate = 0
  old_np = np.load("FILEStest.npy") . ##//almost 200.000 
  
  start_time = time.time()
  
  list_of_files = bc.getListOfFiles("/Users/xxx/Downloads/CS_tt_test/")  ##// almost 1 million files. its fast . 60 sec.
  
  elapsed_time = time.time() - start_time
  
  ##//Comparing here whether is already processed. This part very slow.
  
  for old in range(len(list_of_files)):
      file = list_of_files[old]
      if not np.any(old_np[:, ] == file):
          newFiles.append(file)
      else:
          duplicate += 1
          # print("Seen before ", file)
  
  print("duplicates : " , duplicate)
  
  print(elapsed_time)
  
  </code></pre>