title: error: (-4:Insufficient memory) Failed to allocate 36578304 bytes in function 'cv::OutOfMemoryError'
body: <h2>Intro</h2>
  <p>I want to train a convolutional neural net to predict how many upvotes a picture posted to r/art will get on Reddit as a personal project.</p>
  <p>I've downloaded 30000 pictures -&gt; 18648 valid pictures and I named them using their score and post id so that posts would the same score wouldn't conflict (0_k0un69.jpg for example, only mildly important for the code snippet)</p>
  <h2>Problem</h2>
  <p>I'm trying to create X, the input pictures array and Y, an array of the scores for neural net training. About 5.8% of the way through, I get the title error:</p>
  <pre><code>error: OpenCV(4.4.0) C:\Users\appveyor\AppData\Local\Temp\1\pip-req-build-6sxsq0tp\opencv\modules\core\src\alloc.cpp:73: error: (-4:Insufficient memory) Failed to allocate 36578304 bytes in function 'cv::OutOfMemoryError'
  </code></pre>
  <p>The code that creates this error is the following:</p>
  <pre><code>path = &quot;pics/&quot;
  #Get all file names
  files = [f for f in os.listdir(path) if os.path.isfile(os.path.join(path, f))]
  numpics = len(files)
  #Store all image arrays and image names in a list
  X = []
  Y = []
  counter = 0
  for file in files:
      #Progress bar
      sys.stdout.write('\r')
      sys.stdout.write(&quot;Reading images into DataSets:  {:.1f}%&quot;.format((100/(numpics-1)*counter)))
      sys.stdout.flush()
      counter+=1
  
      X.append(cv2.imread(f'{path}{file}'))
      Y.append(file.split(&quot;.&quot;)[0].split(&quot;_&quot;)[0]
  </code></pre>
  <p>It seems obvious to me that the problem is with how many big pictures I'm trying to process. Nonetheless, I'm wondering if there's anything I can do to avoid turning my pictures into grayscale/downsizing them, or if there's just a memory leak or problem with my code. If it helps, I was looking into using Global Pooling to handle variable image dimensions as <a href="https://medium.com/mindboard/image-classification-with-variable-input-resolution-in-keras-cbfbe576126f" rel="nofollow noreferrer">this medium post</a> and <a href="https://stats.stackexchange.com/a/445201">a stack exchange post suggest</a>. If it turns out that scaling wouldn't change the resolution after global pooling, please let me know because I have no idea.</p>
  <h2>Attempted solutions</h2>
  <p>I heard that going from 32bit -&gt;64 bit would help, but I'm not sure what that looks like for what I'm doing. I'm running this in a Jupyter notebook in Pycharm, and I checked that Pycharm is a 64 bit executable.</p>
  <p>Some extra questions I have for comprehension:</p>
  <ol>
  <li>The error message seems to occur on the Java Virtual Machine behind Pycharm or something like that because appveyor is not a user on my PC. That, or it's opencv related. Would running this without Pycharm maybe help?</li>
  <li>Is this memory issue a RAM issue or a storage issue? I find it strange that the pictures themselves take up only ~22 GB of storage, but 6% of them trip up this error.</li>
  </ol>