title: Numba and guvectorize for CUDA target: Code running slower than expected
body: <p>Notable details</p>
  
  <ul>
  <li>Large datasets (10 million x 5), (200 x 10 million x 5)</li>
  <li>Numpy mostly</li>
  <li>Takes longer after every run</li>
  <li>Using Spyder3</li>
  <li>Windows 10</li>
  </ul>
  
  <p>First thing is attempting to use guvectorize with the following function. I am passing in a bunch of numpy arrays and attempting to use them to multiply across two of the arrays. This works if run with a target other than cuda. However, when switched to cuda it results in an unknown error being:</p>
  
  <blockquote>
    <p>File "C:\ProgramData\Anaconda3\lib\site-packages\numba\cuda\decorators.py", >line 82, in jitwrapper
       debug=debug)</p>
    
    <p>TypeError: <strong>init</strong>() got an unexpected keyword argument 'debug'</p>
  </blockquote>
  
  <p>After following all that I could find from this error, I hit nothing but dead ends. I'm guessing it's a really simple fix that I'm completely missing but oh well. It should also be said that this error only occurs after running it once and having it crash due to memory overload.</p>
  
  <pre><code>os.environ["NUMBA_ENABLE_CUDASIM"] = "1"
  
  os.environ["CUDA_VISIBLE_DEVICES"] = "10DE 1B06 63933842"
  ...
  </code></pre>
  
  <p><em>All of the arrays are numpy</em></p>
  
  <pre><code>@guvectorize(['void(int64, float64[:,:], float64[:,:], float64[:,:,:], 
  int64, int64, float64[:,:,:])'], '(),(m,o),(m,o),(n,m,o),(),() -&gt; (n,m,o)', 
  target='cuda', nopython=True)
  def cVestDiscount (ed, orCV, vals, discount, n, rowCount, cv):
      for as_of_date in range(0,ed):
          for ID in range(0,rowCount):
              for num in range(0,n):
                  cv[as_of_date][ID][num] = orCV[ID][num] * discount[as_of_date][ID][num]
  </code></pre>
  
  <p>Attempting to run the code with nvprofiler in command line results in the following error:</p>
  
  <blockquote>
    <p>Warning: Unified Memory Profiling is not supported on the current
    configuration because a pair of devices without peer-to-peer support
    is detected on this ?multi-GPU setup. When peer mappings are not
    available, system falls back to using zero-copy memory. It can cause
    kernels, which access unified memory, to run slower. More details can
    be found at:
    <a href="http://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#um-managed-memory" rel="nofollow noreferrer">http://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#um-managed-memory</a></p>
  </blockquote>
  
  <p>I realized that I am using SLI enabled graphics cards (both cards are identical, evga gtx 1080ti, and have the same device id), so I disabled SLI and added the "CUDA_VISIBLE_DEVICES" line to try and limit to other one card, but am left with the same results.</p>
  
  <p>I can still run the code with nvprof, but the cuda function is slow compared to njit(parallel=True) and prange. By using a smaller data size we can run the code, but it is slower than target='parallel' and target='cpu'.</p>
  
  <p>Why is cuda so much slower, and what do these errors mean?</p>
  
  <p>Thanks for the help!</p>
  
  <p>EDIT:
  Here is a working example of the code:</p>
  
  <pre><code>import numpy as np
  from numba import guvectorize
  import time
  from timeit import default_timer as timer
  
  
  @guvectorize(['void(int64, float64[:,:], float64[:,:,:], int64, int64, float64[:,:,:])'], '(),(m,o),(n,m,o),(),() -&gt; (n,m,o)', target='cuda', nopython=True)
  def cVestDiscount (countRow, multBy, discount, n, countCol, cv):
      for as_of_date in range(0,countRow):
          for ID in range(0,countCol):
              for num in range(0,n):
                  cv[as_of_date][ID][num] = multBy[ID][num] * discount[as_of_date][ID][num]
  
  countRow = np.int64(100)
  multBy = np.float64(np.arange(20000).reshape(4000,5))
  discount = np.float64(np.arange(2000000).reshape(100,4000,5))
  n = np.int64(5)
  countCol = np.int64(4000)
  cv = np.zeros(shape=(100,4000,5), dtype=np.float64)
  func_start = timer()
  cv = cVestDiscount(countRow, multBy, discount, n, countCol, cv)
  timing=timer()-func_start
  print("Function: discount factor cumVest duration (seconds):" + str(timing))
  </code></pre>
  
  <p>I am able to run the code in cuda using a gtx 1080ti, however, it is much slower than running it in parallel or cpu. I've looked at other posts pertaining to guvectorize, yet none of them have helped me understand what is and isn't optimal to run in guvectorize. Is there any way to make this code 'cuda friendly', or is only doing multiplication across arrays too simple for any benefit to be seen?</p>