title: CUDA Block parallelism
body: <p>I am writing some code in CUDA and am a little confused about the what is actually run parallel.</p>
  
  <ol>
  <li><p>Say I am calling a kernel function like this: <code>kenel_foo&lt;&lt;&lt;A, B&gt;&gt;&gt;</code>. Now as per my device query below, I can have a maximum of 512 threads per block. So am I guaranteed that I will have 512 computations per block every time I run <code>kernel_foo&lt;&lt;&lt;A, 512&gt;&gt;&gt;</code>? But it says <a href="http://www.cc.gatech.edu/~vetter/keeneland/tutorial-2012-02-20/06-cuda-overview.pdf" rel="nofollow noreferrer">here</a> that one thread runs on one CUDA core, so that means I can have 96 threads running concurrently at a time? (See device_query below).</p></li>
  <li><p>I wanted to know about the blocks. Every time I call <code>kernel_foo&lt;&lt;&lt;A, 512&gt;&gt;&gt;</code>, how many computations are done in parallel and how? I mean is it done one block after the other or are blocks parallelized too? If yes, then how many blocks can run 512 threads each in parallel? It says <a href="http://www.cc.gatech.edu/~vetter/keeneland/tutorial-2012-02-20/06-cuda-overview.pdf" rel="nofollow noreferrer">here</a> that one block is run on one CUDA SM, so is it true that 12 blocks can run concurrently? If yes, the each block can have a maximum of how many threads, 8, 96 or 512 running concurrently when all the 12 blocks are also running concurrently?  (See device_query below).</p></li>
  <li><p>Another question is that if <code>A</code> had a value ~50, is it better to launch the kernel as <code>kernel_foo&lt;&lt;&lt;A, 512&gt;&gt;&gt;</code> or <code>kernel_foo&lt;&lt;&lt;512, A&gt;&gt;&gt;</code>? Assuming there is no thread syncronization required.</p></li>
  </ol>
  
  <p>Sorry, these might be basic questions, but it's kind of complicated... Possible duplicates:<br>
  <a href="https://stackoverflow.com/q/3519598">Streaming multiprocessors, Blocks and Threads (CUDA)</a><br>
  <a href="https://stackoverflow.com/q/10460742">How do CUDA blocks/warps/threads map onto CUDA cores?</a>  </p>
  
  <p>Thanks</p>
  
  <p>Here's my <code>device_query</code>:</p>
  
  <pre><code>Device 0: "Quadro FX 4600"
  CUDA Driver Version / Runtime Version          4.2 / 4.2
  CUDA Capability Major/Minor version number:    1.0
  Total amount of global memory:                 768 MBytes (804978688 bytes)
  (12) Multiprocessors x (  8) CUDA Cores/MP:    96 CUDA Cores
  GPU Clock rate:                                1200 MHz (1.20 GHz)
  Memory Clock rate:                             700 Mhz
  Memory Bus Width:                              384-bit
  Max Texture Dimension Size (x,y,z)             1D=(8192), 2D=(65536,32768), 3D=(2048,2048,2048)
  Max Layered Texture Size (dim) x layers        1D=(8192) x 512, 2D=(8192,8192) x 512
  Total amount of constant memory:               65536 bytes
  Total amount of shared memory per block:       16384 bytes
  Total number of registers available per block: 8192
  Warp size:                                     32
  Maximum number of threads per multiprocessor:  768
  Maximum number of threads per block:           512
  Maximum sizes of each dimension of a block:    512 x 512 x 64
  Maximum sizes of each dimension of a grid:     65535 x 65535 x 1
  Maximum memory pitch:                          2147483647 bytes
  Texture alignment:                             256 bytes
  Concurrent copy and execution:                 No with 0 copy engine(s)
  Run time limit on kernels:                     Yes
  Integrated GPU sharing Host Memory:            No
  Support host page-locked memory mapping:       No
  Concurrent kernel execution:                   No
  Alignment requirement for Surfaces:            Yes
  Device has ECC support enabled:                No
  Device is using TCC driver mode:               No
  Device supports Unified Addressing (UVA):      No
  Device PCI Bus ID / PCI location ID:           2 / 0
  </code></pre>