title: gpu slower than cpu for neural networks in matlab
body: <p>I have the following code:</p>
  
  <pre><code>tic;
  H = rand(100, 1000);
      F = rand(1, 1000);
      net = newff(H, F, [30, 10], { 'tansig' 'tansig'}, 'traingdx', 'learngdm', 'mse');
  
      net.trainParam.epochs = 400;
      net.performParam.regularization = 0.05;
  
      net.divideParam.trainRatio = 1;
      net.divideParam.valRatio = 0;
      net.divideParam.testRatio = 0;
  
      net.trainParam.showWindow = 0;
      net.trainParam.showCommandLine = 0;
  
      % net = train(net, H, F, 'useGPU', 'yes', 'showResources', 'yes'); % line 1
      net = train(net, H, F, 'showResources', 'yes'); % line 2
  
  toc;
  </code></pre>
  
  <p>with line 2 uncommented I get </p>
  
  <pre><code>Computing Resources:
  GPU device #1, GeForce 800M
  
  Elapsed time is 5.084222 seconds.
  </code></pre>
  
  <p>and with line 1 uncommented I get </p>
  
  <pre><code>Computing Resources:
  MEX2
  
  Elapsed time is 1.870803 seconds.
  </code></pre>
  
  <p>Why is GPU slower than CPU?
  My GPU properties:</p>
  
  <pre><code>CUDADevice with properties:
  
                    Name: 'GeForce 800M'
                   Index: 1
       ComputeCapability: '2.1'
          SupportsDouble: 1
           DriverVersion: 6
          ToolkitVersion: 5
      MaxThreadsPerBlock: 1024
        MaxShmemPerBlock: 49152
      MaxThreadBlockSize: [1024 1024 64]
             MaxGridSize: [65535 65535 65535]
               SIMDWidth: 32
             TotalMemory: 2.1475e+09
              FreeMemory: 1.9886e+09
     MultiprocessorCount: 1
            ClockRateKHz: 1475000
             ComputeMode: 'Default'
    GPUOverlapsTransfers: 1
  KernelExecutionTimeout: 1
        CanMapHostMemory: 1
         DeviceSupported: 1
          DeviceSelected: 1`
  </code></pre>