title: parallel processing multiple evaluations of a sequential task on a large data set -- a task for GPU computing?
body: <p>I am working on some signal processing code in SciPy, and am now trying to use a numerical optimizer to tune it.  Unfortunately, as these things go, it is turning out to be quite a slow process.</p>
  
  <p>The operations I must perform for this optimization are the following:</p>
  
  <ul>
  <li>Load a large 1-d data file (~ 120000 points)</li>
  <li>Run optimizer, which:
  <ul>
  <li>Executes a signal processing operation, does not modify original data, produces 120000 new data points.</li>
  <li>Examines difference between original signal and new signal using various operations,
  <ul>
  <li>One of which includes FFT-based convolution</li>
  <li>Generates a single "error" value to summarise the result -- this is what should be minimized</li>
  </ul></li>
  <li>Looks at error and re-runs operation with different parameters</li>
  </ul></li>
  </ul>
  
  <p>The signal processing and error functions take under 3 seconds, but unfortunately doing it 50,000 times takes much longer.  I am experimenting with various more efficient optimisation algorithms, but no matter what it's going to take thousands of iterations.</p>
  
  <p>I have parallelised a couple of the optimisers I'm trying using CPU threads, which wasn't too difficult since the optimiser can easily perform several scheduled runs at once on separate threads using <code>ThreadPool.map</code>.</p>
  
  <p>But this is only about a 2x speed-up on my laptop, or maybe 8x on a multicore computer.  My question is, is this an application for which I could make use of GPU processing?  I have already translated some parts of the code to C, and I could imagine using OpenCL to create a function from an array of parameters to an array of error values, and running this hundreds of times at once. -- Even if it performs the sequential processing part slowly, getting all the results in one shot would be amazing.</p>
  
  <p>However, my guess is that the memory requirements (loading up a large file and producing a temporary one of equal size to generate every data point) would make it difficult to run the whole algorithm in an OpenCL kernel.  I don't have much experience with GPU processing and writing CUDA/OpenCL code, so I don't want to set about learning the ins and outs if there is no hope in making it work.</p>
  
  <p>Any advice?</p>