Multiple Blender processes for multiple GPUs? I've got a Ubuntu 14.04 headless server running right now for 3d rendering on Blender's Cycles rendering engine, across 4 GPUs. 

Specs: 

(4) ASUS GTX 970s, 4GB
1300W PSU
Core i7
16GB RAM
250GB SSD


Animations are rendering beautifully across the 4 GPUs in Cycles, but my question is concerning parallelizing processes to those cards:

Is it possible for me to have 4 separate blender processes running in parallel targeting one specific GPU each? 

Running all 4 cards as "one" card (CUDA_MULTI_0 in a .py setup) is not actually a full 4x speed increase over running just one card, though it is still quite fast, and amazing i might add having come from mostly CPU rendering. Anyway, it turns out to be more like 3x speed, and i want to maximize render times further by rendering one animation frame per card in parallel, giving me a true 4x speed. 

Theoretically, it should just be a matter of running 4 separate blender processes on the command line, while having python scripts setup to target specific cards like this:

$ blender -b file.blend -P cuda1.py -s 1 -e 10 -a &amp;
$ blender -b file.blend -P cuda2.py -s 11 -e 20 -a &amp;
$ blender -b file.blend -P cuda3.py -s 21 -e 30 -a &amp;
$ blender -b file.blend -P cuda4.py -s 22 -e 40 -a &amp;


and the cuda[x].py scripts look like this, where each one is targeting a specific CUDA device:

import bpy, _cycles

bpy.context.scene.cycles.device = 'GPU'
bpy.context.user_preferences.system.compute_device_type = 'CUDA'

# this is different in each cuda[x].py file, CUDA_0, CUDA_1, CUDA_2, CUDA_3
bpy.context.user_preferences.system.compute_device = 'CUDA_0'


So, this all technically works right now, but here's the big issue i'm trying to figure out. When i do this, only the first blender process gets GPU access. Processes 2-4 default back the CPU, ignoring the GPU instructions. All four cards work just fine, and as I mentioned, one blender process using the CUDA_MULTI_0 device in the python script is leveraging all four cards as expected.

Is there some known limitation to blender, or CUDA/NVIDIA for that matter, that only lets a single processes access any or all the GPUs at one time, that maybe i'm just not aware of?

Any advice on this is greatly appreciated.
It turns out all I needed was to update the NVIDIA driver to v346.72 for the ASUS GTX 970 Strix cards to be able to handle multiple processes in tandem.

Update - Oct 2019

To those wondering if there was a performance gain in doing this: yes, there was. 

I don't have the specifics with me as this was done a few years back, but I do remember the render times breaking down like this simplified example: 

1 GPU renders 1 frame of an animation in 4 minutes, so in theory bulking up to 4 GPUs should render that one frame in 1 minute. This wasn't what I was seeing in reality though. The actual result took more like 1.2 minutes per frame.

So for a 200 frame animation, the math broke down like this:

All 4 GPUs working as one:


(200 frames / 1 Combined GPU) = 200 frames * 1.2 minutes per frame = 240 mins total


Splitting up the GPUs to independently work on a queue of frames:


(200 frames / 4 GPUs) = 50 frames per GPU * 4 minutes per frame in parallel = 200 mins total