When Blender crashes during rendering
After asking a good number of questions,
I felt it best that I come with some tips that I have discovered and learnt on my own.
One of the things I've heard is that there are people that have tried to render a video
and that Blender (or CUDA, GPU rendering) would crash and their video file would be corrupt.
This usually happens in Cycles.
How to avoid this?
What I'm about to tell you is no miracle worker.
There is still a chance it dies depending on how complicated your scene is
and how much memory your computer or video card has.
To begin with, set all your rendering options according to what you want.
Make a few still image renders here and there to make sure you are satisfied with the quality.
In Output set where you want the result to be saved to and give it a name and a period (.) but without setting an extension (png),
check File Extensions, but uncheck Overwrite, Placeholder and Cache Result.
Set the format to PNG, and Compression to 100%.
Yes, we are writing image sequences here. Even the most professional Blender Gurus
with the most powerful of computers do this. This is to ensure that no matter what happens,
be it that the OS decides it has had enough or power failure/outage,
what you have rendered so far is safe and sound and you can pick it up right where you left off.
Another option you can check in Performance to save on RAM is to check Save Buffers.
Now that all that is setup, Close Blender!
To ensure that there is enough memory to render our video
we do not want Blender's GUI to be using any of it, and if possible, not even the Desktop.
What we are going to do now is write a little script file that is going to render our video.
Linux Bash Script
#!/bin/bash
cd &quot;/path/to/blender/&quot;
for i in {1..250}
do
   foo=$(printf "%04d" $i)
   if [ ! -f &quot;/absolute/path/to/project/imgseq/video.$foo.png" ]
   then
      ./blender -b "/absolute/path/to/project/project.blend" -f $i
   fi
done

Lets go through this step by step.

The first line #!/bin/bash is pretty much the header telling Linux this is a bash script
and should be interpreted as such.
Next we Change Directory to wherever you have Blender installed cd &quot;/path/to/blender/&quot;.
Now we create a for loop in which we iterate over the number of frames your animation has,
in which i holds a number from 1 to 250. Don't forget the do right after that.
The command foo=$(printf &quot;%04d&quot; $i) creates a formatted string where the number i gets 4 leading zeros. Remember that Blender will name the file with 4 leading zeros.
Next we test if a file with your video name that you set in File Output plus the frame number and png does not exists.
The ! in front of the -f is a negation so one would read it as 'if not a-file &quot;video.number.png&quot;, then...'
This will ensure that Blender will not be launched to render this frame if the frame already exists.
So if this frame has not been rendered, launch blender (./blender) in the background (-b) and render the blend file
(&quot;/absolute/path/to/project/project.blend&quot;) on frame i only (-f $i).
Finally, end the if block with fi and our for loop with done.

Save this file somewhere, maybe with your blend file, and call it something simple like render.sh.
Make sure it is executable by doing sudo chmod 755 render.sh.
When you are ready to leave your computer running for hours on end, open a terminal or drop to a tty
(Ctrl+Alt+F1 - F6),
navigate to this render.sh and run it (./render.sh).
How to stop it?
If you want to interrupt your rendering at that moment, hold Ctrl+C
until Blender isn't trying to restart itself anymore.
If you want Blender to finish off its current frame, open a new terminal or jump to a different tty,
type in ps -A |grep render.sh or whatever name you gave your script, make note of its number
and type in kill number, and wait until Blender is finished.
Windows Batch Script
echo off
C:
cd &quot;C:\path\to\blender&quot;
for /L %%i in (1,1,250) do (
  set 'foo=0000%%i'
  set 'foo=!foo:~-4!'
  if not exist 'C:\absolute\path\to\project\imgseq\video.!foo!.png' (
    blender.exe -b &quot;C:\absolute\path\to\project\project.blend&quot; -f %%i
  )
)

Pretty much the same principle as the shell script, but lets go through it anyways.

The first part is optional. This will ensure that the whole script is not printed on screen each time a frame is to be rendered.
After that, C: ensures that we are in the drive that Blender was installed to.
Next we change directory to wherever you have Blender installed with cd &quot;C:\path\to\blender&quot;.
Now we create a for loop in which we iterate over the number of frames your animation has,
where /L tells for that we are looping through numbers, %%i shall hold these numbers,
within the range of (start, step, end). Don't forget do ( at the end of this.
We then set our zero padding to %%i using the two lines set 'foo=0000%%i' and set 'foo=!foo:~-4!'.
The first adds the zeros and the second makes sure there are only 4 digits in total. Remember that Blender will name the files with 4 leading zeros.
Next we test if a file with your video name that you set in File Output plus the frame number and png does not exists.
Notice the ! surrounding the foo which is our zero padded %%i.
Unlike the shell, batch uses not instead of ! to negate what we are testing for.
This will ensure that Blender will not be launched to render this frame if the frame already exists.
So if this frame has not been rendered, launch blender.exe in the background (-b) and render the blend file
(&quot;C:\absolute\path\to\project\project.blend&quot;) on frame i only (-f %%i).
Finally we end the if and for blocks with a ) each.

Save this file somewhere, maybe with your blend file, and call it something simple like render.bat.
If you want the console window to stay open, press the Window Key, type in cmd and enter,
navigate to where this render.bat file is and simply run it by typing in render.bat.
If you don't mind it closing when it's done, simply double-click render.bat.
How to stop it?
Pressing Ctrl+C, Y and Enter will stop the loop, but Blender will continue on its current frame.
To stop Blender itself one would go into Task Manager, find blender among the Processes and End Process.
To stop it all, simply close the terminal (x).
Mac OS
As far as I can tell, Mac uses the same bash script as Linux, but if I'm wrong do let me know.
Additional Thoughts
Some of you may be noticing that this method will close and re-open Blender for each frame rendered,
and you are probably wondering &quot;Won't this put a strain on the computer?&quot;
Not necessarily. I have noticed that if I were to run Blender in the background and include -a
to render the whole animation in one sitting, it won't clear the memory properly between frames
and one would run into memory problems.
By closing Blender after each frame it will clear all of the memory (if not 99%)
and leave plenty for Blender to take care of the next frame.
The only factor that may interfere with this is how well your OS manages memory.
&quot;What if I just give my project to a Render Farm like Sheep-It?&quot;
That is a possibility if you need to use your computer for other things in the mean time,
but there are a few factors to consider which would make this difficult.

In the case of Sheep-It you can only upload 500mb, so if your project has a lot of textures or cached dynamics
that exceed this limit, even compressed, this will not work.
If your project takes a long time to render you'll run out of points,
and based on that system your project will get less attention, and thereby take longer to complete.
You are part of a studio of some kind and doing this may violate any agreements you have with your boss.
You yourself want to keep this project a big secret and you do not want the slightest bit of exposure until it is done.
Your project contains questionable material that may violate the Render Farm's terms.
You want to render your project in a special or non-linear way.

Render Farms like Sheep-It pretty much does the same thing as the scripts above.
Get the project, check what frame hasn't been rendered, then tell Blender to render that one frame in the background, then close.
Although because there is a little shuffling involved, projects will be downloaded and deleted several times
as well as Blender being re-opened and closed.
&quot;My Blender project contains audio. Can I render that in the background?&quot;
As far as I can tell, you cannot. That would've been a handy command to add at the end.
Sadly, you'll have to open Blender and click the Audio button in the Render panel and wait.
So, now that your rendering is complete and you have a long list of PNG files,
you'd like to turn them into a complete video. There are a number of ways to do this.
The first and foremost way would be to put them into your video editing software
so that you can apply any extra visual effects or sounds that are needed.
If you want an mp4 as soon as it is all complete, you could add the following to the end of your script:
ffmpeg -r 24 -start_number 1 -i &quot;/absolute/path/to/project/imgseq/video.%%04d.png&quot; -f mp4 -crf 15 &quot;/absolute/path/to/project/imgseq/video.mp4&quot;

This will of course work if ffmpeg is installed and recognized as an internal command.
If not you'll have to Change Directory to where ffmpeg.exe is hiding just before this command.
The first part of this command -r 24 sets the frame rate. Be sure it is the same one you set in Blender.
The next part -start_number 1 tells ffmpeg that the image sequence starts at 1. If your video actually starts at frame 0, you can skip this part.
The -crf you can change to 0 if you want 100% lossless compression, but that would be a huge video.
For any other video format, feel free to look them up on your own.
Hope this helps you in rendering your massive projects.
If the CUDA still gives up half-way through a single frame then the only thing I can think of is that
there isn't enough VRAM or there is a Shader that your GFX card does not like.
You can experiment and see if CPU rendering works any better, even if it is slower.
/Edward
(blender 2.79)
EDIT It has come to my attention that sometimes the console in Linux (tty) can be a bit slow if it halts execution as it refreshes the screen for a new line. This can be a problem if your project has a lot of objects and textures, and printing out each line takes far too long. One way to speed things up is to add  | grep Remaining to the end of the ./blender line or after typing ./render.sh.