Cycles/ CUDA out of memory (sometimes) like many other posts, I am getting CUDA error: out of memory when I am rendering out many images for an animation.

But- when I close Blender, and restart the rendering process from the stopped image, then the rendering works fine again for a while.  This really confuses me- because if the scene requires larger memory than GPU's memory, shouldn't it have an error message all the time?

I am curious- is there a function in Blender rendering, where the GPU memory starts to save and accumulate for a faster rendering?  In the attached image- I noticed that another memory number does get larger as I render out more and more images in one rendering session.

Can someone explain what this number is? And how I can avoid this out of memory problem when rendering out large number of images (say more than 3000 pictures)?

Thanks!
Memory on the left is your VRAM and on the right it's your RAM.

441Mb of VRAM is not much though, what's your GPU? You probably want to upgrade if you are going to use it to render.

You can also try to optimize your scene, easiest way to do this is to bake as much things as you can, lower texture resolution and your number of polygons, create instances instead of duplicating object data. Also you can probably reduce lights bounces but I'm not certain this would help. 

That or just render with your CPU (it will take longer but will store data in RAM instead of VRAM). How much RAM do you have?

Google 'Cycles Optimization VRAM' for more info on the subject.

Here you'll find a way to launch Blender in debug mode to troubleshoot memory usage: How do I see what is using Memory?