GPU load (far) bigger than memory peak
The GTX 970 and GTX 980 seem to have massive VRAM leaks in Blender 2.77.
The problem is greatly reduced in the 2.78 Test Build.

https://wiki.blender.org/index.php/Dev:Ref/Release_Notes/2.78