Calculation of GPU memory consumption on softmax layer doesn't match with the empirical result
GPU DRAM capacity - 7.5G
the below link explains how nVIDIA GPU cUDNN does memory optimization.
https://devblogs.nvidia.com/optimizing-recurrent-neural-networks-cudnn-5/
the below link has detailed steps to calculate memory required by parameters and data.
http://cs231n.github.io/convolutional-networks/#case
one data point missing in question is number of output classes in softmax layer.
The above two links will help you to calculate memory required and how software handles large matrix multiplication.