What's the advantage of multi-gpu training in real?
I see two main advantages of using multi-GPU instead of one as they distribute certain resources:


using large DNN models - some recent models occupy vast space in memory so they simply cannot fit regular GPU and using multiple GPU allow to distribute some parts of the model to different GPU instances.
speed-up DNN training is also a very positive effect of using multiple GPU but only if you have a high-speed connection among GPUs as NVIDIA came with their NVLink