Pytorch only sees 15GB memory when the device should have more
I just kicked off a g4dn.8xlarge instance on AWS. According to their website, this instance should have 128GB memory, as shown in the screenshot below:

However, I notice my model keeps running out of memory. When I investigated, I see this:
&gt;&gt;&gt; print(torch.cuda.get_device_properties('cuda:0'))
_CudaDeviceProperties(name='Tesla T4', major=7, minor=5, total_memory=15109MB, multi_processor_count=40)

It turns out the device can only use about 15GB of memory! Could someone please explain why this might be? Do I need to use a different AWS instance to get more memory, or is PyTorch just not able to use the rest of the memory that's available?

The g4dn.8xlarge use NVIDIA T4 GPU which have 16GB of RAM. So I think PyTorch shows the memory of the GPU, not the instance.

