| here is a quote from Nvidia Ampere GA102 GPU Architecture: | |
| Third-Generation NVLink® | |
| GA102 GPUs utilize NVIDIA’s third-generation NVLink interface, which includes four x4 links, | |
| with each link providing 14.0625 GB/sec bandwidth in each direction between two GPUs. |