At Hot Chips 2025, NVIDIA was fighting back among a sea of 400G NICs with its already shipping NVIDIA ConnectX-8 SuperNIC. This is a PCIe Gen6 NIC capable of 800GbE speeds. There are other capabilities such as having a built-in PCIe Gen6 switch that are often overlooked.
We are doing these live, so please excuse typos.
NVIDIA ConnectX-8 SuperNIC PCIe Gen6 800G NIC Detailed
NVIDIA says that AI training and inference have different characteristics and, therefore, different network needs.
Like the Enfabrica ACF-S, NVIDIA calls its NIC a SuperNIC. Something to keep in mind here is that ConnectX-8 is designed for both Spectrum-X Ethernet as well as Quantum-X Infiniband.
Here is a neat one, folks see the ConnectX-8 card and see a x16 connector, but there are actually 48 lanes of PCIe Gen6 with a PCIe Gen6 switch built-in.
NVIDIA has the ConnectX-8 RDMA scaling at 800G at different message sizes. Broadcom talked about the importance of 64KB for AI and HPC in its Tomahawk Ultra launch.
NVIDIA showed this throughput chart for showing the performance using NCCL AllReduce and AlltAll with different message sizes, but omitted the 64KB and 128KB on these charts.
Now it is saying that the traditional data center had very well defined components.
Now, as the data center becomes the unit of computing, the ASIC needs to do everything to connect the GPU to the rest of the cluster (presumably other than for the scale-up NVLink connections.)
NVIDIA is saying that the GB300 NVL72 is the first deployment of the PCIe Gen6 SuperNIC. Since NVIDIA Grace operates at PCIe Gen5 speeds, there is a Gen5 x16 link to the Grace CPU, then there is a Gen6 x16 link to the B300 GPU. There is another Gen5 x4 link to a SSD. That solves a challenge for NIVIDA as well because Grace does not bifurcate down to x4 like most server CPUs.
We have shown this before, but NVIDIA is also using this on the NVIDIA MGX PCIe Switch Board with ConnectX-8. There, NVIDIA is able to push Broadcom switch chips from PCIe card platforms, but also it can provide PCIe Gen6 to NIC connectivity to future B300 PCIe GPUs instead of PCIe Gen5. Those PCIe form factors are not announced yet, but NVIDIA has been making PCIe versions for many years, and the purpose of this board is to support the B300-based PCIe Gen6 cards that are coming later this year.
Here is a look at the ConnectX-8 Spectrum-X Ethernet scaling.
That includes the integrated Spectrum-X Ethernet switch for scalability.
The RDMA networking is a big part of what made Mellanox successful before NVIDIA’s acquisition, and it is also what has allowed NVIDIA to scale to large clusters.
Here is a look at the PSA packet processor.
Here is the data path accelerator (DPA) that is a RISC-V event processor.
The NIC also has Spectrum-X Ethernet congestion control and routing.
Here is a slide about how that works with the DPA.
NVIDIA started showing some numbers, like this is what it looks like with training step time.
Here is a look at the tail latency under load
Now we are getting more in to Spectrum-X switches, not just the ConnectX-8 NIC side.
Overall, Spectrum-X is designed to be fast at AI networking using Ethernet.
Final Words
Unfortunately, we could not get ConnectX-8 for our new Keysight CyPerf box. We have the licensing to go up to use four of these ConnectX-8 NICs, but we had to use ConnectX-7 instead. ConnectX-8, and especially the C8240, is still rather hard to get from a supply side because it is the next-generation very fast NIC. Right now, if you want a high-speed NIC, this is faster than the Broadcom “Thor” 400GbE cards and AMD Pollara 400 cards on the market today. By making the entire stack, NVIDIA is able to push 800G NIC speeds and put them into systems before PCIe Geen6 server CPUs are readily available.