Hardware Bandwidth & Benchmark Methodology
Throughput comparisons across shared 200GbE/400GbE storage fabrics, node-local PCIe Gen 4/5 NVMe DMA pools, and Tier 1 DDR5/CXL Shared Memory.
Dedicated Node-Local Bus vs. Shared Cluster Storage
In multi-node AI training and inference clusters (200GbE / 400GbE+), central storage bandwidth is shared across every node and GPU pulling weights or saving checkpoints at the same time. When 32, 64, or 128 nodes request a checkpoint simultaneously, per-node throughput drops and network traffic contends with GPU-to-GPU communication.
By shifting weight and dataset delivery onto local PCIe lanes and host memory channels, each node reads from its own dedicated hardware:
| Data Path / Hardware Tier | Per-Drive / Per-Link Limit | Node Throughput | Cluster Network Contention |
|---|---|---|---|
| Shared Storage over 200GbE / 400GbE (Single Node Idle Fabric) | 25 - 50 GB/s wire limit | ~15 - 35 GB/s | 100% over fabric switches |
| Shared Storage over 200GbE / 400GbE (32+ Nodes Concurrent Load) | Shared origin ceiling | ~1.5 - 4.0 GB/s per node | Saturates shared storage & fabric |
| HyperCache Tier 2: PCIe Gen 4 NVMe (Node-Local) | ~7.0 GB/s per NVMe SSD | ~28 GB/s (4x NVMe) / ~56 GB/s (8x NVMe) | 0 Bytes (100% Local PCIe) |
| HyperCache Tier 2: PCIe Gen 5 NVMe (Node-Local) | ~14.0 GB/s per NVMe SSD | ~56 GB/s (4x NVMe) / ~112 GB/s (8x NVMe) | 0 Bytes (100% Local PCIe) |
| HyperCache Tier 1: Host RAM (DDR5 / CXL) | 50 - 100+ GB/s per stream | 300 - 460+ GB/s (8-12 channel DDR5 bus) | 0 Bytes (100% Memory Bus) |
Reproducing Benchmarks on Your Hardware
Every HyperCache installation includes a built-in parallel DMA benchmark utility (./hypercache bench) that writes a test payload across your discovered NVMe or RAM pool, reads it back using multi-threaded O_DIRECT DMA workers, and verifies end-to-end SHA-256 bit parity:
# 1. Start the local worker daemon
./hypercache -c configs/hypercache-agent.toml start &
# 2. Benchmark Tier 2 (Striped NVMe Pool) with a 5 GB payload
./hypercache bench --tier nvme --size-mb 5000
# 3. Benchmark Tier 1 (Host RAM /dev/shm Pool) with a 2 GB payload
./hypercache bench --tier ram --size-mb 2000
Recommended Node Chassis (x86_64)
HyperCache works on any Linux system (x86_64) with available NVMe drives - including custom builds, GPU workstations, OEM servers, and cloud instances. For 8-GPU cluster deployments, below are recommended enterprise node chassis with high-density local NVMe bays:
| Server Vendor | System Model | GPU Platform | Host CPU Architecture |
|---|---|---|---|
| Supermicro (SMC) | SYS-821GE-TNHR | NVIDIA HGX H100 / H200 (8-GPU) | Dual Intel Xeon (x86_64) |
| Supermicro (SMC) | AS-8125GS-TNMR2 | NVIDIA HGX H100 / H200 (8-GPU) | Dual AMD EPYC (x86_64) |
| Supermicro (SMC) | SYS-A22GA-NBRT / AS-A126GS-TNBR | NVIDIA HGX B200 / B300 (8-GPU) | Dual Intel Xeon / AMD EPYC (x86_64) |
| Dell PowerEdge | XE9680 / XE9780 / XE9785 | NVIDIA HGX H100 / H200 / B200 (8-GPU) | Dual Intel Xeon / AMD EPYC (x86_64) |