Eliminate the AI Storage I/O Wall.
Striped local NVMe and RAM cache that streams weights directly into GPUs at hardware line rate.
Metadata lookup
< 1 ms
Local Unix socket (Moka W-TinyLFU)
Tier 2 NVMe DMA
7 - 14 GB/s
Per PCIe Gen 4/5 drive (4 MiB stripes)
Tier 1 Host RAM
50 - 100+ GB/s
Zero-copy /dev/shm & HugePages
Default exposure
0 TCP ports
No external DB or network ports
Node-local PCIe vs. shared cluster storage
Even on 200GbE and 400GbE fabrics, central storage bandwidth is divided across every host and GPU pulling weights at the same time. HyperCache moves weight and dataset reads onto each node's local PCIe lanes and memory bus.
Data path
Per-node bandwidth
70 GB load time
Shared storage over 200GbE / 400GbE
32+ nodes pulling concurrently - bandwidth shared across hosts & contends with fabric traffic
1.5 - 4.0 GB/s
18 - 45s
HyperCache Tier 2: 4x PCIe Gen 4 NVMe
Node-local O_DIRECT / GPUDirect Storage - 0 bytes over network fabric
~28 GB/s
2.5s
HyperCache Tier 2: 8x PCIe Gen 5 NVMe
Node-local striped NVMe pool - scales linearly with cluster size
~112 GB/s
0.6s
HyperCache Tier 1: Host RAM (/dev/shm)
Zero-copy memory-mapped streaming for hot weights & LoRA adapters
50 - 100+ GB/s
< 0.7s
Drop-in URI routing for Python
Importing hypercache registers the hypercache:// scheme with fsspec. Existing torch.load, torch.save, Safetensors, vLLM, and FSDP checkpoints stream directly from local NVMe or RAM.
example.py
import fsspec
import torch
import hypercache # Registers hypercache:// protocol
# Read weights from local striped NVMe/RAM cache
uri = "hypercache://s3/models/llama-3-70b/weights.pt"
with fsspec.open(uri, "rb") as f:
state_dict = torch.load(f, map_location="cuda:0")
# Save checkpoints at local NVMe speed (async S3 flush)
ckpt = "hypercache://s3/checkpoints/step_1000.pt"
with fsspec.open(ckpt, "wb") as f:
torch.save(model.state_dict(), f)