DISTRIBUTED · GPU-LOCAL · HIGH PERFORMANCE
RedCache
High-performance distributed S3 proxy cache for GPU-local AI/ML training and inference.
Eliminate S3 latency without changing a line of code. RedCache sits transparently between your GPU workloads and object storage, serving cached data at NVMe speed with intelligent prefetching, erasure coding, and cluster-wide consistency — deployed GPU-local so every read is as fast as local disk.
THE S3 LATENCY PROBLEM
Every S3 call is a network trip.
At scale, that's your bottleneck.
Small Object Penalty
AI/ML training reads thousands of small checkpoint files, sharded datasets, and tokenized inputs. Each S3 GET request carries ~100μs API overhead plus network RTT. Multiply by millions of reads and you're looking at hours of wasted compute.
GPU Idle Time
When your data pipeline can't feed GPUs fast enough, multi-million dollar clusters sit idle. S3 throughput limits and request rate caps mean your storage layer becomes the ceiling on training velocity.
No GPU-Local Cache Strategy
Most teams either accept the S3 tax or build fragile homegrown caching that breaks under concurrent access, lacks consistency guarantees, and can't scale beyond a single node. The data simply isn't close enough to the GPU.
THE SOLUTION
GPU-local caching between
your GPUs and S3.
RedCache deploys as a transparent S3 proxy co-located with your GPU compute nodes. Repoint your S3 endpoint, and every subsequent read is served from local NVMe cache at microsecond latency — with intelligent prefetching, erasure coding, and cluster-wide consistency.
RedCache deploys GPU-local — each compute node runs its own RedCache proxy with local NVMe cache. Data is served at disk speed, not network speed.
KEY FEATURES
Built for the demands of
production AI workloads.
Zero Code Changes
Drop-in S3 proxy. Repoint your endpoint_url and every existing Boto3, AWS SDK, or Go client works transparently. No refactoring, no retraining pipelines.
Hardware-First I/O
Built on Linux io_uring for async NVMe and networking. Submission queues scale with your hardware — no thread-per-request overhead, no context switch tax.
Distributed at Scale
1,024 virtual shards across 3 to 1,000+ nodes. Auto-rebalancing when nodes join or leave. BadgerDB-backed metadata keeps startup and scans bounded.
Intelligent Prefetching
Sliding-window frequency analysis, temporal locality detection, and sequential access pattern recognition predict what data you need next — before you ask for it.
Smart Sequencing
Passive straggler detection learns your batch access patterns and auto-prefetches chronic slow objects before they cause cache misses. Reduces epoch stalls by 20-40% on repeat epochs — zero code changes, no dataset rewriting needed.
Resilience Built In
S3 circuit breaker prevents quota exhaustion. Reed-Solomon erasure coding for durability. Merkle tree anti-entropy ensures cluster-wide consistency. Request coalescing via singleflight.
COMPARISON
RedCache vs
NVIDIA AIStore
Both are high-performance storage solutions for AI workloads — but they solve different problems. Understanding the distinction helps you pick the right tool, or use both together.
| Feature | RedCache | NVIDIA AIStore |
|---|---|---|
| Architecture | GPU-local S3 proxy cache layer | Distributed object storage system |
| Why it matters: RedCache runs right on your GPU nodes as a transparent proxy — no new infrastructure to provision. AIStore replaces or sits alongside your existing object store as a dedicated storage cluster, requiring separate hardware and capacity planning. | ||
| Primary Role | Caching tier in front of existing S3 | Primary storage for AI datasets |
| Why it matters: RedCache accelerates whatever object store you already use (S3, OCI, GCS) without migrating data. AIStore becomes your primary dataset home — excellent for petascale workloads but requires data ingestion and management. | ||
| Deployment | Co-located with GPU compute nodes | Flexible — dedicated cluster or co-located on GPU nodes (fast-tier mode with local NVMe) |
| Why it matters: RedCache lives where your GPUs are by design — cached reads never cross the network, giving you NVMe speed instead of S3 latency. AIStore can also run co-located on GPU nodes in fast-tier mode, but its primary strength is distributed storage at scale across a cluster. | ||
| Data Source | Caches from any S3-compatible backend | Local drives + optional S3/GCP backends |
| Why it matters: RedCache works with your existing buckets immediately — no data migration, no reorganization. AIStore needs its own storage pool built up from local drives or synced from cloud backends. | ||
| Integration | Zero code changes (S3 proxy) | S3-compatible + native Go/Python SDKs |
| Why it matters: RedCache requires one config change — repoint your S3 endpoint URL. Your existing Boto3, PyTorch DataLoader, or HuggingFace pipelines keep working as-is. AIStore can also work via S3 compatibility, but to unlock its full ETL and optimization features you need to adopt its native SDKs. | ||
| Scaling | 3 → 1,000+ nodes · virtual sharding | Linear scale-out · no hard limits |
| Why it matters: RedCache scales with your GPU fleet — add a node, get more cache automatically. Virtual sharding (1,024 shards) means rebalancing is smooth even at large scale. AIStore has no theoretical upper limit and can grow to thousands of nodes independently of compute. | ||
| Data Protection | Reed-Solomon erasure coding | Erasure coding (XOR) + replication |
| Why it matters: Both protect against node failure. RedCache uses Reed-Solomon (more space-efficient than XOR, tolerates multiple simultaneous failures). AIStore offers both erasure coding and full replication — useful when you need guaranteed redundancy for primary storage. | ||
| Caching | ML-based prefetching + Smart Sequencing (straggler detection) + adaptive eviction (LRU/LFU/ARC) | LRU with configurable watermarks + local cache (lcache) |
| Why it matters: RedCache actively predicts what your training job needs next using access pattern analysis — sliding-window frequency, temporal locality, and sequential detection. Smart Sequencing adds passive straggler detection that learns which objects are chronically slow in each batch and auto-prefetches them before they cause cache misses. This means data is often already in cache before you request it. AIStore provides effective fast-tier caching via its lcache component, but without predictive prefetching or Smart Sequencing-style straggler detection. | ||
| Smart Sequencing / Straggler Handling | Passive learning + auto-prefetch (no data rewrite, adapts continuously) | dSort — powerful one-time ETL that physically reorders datasets for optimal read patterns; excellent for heavy preprocessing pipelines |
| Why it matters: AIStore's dSort is a powerful tool that physically reorders your dataset for optimal read patterns — but it requires rewriting data, runs once before training, and doesn't adapt if access patterns change. RedCache Smart Sequencing learns straggler patterns passively during training (after just 3 observed batches) and auto-prefetches chronic slow objects before they cause cache misses. No data modification, no ETL step, adapts continuously as your workload evolves. For heavy preprocessing needs, dSort still wins — but for transparent straggler elimination, Smart Sequencing requires zero effort. | License | Proprietary (Cybertronic) | MIT |
| Why it matters: AIStore is fully open-source — free to use, modify, and deploy with no licensing overhead. RedCache is proprietary with commercial support from Cybertronic; contact us for licensing options tailored to your scale. | ||
| Best For | Eliminating S3 latency for GPU training/inference | Petascale primary storage + dataset ETL pipelines |
This comparison reflects typical deployment patterns. Both systems are open-source and can be adapted to various architectures — choose based on your primary need: distributed storage at scale (AIStore) or GPU-local caching with intelligent prefetching (RedCache).
INTERESTED?
Ready to eliminate
S3 latency from your pipeline?
Whether you want to deploy RedCache for your GPU workloads or need help architecting the right caching strategy — let's talk.