As AI Increases Demands on Memory, Storage Steps Up

Surging AI demands are driving the need for massive datasets and context windows that burst past the confines of system memory.  But rising needs aren’t met by simply adding more storage capacity. What’s needed is useful, grounded insights from AI factories and efficient, secure storage architectures that enable those insights.  At this week’s Future of Memory and Storage (FMS) conference, NVIDIA is unveiling new storage advancements and showcasing how the next leap in AI depends as much on the storage infrastructure feeding accelerated computing as on the computing power itself. The pressure on that infrastructure is intensifying as AI agents consume massive amounts of data — and GPUs can now initiate storage requests directly, generating thousands of concurrent operations. To serve those requests, storage systems must continuously encrypt, compress, verify and reconstruct data. These critical data services can become bottlenecks when thousands of agents access storage simultaneously. Benchmarks highlighted in this NVIDIA technical blog show that the NVIDIA Vera CPU, part of NVIDIA Vera BlueField-4 STX, delivers up to 3.21x higher throughput than an x86 CPU in a two-stage compression and encryption pipeline. This means that with Vera, storage platforms can absorb the flood of AI data more efficiently — delivering greater throughput with significantly less compute infrastructure. With accelerated computing, storage stops being a passive place to keep data and becomes an active part of the data path.  This upends the old economics of determining when data belongs in memory (where applications can fetch it faster) versus on a storage drive (where it can be held in cheap and plentiful space). The tradeoff was first framed 40 years ago, when the answer was measured in accessing that data in minutes. On today’s GPUs, paired with AI storage solutions from NVIDIA and partners, the same tradeoff now plays out in microseconds. Closing the gap between AI’s needs and memory shortage depends on extreme codesign across the whole ecosystem, from memory and storage manufacturers to the software built on them.  Open Source NVIDIA cuFile APIs Enable Interoperability for Storage Solutions At FMS, NVIDIA announced it is open sourcing its cuFile application programming interfaces (APIs) — and the vertical storage software stack underneath them — which let GPUs, not just CPUs, read from and write to storage directly. cuFile is an open source component of NVIDIA GPUDirect Storage. Using hundreds of thousands of GPU threads, fast high-bandwidth memory and other methodologies, cuFile enables securely accessing data from storage in just microseconds. This represents how the industry is unifying a security-first storage stack based on Linux best practices, providing interoperability between GPUs and data.  In addition, fast, secure access to data and storage is a foundational element to powering preventive and detective cybersecurity measures. Making cuFile openly available will help make security context, data and storage accessible at the speed AI-powered defenses need. Such open technologies support initiatives such as the new Open Secure AI Alliance. This site is the new home for APIs that are open to contributions — with Google, Intel, NVIDIA and Meta as inaugural maintainers — and can be optimized for use across various software and hardware platforms, driving innovation and efficiency for developers and enterprises. NVIDIA and Industry Leaders Advance New Frontier of AI Storage In addition, NVIDIA and storage industry leaders are optimizing memory and storage solutions through an initiative called Storage-Next. The NVIDIA-driven initiative brings together storage makers, controller vendors, thermal design, cooling and orchestration operators, and standards bodies to align on how GPU-driven storage should behave — then turn these advanc