Key Takeaways

• Rambus and ZeroPoint Technologies are collaborating to enhance memory efficiency in AI and HPC workloads through composable memory solutions.
• Traditional server architectures often lead to over-provisioned memory, resulting in high costs and wasted resources.
• Disaggregation of memory resources and advanced data compression can significantly lower total operational costs in data centers.

Memory Innovations for Data Centers

The rising costs of memory in data centers, especially for artificial intelligence (AI) and high-performance computing (HPC) applications, pose significant challenges. Rambus, under the guidance of Senior Director Lou Ternullo, and ZeroPoint Technologies, represented by VP Nilesh Shah, are tackling this issue with innovative strategies aimed at enhancing memory efficiency.

In traditional server configurations, processing units (CPUs, GPUs, etc.) and memory are tightly integrated, leading to a scenario where memory costs can represent nearly 50% of the total server cost. The prevalent practice of over-provisioning memory to cater to peak workloads often results in approximately 40% of that memory going unused, which translates to substantial financial waste.

The introduction of disaggregated infrastructure, where memory and processing units are separated, offers a promising avenue for optimizing resource allocation. This architecture, described as “composable,” enables data centers to assemble resources dynamically based on workload requirements. When workloads finish, the memory can be returned to a pool, allowing for better resource utilization.

A notable concern with this approach is the latency associated with accessing memory that is physically distant from processing units. The development of Compute Express Link (CXL) technology aims to mitigate this latency by providing high-speed connections between processors and disaggregated memory.

Rambus plays a critical role in the CXL ecosystem, contributing intellectual property essential for creating high-performance interconnects. Their offerings include CXL PHY (physical layer) and controller technologies, which facilitate rapid signaling essential for efficient memory sharing among different processing units, thereby boosting overall performance.

Additionally, ZeroPoint Technologies introduces a state-of-the-art compression solution capable of achieving 2-4 times data compression for memory resources. This compression allows organizations to either reduce required memory capacity by half or store twice the data in the same physical resources, significantly lowering overall memory costs.

Interestingly, compression and decompression traditionally occur within server software, consuming valuable processing cycles that could generate revenue. Companies like Meta and Google have indicated a preference for offloading these tasks to memory hardware, leading to more efficient use of their processing resources.

As an example of their collaboration, Rambus and ZeroPoint developed a turnkey FPGA CXL expander card, combining both companies’ technologies to provide additional memory access. By implementing compression and decompression in hardware rather than software, they ensure that latency remains minimal, ultimately supporting smoother and more efficient operations for data centers.

The potential for significant cost savings and enhanced performance through these innovations represents a transformative shift in how data centers can operate, ensuring better resource management while maintaining the high performance necessary for AI and HPC applications. As these technologies evolve, they promise to reshape the landscape of data center architecture, delivering improved efficiency and reduced operational costs.

The content above is a summary. For more details, see the source article.