Large Memory Servers Designed for Real Time AI Inferencing
One clear metric emerges when a large number of users and datasets are involved with AI model training and AI inferencing: memory capacity.
Big Local Memory Makes All the Performance Difference
When attempting to increase the size of an in-memory database, the answer—once a single CPU’s worth of memory has been maxed out—has always been to add more CPUs and memory.
This solution is problematic, in that as soon as workloads overflow to multiple servers, then network latency and workflow overhead quickly start to degrade the data access response and application communication performance.
However, the larger you can make the memory in a single server running the database, the less chance there is of network delays. Enter Compute Express Link® (CXL®).
Starting with PCIe Gen 5, the memory expansion protocol layer is tightly coupled with the CPU’s memory architecture allowing additional memory or memory modules to be added to a mainstream single or dual socket server.
CXL is designed to solve the memory capacity problem within a single server by allowing additional memory to be added via the CPU’s peripheral I/O bus solving your real time data analytics retrieval and processing large memory needs.
Performance & Consistency
Adding memory headroom avoids the overflowing to persistent storage
Lower Memory Costs
Use lower cost per bit 96GB or 128GB DIMMs vs. 256GB DIMMs
Latency Reduction
Millisecond level latencies when deployed with fast real-time dBase
Ingest Real Time Data
Use the most current data when making key decisions in AI inference
Talk to the CXL Experts at Penguin Solutions
Reach out today and learn more how we help you reach your AI & HPC infrastructure project goals. Our team designs, builds, deploys, and manages high-performance, high-availability enterprise solutions, empowering customers to achieve their breakthrough innovations.