Samsung has built what it says is a first-of-its-kind supercomputer that includes AMD data center GPUs outfitted with its processing-in-memory chips, which the company says can significantly improve performance and power efficiency when training large AI models.
The supercomputer, unveiled at an industry event in South Korea on Tuesday, includes 96 AMD Instinct MI100 GPUs, each equipped with a Processing-in-Memory (PIM) chip, a new type of memory technology that reduces the amount of data switch between CPU and DRAM.
Choi Chang-kyu, the head of the AI research center at Samsung Electronics Advanced Institute of Technology, reportedly said that the cluster was able to use the Text-to-Test Transfer Transformer (T5) 2.5 language model developed by Google. times faster to train while using 2.7 times less power compared to the same cluster configuration not using the PIM chips.
“It’s the only one of its kind in the world,” Choi said.
Samsung has stated that its PIM technology has a significant impact on energy consumption and the environment, reducing a cluster’s annual energy consumption by 2,100 gigawatt hours and consequently 960,000 tons of CO2 emissions.
As always, we should reserve judgment until these claims can be independently tested and verified, but the company said such a reduction in performance is equivalent to the amount of carbon it takes 16 billion urban trees to absorb over a decade .
A big reason the PIM-powered supercomputer packs so much horsepower is that each PIM chip uses high-bandwidth memory (HBM), which the industry is increasingly turning to to handle high-performance computing and AI workloads leaves. Nvidia and AMD have used HBM in data center GPUs for several generations, and Intel plans to introduce HBM in an upcoming variant of Xeon Max-branded server processors and a high-end data center GPU.
What sets Samsung HBM PIM chips apart from other companies’ HBM implementations is that each memory bank on the PIM chip contains a processing unit. According to the South Korean electronics giant, this reduces bottlenecks when moving data between the CPU and memory by offloading some of the computation to memory itself.
Samsung hopes to drive industry adoption of its PIM chips by developing software that will allow companies to use the technology in an integrated software environment. To do this, it relies on SYCL, a royalty-free, cross-architecture programming abstraction layer that underpins Intel’s C++ implementation for its OneAPI parallel programming model.
The company has been pushing PIM for almost three years now, and another way it’s bringing the technology to market is through something called AXDIMM, short for Accelerated DIMM.
We’ll know if Samsung ends up making any headway when PIM shows up in new supercomputers set up by research labs, academic institutions, and other organizations over the next few years. ®
https://www.theregister.com/2022/12/13/samsung_gpu_supercomputer/ Samsung uses processing-in-memory chip on AMD MI100 GPU • The Register