event

SC22

November 13–18, 2022

The International Conference for High Performance Computing, Networking, Storage, and Analysis

Book a meeting with us
Read wrap-up blog

000 days 00 hours 00 minutes 00 seconds

speaker

Cindy Orozco Bohorquez

Machine Learning Solutions Engineer

Session time: Monday, Nov 14, 2022 @9am – 9:30pm CST

Location: C155

Session Title:A Comprehensive Evaluation of Novel AI Accelerators for Deep Learning Workloads

Session description:

Scientific applications are increasingly adopting Artificial Intelligence (AI) techniques to advance science. High-performance computing centers are evaluating emerging novel hardware accelerators to efficiently run AI-driven science applications. With a wide diversity in the hardware architectures and software stacks of these systems, it is challenging to understand how these accelerators perform. The state-of-the-art in the evaluation of deep learning workloads primarily focuses on CPUs and GPUs. In this paper, we present an overview of dataflow-based novel AI accelerators from SambaNova, Cerebras, Graphcore, and Groq.

We present a first-of-a-kind evaluation of these accelerators with a diverse set of workloads, such as deep learning (DL) primitives, benchmark models, and scientific machine learning applications.

We also evaluate the performance of collective communication, which is key for distributed DL implementation, along with a study of scaling efficiency.

We then discuss key insights, challenges, and opportunities in integrating these novel AI accelerators in supercomputing systems.

Show more
Show less

Learn more
Panelist

Cindy Orozco Bohorquez

Machine Learning Solutions Engineer

Session time: Monday, Nov 14, 2022 @1:30pm – 5:30pm CST

Location: D161

Session Title: Programming New AI Accelerators for Scientific Computing

Session description:

Scientific applications are increasingly adopting Artificial Intelligence (AI) techniques to advance science. There are specialized hardware accelerators designed and built to efficiently run AI applications. With diverse hardware and software stacks of these systems, it is challenging to comprehend their capabilities, programming approaches, and measure performance. In this tutorial, we will present an overview of novel AI accelerators namely, SambaNova, Cerebras, Graphcore, Groq, and Habana. This includes presentations on hardware and software features on each system. We present steps on how to program these systems by porting deep learning models with any refactoring of codes implemented in standard DL framework implementations, compiling and running on the accelerator hardware. Next, we conduct a hands-on session on SambaNova and Cerebras systems at ALCF AI Testbed. The tutorial will provide the attendees with an understanding of the key capabilities of emerging AI accelerators and their performance implications for scientific applications.
Show more
Show less

Learn more
speaker

Natalia Vassilieva

Director of Product

Session time: Tuesday, Nov 15, 2022 @2:30pm – 3pm CST

Location: D171

Session Title: Near-Perfect Scaling on 13 Million Core Cluster Made Easy

Session description:

Large Language Models are shifting “what’s possible” in AI, but distributed training across thousands of traditional accelerators is massively complex and always suffers diminishing returns as more compute is added. Always? No longer. In this talk, Natalia Vassilieva from Cerebras Systems, will present a cluster of 16 Cerebras CS-2 nodes that achieves near-perfect linear scaling across more cores than the world’s most powerful supercomputer. And there’s more: the programming model is radically simple: the code for 16 nodes is exactly the same as that for a single node. A new era of easy access to extreme-scale AI has just begun.
Show more
Show less

Learn more
Speaker

Mathias Jacquelin

Member of Technical Staff

Session time: Tuesday, Nov 15, 2022 @ 4:30 PM CST

Location: C141-143-149

Session Title: Scalable Distributed High-Order Stencil Computations

Session description:

Stencil computations lie at the heart of many scientific and industrial applications. Stencil algorithms pose several challenges on machines with cache based memory hierarchy, due to low re-use of memory accesses if special care is not taken to optimize them. This work shows that for stencil computation a novel algorithm that leverages a localized communication strategy effectively exploits the second generation Cerebras Wafer-Scale Engine (WSE-2), which has no cache hierarchy. This study focuses on a 25-point stencil finite-difference method for the 3D wave equation, a kernel frequently used in earth modeling as numerical simulation. In essence, the algorithm trades memory accesses for data communication and takes advantage of the fast communication fabric provided by the architecture. The algorithm —historically memory-bound— becomes compute-bound. This allows the implementation to achieve near perfect weak scaling, reaching up to 503 TFLOPs on a single WSE-2, a figure that only full clusters can eventually yield.
Show more
Show less

Learn more
speaker

Natalia Vassilieva

Director of Product

Session time: Wed, Nov 16, 2022 @10:30am – 12pm CST

Location: C144-145

Session Title: ACM Gordon Bell Special Prize for HPC-Based COVID-19 Research Awards Ceremony

Session description:

A team from Argonne National Laboratory, Cerebras and others has been nominated for the Gordon Bell Special Prize for High Performance Computing-Based COVID-19 Research. The team was nominated for their paper: “GenSLMs: Genome-scale language models reveal SARS-CoV-2 evolutionary dynamics“.

This award recognizes outstanding research achievement towards the understanding of the COVID-19 pandemic through the use of high-performance computing (HPC). The purpose of the award is to recognize the innovative parallel computing contributions towards the solution of the global crisis. Nominations will be selected based on performance and innovation in their computational methods, in addition to their contributions towards understanding the nature, spread and/or treatment of the disease.

Show more
Show less

Learn more
Speaker

Andy Hock

VP Product Management

Session time: Thursday, Nov 17, 2022 @12:15pm – 1:15pm CST

Location: D168

Session Title: Birds of a Feather: Transformers at Scale for Science

Session description:

Transformers and other large language models have shown impressive capabilities as ‘foundation’ models for domains such as natural language processing and computer vision. Requiring huge amounts of scalable compute, self-supervised training on large datasets is leveraged to develop models applicable to a variety of specialized tasks. Recent efforts in areas such as bioinformatics and protein folding indicate the significant potential for Transformer models in domain science applications. In this session, presenters and attendees have the opportunity to discuss new algorithms, software, or hardware for training Transformers on large domain science datasets and novel ideas for applying Transformers in this space.
Show more
Show less
Learn more
speaker

Natalia Vassilieva

Director of Product

Session time: Friday, Nov 18, 2022 @8:30am – 10am CST

Location: C147-148-154

Session Title: Quantum Computing: A Future for HPC Acceleration?

Session description:

Quantum computing is a quickly growing field and we expect viable systems to be available relatively soon. However, it’s becoming clear that quantum computing systems alone are only of limited use unless integrated with HPC systems to enable both efficient pre- and post-processing and to allow for truly hybrid applications. Many questions remain: which applications will benefit from such acceleration? How widely will quantum acceleration be efficient and cost-effective? How large do quantum systems have to be to make an impact and when will this be the case? What is the impact on the operation in HPC centers? Will quantum acceleration augment or replace other accelerators, e.g., for AI? Or will the quantum computing hype bubble just pop? In this panel, we will discuss these questions with experts from the quantum computing community, with HPC architects, representatives from HPC centers as well as vendors of traditional and emerging accelerators.
Show more
Show less

Learn more
blog

Context is Everything: Why Maximum Sequence Length Matters

GPU-Impossible™ sequence lengths on Cerebras systems may enable breakthroughs in Natural Language Understanding, drug discovery and genomics.

Read
Blog

Cerebras Sets Record for Largest AI Models Ever Trained on Single Device

Our customers can easily train and reconfigure GPT-3 and GPT-J language models with up to 20 billion parameters on a single CS-2 system

Read
Blog

TotalEnergies and Cerebras Create Massively Scalable Stencil Algorithm

TotalEnergies used the Cerebras CS-2 system to turn a problem long accepted to be memory-bound into compute-bound. On a benchmark case inspired by a seismic kernel used to image the Earth, the CS-2 delivered more than 200x performance compared to a NVIDIA® A100 GPU.

Read