Whitepapers
May 23, 2023
Training Large Language Models on Cerebras Wafer Scale Engine
In this paper, we survey existing approaches used to scale training to clusters of compute units and explore the limitations of each in the face of giant models. We present a new paradigm for giant model training, called weight streaming, whiche enables the training of models two orders of magnitude larger than the current state-of-the-art, with a simple scaling model. [Updated March 2023 with scaling and sparsity performance results and use cases.]
April 27, 2023
Argonne National Laboratory
At ANL, Cerebras is working with research staff in the Computing, Environment, and Life Sciences (CELS) directorate to accelerate groundbreaking scientific and technical innovation. This partnership has produced award-Winning work, spanning cancer treatment, CoVID-19 drug discovery, and advances in physics.
June 06, 2022
Deep Learning Programming at Scale
Deep learning has become one of the most important computational workloads of our generation, advancing applications across industries from healthcare to autonomous driving. But it is also profoundly computationally intensive. (Updated June 2022.)
April 06, 2022
Powering Extreme-Scale HPC with Cerebras WaferScale Accelerators
In this paper, we will explore the challenges facing HPC developers today and show how the Cerebras architecture can help to accelerate sparse linear algebra and tensor workloads, stencilbased partial differential equation (PDE) solvers, N-body problems, and spectral algorithms such as FFT that are often used for signal processing.
February 07, 2022
The Cerebras Software Development Kit: A Technical Overview
Cerebras has introduced a new software development kit (SDK) which allows anyone to take advantage of the strengths of the CS-2 system. Developers can use the Cerebras SDK to create custom kernels for their standalone applications or modify the kernel libraries provided for their unique use cases. The SDK enables developers to harness the power of wafer-scale computing with the tools and software used by the Cerebras development team.
October 05, 2021
Lawrence Livermore National Laboratory
Lawrence Livermore National Laboratory in Livermore, California, is a federal research facility primarily funded by the US Department of Energy Energy’s National Nuclear Security Administration (NNSA). LLNL’s mission is to strengthen the United States’ security by developing and applying world-class science, technology and engineering.
April 06, 2021
Cerebras Systems: Achieving Industry Best AI Performance Through A Systems Approach
The CS-2 is a system solution that consists of innovations across three dimensions: a) the second generation Cerebras Wafer Scale Engine (WSE-2) — the industry’s largest and only multi-trilliontransistor processor, b) the Cerebras System and c) the Cerebras software platform.