Argonne National Laboratory
At ANL, Cerebras is working with research staff in the Computing, Environment, and Life Sciences (CELS) directorate to accelerate groundbreaking scientific and technical innovation. This partnership has produced award-Winning work, spanning cancer treatment, CoVID-19 drug discovery, and advances in physics.
Training Giant Neural Networks Using Weight Streaming on Cerebras Wafer-Scale Systems
In this paper, we survey existing approaches used to scale training to clusters of compute units and explore the limitations of each in the face of giant models. We present a new paradigm for giant model training, called weight streaming, whiche enables the training of models two orders of magnitude larger than the current state-of-the-art, with a simple scaling model. [Updated March 2023 with scaling and sparsity performance results and use cases.]
June 6, 2022
Whitepaper
Deep learning,chip,software,machine learning,PyTorch,TensorFlow,weight streaming
Deep Learning Programming at Scale
Deep learning has become one of the most important computational workloads of our generation, advancing applications across industries from healthcare to autonomous driving. But it is also profoundly computationally intensive. (Updated June 2022.)
Powering Extreme-Scale HPC with Cerebras WaferScale Accelerators
In this paper, we will explore the challenges facing HPC developers today and show how the Cerebras architecture can help to accelerate sparse linear algebra and tensor workloads, stencilbased partial differential equation (PDE) solvers, N-body problems, and spectral algorithms such as FFT that are often used for signal processing.
Accelerating NLP Model Training and Enabling Higher Accuracy for Financial Services Applications
The benefits of training from scratch using domain-specific datasets can now be realized in an enterprise environment thanks to Cerebras AI accelerator technology.
The Cerebras Software Development Kit: A Technical Overview
Cerebras has introduced a new software development kit (SDK) which allows anyone to take advantage of the strengths of the CS-2 system. Developers can use the Cerebras SDK to create custom kernels for their standalone applications or modify the kernel libraries provided for their unique use cases. The SDK enables developers to harness the power of wafer-scale computing with the tools and software used by the Cerebras development team.
Lawrence Livermore National Laboratory
Lawrence Livermore National Laboratory in Livermore, California, is a federal research facility primarily funded by the US Department of Energy Energy’s National Nuclear Security Administration (NNSA). LLNL’s mission is to strengthen the United States’ security by developing and applying world-class science, technology and engineering.
Cerebras Systems Enables Brain-scale AI
This research paper explores Cerebras System's approach to create a brain-scale AI and the new technologies that could enable that feat. But first, let's put this discussion into the proper context. Just how big is a 120 trillion-parameter model?
Limits to Scale-Out for Training Language Models
Natural language processing has revolutionized how data is consumed, meaning that computational demand has skyrocketed. Companies in every industry are using graphics processing unit (GPU) clusters to keep up. But is this really the best solution?
Train Large BERT Models Faster with Cerebras Systems
Unstructured text is one of the largest human-generated data sources. Web data, academic publications, emails, traditional media, texts, instant messages, digital records, social media — all hold an enormous volume of unstructured text.
Cerebras Systems: Achieving Industry Best AI Performance Through A Systems Approach
The CS-2 is a system solution that consists of innovations across three dimensions: a) the second
generation Cerebras Wafer Scale Engine (WSE-2) — the industry’s largest and only multi-trilliontransistor processor, b) the Cerebras System and c) the Cerebras software platform.