BLOOM: open-access 176B-parameter multilingual language model

The BigScience Workshop released BLOOM, a 176-billion-parameter language model trained across 46 natural and 13 programming languages, made freely available under the Responsible AI License at a scale that had only recently begun to become accessible, and had not previously been available with multilingual training data.

Background

By late 2022, large language models had shown they could learn new tasks from a handful of examples or a plain-language instruction, without being retrained. That was useful enough that organisations across industry and research were building products around the capability. The problem was access. As the BigScience Workshop put it, “most LLMs are developed by resource-rich organizations and are frequently kept from the public.” Researchers who wanted to study how these models worked, or to build on them, generally could not.

The scale involved made this harder to fix. A model with hundreds of billions of parameters requires substantial computing infrastructure to train, and the teams that had done it tended not to share the results. Multilingual coverage was a further gap that the project explicitly set out to address.

What happened

The BigScience Workshop, a collaboration of hundreds of researchers, built and released BLOOM: a decoder-only Transformer language model with 176 billion parameters. Decoder-only means the model generates text left to right, predicting each next word from what came before, which is the same broad approach used in the GPT family of models.

Training data came from the ROOTS corpus, described in the paper as “a dataset comprising hundreds of sources in 46 natural and 13 programming languages (59 in total).”

The model was evaluated on a wide range of benchmarks. The authors found that BLOOM achieves competitive performance on multilingual benchmarks, with stronger results after multitask prompted finetuning. Multitask prompted finetuning means continuing to train the model on many different tasks, each described in natural language, so the model gets better at following instructions.

Models and code were made available under the Responsible AI License, which the authors described as a step “towards democratizing this powerful technology.” The licence placed some conditions on use, distinguishing it from a fully permissive release, but the model weights were accessible at this scale with multilingual coverage in a way that had not previously been offered.

Why it mattered

Most large language models at this scale were developed by resource-rich organisations and kept from the public. BLOOM gave researchers outside those organisations access to a model of comparable size, trained on genuinely multilingual data, at a time when open access at this scale was only just beginning. The authors released models and code under the Responsible AI License to help make that access real rather than nominal.

Sources

• BLOOM: A 176B-Parameter Open-Access Multilingual Language Model. arxiv.org. Primary source
• BLOOM: A 176B-Parameter Open-Access Multilingual Language Model. arxiv.org. Primary source

Cite this page

AI Achievements. (2022). BLOOM: open-access 176B-parameter multilingual language model. Retrieved 2026-08-29, from https://achievements.ai/milestone/bloom-open-access-176b-parameter-multilingual-language-model

@misc{achievements_bloom_open_access_176b_parameter_multilingual_language_model,
  title  = {BLOOM: open-access 176B-parameter multilingual language model},
  author = {{AI Achievements}},
  year   = {2022},
  url    = {https://achievements.ai/milestone/bloom-open-access-176b-parameter-multilingual-language-model}
}