top of page

Estimating the Carbon Footprint of BLOOM, a 176B Parameter Language Model

Reference Type: 

Journal Article

Luccioni, Alexandra Sasha, Sylvain Viguier, and Anne-Laure Ligozat. 2022. “Estimating the Carbon Footprint of BLOOM, a 176B Parameter Language Model.” Journal of Machine Learning Research. https://doi.org/10.48550/ARXIV.2211.02001.

Progress in machine learning (ML) comes with a cost to the environment, given that training ML models requires significant computational resources, energy and materials. In the present article, we aim to quantify the carbon footprint of BLOOM, a 176-billion parameter language model, across its life cycle. We estimate that BLOOM's final training emitted approximately 24.7 tonnes of~\carboneq~if we consider only the dynamic power consumption, and 50.5 tonnes if we account for all processes ranging from equipment manufacturing to energy-based operational consumption. We also study the energy requirements and carbon emissions of its deployment for inference via an API endpoint receiving user queries in real-time. We conclude with a discussion regarding the difficulty of precisely estimating the carbon footprint of ML models and future research directions that can contribute towards improving carbon emissions reporting.

Download Reference:

Search for the Publication In:

Formatted Reference:

bottom of page