> ## Documentation Index
> Fetch the complete documentation index at: https://preview.methodology.scope3.com/llms.txt
> Use this file to discover all available pages before exploring further.

# NVIDIA A100

## Carbon Footprint

From Andrew A. Chien, Liuzixuan Lin, Hai Nguyen, Varsha Rao, Tristan Sharma, and Rajini Wijayawardana. 2023. [Reducing the Carbon Impact of Generative AI Inference (today and in 2035)](https://dl.acm.org/doi/pdf/10.1145/3604930.3605705): In 2nd Workshop on Sustainable Computer Systems (HotCarbon ’23), July 9, 2023, Boston, MA, USA. ACM, New York, NY, USA, 7 pages.

𝑇𝐷𝑃 = 0.428 kW per GPU (1/8 of 3.43 kW for the instance) x 1.1 PUE

𝑂𝐼 = 0.35 is TFLOPS per inference assuming GPT-3 model (around 175 billion weights) processed with BF16 operations.

𝐼𝑊 = 5 is the number of inferences per output word (assumed window/sampling of 5 for each output word)

𝑊𝐶 is the output word count (measured average of 185 output words/request)

𝐶 = 156 TFLOPS is the GPU capacity assuming 50% efficiency

𝐸h𝑤 is per-GPU emission calculated as 1/8 of estimated per-instance emissions:
𝐸h𝑤 = 1/8 (𝑃𝐹 +𝐸𝐺𝑃𝑈 +𝐸𝐶𝑃𝑈 +𝐸𝐷𝑅𝐴𝑀 +𝐸𝑆𝑆𝐷 +𝐸𝐻𝐷𝐷)
where 𝑃𝐹 is IC packaging Carbon footprint while 𝐸𝐺𝑃𝑈 , 𝐸𝐶𝑃𝑈 , 𝐸𝐷𝑅𝐴𝑀, 𝐸𝑆𝑆𝐷, and 𝐸𝐻𝐷𝐷 are GPU, CPU, memory, and storage emissions, respectively. We estimate these emissions based on previous reports \[26] and instance hardware specifications \[1, 3, 11], yielding 𝐸h𝑤 = 318 kgCO2 per GPU

## Water Use

Data for A100:

* Water consumption per wafer-layer (Liter/12-inch equivalent wafer mask layer) from [TSMC 2022 ESG report](https://esg.tsmc.com/en-US/file/public/e-all_2022.pdf): 137.3 L per 12-inch equivalent wafer mask layer
* Mask layers for the TSMC 7nm process: [87](https://www.3dincites.com/2021/08/sustainability-in-the-semiconductor-fab-and-sub-fab/#)
* A100 chips per wafer: the A100 is 826 mm² which is similar to the H100's 814 mm², and the H100 yields [29 sets per 12" wafer](https://www.nextbigfuture.com/2024/06/growing-size-of-ai-chips-forces-tsmc-to-work-on-tripling-the-size-of-wafers.html)

Using the [manufacturing water use formula](https://localhost:3000/cluster#embodied-water-use):

```
(water use per chip) = (water use per wafer mask layer) x (mask layers) / (chips per wafer)

(water use per A100) = (137.3 L/layer) x (87 layers/wafer) / (29 chips/wafer) = 411.9 L/chip
```

Note: this doesn't include the memory chips that are also on the A100... need to find a source for the water use there