The AI Energy Footprint: An Explainer on Data Centers, Water Use, and Carbon

Ask how much energy a language model uses and you will get an answer that depends almost entirely on where the line was drawn. Was it the training run? The fine-tune? The million replies served last Tuesday? The chips that had to be fabricated first, or the power plants upstream of the servers? Move the boundary and two honest analyses can land a factor of ten apart.

That gap is not academic. Data center construction now shows up in utility planning meetings, water district agendas, and household electricity rates. A few concepts make it much easier to separate a real disclosure from a press release.

Two power bills, one model

Training is the part that gets the headlines, and it deserves some of them. A frontier run means weeks of near-peak load across a hall of accelerators, all pulling power simultaneously, all needing to be cooled. That is a genuine, concentrated draw on a regional grid, which is why training clusters tend to appear near cheap, abundant, or low-carbon power.

It is also, usually, a one-time cost. A large run might consume tens of gigawatt-hours — on the order of the annual electricity of several thousand households — and then it is done. That figure is enormous in isolation and moderate when spread across years of use.

The quieter bill is inference. Every prompt, every generated token, every autocomplete suggestion is a small computation on a real machine somewhere. Individually the cost is trivial. Multiplied by hundreds of millions of daily users, it usually outgrows training within the first couple of years of a popular service.

Then there are the loads nobody charts: evaluation sweeps, data cleaning, embedding jobs, search indexes rebuilt overnight, and the reserved capacity kept warm so peak traffic does not hit a wall. Idle accelerators still draw power, and a cluster sized for its busiest hour spends most of the year underused.

That is why "the energy cost of a query" is such a slippery number. A well-batched request on a nearly full server can use a small fraction of the energy of the identical request on a machine running at 20 percent utilization. Same model, same answer, very different joules.

Water: a local cost

Cooling is what makes data centers thirsty. Some halls use outside air, which is cheapest where the climate cooperates. Others run chilled water loops. Many rely on evaporative cooling, which trades water for electricity: it is often the most efficient option in a hot, dry place, and it works by letting water evaporate into the air.

A big evaporatively cooled campus can consume hundreds of thousands of liters a day, and several times that during a summer heat wave. In a water-rich region that is a rounding error. In a drought-prone one it is a fight.

The distinction between withdrawal and consumption matters here. Water pulled from a river, run through a heat exchanger, and returned is warmer but still there. Water that evaporates is gone from the watershed. Most breathless coverage of computing's water use collapses the two.

Two numbers are worth tracking: on-site consumption, and the water consumed off-site to make the electricity. Thermal power plants need cooling too, so a facility drawing on a thirsty grid carries a water footprint even when it cools with air. And liters are not interchangeable — a million liters from a wet basin in a rainy season is a different problem than a million from a stressed aquifer in August.

Which makes water a siting question more than a global one. There is no meaningful "water footprint of AI" that tells you whether a particular building belongs where it is being built. The answer changes with the watershed, the season, and what else is already pulling from the same source.

Carbon depends on the hour

A kilowatt-hour has no fixed carbon content. Its footprint depends on which generators are running when demand arrives. The same training job scheduled at 2 a.m. on a coal-heavy grid and at midday on a solar-heavy one can produce very different emissions for identical computation.

Hardware counts too. Manufacturing a high-end accelerator is carbon-intensive, and on cleaner grids the embodied emissions of the chip can rival everything it consumes while running.

Then comes accounting. Companies report direct emissions, purchased energy, and value-chain emissions separately, and they can present purchased electricity two ways: based on what the local grid actually emits, or based on the renewable certificates and contracts they have bought. Annual certificate matching lets an operator claim clean power on paper even when the electrons arriving at its meters came from a gas plant at 6 p.m. on a still evening. Hourly matching, measured against real generation and real load at each moment, is a stricter standard — and the one that actually pulls new clean capacity onto the grid.

There is no single number for the energy cost of AI. There is only a number, plus the boundary it was measured against.

The units behind the arguments

PUE

Power usage effectiveness is total facility power divided by the power delivered to the IT equipment. A PUE of 1.1 means about 10 percent of the campus's electricity goes to cooling, pumps, and losses; a PUE of 1.5 means a third does. It says nothing about how hard the chips are working — a very efficient building can host very idle servers.

WUE

Water usage effectiveness is liters consumed per kilowatt-hour of IT load, usually measured on site. Read it alongside the local water picture, because one liter per kilowatt-hour in a wet region and one liter per kilowatt-hour in a desert are not the same decision.

Carbon intensity

Grams of carbon dioxide equivalent per kilowatt-hour, varying by region and by hour. This is the number that decides whether a flexible workload should run now or wait six hours.

How to read a sustainability report

Most disclosures are not false. They simply answer a narrower question than the one being asked of them.

• Match the boundary to the claim. A figure covering only on-site electricity says nothing about the grid, the chip fabrication, or upstream water.
• Look for utilization. Energy per query is close to meaningless without knowing how full the machines were.
• Separate withdrawal from consumption. Only one of them leaves the watershed.
• Check whether certificates are annual or hourly. The difference is often the whole story.
• Ask about location and season. A global average hides the basin and the grid that actually carry the load.

What actually moves the number

The levers are unglamorous, and they mostly act on the same few variables.

1. Siting. Put flexible compute where the grid is clean and the water is not already spoken for.
2. Scheduling. Training runs and batch jobs can often wait for the hours when wind and solar are abundant.
3. Utilization. Batching, smarter routing, and consolidating workloads onto fewer, fuller machines cut energy and hardware at the same time.
4. Model efficiency. Distillation, quantization, caching, and smaller models for smaller questions reduce the work required per answer.
5. Cooling and heat reuse. Liquid cooling, free cooling where the climate allows, and piping waste heat into district heating systems all shift the balance.

None of these is a fix on its own, and several trade against each other — water for electricity, latency for carbon, capital for efficiency. What they have in common is that they are engineering and procurement decisions made long before anyone writes a press release about them.

Which is the useful

Photo: cookieone / Pixabay