I ask AI dumb questions all day. How many r's in strawberry? Can you make this email less weird? Every answer comes back instant and weightless, so it feels free.
Well, it isn't. Behind the chat box, a building is dragging its entire memory across itself to hand me a single word. Then it does it again for the next word. I feel none of it. That's what a clean interface is for.
The argument people are having about AI and energy is a fight over the price of one of those dumb questions. Both sides are profiling the interface and ignoring the datapath. The real bill is the distance the machine has to haul its own memory before a single word comes back.
Let's start with something smaller than a prompt. One bit. Easy, right?
On the chip, the math is almost free. Arithmetic got solved decades ago, back when CPUs turned into serious math engines. GPUs took it further, thousands of those calculations running at once, and every process node since has driven the cost down toward nothing. Adding two 32-bit numbers runs about a tenth of a picojoule. Then look at what it costs just to move the data those numbers are made of. One bit into the memory stacked against the chip, roughly 4 pJ. Across the board to commodity memory, 20 pJ. Across the data center, through optics and switch after switch, into the tens and hundreds. Moving a single bit to where the math happens can cost more than the whole operation it feeds. The bit didn't get smarter. It just got farther from the math.
That distance is the whole bill. Each hop outward multiplies it again, and a bit that has to cross the whole building can cost hundreds to thousands of times the arithmetic it feeds. The exact number moves with the process and the packaging. The direction doesn't. The math got cheap while getting to the data stayed as expensive as it ever was.
The machine is slow for one reason. Every piece of information and instruction it needs to think lives somewhere else, and it has to go fetch it, every single time.
Somewhere else is HBM. Stacks of high-bandwidth memory bonded right onto the GPU package, about as close as memory gets to the math without being the math. That's where the model actually lives, all of its weights, plus whatever context you handed it. The cores that do the multiplying are tiny next to it. They hold almost nothing. Everything they work on has to be pulled out of that memory and pushed through them first.
And the pulling is the slow part. A modern GPU can run the math far faster than the memory can feed it, so the cores spend most of their time starved, stalled on data still in flight. There's a name for the gap between how fast you can compute and how slowly you can feed it. The memory wall.
To write a single word, really a token, a fragment of one, a dense 70-billion-parameter model reads its entire brain. All of it. Every weight, about 140 gigabytes, pulled out of memory and shoved past the math just to decide what comes next. Then it throws the reading away and does it again for the second token. And the third. It doesn't think a sentence. It hauls the whole library across the room, word by word, and the thinking waiting at the end of each trip is the cheapest part of the day.
That is the simplest way to run it. One user, one dense model reading all of itself for every token. It is also the version the whole field spends its cleverness trying to escape. The escape routes are real, and they prove the point instead of breaking it.
Run the arithmetic on that one token and it comes out near 4.5 J spent moving, 0.2 J spent computing.
Batch a thousand users together and the brain gets read once for all of them, so each one pays only a sliver of that read. Wake only part of the network, the way the sparse models do, and there's less brain to read in the first place. Neither one changes the shape of it. Every trick the field is proud of is the same move in a different coat: pay the distance less often.
Back to the strawberry question. I knew the answer. I asked anyway, out of the same habit I've had my whole life.
In school I could add 57 and 23 in my head. I still checked it on the calculator, every time, because the thing was right there and trusting the machine was easier than trusting myself. The calculator never made me worse at math. It just made me stop asking whether I needed it. The chat box is the same reflex, one layer up. I know how many r's are in strawberry. I ask anyway.
End to end, wall plug and cooling included, a short prompt like that runs about a third of a watt-hour, 0.3 Wh. That is the whole system on a model bigger than the 70B, not the bare joules of one word. Google measured its own median text prompt at 0.24 Wh. Altman put an average ChatGPT query near 0.34 Wh. Ten seconds of a television. A second of a microwave. Three thousandths of a cent.
The people who say one prompt is no crisis are right. The viral line that a single AI query costs ten times a Google search was about ten times too high. Both of those are correct, and both of them are looking at the wrong number. Because answering it still dragged something like a terabyte of memory through the machine, to hand me an answer I already knew. A terabyte moved, so I wouldn't have to count to three.
Now ask it to read a book. Feed it Harry Potter and the Philosopher's Stone, about a hundred thousand tokens, and ask what happens in it.
Summarizing a children's book is not the hard part. But to do it, the machine has to move the entire book through itself, every word dragged past every weight, more memory traffic, more cache to keep warm, more attention to pay across more tokens, all of it hauled the same distances as before, just far more of it. The cost climbs to something like 40 Wh, over a hundred times the short prompt. Feed it the whole series, past a million tokens, and you are on the order of a kilowatt-hour, extrapolated out past where anyone has cleanly measured. Three thousand times the freight. Roughly forty-five minutes of an average home's electricity, to summarize books a ten-year-old reads for fun, because you gave it that much more to move.
One more step, and it isn't the question anymore. It's the habit.
About 900 million people now use ChatGPT every week, close to a tenth of the people alive. Between them they send around 2.5 billion prompts a day, plenty of them this trivial. What should I make for dinner. Does this text make me sound needy. Say hi to check it's awake. Put each of those at 0.3 Wh, and it comes to roughly 750 MWh a day, enough to run 25,000 homes, a small city drawing power for nothing but a planet half-wanting to know something. None of it is heavy. Not one query is a thing you could feel in your hand.
No single person is wasting anything. That is the whole problem. The power comes a fraction of a watt at a time, from everyone, and no one can feel their share.
People fight over whether a prompt is expensive. In money it barely registers, which is why that fight goes nowhere. Weigh it in joules instead and follow where they go, and the machine turns out to be bottlenecked on hauling its own memory, not on the math. Batch the users, shrink the weights, do everything right, and it stays that way. Movement is the thing it runs out of first. The math is the visible part. Movement is the machine.
Training a model burns real math. But serving it, the part you touch every day, keeps getting called compute too, as if raw math were the scarce thing there. What actually gates it, once the model is answering you word by word, is movement. Fast memory in short supply. The packaging that bonds memory against the die. Optical links between racks. And now the grid itself. Nobody is short of ways to multiply numbers. They are short of ways to feed them fast enough.
At the frontier, this never trends toward smaller. The newest models don't answer faster, they think longer, a reasoning model can burn tens of thousands of tokens muttering to itself before it says a word back. Add agents calling other agents, context windows you fill just because you can, retries, tools, the reflex to paste in the whole repo. The questions stay as dumb as ever, and the freight behind each one only climbs.
The IEA expects the world's data centers to more than double their electricity use by 2030, toward a thousand terawatt-hours a year, plants and substations and new lines run out to the server farms. Stand close to where that power goes when the machine is answering you, and the strange part is how little of it is for thinking. It is for moving.
Watch the next answer land instant, and it isn't a mind lighting up. It's a warehouse getting emptied and refilled, one shelf at a time, fast enough that you never see the forklift.
None of this is a secret. Moving data is the problem the whole field is actually working on. Faster memory, co-packaged optics, chiplets, all of it one long campaign to move fewer bits a shorter way. And every generation, the cost of a bit falls.
Every generation, we also move more of them. Cheaper movement just buys longer context, more agents, more retries, more people asking more often, so the total keeps climbing. Physics sets a floor nobody gets under. Moving a bit across a distance costs energy, and no amount of cleverness makes that free. So the best people in the world keep making the bit cheaper, and the grid keeps filling up behind them, because the moment moving gets cheaper, we move more.
You could argue the moving is the thinking. That the mind is those 140 gigabytes, and dragging them through is what a thought physically is. Maybe so. It only sharpens the point, because then intelligence and movement are the same act, and the act is priced in distance, not in math.
There's something almost funny in it. We built the closest thing to a mind we've ever made, and it ran headfirst into the dumbest law in the universe. It costs energy to move anything from here to there. That was true before chips, before people, and it'll be true long after. The intelligence is the new part. The distance is as old as everything, and it's still what wins.
References
- Horowitz, M. Computing's Energy Problem (and what we can do about it). ISSCC 2014. link
- O'Connor, M. et al. Fine-Grained DRAM: Energy-Efficient DRAM for Extreme Bandwidth Systems. MICRO 2017. link
- Ivanov, A., Hoefler, T. et al. Data Movement Is All You Need: A Case Study on Optimizing Transformers. arXiv:2007.00072. link
- Google. Measuring the environmental impact of AI inference. August 2025. link
- Epoch AI. How much energy does ChatGPT use? 2025. link
- Altman, S. The Gentle Singularity. 2025. link
- Masley, A. A cheat sheet for conversations about AI energy use. 2025. link
- TechCrunch. ChatGPT users send 2.5 billion prompts a day. July 21, 2025. link
- TechCrunch. ChatGPT reaches 900M weekly active users. February 27, 2026. link
- International Energy Agency. Energy and AI. 2025. link