The next 1,000x of AI inference won’t fit in giant data centers. About 40% of it will run on devices and small sites under 20 MW, and the software that makes those sites work as one network is where the opportunity is.
Last week’s midterms were as much a vote on data centers as anything else. Democrats took both chambers and flipped the governorships of Texas, Ohio and Georgia. Hinojosa ran on blocking “unfair” data center permits. Voters in 10 states decided 41 local measures on data centers, 28 of them to limit development.
The build-out doesn’t stop. Hyperscaler capex is tracking above $800B this year, and Amazon says 2027 will be “meaningfully more.” What changes is where the fight happens: statehouses, utility commissions and town halls.
We think the election is a good prompt to ask a bigger question than which gigawatt campus gets its permit. In five years, inference will be 1,000x or more what it is today. Where does all of it run? On phones, laptops and robots? In a rack in an office closet, a 5 MW pod on a solar farm, a 100 MW regional site, a gigawatt campus? What decides which, and how does it spread across countries, cities and homes?
This post is our attempt at an answer. We built a simple model of cost per token at each kind of site and mapped today’s workloads onto them. The short version: about a third of 2031’s tokens still run on giant campuses, a quarter on regional sites, and 41% below the 20 MW line: on devices, in buildings and in small metro sites. Because the total is 1,000x bigger, even the smallest of those slices is many times all the inference running today.
That’s where we think the next set of AI infrastructure companies gets built. It may also be the first place crypto has a real job in AI infrastructure. We’ll be specific about both.
Tokens are outgrowing the power behind them
Google served 9.7 trillion tokens a month in May 2024, 480 trillion in May 2025, and 3.2 quadrillion in May 2026. That’s 330x in two years, and still about 7x in the past year alone. Our base case of 1,000x over five years assumes growth slows to about 4x a year. It isn’t even the bull case.
But power doesn’t move that fast. EPRI and Epoch put US AI power capacity at ~5 GW today and >50 GW by 2030. The IEA expects US data center electricity use to rise ~130% by 2030. Those are big numbers for the grid, but small ones vs. the token curve.
Three things close the gap:
Each watt does more work: AI chip performance per dollar improves ~49% a year, and the price of a fixed level of capability falls 10x a year by a16z’s count and a median 50x by Epoch’s.
Small models get good enough fast: open models that fit on one consumer GPU match the frontier of 6 to 12 months earlier.
A lot of the hardware is already bought: billions of phones, laptops and cars carry accelerators that nobody counts as data center capacity.
None of this shrinks demand. Every time the price of a capability drops 10x, new uses show up and total spend rises. Reasoning models went from almost nothing to over half of OpenRouter’s tokens in 2025, and programming went from 11% to >50%. Agents burn tokens the way humans never did. The question isn’t whether 1,000x shows up. It’s where it runs.
Almost every new rule starts at 20 megawatts
The election didn’t change the demand curve. It changed what it costs to build the biggest sites. Look closely and nearly every new rule targets them.
The rules already in force (New York’s executive order, Texas SB 6, FERC’s large-load orders, New Jersey’s tariff) start at between 50-75 MW. The proposals (New York’s bill, Maine’s vetoed LD 307, the Sanders and Ocasio-Cortez moratorium) start at 20 MW.
The physical queue tells the same story: a big site now waits years for grid capacity, transformers and turbines, then fights the town for its permit. A 12.5 kW node or a 1 MW pod skips that line.
Small isn’t unregulated. Urbana, Ohio voted on a 7.5 MW cap, and Monterey Park’s citywide ban carves out sub-acre sites that serve a business on the same lot. New Jersey’s tariff adds up sites under common ownership, which is how a regulator eventually catches a network of small sites. But today, a 5 MW site and a 500 MW campus live under different rulebooks, on different timelines.
Below 20 MW, you can build faster. But a fast build is worthless if nothing needs to run there. That’s the real question.
Three questions decide where a token runs
Every request passes three filters before price matters. Model size decides what can run it, latency decides how far away it can be, and data rules decide where it’s allowed to go.
Does the model fit? Weights and KV cache have to sit in fast memory. A phone holds a model of a few billion parameters. A Mac Studio with M5 Ultra holds 512GB at 1.2 TB/s, enough for a 750B-parameter MoE at low precision, and four Mac Studios have run a 1T-parameter model. A Jetson Thor gives a robot 128GB and 2,070 FP4 TFLOPS on 40 to 130 W. Serving a frontier model with long context to many users takes terabytes of HBM in one NVLink domain. That means NVL72-class racks, and those mean a big site.
How fast does the answer have to come back? A voice agent, a robot arm or a car has a budget of tens of milliseconds, and a cross-country round trip can spend most of it on its own. A coding agent can wait a second. An overnight batch job can wait hours.
Where is the data allowed to go? Patient records, a factory’s camera feeds, a bank’s code base, a government’s citizen data. Rules and contracts pin a lot of work to a building, a company or a border.
Whatever passes all three goes to the cheapest place that can run it. Put together, that gives five tiers.
Frontier training and the largest frontier models stay at the top. Almost everything else could run on more than one tier, which is why cost decides most of the split.
What a token costs at each tier
For the same job on the same chip, a small site costs 1.3x to 3x what a campus does. That premium is smaller than most people assume, and most of it is idle hardware, not expensive power.
We built a simple model to check. The same GPU slot sits in every tier: $20K, a four-year life, 2,000 tokens a second at peak. Only the building, the power price, cooling overhead, upkeep and how busy the GPU stays change by tier. Power prices come from EIA: about 18¢/kWh for homes and 9¢ for industry this year.
Four things fall out of it.
Hardware is about two thirds of the cost at every tier. So the expensive thing about a small site is a GPU sitting idle. Moving a metro pod onto stranded solar at 3¢ only takes it from about 2x to 1.8x a campus.
Utilization is the lever. A home node busy 30% of the time costs about 3x a campus. The same node kept 55% busy by a router that sends it work from other places costs about 1.7x.
Devices play a different game. The owner has already paid for the hardware, so the marginal cost is the electricity: about a cent per million tokens on a phone. Whenever a model fits on the device, it’s the cheapest place in the system.
The absolute numbers are low on purpose. They leave out networking, staff and margin; an H100 rents for a median $3.49 an hour on demand today. The ratios between tiers are the point.
A workload that needs low latency or has to stay in the building will happily pay a 1.3x to 2x premium. A workload with neither will go to the campus. That’s how we split the 1,000x.
How the 1,000x splits in 2031
In our base case, 32% of 2031 tokens run on campuses, 27% on regional sites, and 41% below the 20 MW line: 23% on devices, 9% in homes and buildings, 9% in metro pods.
We got there by sorting tomorrow’s tokens into seven workloads, running each through the three filters, then letting cost settle the rest. The shares are our judgment, anchored on what OpenRouter and Google report about usage today.
Our base case is a judgment call, so we tested what would move it most. Two things matter: how good small models get, and how easy it is to connect small sites. If small models fall further behind and small sites get hard to connect, the share of 2031 tokens running below the 20 MW line drops from 41% to 24% (the centralized case in the chart). If small models catch up faster and new campuses get delayed, that share rises to 54% (the distributed case).
The shares matter less than the size of the pie. If total inference grows 1,000x, the 9% that lands on metro pods is about 90x everything running today. Homes and buildings get about the same. Even in the centralized case, metro pods carry roughly 47x today’s total. None of these tiers has to win for it to be a very large market. They only have to exist.
Small sites only work as a network
One small site on its own isn’t much of a business. A 1 MW pod or a home node goes down, runs out of room, or sits idle when nobody nearby needs it. The cost model says the same thing in numbers: idle hardware is what makes small sites expensive. The value comes from running thousands of them as one network, so each request goes to whichever site is free, close and cheap at that moment. Call it a fleet.
The first fleets are live, shipping or in the works:
Nvidia has pitched Apollo on turning 130+ old telephone buildings, owned by Apollo’s telecom company Brightspeed, into small AI data centers across up to 20 states. Nvidia puts the spare power at ~1 GW, under 10 MW a building on average.
Akamai runs one of the world’s largest networks for delivering websites and video. It is now putting Nvidia GPUs into that network and selling inference from it across 4,400+ edge locations without building a new site, because the buildings, fiber and customers were already there.
SPAN makes smart electrical panels. With homebuilder PulteGroup, it is putting 16-GPU, 12.5 kW nodes into new homes, starting with a 100-home pilot. Each home hosts a slice of a data center, and 8,000 of them add up to 100 MW without ever asking a utility for 100 MW.
Nodiac builds modular data centers and, through a deal with PowerBank, is placing 1-20 MW pods on solar and battery sites across North America. Those sites already have land, permits and a grid connection, the parts that usually take years.
Liquid Energy ships NanoDC, a self-powered container with no grid connection, carrying its own solar panels, batteries and Starlink link. It says a unit can be running anywhere in four weeks. The first units ship this month, with thousands planned from 2027.
Each one gets around the power queue a different way. None of them solves keeping the hardware busy, and that’s where the network comes in.
A campus has one owner, one network and one power contract. A fleet has thousands of each, plus uneven power and nobody on site. Three pieces of software are missing:
Routing. Placing models and sending each request to the right site by latency, power price and what’s already in cache. This is where utilization comes from, so it’s where the margin is. Akamai and SPAN are each building their own. Nobody sells it to everyone else.
Flexibility. Emerald AI raised at a $1.05B valuation selling verified curtailment to big sites, and Silicon Valley Power now trades faster interconnection for it. Nobody does this across 10,000 small sites.
Attestation. Proving a given model ran on a given chip, with the host blind to the data. Confidential computing on Hopper and Blackwell makes it possible. Nobody has packaged it into something an enterprise or a utility will sign off on.
The third is the gate. The enterprise and vision rows in the table above only move off the campus if the buyer can trust a box they don’t own. Without attestation, distributed inference only gets the workloads nobody cares about.
Who owns the edge
The further a tier sits from the campus, the more people own it. That matters more than any share in our model, because the number of owners decides how hard a tier is to coordinate and who gets paid.
Rooftop solar is the closest precedent we have. In 2025, small-scale solar made 2.06% of US electricity against 6.54% for utility-scale, so ~25% of all solar came from roofs and small sites. Getting there took 15 years, net metering, lease financing and, later, aggregators that bundled homes into VPPs. Our base case puts homes, buildings and metro pods at about 18% of 2031 tokens, in the same range.
The analogy has limits in both directions. Compute moves over fiber in milliseconds, so a home node can serve a customer three states away; electrons from a roof mostly don’t leave the street. But a GPU depreciates in 4 years where a solar panel lasts 25, so idle compute is far more expensive than idle panels.
Geography follows the same logic. Metro pods go where spare distribution capacity, cheap or stranded power and enough nearby users overlap. Premises nodes go where people already are, which is usually where power costs the most. And outside the US, data residency rules push more inference onto regional and metro sites inside each border, especially in countries without hyperscale campuses of their own.
Where crypto earns its place
Crypto has no job on the campus or regional tiers. A few owners, long contracts and bank financing work fine there. It might have one on the tiers with thousands or millions of owners: homes, buildings, metro pods, and the devices and agents paying into them.
Bitcoin is the best proof we have that a token can build physical infrastructure. Nobody planned it. A protocol paid strangers for verified work, and they built a global industry of mining companies, data centers and power deals around it, behind an asset that has been worth more than $2T. It has run for 17 years with nobody in charge.
A fleet of small inference sites has the same three problems Bitcoin solved for hashing. Someone has to fund hardware in thousands of places, often before local demand shows up. Someone has to prove the work was done. And someone has to pay every owner, automatically, in amounts too small for an invoice. Concretely:
Paying many owners. Millions of nodes and hundreds of operators each get paid per job, often across borders. Agents paying per call for inference are the same problem from the demand side. Stablecoins fit both.
Financing hardware. Hardware is two thirds of cost per token, so the cost of capital is the biggest lever after utilization. On-chain credit against GPUs and pods is an early but real market.
Proving what ran. A buyer routing work to a stranger’s box needs evidence that the right model ran on the right data, untouched. Gensyn and OpenGradient, both Canonical portfolio companies, work on verifying machine learning work and inference.
The difference is that Bitcoin mining has no customer: the protocol itself is the buyer. Inference needs real demand, and that’s where today’s networks fall short. DePIN as a whole made ~$72M of onchain revenue in 2025, and decentralized compute earns ~$200M a year, most of its supply paid for with tokens. Generic GPU marketplaces compete with neoclouds on price and mostly lose.
The networks that work will pay hosts only for verified work and sell to customers who never touch the token. We’d drop crypto from this thesis if routing and payments consolidate into a few platforms that settle on cards, if chip vendors’ secure enclaves make attestation good enough without a public ledger, or if homes and buildings stay a rounding error. We’ll know within a couple of years.
What could make us wrong
The strongest case against us is that density wins. Industrial power, high utilization and the biggest models all favor the campus, and our own model says a campus is the cheapest place to serve any token that doesn’t have a reason to be somewhere else. Here’s what would push the world toward our centralized scenario, where only 24% of tokens run below the 20 MW line:
Small models stall. If the gap between frontier and small models widens instead of closing, agents and chat escalate to the campus and stay there.
Memory runs out. Devices are limited by DRAM, and DRAM is competing with HBM for the same fabs. Expensive memory caps what fits on a phone or laptop.
Hyperscalers build the fleet. They already run edge networks and modular builds, and could own routing before a startup does.
Regulators aggregate. New Jersey already adds up sites under common ownership. If that spreads, the 20 MW line stops mattering.
The unit economics don’t close. Retail power, truck rolls and theft across thousands of sites are real costs, and today’s numbers come from pilots.
Demand grows 100x, not 1,000x. Then campuses have room to spare, and the pressure to put tokens elsewhere fades. The workloads with latency or data constraints still move, but the market is a tenth the size.
The election doesn’t matter. Capex keeps rising, and builders move to friendlier states, federal land or the Gulf.
The last one is mostly right, and it doesn’t hurt us. The thesis doesn’t need a slowdown at the campus, only for big sites to get slower and costlier at the margin while small ones stay easy. Both are already true. Small-model progress and regulatory aggregation are the two we’ll watch most closely.
Underneath all of it, the bet is simple: the next 1,000x won’t fit in giant data centers, and whoever makes small sites work as one network gets paid for it.
What we’d fund
We’re looking for founders building the layers that make thousands of small sites, and billions of devices, work like one cloud:
Routing across tiers. Software that decides where each request runs, from the phone to the campus, by latency, power price, data rules and what’s already loaded. Whoever routes sets the utilization, and utilization sets the cost.
On-device runtimes. Compression, scheduling and model tooling that let app developers use the 23% of tokens that will run on hardware their users already own.
Proof. Attestation that a model ran where and how it was claimed, good enough for an enterprise buyer or a utility.
Power flexibility. Software that lets small sites prove to a utility they’ll cut back on demand, and get connected sooner for it.
Modular pods sold with secured power, remote-operations software and financing, not as bare boxes.
Inference hardware built for sites that can’t take a liquid-cooled training rack.
The physical AI stack. Inference for robots, vehicles and cameras, where the control loop can’t wait for a data center.
Crypto-native rails for paying hosts, financing GPUs and settling agent payments, with a product people would buy without the token.
If you’re building any of this, we’d love to talk.






