Frontier AI You Can Own: The Kimi K3 Math
There's a lot of hoo-ha about the Kimi K3 release, so I broke it down.
The headline: Moonshot says the weights land Monday, July 27. From that day, you have theoretically "unlimited" access to an Opus 4.8-class model. Frontier AI becomes something you can own and tweak, not just rent.
The asterisk on "unlimited" is the whole story. Let's price it.
The Three Ways to Consume It
Own it. K3 is 2.8T parameters, shipped natively in 4-bit: about 1.4TB of weights. A single B300 (288GB) can't hold it; an 8x B300 node can, and that's one serving copy at roughly $400–500K of hardware before opex. An enterprise floor with failover is 16x. A fleet for real product traffic is 64x, $3–4M.
Rent it. This tier splits in two, and the split matters. Server rental is per GPU-hour (EC2, the neoclouds): the hardware is yours to load, and so is the idle time. That same 8-GPU node runs $230–470K a year reserved, at $3.27–6.71 per GPU-hour, and the 64-GPU fleet is $2–4M a year. Renting for a year costs about what buying does; that IS the neocloud business model. Serverless rental is the other sub-option: billed per active capacity-time, autoscaled, with the provider pooling idle across tenants and charging a premium for the privilege. AWS Bedrock's custom model import, for instance, bills per active model copy per 5-minute window, not per token.
API it. Moonshot's per-token pricing is substantially cheaper than the closed frontier, before caching and batch discounts narrow the gap. And the API tier turns out to be host-agnostic: hyperscalers already serve open models per-token (Kimi K2 sits on Bedrock today), which is still an API, just with a different counterparty and enterprise wrapping.
The through-line: the billing unit tracks who bears idle risk. Per GPU-hour means you do. Per token means they do. Capacity-time is the negotiation in between.Here's the full grid:
The Break-Even
One 8x B300 node costs $685–959 a day, depending on whether you own (3-year amortization plus opex) or rent (reserved rates). Divide by roughly $9 per million tokens of blended closed-frontier list pricing and the crossover lands at 75–110M tokens a day. Against Moonshot's own API the bar is about 1.7x higher.
Translated into humans, at ~100K tokens per heavy chat user per day:
| Config | Own | Rent / year | Breaks even around |
|---|---|---|---|
| 8x B300 (one copy) | $400–500K | $230–470K | ≈750–1,100 daily users |
| 16x B300 (HA pair) | $850K–1M | $460–940K | ≈1,500–2,200 daily users |
| 64x B300 (fleet) | $3–4M | $1.8–3.8M | ≈6,000–14,000 daily users |
Why This Is a Big Deal for the American Model Giants
Not using them now has a visible price. Self-hosting starts to pencil around 100M tokens a day of steady load; a threshold one node can serve. And tweakable weights is a strategic argument, not just a financial one: fine-tuning on your own data, in your own walls, is something no closed API sells you.
That pressure lands on recurring API revenue at the exact moment these companies are raising.
Why the Giants Still Sleep Fine at Night
You're not buying a model, you're buying a system. Harnesses, tool scaffolds, caches, retrieval, safety and enterprise controls. Self-hosting strips those away, and rebuilding them becomes your job.
Reliability is a product. SREs, outages, power, parts, maintenance: all yours now. One node is an outage away from zero; real production means redundancy, which doubles the bar.
Models have a shelf life, and it isn't long. 3 to 6 months at the frontier. Every swap breaks prompts, integrations, and tests, and fine-tuning at this scale runs hundreds of thousands to low millions. The cost of keeping up is yours to bear, or you're serving 2024 intelligence to 2026 customers.
Fine Print & Open Questions
Is 4-bit a quality haircut? No. K3 is quantization-aware trained in MXFP4 from the supervised fine-tuning stage. Four-bit is the shipped format, and the published benchmarks already reflect it. This is not a community squeeze of a bigger model.
Why 8x, 16x, and 64x? 8x B300 gives 2.3TB of HBM against 1.4TB of weights: one serving copy with KV-cache headroom. 16x is what production actually requires, because one node is an outage away from zero. 64x matches Moonshot's own serving recommendation of "at least 64 accelerators" (which is written for 80GB-class cards, where 64 is the minimum; on 288GB B300s it buys you a fleet of replicas).
Where does serverless fit? Inside the rent tier, as the capacity-time sub-option, and at K3 scale it carries an asterisk: cold-loading 1.4TB of weights per request is a challenge, so scale-to-zero mostly collapses back toward keeping a server warm. Related: true per-token pricing on heavily tuned models doesn't exist to my knowledge. Thin LoRA-style tunes riding shared fleets are the exception, and only because the base weights are shared across tenants; heavy customization forces capacity-time billing. The day someone serves your fully custom multi-terabyte model per-token, this tier map gets redrawn.
How solid are the serving numbers? The dollar figures are list prices and published rental rates. The concurrency figures are modeled: memory-bandwidth-bound decode with batch serving. Nobody outside Moonshot has served K3 yet, and K3 currently runs reasoning at "max" only, which can cut effective throughput 3–10x on reasoning-heavy traffic. Treat that column as estimates, which is exactly how the graphic labels it.
License? Modified-MIT expected; final terms ship with the weights.
Big Picture
Self-hosting still won't make sense for most startups. But per-token spend shifting toward open models? That can move fast, depending on how the regulation plays out.
Updated July 23: split the rent tier into server rental and serverless rental, and added the idle-risk through-line, after good pushback from my Wharton AI peers.