Why aren't open-source models a hundred times cheaper? Let's explore the Pareto frontier.

To start with, I'd separate two economic layers in an API price.

The first is running the model. GPUs, memory, interconnects, power, the datacenter, software, and the people keeping it all running. Plus compensation for the provider that handles all this and bears the risk of unused capacity. In this respect, model serving is a fairly conventional cloud business.

The second is developing the model and the value of its weights. Someone paid for research, data, training, and post-training. With a closed model, I buy access to the result from its owner or a partner. With an open model, I may have the right to use the weights without an ongoing license payment, if its license allows it.

That doesn't mean developing the open model cost nothing. It means I don't have to pay for it in the same way. Openness can remove a license payment for the weights. It doesn't remove the work the hardware has to do when I use the model.

Besides, I usually won't see these two layers itemized on the invoice. An API price doesn't have to be "the cost of generating tokens plus a margin for the weights." Competition, capacity, business strategy, and efforts to attract customers all affect it. It can even be below cost temporarily.

I think economies of scale will play a crucial role in the operational layer. A large provider has demand from multiple applications and time zones. It can combine requests into larger batches, make better use of caching, and spread the cost of serving specialists across enormous volumes.

In our previous measurements, going from one to fifty concurrent requests with caching gave us about 5.5 times the aggregate capacity. But each request took longer. And alongside batch size, I also need utilization over time: ideally, work that approaches continuous operation, not a GPU waiting every evening for morning to come.

That doesn't mean self-hosted inference has to run exactly 24×7 or it can never pay off. It means its economics are determined by the amount of useful work at an acceptable latency.

Providers of both open and closed models can take advantage of these benefits. Removing their margin doesn't automatically give me their operating costs.

This, I think, is where much of the talk about hundredfold savings comes from. I take a relatively small open model and compare it with an expensive frontier model. The difference on the price list can be enormous. But I haven't just changed the license. I've also changed what I'm buying.

Just to be clear about the units: B means billion, T means trillion. A 2T model therefore has two trillion parameters.

These aren't imaginary size categories detached from reality. According to its published model card, Kimi K3 has 2.8T total parameters and 104B active per token. With Mixture of Experts, active parameters have a major effect on compute requirements, but all the weights still have to fit somewhere. And it's not just about memory capacity: memory bandwidth when loading the necessary weights and working with the cache is another crucial limit. That doesn't mean I load every expert for every token.

That's also one reason why, unlike home use, large-scale serving benefits from disaggregating prefill and decode. Processing the input tends to be more compute-bound, while generating tokens one by one is more constrained by memory and its bandwidth. A provider can send each phase to differently configured hardware and scale them independently. But transferring the KV cache between them has to be worth the cost, too.

Nor can I increase sparsity — how small a fraction of the experts I activate — arbitrarily without affecting capabilities. Experiments with MoE and reasoning show that more parameters at high sparsity help with knowledge storage, but reasoning also needs enough active computation and training data per parameter. So what's efficient for knowledge isn't necessarily efficient for reasoning. This isn't a universally optimal ratio for every model, but it is a good reason why simply adding more and more inactive experts isn't enough.

From single-digit billions to single-digit trillions, we're looking at roughly three orders of magnitude. Size isn't a precise price list or an automatic measure of quality, though. Architecture, training, quantization, and the specific task all matter. A smaller specialized model can outperform a larger general-purpose one on my task.

So I can't take a hundred times as many parameters and declare that inference costs a hundred times as much. But I'm still comparing significantly different hardware and capability classes, not the same product with its license price tag peeled off.

A cheap small model can be a great choice. It just doesn't need open weights to be one. Cheap closed-weight models exist too — and I mustn't forget to include those in the price comparison.

At the very top of this chart snapshot are the closed models Astra and Fable. A little further down, we already find very capable open models such as GLM-5.3 and Kimi K3.

My working hypothesis is that a team at the technological frontier can also turn some of its lead into efficiency: achieving a certain quality with less computation, shorter reasoning, or better-optimized serving. A closed model might then need less hardware for comparable work while charging me more for access to its weights.

What sort of margins are we even talking about? Media estimates for 2025 put a leading model provider at around 50% gross margin. Reports from September 2026 already talk about more than 80%, but before distribution partners' shares and training costs. I treat these as ballpark estimates, not an audited time series for a single API; Reuters also did not independently verify the September report.

For context on the cloud side, we can look at AWS's 2025 revenue of $128.7 billion and operating income of $45.6 billion, or an operating margin of about 35%. But that isn't the gross margin on renting out a single GPU. I can't simply subtract these differently defined percentages and call the difference the price of the weights.

So for a back-of-the-envelope calculation, I'll use 50% or 80% as the total modeled margin of the closed service, including compensation for serving and the model premium. For the open alternative, I'll keep 30% for serving. This isn't a reconstruction of any particular company's accounts: if it buys compute from a cloud provider, some infrastructure margin may already be included in its costs. Here I'm calculating both options from the same cost basis without counting anything twice.

The direct inference cost of the same work will be 100 units for the closed model. In our example, the open alternative will cost 15% more, or 115. A 30% provider margin means costs make up 70% of the selling price: the open API therefore costs 164.29 units, not 149.50 as it would with a simple 30% markup.

How do I separate the two components in this model? At a direct cost of 100, serving alone with a 30% margin would cost 142.86: a hundred for inference and 42.86 as the operator's compensation. That leaves 57.14 of the final 200 as the model premium; at 500, it leaves 357.14. This isn't pure profit on the weights — it also funds development, training, and other costs.

The 15% efficiency difference and the 30% serving margin remain assumptions in this example, not measurements of a specific endpoint. If both options were equally efficient to run, the price ratio would be 1.4× to 3.5×. With our efficiency difference, it's approximately 1.2× to 3×. For a company, that could be a worthwhile saving, but we're still nowhere near the order of magnitude of 100×.

When I compare models through APIs, the price per million tokens isn't enough. I need to know how many tokens they'll use for the same work. Including reasoning, input, repeated context, and any additional attempts.

A very simple example: one model's output tokens cost half as much, but it needs three times as many. Its output alone then costs me 50% more. I'm leaving input and caching aside for now.

With Kimi K3, this isn't just an academic question. In the Artificial Analysis data available while preparing this article, Kimi K3 at max settings uses approximately 160 million output tokens for an Intelligence Index run, while GPT-5.6 Sol at max settings uses about 90 million. Yet the listed output price for Kimi is $15 per million, versus $20 for Sol. Cheaper tokens, but higher consumption; the weighted cost per benchmark task comes out to around two dollars for both.

Compared with Opus 5 at max settings, which is listed at about 140 million output tokens, the consumption difference is much smaller. So I wouldn't make a blanket claim that Kimi needs several times more tokens than all closed models. It depends on what I compare it with and at what settings.

Nor can I work out the chart's horizontal axis just by dividing these aggregate token counts. Cost per Intelligence Index Task is a weighted average of the per-task costs of individual evaluations. It accounts for their weights in the index and the relevant billable tokens — input, cache, reasoning, and the answer.

That's why I like this chart more than a price list on its own. But it still shows the cost of a benchmark task, not the cost of a successfully solved business case. Quality is on the other axis. If a cheap agent messes up the task, its tokens still get billed just fine, but my work still isn't done.

And with self-hosting, I have to replace the API price with my own full operating cost. This picture alone doesn't tell me what Kimi would cost on my cluster.

Coming back to the picture, I see three important things.

The very highest level of capability isn't automatically available in an open version. Astra and Fable are at the top, scoring around 53. GLM-5.3 and Kimi K3 are around 44–45, Sol around 47, and Opus 5 around 51. That doesn't mean an open model can't handle a specific task just as well or better, but this is how the benchmark comes out. Your use case can of course be different — though I don't expect dramatically different results.

A little further down, open models really do compete, but I don't see an automatic hundredfold discount. Kimi K3, GLM-5.3, and Sol are all roughly around two dollars per task here. Opus costs around six, but it also scores higher. Choosing Kimi instead of Opus may be the right decision. In this chart, though, it's a tradeoff between price and quality, not proof of the same result at a third of the price.

The cheap end of the market isn't reserved for open models. GPT-5.6 Luna at medium settings sits all the way to the left, at approximately $0.016 per task. That's a price difference of more than a hundredfold compared with Sol — within the range of a single closed-model vendor. But Luna's score here is around 25.5, not Sol's 47. Meanwhile, the open GLM-5.3-Flash is an interesting point on the frontier at around $0.25 and a score of 42. Careful: Flash isn't the same variant as GLM-5.3 max from the previous paragraph.

It's interesting to see how some vendors are doing as of the day I wrote this article. OpenAI has a very well-balanced portfolio, from small models (Luna, Terra) to large, highly capable ones (Sol, Astra). Anthropic can excel at the high end, but its more affordable variants can't make the Pareto frontier today (Haiku is way off, Sonnet isn't great, and Opus is also running out of steam on price optimality). Google is currently in a position where its high end, Gemini Pro, is out of the running, but the midrange Gemini 3.8 Flash is on the Pareto frontier, while the lower-end 3.5 Flash-Lite is currently out of the running.

For me, this is exactly what breaks the shortcut of "open equals cheap, closed equals expensive." Both groups occupy interesting positions. Some other models have an alternative that's both cheaper and more capable on the two criteria we're tracking, regardless of the license.

I'd use this chart as an excellent starting point for choosing a model, not the end of the process. The Intelligence Index combines different types of tasks and their weights. But my application might spend all day doing one thing that a smaller model is excellent at.

I'd start with my own examples and a clear bar: what the model must handle and what counts as an error. Then I'd compare several cheap and more capable models, both open and closed, including sensible reasoning settings. Alongside the results, I'd measure the entire bill for completed work: every attempt, tool, possible fallback, and human correction.

This shows once again why our own evaluation suites and training environments (RLE) are so important for the future, as I already wrote in my article about evals as the most valuable asset.