Price what you'd own before you switch to open weights.

Someone has already forwarded the link

On Sunday, October 4, Axios reported that Reflection, the Nvidia-backed startup, is getting ready to release its first open-weight model. According to the report it's expected soon, it should trail the best US closed models at first, and it's pitched as a cheaper alternative to buying from Anthropic, OpenAI and Google. Axios also said other Western open-weight releases are coming this month.

What hasn't been published yet matters just as much. As Siften pointed out, there's no model name, size, benchmark, license, or release date on the record. So nobody can tell you today whether this particular model is right for you.

But somebody on your team has probably already dropped the link in a channel with "should we switch?" underneath. That's a fair question, and you can do most of the work to answer it before any model ships. It starts with being honest about what "free" means.

The download is the cheap part

When you call a hosted model through an API, you're paying for a lot more than the model. The vendor runs the hardware. They scale it when your traffic spikes. They patch the serving software, keep it up, and take the call when it's slow. When they ship a new version, they decide when the old one goes away, which is annoying, but it's also their problem to run.

When you download open weights, all of that comes with you. Somebody has to pick where it runs, whether that's your own GPUs, a cloud provider, or a hosting company that serves open models. Somebody has to set up the inference server, decide how many requests it handles at once, and notice when latency creeps up at ten in the morning. Somebody has to choose which version you're on, and keep you on it until you've decided to move.

None of that is exotic. Teams do it every day. But it's work, it has a cost, and it doesn't show up when you compare the price per million tokens on a pricing page with zero.

The weights are the cheapest thing you'll own. Everything around them is the bill.

Write down what changes hands

Before anyone runs a benchmark, make a list with two columns. On the left, what your API vendor carries for you today. On the right, who on your side would carry it after the switch. Be specific about names. "The platform team" is not a name.

The list usually includes:

• Serving and scaling. Where the model runs, how it handles your busiest hour, and what happens when a GPU node disappears.
• Version control. Which exact weights, quantization, and settings are in production, and how you roll back.
• Evaluation. Every time you change the model, the quantization, or the prompt, you re-run your tests. With an API you could lean on the vendor's release notes. Now the release notes are yours.
• Safety and abuse handling. Hosted APIs usually ship with some filtering and monitoring. With weights, you decide what to add.
• License compliance. Open-weight doesn't always mean "do anything." Some licenses restrict certain uses, require attribution, or add conditions as you grow. Reflection's terms aren't public yet, so read them the day they are.
• On-call. When it's slow at a bad moment, whose phone rings?

If the right-hand column is mostly blank, you're not ready to switch, whatever the model scores.

Price the job, not the token

The useful comparison isn't cost per token. It's cost per piece of work your business accepts.

Pick one real job the model does for you today: summarizing a support thread for the next agent, sorting an order exception into the right queue, drafting a product description someone reviews. Build a small set of real examples with the answer you'd accept for each. Then run the hosted model and the open-weight candidate against the same set.

Count what it costs to get an acceptable result, not just to get a result. If the cheaper model needs a person to fix one answer in five where the current one needed a fix in twenty, the human time can eat the savings fast. We wrote about this kind of test in Score the job before you buy the model. The same approach works here, with the running costs from the last section added in.

You don't need a big program to do this. One job, a few dozen examples, and a spreadsheet of results will tell you more than any leaderboard.

When owning it is worth it

There are good reasons to run your own model, and they're usually not about the token price.

Your data can't leave. If contracts, regulators, or your own customers mean prompts and documents have to stay inside your environment, open weights may be the cleanest answer. Reflection's own pitch leans this way, toward institutions running their own AI on their own data and compute.

Your volume is steady and high. Paying for capacity you use all day can beat paying per call. Spiky, unpredictable traffic usually favors the API.

You need to change the model itself. Fine-tuning on your own examples, or controlling exactly when the model changes, is much easier when you hold the weights.

And there are good reasons not to. If you have one or two low-volume jobs, no one who wants to run inference infrastructure, and no data restriction pushing you, the API is probably still the better deal. That's not a failure of nerve. It's a sensible division of labor.

Try it on one job before you move everything

If the numbers look promising, don't flip everything at once. Put a thin layer between your application and the model so you can send one job to the open-weight candidate while everything else stays put. Run them side by side for a few weeks. Keep the hosted model as your fallback until the new one has handled a bad day.

That layer is worth building even if you never switch. It means the next headline, about Reflection or anyone else, is a test you can run in a week rather than a migration you have to plan.

This is the kind of decision we help with in AI applications and agents: which jobs a model should own, what it costs to run them properly, and how to keep the choice reversible. If an open-weight release has your team asking whether to switch, bring us the job you'd move first.

Related work: Howcast.