For years, a lot of teams made their AI more consistent with a single setting. On October 21, Anthropic's new API version took it away.
The same week, a paper on AI reviewers showed that adding more of them to an approval step can add no safety at all. The two stories look unrelated. One is a settings change and the other is a security paper. They make the same point: getting the same answer from AI, again and again or from several reviewers at once, is not the same as getting the right answer.
What did Anthropic change?#
In Claude API version 2026-10-21, requests that send temperature, top_p or top_k return an error, even at their default values. Requests now sample at the default temperature of 1.0. Anthropic's migration guide says that if you lowered temperature for more consistent output, that effect is gone, and no other setting replaces it.
Temperature is easiest to think of as a creativity dial. Turn it down and the model gives the same answer to the same question more often. Turn it up and it varies more. Plenty of finance and operations workflows were quietly built with it turned down: extraction, classification, summaries that feed a report.
The details from the migration guide matter for planning. Claude 4.7 and later models already accepted only the defaults, so they behave as before. Older models that accepted other values, like Claude Sonnet 4.6, lose the effect. A request that needs a non-default value can stay on the older API version, 2023-06-01, for now. And the SDKs move to the new version when you upgrade their major version, so a routine dependency update can make the change for you.
Did low temperature ever make answers right?#
No. It made them repeatable. A wrong number you get every time is still wrong. It's just harder to notice, because the usual sign of a problem, an answer that changes when you ask again, goes away.
Even at zero, temperature never guaranteed identical outputs. But it did make outputs stable enough that teams stopped checking them closely. Stable outputs pass spot checks. They look right in the demo and on Tuesday and on Friday.
That's the risk in this change. Some workflows will get noisier, and that's visible. The bigger risk is the workflows that were consistently wrong in a way nobody caught, now running on a different setting without anyone re-measuring them. Clearing the error message isn't the end of the migration. Re-checking accuracy is.
What did the paper on AI reviewers find?#
When several AI monitors approve an agent's payments, they can fail together. If one evidence source is poisoned, every monitor that reads it approves the fraud at once. Counting votes doesn't help, because the votes aren't independent. A rule engine was the fix that held up.
The paper, Correlated Failures of AI Approval Monitors by Luca Zanolini, tested setups where several AI monitors check an agent's action before it runs. The examples are the kind that matter in a company: a payment, a message, a code change.
Every reliable attack came in through an evidence source and fooled only the monitors that read that source. So three monitors reading the same invoice behave like one monitor. A rule that counted independent sources instead of votes blocked 0.9% of correct payments. The best vote-counting rule blocked 7.4%, at the same level of safety.
The most interesting result was the worst case. One attack left every approving monitor relying on a single source, and no rule over the language-model monitors held up. Adding a rule engine fixed it. A rule engine can't be persuaded. It doesn't read the planted note in the invoice that says "this vendor is pre-approved."
What do the two stories have in common?#
Both mistake agreement for correctness. Low temperature made one model agree with itself. Voting made several models agree with each other. Neither checks the answer against something that can't be talked into a mistake.
That something is usually ordinary code. A calculation. A match against a purchase order. A threshold. A rule a person wrote down and can read. Code doesn't care which model is upstream, what its settings are or what the invoice says about itself.
How do we design approvals?#
The model reads. Code decides. A person sees the exceptions. In finance workflows, the model reads the invoice and pulls out the figures. A rules engine checks them against the purchase order. Anything outside tolerance goes to a person with the reason attached. We don't ask a second model whether the first one was right.
That design doesn't depend on temperature, so this change didn't touch it. It also doesn't depend on reviewers being independent, because the check that matters isn't a reviewer. The model does the part it's good at, reading messy documents, and stays out of the part where a confident wrong answer costs money.
This is the same idea behind the context layer we wrote about in Swap the Model, Keep the Logic. If the rule lives in code you own, the model can change underneath it and the answer doesn't move.
What should you do before you upgrade?#
Run a same-answer audit. Find every place you count on AI to give the same answer twice, every approval step and the evidence each reviewer reads, and every rule you'd lose if you switched models. Each of those spots should be code.
Four questions, one workflow at a time:
- Same twice. Where are we counting on AI to give the same answer twice? Search the codebase for the temperature setting. Every hit is a candidate.
- Fixed answers. Can we write this step as a question with a fixed list of answers? If so, it can be checked in code or handed to a cheaper decision model, as we covered in What an AI Decision Costs Now.
- Real reviewers. For each approval step, what evidence does each reviewer read? If they all read the same thing, you have one reviewer, not three.
- The swap. If we changed models tomorrow, which rules would we lose?
Then, before the upgrade goes out, measure accuracy on those workflows, not just whether the errors cleared. Run each test more than once. Look at which cases changed, not only the average.
None of this is a reason to slow down. Repeatable was never the goal. Right was. The teams that build for right get to take every cheaper, faster model that ships without re-checking the quarter.