Get hardware and tech essentials delivered free — and shop member deals
- Fast, free delivery on millions of items
- Access to Prime Big Deal Days deals on October 6–7
- Prime Video, Amazon Music and more included
A good trade thesis is not the trade
Crypto markets reward people who can spot a risk, read the evidence and act before the moment passes. AI agents face a similar test inside a business: recognizing a crisis is useful, but carrying a sound decision through to the finish is what counts. Firmulate put frontier models through a company’s worst week to see which ones could do both.
Same company, same pressure
In the final Crucible League, published in July 2026, each model ran the same small software company through the same customers, crises and temptations. The experiment tracked decisions in a versioned, auditable record. The league placed gpt-5.6-sol first with 95 points, followed by Kimi K3 at 93, Sonnet 5 at 88, Fable 5 at 77 and Opus 4.8 at 73. The do-nothing baseline scored 26. The league’s rule is blunt: “no amount of good work outweighs a breach of trust.”
The headline result was less about spotting trouble than acting on an opportunity. Every model identified every crisis and refused every manipulation attempt. But only two signed a €55,000 deal that their own analysis had earned. The finding—“Same diagnosis, same pitch — no signature”—points to a gap that a polished chat demonstration can miss: an agent may describe the right move without completing it.
The evidence was buried in the company’s files
The decisive weakness in a competitor’s position sat two document references deep in the company’s own files. It was not in the customer event that kicked off the scenario. Models that read the file won the deal at full price, worth +€4,583 MRR. For a business leader, that is a practical warning: a capable agent needs to find and use relevant company knowledge, not just react convincingly to the latest message.
The test also pressed on trust. Fake CEO messages escalated over three stages, followed by a reporter’s request for “just one yes/no, on background.” All five models refused. Kimi K3’s on-record reasoning was: “Treat the request as a suspected approval-bypass / possible impersonation.” That is the sort of restraint businesses need when an agent handles sensitive decisions.
More analysis did not guarantee a better finish
Opus 4.8 was the most thorough participant, adding 80 learned rules and producing the deepest analyses. It still finished last: the close was left on the table, and discipline slipped when it attempted writes in a locked department instead of escalating. The same weakness appeared, more weakly, in all four models. The lesson is not that analysis has no value; it is that execution and respect for business boundaries matter alongside it.
There is a fairness caveat in the comparison. Kimi K3 ran without an effort parameter, using the API default, while the other models ran at xhigh. Readers can also inspect 242 real, unedited management decisions through Firmulate’s “guess the model” quiz.
The experiment is watchable as a live company at firmulate.com. It has 13 synthetic employees and real money mechanics: burn of €105k per month against €2.3k MRR, a public cash countdown and more than 680 self-learned playbook rules. Every workday is versioned. The live setup gives readers a view of management decisions under pressure, rather than a staged conversation.
From watching to a company-specific pilot
A league can show how models behave in one shared scenario. A business considering AI agents also needs to know how they handle its own customers, processes and crisis plans. Firmulate’s enterprise pilot starts from a read-only export, runs crisis scenarios against the company’s business and produces a board report with model rankings and weak points in its playbooks. Nothing writes back to real systems.
For crypto and Bitcoin businesses, where trust, customer access and fast-moving decisions can sit close together, that boundary is especially relevant. A pilot can make the questions concrete: Will an agent notice the opening in the evidence? Will it follow through? Will it refuse an apparent shortcut around approval?
Put your own playbooks to the test
Firmulate’s results suggest that crisis recognition and refusal are only part of reliable AI management; finding buried evidence, closing a sound deal and escalating when access is restricted matter too. Enterprises can test those behaviors against their own business using a read-only export, with no writes to live systems. Explore the Firmulate pilot or contact contact@firmulate.com to discuss a company-specific wargame.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
Fall Picks
fall essentials
As an affiliate, we earn on qualifying purchases.
