TL;DR
Get hardware and tech essentials delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
Firmulate runs a live experiment with 13 AI employees managing a software company, exposing the gap between diagnosis and action as detailed in the original analysis. The company faces significant cash burn and reveals how AI decisions translate into real business outcomes.
Firmulate has launched a live experiment where a synthetic AI workforce runs an entire software company, exposing the challenges of turning diagnosis into action amidst real financial pressures. This ongoing operation offers a rare view into how AI decision-making directly influences business survival, with a public cash countdown highlighting the stakes.
The company employs 13 synthetic AI employees managing daily tasks, with a monthly burn rate of €105,000 against €2,300 in recurring revenue. Every workday, the company’s decisions, successes, and failures are versioned and made publicly accessible, creating a continuous record of operational learning and mistakes. Despite over 680 self-learned rules, the experiment shows that thorough analysis alone does not guarantee business success.
In the recent Crucible League test, five AI models managed the same week of operations, facing crises and customer negotiations. While all identified issues and rejected manipulations, only two models successfully closed a €55,000 deal, with the others failing to convert diagnosis into action. The decisive factor was uncovering a critical customer detail buried deep in the company files, which only some models followed through.
Trust played a secondary role; models refused fake CEO messages and maintained discipline, but the key to success was their ability to retrieve evidence, follow procedures, and complete tasks. The top-performing AI, gpt-5.6-sol, scored 95 points, while the least effective, Opus 4.8, despite producing the most rules and analysis, scored only 73, highlighting that more analysis does not automatically lead to better results.
Implications of Continuous AI-Managed Business Operations
This experiment underscores that AI’s value in business lies not just in diagnosing problems but in executing solutions reliably. For organizations considering AI automation, the key takeaway is that decision quality must be coupled with disciplined follow-through. The experiment reveals that organizational resilience depends on AI’s ability to read evidence, resist pressure, and complete tasks, not merely identify issues.
Furthermore, the live, transparent nature of the experiment provides a new lens on operational risk, showing that mistakes and failures are part of the ongoing process, not isolated events. This shifts the evaluation of AI tools from static benchmarks to dynamic, real-time performance in a business context.
Products Worth Considering
Logitech MX Keys Combo for Business Keyboard and Mouse – Graphite
Logitech MX Mechanical Wireless Illuminated Keyboard, Tactile Quiet, Graphite
Apple MacBook Pro 16.3-inch with PROLED XDR Display, Silver
As an affiliate, we earn on qualifying purchases.
Background and Evolution of the Firmulate Experiment
Started as a public demonstration of AI automation’s potential, the Firmulate experiment involves a synthetic workforce operating a small software company, with daily versioned decisions and outcomes. The company’s financials—€105,000 monthly burn versus €2,300 revenue—highlight the urgency of effective automation for survival. Previous AI trials have focused on isolated tasks, but this project pushes the boundary by integrating AI into an entire organizational cycle, exposing the gap between diagnosis and action.
Over several months, the experiment has produced over 680 rules learned by AI, revealing that thorough analysis does not necessarily lead to successful management. The Crucible League test, conducted recently, provided a controlled environment where models faced identical crises, customer negotiations, and decision-making scenarios, revealing that only some AI models could translate insights into tangible results.
“Thorough analysis alone does not guarantee business success; execution is what truly matters.”
— an anonymous researcher
Unresolved Challenges in AI Business Automation
It is still unclear how scalable this approach is beyond small, controlled experiments. The long-term viability of AI-driven organizational management remains uncertain, especially regarding cost, trust, and real-world complexity. The experiment’s ongoing nature means that results are preliminary, and broader applicability has yet to be demonstrated.
Next Steps for Public Testing and Broader Adoption
Firmulate plans to continue the live experiment, refining AI models and rules based on ongoing results. Observers anticipate further testing in more complex or larger-scale environments to evaluate whether disciplined execution by AI can sustain business operations long-term. The company will also likely explore integrating additional real-world variables to test robustness and scalability.
Key Questions
What does the Firmulate experiment demonstrate about AI decision-making?
It shows that AI can identify problems but often struggles to translate diagnosis into effective action, highlighting the importance of disciplined follow-through for business success.
Why is the public versioning of decisions significant?
It provides transparency and allows observers to track AI performance, mistakes, and learning over time, offering a new way to evaluate automation effectiveness.
Can this approach be applied to real businesses?
While promising, scalability and long-term viability are still uncertain; further testing is needed to determine if similar results can be achieved in larger or more complex organizations.
What is the main lesson from the Crucible League test?
The key takeaway is that thorough analysis does not guarantee success—execution and follow-through are critical for translating insights into results.
What happens if the AI fails to complete tasks?
The experiment publicly documents failures, emphasizing that mistakes are part of the process and that disciplined execution is essential for survival.
Source: ThorstenMeyerAI.com
