📊 Full opportunity report: Engineering Is Automated. Research Is the Residual. on ThorstenMeyerAI.com — validation score, market gap, and execution plan.
TL;DR
Recent evidence indicates AI has achieved near-complete automation of engineering tasks in AI R&D. However, the extent to which AI can automate research itself remains unclear, raising questions about future innovation processes.
Recent developments in AI capabilities demonstrate that AI systems can now automate the core engineering tasks involved in AI research, with benchmarks nearing saturation. However, the question of whether AI can fully automate AI research itself remains unresolved, making this a pivotal moment for understanding the future of AI-driven innovation.
Multiple benchmarks tracking AI performance on core AI R&D skills—such as research reproduction and Kaggle competition performance—show rapid progress towards saturation. For example, the CORE-Bench, which measures the ability to reproduce research papers, reached a 95.5% success rate by December 2025, with some authors declaring it ‘solved.’ Similarly, the MLE-Bench, assessing AI on Kaggle competitions, hit 64.4% in early 2026, surpassing mid-tier human performance.
These benchmarks indicate that AI can now handle many engineering tasks involved in research, such as reproducing experiments, optimizing kernels, and managing dependencies, at a level comparable to competent researchers. The pattern across these metrics suggests that engineering automation is nearing completion, with the primary bottleneck shifting from capability to strategic and creative aspects of research.
However, the critical unresolved question is whether AI can automate the creative and hypothesis-driven parts of research—such as formulating novel ideas, designing experiments, and interpreting results—beyond engineering replication and optimization. Experts like Thorsten Meyer highlight that while engineering is approaching full automation, research may involve distinct cognitive processes that remain resistant to automation, at least for now.
Engineering is automated.
Research is the residual.
Six skill benchmarks. Edison’s framing. The question Clark leaves open is whether research is just engineering at scale.
Jack Clark’s Import AI #455 catalogs six benchmarks measuring AI capability on AI R&D tasks and concludes “AI can today automate vast swatches, perhaps the entirety, of AI engineering.” The residual question is research. The structural read on the residual: it may not be a permanent moat.
Six skills. One trajectory.
Clark catalogs six benchmarks measuring AI capability on AI R&D-relevant tasks. Each individual benchmark could be noise. Six benchmarks moving together is a curve. The pattern is the cascade observed across the broader Clark series — visible here in the specific R&D-skill domain.
CLAUDE AI UNLEASHED From First Prompts to Pro: The Complete Guide to Claude AI for Writing, Research, Coding, and Business (The Claude AI Mastery Series)
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Three data points. Mixed signal.
Clark provides three data points on the creative-spark question. Yes-evidence: Erdős-1051, centaur math discovery, sporadic Move-37-style moments. No-evidence: low yield, framing dependence, absence of acceleration. The mixed signal is the honest read.
The data supports two readings. Pessimistic: rare moments suggest creative insight is qualitatively distinct from engineering work. Optimistic: rare moments are an artifact of low-volume exploration; more shots on goal yields more discoveries. Both readings are consistent with Clark’s “vast swatches, perhaps the entirety” claim. They differ on the residual.
Innovation in Music: Current Research Perspectives (Perspectives on Music Production)
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Five dimensions Clark gestures at but leaves underdeveloped.
Clark’s section is rigorous on the empirical evidence. Five strategic dimensions matter for the institutional response that the Clark series synthesis argues is structurally inadequate.
Architecting Data and Machine Learning Platforms: Enable Analytics and AI-Driven Innovation in the Cloud
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Two readings. Different equilibria.
The structural question Clark leaves open: is research a permanent moat that bounds automated AI R&D, or is it engineering at scale that dissolves with more shots on goal? Both readings are consistent with the current data. They differ by orders of magnitude in consequences.
Productivity multiplier years
Recursive loop operational
AI benchmarking tools for research
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Five audiences. Asymmetric cost of being wrong.
The institutional response should not bet on inspiration being a permanent moat. If the distinction holds, capacity built is still useful. If it closes, capacity is necessary. Asymmetric cost-of-being-wrong points toward building now.
IN INDUSTRY
IN ACADEMIA
POLICYMAKERS
INVESTORS
EVERYONE ELSE
Engineering is automated. The residual is the question. The institutional response should not bet on inspiration being a permanent moat.
Implications for AI Development and Innovation
The near-complete automation of engineering tasks could dramatically accelerate AI research and development, reducing costs and timeframes for new breakthroughs. If research itself remains partly human-driven, institutions may need to rethink how they allocate human talent, focusing more on strategic and creative roles. Alternatively, if AI advances enable full automation of research, the pace of innovation could increase exponentially, fundamentally altering the landscape of scientific discovery and technological progress.
Progress in AI Capabilities and Benchmark Saturation
Over the past two years, multiple independent benchmarks—such as CORE-Bench and MLE-Bench—have shown rapid improvements in AI capabilities relevant to AI R&D. The CORE-Bench, measuring research reproduction, improved from 21.5% in September 2024 to 95.5% in December 2025. The MLE-Bench, assessing Kaggle competition performance, moved from 16.9% to 64.4% over the same period. Additionally, advances in kernel design and infrastructure automation indicate that AI is becoming capable of handling increasingly complex engineering tasks.
This pattern suggests that AI systems are approaching the limits of what can be measured with current benchmarks, signaling a saturation point in engineering automation. However, there is less clarity on whether these capabilities extend to the creative, hypothesis-driven aspects of research, which are less easily quantified and benchmarked.
“Engineering is now largely automated; the residual challenge is whether AI can also automate the creative research process.”
— Thorsten Meyer
Unresolved Questions About AI-Driven Research
It remains unclear whether AI can fully automate the creative and hypothesis-driven aspects of research, such as generating new theories, designing novel experiments, and interpreting complex data. While engineering tasks appear to be approaching full automation, the cognitive and strategic elements of research may still require human insight. The timeframe and feasibility of overcoming this residual challenge are still under debate among experts.
Next Steps in Monitoring AI Research Capabilities
Researchers and institutions will closely track ongoing benchmark developments and real-world applications to assess whether AI can handle more abstract research tasks. Expect further efforts to develop benchmarks that evaluate creative and hypothesis-driven research skills. Additionally, policy discussions may emerge around the implications of fully automated research for scientific integrity, intellectual property, and innovation strategies.
Key Questions
What does it mean that engineering is now automated?
It means that AI systems can now perform most engineering tasks involved in AI research, such as reproducing experiments, optimizing code, and managing dependencies, at a level comparable to human experts.
Can AI fully automate AI research?
It is still uncertain whether AI can automate the creative, hypothesis-driven aspects of research, such as formulating new theories or designing novel experiments. Current benchmarks suggest engineering is close to full automation, but research remains partly human-driven.
What are the implications if AI automates all research tasks?
If AI can fully automate research, it could significantly accelerate scientific discovery, reduce costs, and shift the role of human researchers toward strategic oversight and creativity. It may also raise questions about research quality and oversight.
How soon might AI fully automate research?
The timeline is uncertain. While engineering tasks are nearing saturation, the automation of creative research processes could take longer, and experts disagree on how quickly this might happen.
Source: ThorstenMeyerAI.com
