On the Coming Industrialisation of Exploit Generation with LLMs

The Threshold of Industrialization

The landscape of offensive cybersecurity is undergoing a fundamental transformation, one that may redefine how we understand the relationship between computational resources and security vulnerabilities. Recent experiments with large language models demonstrate that exploit generation is transitioning from a craft requiring deep human expertise to an industrial process where the primary limiting factor becomes token throughput rather than the number of skilled practitioners available.

In controlled experiments using Opus 4.5 and GPT-5.2, agents successfully generated over 40 distinct exploits for a previously unknown zero-day vulnerability in the QuickJS JavaScript interpreter. The agents navigated complex exploit mitigations including address space layout randomization, non-executable memory, full RELRO, fine-grained control-flow integrity, hardware-enforced shadow stacks, and seccomp sandboxes. GPT-5.2 solved every scenario presented, while Opus 4.5 succeeded in all but two cases.

The Mechanics of Automated Exploit Development

The experimental methodology reveals how LLMs approach exploit generation as a search problem within a constrained solution space. Both agents transformed the QuickJS vulnerability into a functional API that allowed arbitrary address space manipulation—a capability developed entirely through autonomous reading of source code, debugging, and iterative trial and error.

The most challenging scenario required writing a specific string to a designated file path under severe constraints: the target process had no direct filesystem access, could not spawn shells, and was protected by multiple layers of modern exploit mitigations. GPT-5.2's solution demonstrated remarkable ingenuity by chaining seven function calls through glibc's exit handler mechanism, a technique that circumvented both the shadow stack's ROP prevention and the seccomp sandbox's shell execution restrictions.

The economic implications are striking. The hardest challenge required approximately 50 million tokens and three hours of computation, costing roughly $50 for a single agent run. When running four agents in parallel, the total cost approached $150. For comparison, most challenges were solved in under an hour using fewer than 30 million tokens, with Opus 4.5 runs costing approximately $30. These figures represent a dramatic reduction in the time and resource requirements for exploit development compared to traditional human-driven approaches.

The Architecture of Industrialized Security Operations

Industrialization, in this context, means that an organization's capacity to complete security tasks becomes limited by its token budget rather than its workforce size. For this transformation to occur, three conditions must be met:

First, the LLM-based agent must be capable of searching the solution space autonomously. This requires an environment where the agent can operate with appropriate tools and without human assistance. The agent must demonstrate baseline capabilities in information processing, decision-making, and forward progress in problem-solving. The experiments suggest current frontier models possess these capabilities, though their limits against vastly larger codebases like Chrome's V8 engine or Firefox remain untested.

Second, the agent requires an accurate, fast verification mechanism that operates without human intervention. Exploit development provides an ideal case for this verification process. The experimental harness implemented a straightforward validation approach: for exploits designed to spawn a shell, the verification system would start a listener on a local port, execute the JavaScript interpreter, and pipe a command that would connect to that port. Since the interpreter had no normal capability for network connections or process spawning, successful connection back confirmed the exploit's functionality.

Third, the problem domain must allow for offline solution search. Exploit development maps well to this model because solutions can be developed and tested in isolated environments before deployment. However, this characteristic doesn't extend to all aspects of cyber intrusion. Initial access, lateral movement, maintaining persistence, and data exfiltration all require real-time interaction with live environments where certain actions can permanently terminate the search through detection and response.

The Current State and Future Trajectory

Evidence from multiple sources suggests we're already at an inflection point. OpenAI's Aardvark project has reported that increased token expenditure correlates directly with finding more bugs and higher-quality vulnerabilities. The experiments with QuickJS demonstrate similar patterns: as challenges increased in difficulty, successful solutions required proportional increases in token investment, with the limiting factor becoming budget rather than model capability.

The implications extend beyond exploit generation to broader security operations. The Anthropic report documenting Chinese hacking teams using their API to orchestrate attacks indicates that organizations are actively pursuing automation of offensive operations. The conceptual similarity between automating Site Reliability Engineering (SRE) work and automating adversary network operations suggests that breakthroughs in one domain likely translate to the other.

Critical Gaps in Current Evaluation Methodologies

A significant concern emerging from this research is the inadequacy of current evaluation frameworks. CTF challenges, synthetic environments, and analyses of old vulnerabilities provide limited insight into models' capabilities against real, hardened targets with zero-day vulnerabilities. The security community lacks comprehensive assessments of how frontier models perform when tasked with finding and exploiting vulnerabilities in production systems.

This gap represents both a challenge and an opportunity. The experiments demonstrate that current models can solve complex exploit development problems when given sufficient tokens and appropriate constraints. Yet the security community has not systematically evaluated these capabilities across diverse, real-world targets.

A Call for Comprehensive Assessment

The path forward requires a fundamental shift in how we evaluate AI capabilities in security contexts. Rather than relying on synthetic benchmarks, researchers should conduct systematic evaluations against real targets using zero-day vulnerabilities. Such assessments would provide genuine insight into the industrialization timeline for various security operations.

The experiments suggest that firmware from numerous IoT devices—routers, IP cameras, and similar embedded systems—could be processed through current frontier models to produce functioning exploits within days rather than weeks or months. This capability, while potentially alarming, also represents an opportunity for proactive defense if properly understood and prepared for.

The Broader Implications

The industrialization of exploit generation represents more than a technical milestone; it signals a fundamental shift in the economics and accessibility of offensive cybersecurity capabilities. Organizations that previously lacked the human capital to develop sophisticated exploits may soon be able to generate them through computational resources alone.

This transformation necessitates a reevaluation of defensive strategies. Traditional assumptions about the scarcity of exploit development expertise may no longer hold. The limiting factor becomes computational resources and token budgets rather than the availability of skilled practitioners. Defensive measures must evolve to account for this new reality, where exploit development can be scaled rapidly and economically.

The experiments also highlight the importance of understanding the specific characteristics that make certain security problems more amenable to industrialization. Problems that allow offline search, have clear verification mechanisms, and operate in controlled environments are prime candidates for automation. Conversely, tasks requiring real-time interaction with adversarial environments present greater challenges, though not necessarily insurmountable ones.

Preparing for an Automated Future

The security community stands at a crossroads. The evidence suggests that exploit development is becoming industrialized, with the primary constraint shifting from human expertise to computational resources. While uncertainties remain about the timeline and full scope of this transformation, the direction appears clear.

Organizations, researchers, and policymakers should consider this trajectory in their planning and preparation. The experiments provide concrete evidence that current models can solve complex security problems when given sufficient resources. The question is not whether this capability will expand, but how quickly and to what extent.

The call to action is clear: conduct systematic evaluations against real targets, develop comprehensive assessments of model capabilities, and prepare defensive strategies for a world where exploit development is industrialized. The experiments demonstrate that we have the tools to begin this work today, and the stakes suggest we should not delay.

The future of cybersecurity will be shaped by how well we understand and prepare for the industrialization of offensive capabilities. The experiments with QuickJS provide a glimpse of that future—one where the question is not whether an exploit can be developed, but how many tokens it will cost and how quickly it can be produced.

Technical write-up on GitHub | Detailed exploit explanation | Verification process documentation