Recursive Code Execution: A Paradigm Shift for Grounding Large Language Models

The persistent challenge of unreliable outputs from large language models—even with multi-million-token context windows—reveals a fundamental architectural limitation. When instructed to analyze complex documents like financial reports, most LLMs shortcut genuine comprehension in favor of probabilistic pattern matching, often hallucinating plausible but incorrect figures. This flaw becomes particularly acute in local deployments where smaller models trade capability for efficiency. Traditional Retrieval Augmented Generation (RAG) systems offer partial solutions by leveraging semantic similarity through vector databases, yet they remain fundamentally constrained: embeddings cannot perform arithmetic operations, reconcile scattered data points, or distinguish contextually similar terms like 'Projected Sales' versus 'Actual Sales'.

The Recursive Language Model (RLM) methodology introduces a transformative alternative by reframing text analysis as a programmable interface. Rather than interpreting documents directly, the LLM generates executable code to interact with content through a read-eval-print loop (REPL) environment. This approach leverages the model's strength in code generation while offloading precision tasks to deterministic computation. For example, when extracting sales figures, an LLM might write a regex pattern to identify currency values; the sandbox executes this code and returns verified results like ['$2,340,000'], transforming speculative guesses into grounded facts.

Security concerns inherent to executing arbitrary code are addressed through isolated-vm, a Node.js library that creates hardened sandboxes. These environments prevent filesystem access, network calls, or infinite loops while maintaining document immutability. The system provides exploration tools via strictly typed interfaces defined in TypeScript—such as fuzzy_search(query: string)—using the Universal Tool Calling Protocol (UTCP) to enforce structured inputs and outputs. To counter LLMs' coding inconsistencies, a self-healing layer automatically corrects syntax errors before re-execution, preserving the reasoning chain.

In practical validation, a 4,700-character document containing five sales figures obscured by Lorem Ipsum filler and jargon was processed. A standard LLM prompt returned a hallucinated total of $480,490 after surface-level scanning. By contrast, the RLM approach required four iterative steps:

1. Measuring document metrics via text_stats()
2. Locating relevant lines using fuzzy_search("SALES_DATA")
3. Parsing values with regex
4. Summing verified integers

The model converged on the correct total of $13,000,000 by delegating arithmetic and pattern matching to code execution. This demonstrates RLM's core advantage: shifting the LLM's role from direct interpretation to meta-programming, where it constructs tools to derive answers rather than guessing them.

Performance trade-offs emerge in latency—each query requires multiple model calls—but yield context efficiency for large documents by avoiding full-text ingestion. The system integrates with agent frameworks via a Model Context Protocol (MCP) server, enabling tools like Crush Agent to delegate precision tasks. For instance, an agent might invoke analyze_document("sum Q3 sales", report.pdf) while the RLM backend handles iterative exploration using local models like Ollama or cloud services like DeepSeek.

Available as open-source under the Matryoshka project, this approach redefines document interaction. By treating text as queryable datasets rather than prompts, recursive code execution provides the missing scaffolding for reliable, verifiable LLM outputs—a critical advancement for applications demanding mathematical rigor or distributed data synthesis.