Translating academic research into practical AI Agent skills through systematic experimentation.

A few weeks ago, I stumbled upon a fascinating paper on X: “Recursive Language Models” by Zhang, Kraska, and Khattab from MIT CSAIL. The core insight was elegant—treat the prompt as a Python variable rather than neural network input, allowing models to work with unbounded context through recursive sub-calls. The obvious question: could we translate these theoretical patterns into executable instructions for AI Agents?

This post documents the experiment. We took the RLM paper, extracted its core concepts, and transformed them into skill files that AI Agents can follow. Then we validated the approach by having multiple AI Agents—Claude Code, Codex, and Gemini—cross-validate each other’s work. The results reveal both the promise and the challenges of multi-agent orchestration.

The Problem: Academic Papers Don’t Compile

Research papers describe algorithms and results. They don’t provide executable instructions that an AI Agent can follow step-by-step. The gap between “here’s the theory” and “now do this” is where most knowledge transfer fails.

Consider RLM’s key insight about context rot—performance degrades as context length increases. The paper quantifies this beautifully with benchmarks on OOLONG and BrowseComp+. But an AI Agent facing a 100K token codebase doesn’t know to apply partition+map+reduce unless explicitly told when and how.

Our approach was systematic:

1. Extract - Identify actionable patterns from the paper
2. Codify - Transform into explicit decision trees and templates
3. Validate - Cross-check with multiple AI models
4. Iterate - Refine based on experimental feedback

The Translation Process

Phase 1: Core Concept Extraction

The RLM paper’s abstract mentions “treating the prompt as a Python variable.” What does this mean operationally?

After multiple read-throughs (some by humans, some by AI Agents), we extracted these key patterns:

Phase 2: Decision Tree Construction

Academic papers describe what works. Skill files need to specify when to apply each technique. We built decision trees:

START
  |
  +-- Context > 50K tokens? -------- YES --> ACTIVATE RLM
  |     |
  |     NO
  |     |
  +-- Context > 16K AND complexity > O(1)? -- YES --> CONSIDER RLM
  |     |
  |     NO
  |     |
  +-- Files to process > 5? -------- YES --> ACTIVATE RLM
  |     |
  |     NO
  |     |
  +-- Multi-hop reasoning needed? -- YES --> ACTIVATE RLM
  |     |
  |     NO
  |     |
  +-- PROCEED DIRECTLY

This transforms a theoretical concept into an operational rule. Note the complexity-aware middle branch—the full RLM.md includes additional nuance for moderate-sized contexts where task complexity (O(n) vs O(n²)) determines whether RLM overhead is worthwhile.

Phase 3: Strategy Templates

The paper describes emergent strategies like “Grep First” and “Preview + Partition.” We codified these into templates:

STRATEGY: Grep First
TRIGGER: Searching for specific patterns across many files
PROCEDURE:
1. Use grep/search to identify relevant file subset
2. Read only matching files
3. Process focused context

STRATEGY: Partition + Map + Reduce
TRIGGER: Single large file exceeding context limit
PROCEDURE:
1. Assess document structure (chapters, sections, functions)
2. Split at natural boundaries with 10-20% overlap
3. Process each chunk with identical prompt
4. Aggregate results, resolving conflicts

The Multi-Agent Experiment

Experimental Design

With the RLM skill file (RLM.md) drafted, we needed validation. Our hypothesis: multiple AI models would catch different issues, improving overall quality.

Methodology:

• Create initial draft using Claude Code with RLM patterns
• Have three agents review independently: Claude (via Task tool), Codex (via CLI), Gemini (via CLI)
• Collect and compare findings
• Measure agreement and divergence

Agent Configuration

# Claude Code sub-agent (pseudo-API, actual tool invocation)
Task(subagent_type="Explore", prompt="Review RLM.md for...")

# Codex CLI (replace model with your available model)
codex exec -m <your-model> -i RLM.md "Review for..."

# Gemini CLI (use here-doc for large files to avoid shell arg limits)
GEMINI_API_KEY="$YOUR_API_KEY" gemini "Review RLM.md content: ..."

Note: The Task(...) syntax is pseudo-API representing Claude Code’s internal subagent mechanism. The actual invocation happens through tool use, not shell commands.

Experiment Log

The following logs are from actual execution on 2026-01-05:

Claude Agent Review (Task ID: a642a60)

[EXPERIMENT LOG - Claude Code Sub-Agent]
Timestamp: 2026-01-05T13:52:XX UTC
Task ID: a642a60
Duration: ~90 seconds
Tools used: 31 (Read, Grep, Glob, Bash)

Key Findings:
1. ACCURACY: Paper claims "+12.5 points" on OOLONG verified against RLM.md
2. ACCURACY: BrowseComp+ figure (91.33%) verified consistent
3. COMPLETENESS: Decision tree simplification noted - missing complexity branch
4. STYLE: File line counts needed correction (GEMINI.md: ~300 → 423)
5. ISSUE: Paper year citation inconsistency (arXiv 2512 = Dec 2025, not 2024)

Recommendation Score: 7/10

Codex Agent Review (Session: 019b8e72)

[EXPERIMENT LOG - Codex CLI]
Timestamp: 2026-01-05T13:57:XX UTC
Session ID: 019b8e72-fbae-7910-bbb9-34f4fabfec3e
Model: gpt-5.2-codex (reasoning: xhigh)
Tokens used: 51,162

Key Findings:
1. "Recursive sub-calls" framing overstates RLM - paper uses LM-in-REPL pattern
2. "Grep First" presented as paper claim but is derived heuristic - needs citation
3. Task(...) syntax is pseudo-API - should be labeled clearly
4. Quality scores/time multipliers lack methodology definition
5. Shell arg limits risk with CONTENT=$(cat ...) pattern

Rating: 7/10

Gemini Agent Review

[EXPERIMENT LOG - Gemini CLI]
Timestamp: 2026-01-05T14:05:XX UTC
Model: gemini-2.0 (via API key)

Key Findings:
1. Factual claims about RLM paper VERIFIED
2. GAP: Emulation vs Implementation distinction needed
3. GAP: Cost implications of recursive API calls not addressed
4. GAP: Infinite recursion safeguards not discussed
5. RECOMMENDATION: Add latency warning in introduction

Rating: 8.5/10

Cross-Validation Analysis

Key observation: Each agent identified unique issues. Cross-validation caught 6 distinct improvement areas. Agreement on core accuracy (3/3) validates the foundational content. Disagreement on gaps (1/3 each) demonstrates the value of diverse model architectures.

Important clarification (from Gemini review): This experiment emulates RLM architecture using standard AI Agents and skill files, rather than running the paper’s actual codebase. The skill files translate RLM concepts into actionable patterns, but the underlying mechanism differs from the paper’s REPL-based recursive system.

Results: The Skill File Ecosystem

The translation process produced five interconnected skill files:

Dependency Graph

CLAUDE.md (repository root)
    |
    +-- MULTI-AGENT.md (orchestration)
    |       |
    |       +-- RLM.md (when/how to decompose)
    |       +-- CODEX.md (Codex sub-agent)
    |       +-- GEMINI.md (Gemini sub-agent)
    |
    +-- SKILL.md (writing style)

Usage Pattern

An AI Agent encountering this repository:

1. Reads CLAUDE.md for repository context
2. For complex tasks, references MULTI-AGENT.md
3. For large context, applies RLM.md patterns
4. For cross-validation, uses CODEX.md and GEMINI.md
5. For content creation, follows SKILL.md

Quantitative Findings

Task Complexity vs. RLM Benefit

From our experiments aligning with paper Table 1:

Critical insight: RLM overhead hurts simple tasks but dramatically improves complex ones. The skill file now includes explicit guidance on when NOT to use RLM.

Multi-Agent Overhead

Parallel cross-validation provides the best quality/time tradeoff.

Lessons Learned

What Worked

1. Explicit decision trees - AI Agents follow clear conditionals reliably
2. Structured templates - Standard formats reduce interpretation errors
3. Cross-validation - Different models catch different issues
4. Parallel execution - Background CLI calls minimize overhead

What Didn’t Work

1. Implicit assumptions - Anything unstated gets interpreted randomly
2. Unbounded outputs - Must specify exact format expectations
3. Single-model validation - Creates blind spots
4. Sequential pipelines - Much slower than parallel approaches

Limitations

Latency: Multi-agent orchestration is significantly slower than single-agent execution. Our parallel cross-validation took ~3 minutes vs ~30 seconds for a single review. Budget time accordingly.

Cost: Recursive patterns multiply API costs. A three-agent cross-validation costs ~3x a single review. For O(n²) complexity tasks, costs can grow quadratically.

Recursion safety: The current skill files don’t include explicit recursion depth limits. Production deployments should add safeguards against infinite loops.

Open Questions

1. How do we validate that skill files remain accurate as models evolve?
2. What’s the optimal number of cross-validation agents?
3. Can we automate the paper-to-skill translation process itself?
4. What recursion depth is safe for various task types?

The Meta-Layer

This post was written using the very patterns it describes. The initial draft was created with Claude Code following SKILL.md style guidelines. It was then reviewed by Codex and Gemini sub-agents following MULTI-AGENT.md Pattern D (cross-validation).

The experiment logs above are real. The findings were incorporated into revisions. This recursive application of RLM principles to document RLM principles is intentional—it validates the approach while demonstrating it.

Practical Applications

For AI-Assisted Development

If you’re using AI coding tools, consider building your own skill files:

1. Start with CLAUDE.md or AGENTS.md as templates
2. Add project-specific patterns and conventions
3. Include decision trees for common tasks
4. Cross-validate with multiple models before finalizing

For Research Translation

The paper-to-skill methodology generalizes:

1. Extract actionable patterns (not just results)
2. Codify as explicit decision trees
3. Validate with multiple independent agents
4. Iterate based on experimental feedback

Conclusion

Translating academic research into executable AI skills requires more than summarization. It requires extracting decision procedures, codifying trigger conditions, and validating through multi-agent cross-checking.

The RLM paper provided theoretical foundations. Our skill files provide operational instructions. The gap between theory and practice is bridged by explicit, testable, actionable specifications.

The source files are available in this repository:

• RLM.md - Core recursive patterns
• MULTI-AGENT.md - Orchestration patterns
• CODEX.md - Codex CLI usage
• GEMINI.md - Gemini CLI usage

Fork them, extend them, improve them. The skill file approach works best when it evolves with use.

Experimental Setup

This experiment used the following AI models and tools:

Hardware: MacOS Darwin 24.6.0, Apple Silicon

CLI Versions:

• Codex CLI: v0.77.0
• Gemini CLI: v0.22.5
• Claude Code: Latest (Jan 2026)

Cost Breakdown (approximate):

• Claude Code orchestration: ~$2-3 (context + tool calls)
• Codex review: 51,162 tokens (~$0.50)
• Gemini review: ~10K tokens (~$0.05)
• Total experiment: ~$3-4

The cross-validation pattern used three different model architectures (Anthropic Claude, OpenAI GPT, Google Gemini) to maximize blind spot detection.

References

1. Zhang, A.L., Kraska, T., Khattab, O. (2025). Recursive Language Models. arXiv:2512.24601
2. Original RLM blog post: https://alexzhang13.github.io/blog/2025/rlm/
3. RLM implementation: https://github.com/alexzhang13/rlm

Cross-validation experiment conducted 2026-01-05. All experiment logs represent actual CLI execution outputs.

Support my work