Compare RLM task decomposition with repo implementation

Summary

In Progress

Understand
Researched recursive language models and their task decomposition strategy.
Understand
Understand
Explored the Sovereign Hive repo's orchestration core and decomposition approach.
Understand
Verify
Confirmed the repo's core workflow is a stub with no nested LLM calls.
Verify

What are recursive language models? You might already have some information from your training data but I would like you to augment that knowledge with the latest developments, as there have been many new developments on this front in recent weeks.

I'll research this for you. Recursive language models (RLMs) is an active research area, and since you want the latest developments, let me pull current information from the web.

I'll search for the latest on recursive language models.

Let me dig into the primary sources, including the most recent papers, to get the details.

Let me get the cost/benchmark specifics and the RL training developments from a couple more sources.

Recursive Language Models (RLMs)

Recursive Language Models are an inference strategy (not a new model architecture) for handling near-unbounded context. The core move: instead of cramming a giant prompt into the model's context window, you treat the prompt as data in an environment that the model interacts with programmatically, recursively calling itself (or a smaller sibling) over chunks of it.

The paradigm was introduced by Alex L. Zhang and Tim Kraska at MIT CSAIL (arXiv 2512.24601, late 2025, updated Jan 2026). It's gotten enough traction that Prime Intellect dubbed it "the paradigm of 2026."

How it works

The long input is loaded as a context string variable inside a Python REPL — it never goes into the model's prompt directly.
The root LM sees only the REPL, not the raw context. It writes Python to inspect, slice, grep, and decompose the data (print(), string ops, etc.), using variables as scratch buffers.
When it needs an LM to reason over a chunk, it calls llm_query() — a recursive sub-call to an LM instance (handling ~500K chars per call). Those sub-calls can themselves be RLMs.
Results are aggregated in the REPL, and the root emits a final answer (originally via brittle FINAL() tags).

The key philosophical claim: never summarize/compact the context (which loses information). Instead, delegate it to scripts and sub-LMs that read the relevant parts on demand.

Headline results (original paper)

Handles inputs ~100× beyond the model's context window (two orders of magnitude).
Even on short prompts, beats base models and common long-context scaffolds on four tasks: S-NIAH, BrowseComp-Plus, OOLONG / OOLONG-Pairs, and CodeQA (LongBench-v2).
vs. baselines: +26% median over compaction, +130% over CodeAct-with-subcalls, +13% over Claude Code.
A fine-tuned RLM-Qwen3-8B beat base Qwen3-8B by 28.3% and approached vanilla GPT-5 on three tasks.
Cost: typically competitive with base runs (e.g. RLM-GPT-5 on BrowseComp-Plus ~$0.99 vs ~$1.50–2.75 extrapolated base), and ~3× cheaper than summarization agents — but with a long, expensive tail when trajectories balloon. Setup used GPT-5 as root + GPT-5-mini for sub-calls.

Documented limitations

Smaller models need coding ability (Qwen3-8B required fine-tuning); synchronous/blocking sub-calls hurt latency (async would help a lot); thinking models can blow past output-token limits; system-prompt sensitivity; brittle answer/thought separation; and notably the original paper only evaluated recursion depth of 1.

What's new in recent weeks (post-cutoff developments)

The two most significant recent additions:

1. "Think, But Don't Overthink: Reproducing Recursive Language Models" (arXiv 2603.02615). An independent reproduction that (a) confirms RLMs work and open-sources the reproduction, and crucially (b) probes the recursion-depth axis the original left unexplored. Finding: there's a sweet spot — some recursion helps, but deeper recursion degrades performance and wastes compute rather than improving it. This pushes back on the intuition that "more recursive thinking = better" and argues depth must be calibrated, not maximized.

2. "Reinforcing Recursive Language Models" (alphaXiv). Moves RLMs from a prompting scaffold to a trained capability. They RL fine-tune a small (4B) model to be a native RLM, with the key trick of a single shared policy playing both roles — parent (decomposer) and child (sub-agent) — rather than two separate models. Result: on an evidence-selection task over scientific documents, the RL'd 4B model matched Claude Sonnet 4.6 under an identical RLM harness, at a fraction of the size/cost. Notably, they report this RLM behavior can't be elicited by prompting or even SFT — it requires RL.

The throughline of these recent works: RLMs are shifting from a clever inference-time hack toward a trainable, optimized behavior, while the community simultaneously maps out where it breaks (depth/overthinking).

Ecosystem

Beyond the academic libraries (alexzhang13/rlm — a plug-and-play inference library supporting various sandboxes), there are now community implementations like grishahq/recursive-llm for running RLMs over 100k+ token contexts with arbitrary LLMs.

A note on confidence: today is 2026-05-31 and some of these (the 2603.xxxxx reproduction, the RL work) postdate my training cutoff, so I'm relaying them from the sources rather than prior knowledge — worth reading the primaries directly before relying on specific numbers.

Sources:

Want me to go deeper on any thread — e.g. the RL training recipe, the depth/overthinking analysis, or how RLMs compare to alternatives like RAG and context-compaction agents?

How does this strategy of task decomposition compare with the implementation (and aspirations) in this repo? What are the tradeoffs?

Good question — but I need to actually understand this repo first. Let me explore it.

A "Sovereign Hive" orchestration CLI. Let me look at how it actually decomposes/orchestrates work, and what it aspires to — focusing on the orchestration core, the "swarm," and context handling (the part most comparable to RLMs).

The exploration gave me concrete file references. Let me verify the two load-bearing claims directly before I assert them — the orchestrator "stub" and whether USE_SUBAGENTS actually has an executor.

Verified. The two crux claims hold: USE_SUBAGENTS exists only as an enum entry (tools.ts:33,69) with no executor anywhere; executeCoreWorkflow is literally commented // Placeholder for core workflow logic and returns {status:'completed', input}. The orchestrator is wired into index.ts:250, but its core path is a stub. LLM calls are one generateContent() per createMessage, never nested.

Here's the comparison.

Where "decomposition" lives — the fundamental divergence

These two systems put task decomposition in opposite places, and that's the whole story.

RLM: decomposition is inside the model's reasoning. The root LM sits in a Python REPL, sees the context as a context variable, and decides — token by token — to slice it and fire llm_query() sub-calls over chunks. The orchestration logic is learned/emergent, expressed as code the model writes at inference time.
DietCode: decomposition is aspirationally in the framework — a [LAYER: CORE] Orchestrator, an AgentRegistry with a "Swarm Architect," a USE_SUBAGENTS tool. It's imperative and external: the harness, not the model, is meant to route work. But the routing isn't implemented — executeCoreWorkflow is a placeholder, USE_SUBAGENTS has no executor, and the three registered agents are never invoked.

So the comparison is really RLM's working emergent recursion vs. DietCode's declared-but-hollow imperative orchestration, plus a real, working context layer underneath.

Implementation: what DietCode actually does vs. RLM

Axis	RLM	DietCode (as implemented)
Sub-task spawning	`llm_query()` recursive calls, depth ≥1	None. Single `generateContent()` per turn (`GeminiProvider.ts:41`)
Context strategy	Loaded as a variable; model queries it programmatically, never summarizes	Semantic folding — keep head 60% + tail 30%, discard the middle (`ContextPruner.ts:154`)
Long-context limit	~100× the window	Bounded by the prompt; folding loses the middle
Who orchestrates	The model (in-call)	The framework (between turns) — but the core path is a stub
Tool/code execution	Real Python REPL the model drives	Tools are intent declarations; execution delegated to an external harness

The irony worth naming: DietCode's context handling is the exact thing RLM was built to avoid. RLM's central thesis is "never compact, because compaction loses information." DietCode's ContextPruner does precisely that — it folds away the middle of long files for "cognitive focus." On the one axis where DietCode has working code comparable to RLM, it's made the opposite design bet.

Aspirations vs. RLM

The vision docs (FEATURES.md, proto.md) actually aim at something RLM-adjacent but more imperative:

"Task Sharding: Complex goals are automatically broken down into atomic, verifiable sub-tasks" — this is decomposition, but state-machine-driven (BACKLOG → SHADOW_SIM → SOVEREIGN_DOING → VERIFYING → DONE), not recursive model calls.
"Agent Shadows: asynchronous background agents" + the "Swarm Architect" — a multi-agent hive, closer to a fixed orchestration graph than to RLM's uniform self-recursion.
proto.md's "Shadow Simulation" with a 0.95 verification score gate — a verify-before-act loop RLM doesn't have at all.

So even the aspiration differs from RLM in a meaningful way: DietCode wants heterogeneous, role-specialized agents on an explicit pipeline with verification gates; RLM is homogeneous self-recursion with no built-in verification. The recent RL-trained RLM work (shared parent/child policy) actually moves RLM slightly toward DietCode's "one system, multiple roles" idea — but keeps it learned rather than wired.

Tradeoffs

RLM's approach buys:

True unbounded context without information loss (the middle survives).
Flexibility — the model invents the decomposition per task; no need to pre-build an orchestration graph.
Less framework code.

…at the cost of:

A long, expensive cost tail (runaway trajectories), latency from synchronous sub-calls, and brittleness (needs a capable coding model, output-token headroom, careful prompts). No verification gates — it can confidently recurse into a wrong answer.

DietCode's intended approach buys:

Auditability and safety — the ExecutionService/SafetyGuard envelope (which is real) intercepts and risk-rates every tool call; proto.md adds verification gates. RLM has nothing like this.
Specialization — distinct agents (distiller, architect) tuned per role.
Determinism — an explicit pipeline is debuggable in a way emergent recursion isn't.

…at the cost of:

You have to build and maintain the orchestration graph — and DietCode mostly hasn't. The cost shows up as the aspiration/implementation gap: a hollow orchestrator, an unused agent registry, a USE_SUBAGENTS tool with no executor, and ~43 markdown docs / 8.6K lines describing behavior the code doesn't yet do.
Compaction-based context loses information by construction.

The synthesis

If you wanted to close the gap, RLM is a surprisingly good fit for DietCode's stated goals — and it would attack DietCode's two real weaknesses at once:

Replace ContextPruner's lossy folding with an RLM-style "context-as-queryable-variable" approach, so long files aren't gutted in the middle.
Give USE_SUBAGENTS a real executor shaped like llm_query() — recursive createMessage calls over context chunks — which is genuinely small code (the original RLM scaffold is a thin REPL loop), turning the "Swarm" from three static records into something that runs.

The philosophical tension to resolve: RLM has no verification step, while DietCode's whole identity (SafetyGuard, the 0.95 SHADOW_SIM gate) is verification-first. The interesting design move would be RLM recursion wrapped in DietCode's safety envelope — emergent decomposition, but every sub-call's tool actions still pass through the risk-rated execution path. That's a combination neither the RLM papers nor this repo currently has, and it plays to each side's strength.

Want me to sketch what a real llm_query()-style executor for USE_SUBAGENTS would look like in this codebase (wired through GeminiProvider + the ExecutionService safety envelope)?