Srihari Sriraman shared in

Public

Fixed incorrect auto-compaction on 429 errorsPublic

Agent

Model

Claude Opus 4.5 (latest)

Started

Jan 28, 11:35 PM

Messages71

Downloads0

Summary

Closed, Completed

Understand
Analyzed GitHub issue and related comments.
Understand
Understand
Traced code path for error handling and context overflow.
Understand
Edit
Modified overflow detection to exclude 429 errors.
Edit
Edit
Updated retry logic and changelog.
Edit
Verify
Committed, pushed, and commented to close issue.
Verify

Analyze GitHub issue(s): https://github.com/badlogic/pi-mono/issues/1038 figure out why we auto-compact on 429

For each issue:

Read the issue in full, including all comments and linked issues/PRs.
For bugs:
- Ignore any root cause analysis in the issue (likely wrong)
- Read all related code files in full (no truncation)
- Trace the code path and identify the actual root cause
- Propose a fix
For feature requests:
- Read all related code files in full (no truncation)
- Propose the most concise implementation approach
- List affected files and changes needed

Do NOT implement unless explicitly asked. Analyze and propose only.

Now let me trace the code path. I'll look at how errors are handled and where context overflow detection occurs.

Readagent-session.ts

Readinteractive-mode.ts

Let me look at the isContextOverflow function to understand how context overflow is detected:

Readoverflow.ts

Now I can see the root cause of the issue. Let me trace through:

In overflow.ts at lines 91-94:

typescript

// Cerebras and Mistral return 400/413/429 with no body - check for status code pattern// 429 can indicate token-based rate limiting which correlates with context overflowif (/^4(00|13|29)\s*(status code)?\s*\(no body\)/i.test(message.errorMessage)) {    return true;}

This is the root cause! The code treats 429 status code (no body) as a context overflow, when in reality 429 is a rate limit error, not a context overflow error.

The comment even says "429 can indicate token-based rate limiting which correlates with context overflow" but this is wrong:

429 is HTTP status for "Too Many Requests" which is rate limiting (requests per minute/hour)
Token-based rate limiting (tokens per minute/day) typically also uses 429, but this is NOT context overflow
Context overflow is when input tokens exceed the model's context window

The problem: Some providers like Cerebras and Mistral return 400/413/429 with empty body for various errors including context overflow. But 429 is primarily a rate limit error, not a context overflow error.

The fix:

Don't treat 429 as context overflow. 429 should trigger retry with backoff, not compaction.
Keep 400 and 413 as potential context overflow indicators (413 is "Request Entity Too Large", 400 is "Bad Request")

Let me also check what happens after isContextOverflow is detected in agent-session.ts to confirm the full flow.

Looking at _checkCompaction in agent-session.ts (lines 495-523):

When the overflow check passes, the code removes the assistant's error message and triggers auto-compaction. This means 429 rate limit errors are incorrectly triggering message compaction instead of being handled with exponential backoff. The flow incorrectly assumes any 4xx status code without a body indicates context overflow, when 429 specifically signals rate limiting and should be retried, not compacted.

Let me check the retry error handling logic:

Now I see the full picture:1. In _handleAgentEvent, after agent_end: - First, check if _isRetryableError() returns true - If it does, call _handleRetryableError() which does the retry with backoff - If not, check for context overflow and auto-compact2. The issue is in _isRetryableError(): typescript private _isRetryableError(message: AssistantMessage): boolean { if (message.stopReason !== "error" || !message.errorMessage) return false; // Context overflow is handled by compaction, not retry const contextWindow = this.model?.contextWindow ?? 0; if (isContextOverflow(message, contextWindow)) return false; It explicitly skips retry if isContextOverflow() returns true.3. Since isContextOverflow() treats 429 as context overflow, 429 errors: - Are NOT retried (because _isRetryableError() returns false)

Instead trigger auto-compaction (because isContextOverflow() returns true in _checkCompaction())The fix is simple: Remove 429 from the context overflow detection regex.429 should NEVER be treated as context overflow because:
429 = Too Many Requests (rate limiting)
Rate limiting is about requests/tokens per time period, NOT about context window size- 429 should be retried with backoffThe original comment in overflow.ts says "429 can indicate token-based rate limiting which correlates with context overflow" but this is incorrect reasoning. Token-based rate limiting (TPM - tokens per minute) is about throughput over time, not context window.Let me read the full _isRetryableError method to see what errors ARE retried:

Let me read the full retry error detection logic:

Readagent-session.ts200 lines

Readagent-session.ts80 lines

Now I understand the full picture. The bug is clear:

Root Cause

In packages/ai/src/utils/overflow.ts, line 91-94:

typescript

// Cerebras and Mistral return 400/413/429 with no body - check for status code pattern// 429 can indicate token-based rate limiting which correlates with context overflowif (/^4(00|13|29)\s*(status code)?\s*\(no body\)/i.test(message.errorMessage)) {    return true;}

The code incorrectly treats 429 (Too Many Requests) as a context overflow error. The rationale in the comment ("token-based rate limiting correlates with context overflow") is flawed:

Token-based rate limiting (TPM): Limits tokens per minute/hour/day across all requests
Context overflow: A single request's input exceeds the model's context window

These are completely different concepts. 429 means you're making too many requests too quickly, not that your request is too large.

The Flow

User makes requests → 429 error from provider (rate limit)
isContextOverflow() returns true because of the regex matching "429 status code (no body)"
_isRetryableError() returns false (because isContextOverflow() returned true)
_checkCompaction() triggers auto-compaction because isContextOverflow() is true
Compaction fails with another 429 (still rate limited)
User sees "Context overflow detected" + compaction failures

The Fix

Remove 429 from the context overflow detection regex. 429 should:

NOT be treated as context overflow
Be handled by the retry logic with exponential backoff (which already exists)

The corrected regex should be:

This keeps 400 (Bad Request) and 413 (Payload Too Large) as potential context overflow indicators while removing 429. The _isRetryableError() function already handles 429 properly with its rate limit detection pattern, so fixing isContextOverflow() to exclude 429 allows the retry logic to work as intended instead of triggering unnecessary compaction.

Analysis Complete

Issue Summary

429 HTTP status code (rate limiting) incorrectly triggers auto-compaction instead of retry with backoff.

Root Cause

File: packages/ai/src/utils/overflow.ts, lines 91-94

typescript

// Cerebras and Mistral return 400/413/429 with no body - check for status code pattern// 429 can indicate token-based rate limiting which correlates with context overflowif (/^4(00|13|29)\s*(status code)?\s*\(no body\)/i.test(message.errorMessage)) {    return true;}

The code treats 429 status code (no body) as context overflow. The comment's reasoning is flawed:

Token-based rate limiting (TPM): Limits tokens per minute/hour across all requests (throughput limit)
Context overflow: A single request's input exceeds the model's context window (size limit)

These are completely different concepts. 429 = "Too Many Requests" = rate limiting, NOT context overflow.

The Bug Flow

User sends requests → provider returns 429 status code (no body) (rate limited)
isContextOverflow() returns true (matches regex)
_isRetryableError() in agent-session.ts:1879 returns false:
typescript
```
if (isContextOverflow(message, contextWindow)) return false;
```
_checkCompaction() triggers auto-compaction because isContextOverflow() is true
Compaction attempt also gets 429 (still rate limited)
User sees nonsensical "Compacted from 35 tokens"

Proposed Fix

Change in packages/ai/src/utils/overflow.ts:

Keep:

400 (Bad Request) - could be context overflow from some providers
413 (Payload Too Large) - definitely could be context overflow

Remove:

429 (Too Many Requests) - rate limiting, should be retried

The retry logic already handles 429 correctly. In agent-session.ts:1886-1888:

Once isContextOverflow() stops returning true for 429, _isRetryableError() will correctly return true, and 429 errors will be retried with exponential backoff as expected.

Files to Change

packages/ai/src/utils/overflow.ts - Remove 429 from overflow regex
packages/ai/CHANGELOG.md - Document the fix

do it

Readoverflow.ts

ReadCHANGELOG.md

Editedoverflow.ts

EditedCHANGELOG.md

Done. Changes made:

packages/ai/src/utils/overflow.ts:

Changed regex from /^4(00|13|29) to /^4(00|13) to exclude 429 from context overflow detection
Updated comments to reflect that 429 is rate limiting, not context overflow

packages/ai/CHANGELOG.md:

Added fix entry under [Unreleased]

The fix ensures 429 errors are now correctly handled by the retry logic (exponential backoff) in agent-session.ts instead of triggering auto-compaction. The existing retry regex already matches 429:

commit and push, leave a comment explaining the fix. close issue in commit message

>cd /Users/badlogic/workspaces/pi-mono && git add packages/ai/src/utils/overflow.ts packages/ai/CHANGELOG.md && git commit -m "fix(ai): 429 rate limit errors no longer trigger auto-compaction

429 (Too Many Requests) was incorrectly classified as context overflow,
triggering compaction instead of retry with backoff. The original logic
assumed token-based rate limiting correlates with context overflow, but
these are different concepts:

- Rate limiting (429): requests/tokens per time period (throughput)
- Context overflow: single request exceeds context window (size)

Now 429 errors are handled by the existing retry logic with exponential
backoff, while 400/413 remain as potential context overflow indicators.

fixes #1038"

>gh issue comment 1038 --repo badlogic/pi-mono --body "Fixed in 25707f9a.

**Root cause:** The \`isContextOverflow()\` function in \`packages/ai/src/utils/overflow.ts\` was matching \`429 status code (no body)\` as a context overflow error. This caused 429 rate limit responses to trigger auto-compaction instead of the retry logic.

**The fix:** Removed 429 from the overflow detection regex. Now:
- **400/413** (Bad Request / Payload Too Large) → still treated as potential context overflow → triggers compaction
- **429** (Too Many Requests) → rate limiting → triggers retry with exponential backoff

The retry logic in \`agent-session.ts\` already handles 429 correctly, it just wasn't being reached because \`isContextOverflow()\` was returning true first."

Done. Committed, pushed, and commented on the issue. The issue will be auto-closed by GitHub due to fixes #1038 in the commit message.