Srihari Sriraman shared in

Public

Analyzed and fixed GitHub issue #967Public

Agent

Model

Claude Opus 4.5 (latest)

Started

Jan 29, 1:06 AM

Messages201

Downloads0

Summary

Closed, Completed

Understand
Reviewed GitHub issue and related code.
Understand
Understand
Checked SDK cache control options.
Understand
Edit
Committed and pushed fix with auto-close message.
Edit

Analyze GitHub issue(s): https://github.com/badlogic/pi-mono/issues/967

For each issue:

Read the issue in full, including all comments and linked issues/PRs.
For bugs:
- Ignore any root cause analysis in the issue (likely wrong)
- Read all related code files in full (no truncation)
- Trace the code path and identify the actual root cause
- Propose a fix
For feature requests:
- Read all related code files in full (no truncation)
- Propose the most concise implementation approach
- List affected files and changes needed

Do NOT implement unless explicitly asked. Analyze and propose only.

Let me read the relevant code files to understand the current implementation.

Let me check the Anthropic SDK types for cache control options and OpenAI's cache retention options:

Now let me check the env-api-keys.ts file to see the pattern for environment variable handling:

Analysis: Issue #967 - Add configurable cache retention

Summary

Type: Feature request

Request: Add support for extended cache retention for Anthropic and OpenAI APIs, controlled via environment variable (not user-facing setting).

Agreed approach (from discussion): Environment variable to keep it hidden from users who don't understand cost implications.

Provider Support

Provider	Default	Extended
Anthropic	`5m`	`1h` (costs more for cache writes)
OpenAI	`in_memory`	`24h` (no extra cost)

Current Implementation

Anthropic (anthropic.ts):

Uses cache_control: { type: "ephemeral" } without TTL (defaults to 5m)
Applied to: system prompt (line ~319-335) and last user message (line ~454-460)

OpenAI Responses (openai-responses.ts):

Uses prompt_cache_key: options?.sessionId (line 103)
No prompt_cache_retention field set (defaults to in_memory)

Proposed Implementation

Environment variable: PI_CACHE_RETENTION

Values: unset (default behavior) or persistent (extended retention)
Browser-safe check: typeof process !== "undefined" && process.env.PI_CACHE_RETENTION === "persistent"

Files to modify:

packages/ai/src/providers/anthropic.ts
- Add helper function to get cache control TTL from env
- Update buildParams() (~line 319-335): Add ttl: '1h' to system prompt cache_control when persistent
- Update convertMessages() (~line 454-460): Add ttl: '1h' to last user message cache_control when persistent
packages/ai/src/providers/openai-responses.ts
- Update buildParams() (~line 97-127): Add prompt_cache_retention: '24h' when persistent
packages/ai/README.md
- Document PI_CACHE_RETENTION environment variable
packages/ai/CHANGELOG.md
- Add entry under [Unreleased]

Code Changes

anthropic.ts - Add helper and modify cache_control:

typescript

// Helper to get cache TTL based on envfunction getCacheTtl(): '5m' | '1h' {  if (typeof process !== "undefined" && process.env.PI_CACHE_RETENTION === "persistent") {    return '1h';  }  return '5m';}
// In buildParams() and convertMessages():cache_control: { type: "ephemeral", ttl: getCacheTtl() }

openai-responses.ts - Add cache retention:

Alternative: Provider-specific env vars

Could also support provider-specific overrides:

ANTHROPIC_CACHE_TTL=1h
OPENAI_CACHE_RETENTION=24h

With PI_CACHE_RETENTION as unified fallback. This gives power users more control.

Add prompt_cache_retention: '24h' when persistent what does this mean? store: true? can't do that

Let me check the OpenAI types more carefully:

You're right to question. prompt_cache_retention is independent from store. It's specifically about prompt caching duration:

in-memory (default): short-lived cache
24h: keeps cached prefixes active up to 24 hours

No store: true required. The implementation for OpenAI is just:

The store field is about storing responses for later retrieval/fine-tuning, which is a different feature entirely.

PI_CACHE_RETENTION shoule be either "short" or "long". no provider has any more granular shit than that. then @packages/ai/src/providers/openai-responses-shared.ts (?) and anthropic should interpret that env var.

Got it. The cache retention param belongs in openai-responses.ts (in buildParams), not the shared file. The shared file is just for message conversion and stream processing.

Revised Analysis

Environment variable: PI_CACHE_RETENTION

Values: short (default behavior) or long (extended retention)
Unset = short

Files to modify:

packages/ai/src/providers/anthropic.ts
- Add helper to check env var
- When long: set cache_control: { type: "ephemeral", ttl: "1h" }
- When short/unset: keep current cache_control: { type: "ephemeral" } (defaults to 5m)
- Locations: buildParams() (system prompt) and convertMessages() (last user message)
packages/ai/src/providers/openai-responses.ts
- In buildParams(): add prompt_cache_retention: '24h' when env var is long
- When short/unset: don't set the field (defaults to in-memory)
packages/ai/README.md - Document PI_CACHE_RETENTION
packages/ai/CHANGELOG.md - Add entry

Helper pattern (browser-safe):

we should also only do this if the baseUrl matches the OG providers

Good point. Need to check the base URLs to avoid sending provider-specific params to proxies/third-parties.

Final Analysis

Environment variable: PI_CACHE_RETENTION=long

Files to modify:

1. `packages/ai/src/providers/anthropic.ts`

Add helper and apply only for OG Anthropic:

typescript

function getCacheTtl(baseUrl: string): '5m' | '1h' | undefined {  if (    typeof process !== "undefined" &&    process.env.PI_CACHE_RETENTION === "long" &&    baseUrl.startsWith("https://api.anthropic.com")  ) {    return '1h';  }  return undefined; // SDK defaults to 5m}

In buildParams() and convertMessages():

2. `packages/ai/src/providers/openai-responses.ts`

In buildParams():

typescript

function getPromptCacheRetention(baseUrl: string): '24h' | undefined {  if (    typeof process !== "undefined" &&    process.env.PI_CACHE_RETENTION === "long" &&    baseUrl.startsWith("https://api.openai.com")  ) {    return '24h';  }  return undefined; // defaults to in-memory}
// In params:prompt_cache_retention: getPromptCacheRetention(model.baseUrl),

3. `packages/ai/README.md` - Document env var

4. `packages/ai/CHANGELOG.md` - Add entry

Behavior:

`PI_CACHE_RETENTION`	Anthropic (api.anthropic.com)	OpenAI (api.openai.com)	Other providers
unset / `short`	5m (default)	in-memory (default)	unchanged
`long`	1h	24h	unchanged

baseUrl.startsWith("https://api.anthropic.com") should be an includes without the https i think? do we check for that url in @packages/ai/src/providers/anthropic.ts already? sould mirror that then, same for openai

No existing checks in those files, but openai-completions.ts uses the pattern:

Updated Analysis

Use .includes() with domain:

Helper functions:

typescript

// anthropic.tsfunction getCacheTtl(baseUrl: string): '1h' | undefined {  if (    typeof process !== "undefined" &&    process.env.PI_CACHE_RETENTION === "long" &&    baseUrl.includes("api.anthropic.com")  ) {    return '1h';  }  return undefined;}
// openai-responses.ts  function getPromptCacheRetention(baseUrl: string): '24h' | undefined {  if (    typeof process !== "undefined" &&    process.env.PI_CACHE_RETENTION === "long" &&    baseUrl.includes("api.openai.com")  ) {    return '24h';  }  return undefined;}

lgtm, implement, set up a simple etst in pi-ai package for both providers, see stream.test.ts as an example

Now let me implement the changes. First, I'll update both provider files:

Now let me update the buildParams function to use the cache TTL:

Now I have all the locations. Let me update the buildParams function to use the cache TTL:

Now update the convertMessages function for the last user message:

Now let me update the openai-responses.ts file:

Now I'll add the helper function and update buildParams in openai-responses.ts:

Now update the buildParams function to include the cache retention:

Now let me create a test file for cache retention:

Now let me run the check to ensure everything compiles:

Let me run the cache retention test:

Let me run the test directly from the ai package:

All tests pass. Now let me update the CHANGELOG and README:

Now let me update the README:

Let me run the check one more time to ensure everything is good:

Implementation complete. Here's a summary:

Changes Made

1. `packages/ai/src/providers/anthropic.ts`

Added getCacheTtl() helper function that returns '1h' when PI_CACHE_RETENTION=long and baseUrl includes api.anthropic.com
Updated buildParams() to add ttl to cache_control for system prompts (both OAuth and non-OAuth)
Updated convertMessages() to add ttl to cache_control for the last user message

2. `packages/ai/src/providers/openai-responses.ts`

Added getPromptCacheRetention() helper function that returns '24h' when PI_CACHE_RETENTION=long and baseUrl includes api.openai.com
Updated buildParams() to include prompt_cache_retention in the request params

3. `packages/ai/test/cache-retention.test.ts`

New test file with 6 tests covering:
- Anthropic: default TTL, long TTL, proxy URL (no TTL)
- OpenAI: default retention, 24h retention, proxy URL (no retention)

4. `packages/ai/CHANGELOG.md`

Added entry under [Unreleased] documenting the new feature

5. `packages/ai/README.md`

Added "Cache Retention" section under Environment Variables explaining the feature

Usage

ok, please also pudate the env var section in @packages/coding-agent/README.md

Done. Added PI_CACHE_RETENTION to the Environment Variables table in packages/coding-agent/README.md.

commit and push, leave a comment on the issue explaining things concisely in my tone

Done. Committed, pushed, and commented on the issue. The fixes #967 in the commit message will auto-close it.

Summary

Analysis: Issue #967 - Add configurable cache retention

Summary

Provider Support

Current Implementation

Proposed Implementation

Code Changes

Alternative: Provider-specific env vars

Revised Analysis

Final Analysis

1. packages/ai/src/providers/anthropic.ts

2. packages/ai/src/providers/openai-responses.ts

3. packages/ai/README.md - Document env var

4. packages/ai/CHANGELOG.md - Add entry

Updated Analysis

Changes Made

1. packages/ai/src/providers/anthropic.ts

2. packages/ai/src/providers/openai-responses.ts

3. packages/ai/test/cache-retention.test.ts

4. packages/ai/CHANGELOG.md

5. packages/ai/README.md

Usage

Summary

Analysis: Issue #967 - Add configurable cache retention

Summary

Provider Support

Current Implementation

Proposed Implementation

Code Changes

Alternative: Provider-specific env vars

Revised Analysis

Final Analysis

1. packages/ai/src/providers/anthropic.ts

2. packages/ai/src/providers/openai-responses.ts

3. packages/ai/README.md - Document env var

4. packages/ai/CHANGELOG.md - Add entry

Updated Analysis

Changes Made

1. packages/ai/src/providers/anthropic.ts

2. packages/ai/src/providers/openai-responses.ts

3. packages/ai/test/cache-retention.test.ts

4. packages/ai/CHANGELOG.md

5. packages/ai/README.md

Usage

1. `packages/ai/src/providers/anthropic.ts`

2. `packages/ai/src/providers/openai-responses.ts`

3. `packages/ai/README.md` - Document env var

4. `packages/ai/CHANGELOG.md` - Add entry

1. `packages/ai/src/providers/anthropic.ts`

2. `packages/ai/src/providers/openai-responses.ts`

3. `packages/ai/test/cache-retention.test.ts`

4. `packages/ai/CHANGELOG.md`

5. `packages/ai/README.md`

1. `packages/ai/src/providers/anthropic.ts`

2. `packages/ai/src/providers/openai-responses.ts`

3. `packages/ai/README.md` - Document env var

4. `packages/ai/CHANGELOG.md` - Add entry

1. `packages/ai/src/providers/anthropic.ts`

2. `packages/ai/src/providers/openai-responses.ts`

3. `packages/ai/test/cache-retention.test.ts`

4. `packages/ai/CHANGELOG.md`

5. `packages/ai/README.md`