Analyzed and fixed GitHub issue #967
Public
Analyzed and fixed GitHub issue #967
Agent
Pi
Model
Claude Opus 4.5 (latest)
Started
Jan 29, 1:06 AM
Messages201
Downloads0
Reviewed GitHub issue and related code.
UnderstandChecked SDK cache control options.
UnderstandCommitted and pushed fix with auto-close message.
Edit
Analyze GitHub issue(s): https://github.com/badlogic/pi-mono/issues/967
For each issue:
Read the issue in full, including all comments and linked issues/PRs.
For bugs:
For feature requests:
Do NOT implement unless explicitly asked. Analyze and propose only.
The user wants me to analyze GitHub issue #967. Let me fetch the issue details first.
>gh issue view 967 --json title,body,comments,labels,state --repo badlogic/pi-mono{"body":"### What do you want to change?\n\nExtend pi to utilize the extended/persistent caching mechanisms available for OpenAI and Anthropic APIs. (Configurable as a opt-in setting of course)\n\n\n\n### Why?\n\nBoth Anthropic and OpenAI support prompt caching to reduce costs and latency, but with different retention options:\n\n- **Anthropic**: Cache TTL of `5m` (default) or `1h`\n- **OpenAI**: Cache retention of `in_memory` (default) or `24h`\n\nCurrently, pi-ai uses the default cache settings for both providers.\nThe current default 5min cache is (atleast for my apparently too slow brain) a bit short. When coding with pi, I'm often running into situations where I want to check what the agent did in the last turn, and blow way past the 5 min timer to continue making use of the cache. Using much more of my quota than necessary this way.\n\n(I'm also building an extension (\"cache watchdog\") for that, that sends regular \"keep-alive pings\" to the api to keep the current content fresh. But in a lot of cases it would be more efficient and cheaper to use the longer cache retention. Especially for OpenAI endpoints, as the longer 24h cache doesnt cost anything extra, which should be taken advantage off.)\n\n### How? (optional)\n\n\nAdd a unified `cacheRetention` option to `SimpleStreamOptions` with two values:\n- `\"ephemeral\"` - short-lived cache (Anthropic: 5m, OpenAI: in-memory)\n- `\"persistent\"` - longer-lived cache (Anthropic: 1h, OpenAI: 24h)\n\nThis follows the same pattern as `reasoning` - a unified abstraction that maps to provider-specific settings, silently ignored by providers that don't support it.\n\n**Scope**\n\n- **pi-ai**: Add `CacheRetention` type, extend `SimpleStreamOptions`, update Anthropic and OpenAI Responses providers\n- **pi-agent**: Pass through `cacheRetention` from `AgentOptions` to `AgentLoopConfig`\n- **pi-coding-agent**: Add `cacheRetention` to settings, wire through to Agent","comments":[{"id":"IC_kwDOPbFNk87imlaz","author":{"login":"badlogic"},"authorAssociation":"OWNER","body":"For most people > 5 min retention will cost more, so I'm hesitant exposing it as a in your face setting. People don't read and don't understand the implications.\n\nI think I'd rather have this as an environment variable that the providers check (if process is defined, as they are also meant to run in the browser if possible, which both the current implementations are capable of).\n\nWould that fit your needs?","createdAt":"2026-01-26T21:09:04Z","includesCreatedEdit":false,"isMinimized":false,"minimizedReason":"","reactionGroups":[],"url":"https://github.com/badlogic/pi-mono/issues/967#issuecomment-3801765555","viewerDidAuthor":true},{"id":"IC_kwDOPbFNk87ir1rn","author":{"login":"StaticShielding"},"authorAssociation":"NONE","body":"Yeah, you're probably right. That might be a more sensible option. \nAnd, if I'm following correctly, it would mean only changes to the pi-ai package. \n\nI can then control it manually or via an extension, if I want more fine-grained control over the duration. Should work just as good for my usecase. \n\nHappy to provide a PR, if you can wait a few days, as I'm on the road. ","createdAt":"2026-01-27T05:07:44Z","includesCreatedEdit":false,"isMinimized":false,"minimizedReason":"","reactionGroups":[],"url":"https://github.com/badlogic/pi-mono/issues/967#issuecomment-3803142887","viewerDidAuthor":false},{"id":"IC_kwDOPbFNk87jDXGF","author":{"login":"algal"},"authorAssociation":"NONE","body":"For what it's worth, I learned all about the 5min vs 1h ttl today, when I realized that the 30 min heartbeat of automatic prompts used by Molty (once, Clawdbot) falls outside the 5 min ttl window.\n\nAs a result, if you have a long chat session with Opus-4.5, and a 30 min heartbeat, you get an expensive cache miss every 30 minutes. I end up spending a couple hundred dollars this way. What's a little confusing is that clawdbot.json has `params.cacheControlTtl:1h` configured by default, IUUC, so one might be mislead into thinking that value has an effect.","createdAt":"2026-01-28T06:45:35Z","includesCreatedEdit":false,"isMinimized":false,"minimizedReason":"","reactionGroups":[],"url":"https://github.com/badlogic/pi-mono/issues/967#issuecomment-3809309061","viewerDidAuthor":false},{"id":"IC_kwDOPbFNk87jYD3k","author":{"login":"badlogic"},"authorAssociation":"OWNER","body":"While clawdbot's heartbeat is kind of related to prompt caching, it's not intended as a mechanism to prevent cache invalidation (I don't work on clawdbot, so I don't know all the internals, this is just what I remember from discussing this with Peter a while ago). Assuming you stuck to the default heartbeat of 30 minutes, you have 48 wake ups per day. Assuming your context is 160tokens long, and you use Anthropic API with Opus 4.5 or Sonnet 4.5 and these prices:\n\n<img width=\"894\" height=\"557\" alt=\"Image\" src=\"https://github.com/user-attachments/assets/27ba313c-2186-4ab6-8937-0ac0adbb86d3\" />\n\nThat's\n- 48 * 0.16 * $10 = $76 / day for Opus\n- 48 * 0.16 * $6 = $46 / day for Sonnet\n- 48 * 0.16 * $3 = $23 / day for Haiku\n\njust for cache writes. I posit that clawdbots heartbeat mechanism is ill-conceived, as it sends full context for what's basically something that does not require full context. I also posit that a fixed rate heartbeat is a very bad event model for the tasks that are listed here:\n\nhttps://github.com/moltbot/moltbot/blob/fdcac0ccf44704f384f891f8c1016af531e88cb9/docs/automation/cron-vs-heartbeat.md?plain=1#L61\n\nNeither 1h TTL not 5 minute TTL can fix the cost issue with this design.","createdAt":"2026-01-29T00:55:26Z","includesCreatedEdit":false,"isMinimized":false,"minimizedReason":"","reactionGroups":[],"url":"https://github.com/badlogic/pi-mono/issues/967#issuecomment-3814735332","viewerDidAuthor":true}],"labels":[],"state":"OPEN","title":"Add configurable cache retention (for Anthropic and OpenAI)"}Let me analyze this issue. It's a feature request to add configurable cache retention for Anthropic and OpenAI APIs.
The discussion concluded that instead of exposing it as a user-facing setting, it should be an environment variable that the providers check (to avoid confusion for users who don't understand the cost implications).
Let me read the relevant code files to understand how the streaming works for Anthropic and OpenAI providers.
Let me read the relevant code files to understand the current implementation.
Now I have a good understanding of the current implementation. Let me look at where the cache control is set in the Anthropic provider. I can see it's using cache_control: { type: "ephemeral" } in several places:
buildParams() for system promptsconvertMessages() for the last user messageFor OpenAI Responses, I see prompt_cache_key: options?.sessionId but I don't see any cache retention settings. Let me check what OpenAI supports for this.
The issue discussion concluded that:
Let me check the Anthropic SDK types for cache_control to see what TTL options are available.
Let me check the Anthropic SDK types for cache control options and OpenAI's cache retention options:
>cd /Users/badlogic/workspaces/pi-mono && rg -n "cache_control" node_modules/@anthropic-ai/sdk/resources/messages.d.ts 2>/dev/null | head -50(no output)>cd /Users/badlogic/workspaces/pi-mono && rg -n "prompt_cache" node_modules/openai/resources/responses/responses.d.ts 2>/dev/null | head -20449: prompt_cache_key?: string;
456: prompt_cache_retention?: 'in-memory' | '24h' | null;
520: * `prompt_cache_key`. Use `prompt_cache_key` instead to maintain caching
5060: prompt_cache_key?: string;
5067: prompt_cache_retention?: 'in-memory' | '24h' | null;
5181: * `prompt_cache_key`. Use `prompt_cache_key` instead to maintain cachingI can see OpenAI has prompt_cache_retention?: 'in-memory' | '24h' | null; in the responses API.
Now let me check Anthropic's types for the cache_control TTL options:
>cd /Users/badlogic/workspaces/pi-mono && rg -n "ephemeral" node_modules/@anthropic-ai/sdk --include="*.ts" --include="*.d.ts" 2>/dev/null | head -30(no output)>cd /Users/badlogic/workspaces/pi-mono && rg -n "ttl" node_modules/@anthropic-ai/sdk --include="*.ts" --include="*.d.ts" -i 2>/dev/null | head -30(no output)Let me search more broadly in the Anthropic SDK:
>cd /Users/badlogic/workspaces/pi-mono && rg -n "cache" node_modules/@anthropic-ai/sdk/resources/messages.d.ts 2>/dev/null | head -30(no output)Let me try a different approach:
>cd /Users/badlogic/workspaces/pi-mono && find node_modules/@anthropic-ai -name "*.d.ts" 2>/dev/null | head -10node_modules/@anthropic-ai/sandbox-runtime/dist/cli.d.ts
node_modules/@anthropic-ai/sandbox-runtime/dist/utils/debug.d.ts
node_modules/@anthropic-ai/sandbox-runtime/dist/utils/ripgrep.d.ts
node_modules/@anthropic-ai/sandbox-runtime/dist/utils/platform.d.ts
node_modules/@anthropic-ai/sandbox-runtime/dist/sandbox/linux-sandbox-utils.d.ts
node_modules/@anthropic-ai/sandbox-runtime/dist/sandbox/generate-seccomp-filter.d.ts
node_modules/@anthropic-ai/sandbox-runtime/dist/sandbox/sandbox-manager.d.ts
node_modules/@anthropic-ai/sandbox-runtime/dist/sandbox/socks-proxy.d.ts
node_modules/@anthropic-ai/sandbox-runtime/dist/sandbox/sandbox-config.d.ts
node_modules/@anthropic-ai/sandbox-runtime/dist/sandbox/sandbox-schemas.d.tsThe SDK types might be in a different location. Let me search more broadly.
>cd /Users/badlogic/workspaces/pi-mono && rg -l "cache_control" node_modules/@anthropic-ai/sdk 2>/dev/null | head -5node_modules/@anthropic-ai/sdk/src/resources/messages/messages.ts
node_modules/@anthropic-ai/sdk/src/resources/beta/messages/messages.ts
node_modules/@anthropic-ai/sdk/resources/messages/messages.d.ts
node_modules/@anthropic-ai/sdk/resources/messages/messages.d.mts
node_modules/@anthropic-ai/sdk/resources/beta/messages/messages.d.mtsLet me check the type definition file:
>cd /Users/badlogic/workspaces/pi-mono && rg -n -A5 "cache_control" node_modules/@anthropic-ai/sdk/resources/messages/messages.d.ts 2>/dev/null | head -50197: cache_control?: CacheControlEphemeral | null;
198- citations?: CitationsConfigParam | null;
199- context?: string | null;
200- title?: string | null;
201-}
202-export interface ImageBlockParam {
--
208: cache_control?: CacheControlEphemeral | null;
209-}
210-export interface InputJSONDelta {
211- partial_json: string;
212- type: 'input_json_delta';
213-}
--
440: cache_control?: CacheControlEphemeral | null;
441- citations?: CitationsConfigParam;
442-}
443-export interface ServerToolUsage {
444- /**
445- * The number of web search tool requests.
--
463: cache_control?: CacheControlEphemeral | null;
464-}
465-export interface SignatureDelta {
466- signature: string;
467- type: 'signature_delta';
468-}
--
488: cache_control?: CacheControlEphemeral | null;
489- citations?: Array<TextCitationParam> | null;
490-}
491-export type TextCitation = CitationCharLocation | CitationPageLocation | CitationContentBlockLocation | CitationsWebSearchResultLocation | CitationsSearchResultLocation;
492-export type TextCitationParam = CitationCharLocationParam | CitationPageLocationParam | CitationContentBlockLocationParam | CitationWebSearchResultLocationParam | CitationSearchResultLocationParam;
493-export interface TextDelta {
--
558: cache_control?: CacheControlEphemeral | null;
559- /**
560- * Description of what this tool does.
561- *
562- * Tool descriptions should be as detailed as possible. The more information that
563- * the model has about what the tool is and how to use it, the better it will
--
595: cache_control?: CacheControlEphemeral | null;
596-}
597-/**
598- * How the model should use the provided tools. The model can use a specific tool,
599- * any available tool, decide by itself, or not use tools at all.
600- */
--
657: cache_control?: CacheControlEphemeral | null;Now let me see what CacheControlEphemeral is:
>cd /Users/badlogic/workspaces/pi-mono && rg -n "CacheControlEphemeral" node_modules/@anthropic-ai/sdk/resources/messages/messages.d.ts 2>/dev/null | head -1067:export interface CacheControlEphemeral {
197: cache_control?: CacheControlEphemeral | null;
208: cache_control?: CacheControlEphemeral | null;
440: cache_control?: CacheControlEphemeral | null;
463: cache_control?: CacheControlEphemeral | null;
488: cache_control?: CacheControlEphemeral | null;
558: cache_control?: CacheControlEphemeral | null;
595: cache_control?: CacheControlEphemeral | null;
657: cache_control?: CacheControlEphemeral | null;
672: cache_control?: CacheControlEphemeral | null;Let me see the actual definition of CacheControlEphemeral:
>cd /Users/badlogic/workspaces/pi-mono && rg -n -A10 "^export interface CacheControlEphemeral" node_modules/@anthropic-ai/sdk/resources/messages/messages.d.ts 2>/dev/null67:export interface CacheControlEphemeral {
68- type: 'ephemeral';
69- /**
70- * The time-to-live for the cache control breakpoint.
71- *
72- * This may be one the following values:
73- *
74- * - `5m`: 5 minutes
75- * - `1h`: 1 hour
76- *
77- * Defaults to `5m`.Now I see the Anthropic TTL options. Let me also check if there's a ttl field:
>cd /Users/badlogic/workspaces/pi-mono && rg -n -A15 "^export interface CacheControlEphemeral" node_modules/@anthropic-ai/sdk/resources/messages/messages.d.ts 2>/dev/null67:export interface CacheControlEphemeral {
68- type: 'ephemeral';
69- /**
70- * The time-to-live for the cache control breakpoint.
71- *
72- * This may be one the following values:
73- *
74- * - `5m`: 5 minutes
75- * - `1h`: 1 hour
76- *
77- * Defaults to `5m`.
78- */
79- ttl?: '5m' | '1h';
80-}
81-export interface CacheCreation {
82- /**Now I have all the information I need. Let me also check where the current cache_control is set in the anthropic.ts file to understand all the places that need to be modified:
Now let me check the env-api-keys.ts file to see the pattern for environment variable handling:
Now I have a complete understanding of the issue and the codebase. Let me summarize my analysis:
Issue #967: Add configurable cache retention for Anthropic and OpenAI
This is a feature request to add support for extended cache retention:
5m (default) or 1h TTLin_memory (default) or 24hDiscussion outcome: Instead of exposing this as a user-facing setting (which could confuse users about cost implications), it should be controlled via environment variables that the providers check.
packages/ai/src/providers/anthropic.ts
PI_CACHE_RETENTION or ANTHROPIC_CACHE_TTL)cache_control objects to include ttl: '1h' when persistent cache is enabledbuildParams() - system prompt cache_controlconvertMessages() - last user message cache_controlpackages/ai/src/providers/openai-responses.ts
PI_CACHE_RETENTION or OPENAI_CACHE_RETENTION)buildParams() to add prompt_cache_retention: '24h' when persistent cache is enabledbuildParams() - add prompt_cache_retention fieldOptions:
PI_CACHE_RETENTION=persistent (maps to 1h for Anthropic, 24h for OpenAI)ANTHROPIC_CACHE_TTL=1h and OPENAI_CACHE_RETENTION=24hA unified approach makes more sense since both providers distinguish between ephemeral (default, short-lived) and persistent (longer-lived) cache modes. The implementation can map a single PI_CACHE_RETENTION value to the appropriate TTL for each provider. The variable needs browser-safe access via typeof process !== "undefined" ? process.env.PI_CACHE_RETENTION : undefined, should default to current behavior (ephemeral), and should follow the pattern established in env-api-keys.ts for compatibility.
Type: Feature request
Request: Add support for extended cache retention for Anthropic and OpenAI APIs, controlled via environment variable (not user-facing setting).
Agreed approach (from discussion): Environment variable to keep it hidden from users who don't understand cost implications.
| Provider | Default | Extended |
|---|---|---|
| Anthropic | 5m | 1h (costs more for cache writes) |
| OpenAI | in_memory | 24h (no extra cost) |
Anthropic (anthropic.ts):
cache_control: { type: "ephemeral" } without TTL (defaults to 5m)OpenAI Responses (openai-responses.ts):
prompt_cache_key: options?.sessionId (line 103)prompt_cache_retention field set (defaults to in_memory)Environment variable: PI_CACHE_RETENTION
persistent (extended retention)typeof process !== "undefined" && process.env.PI_CACHE_RETENTION === "persistent"Files to modify:
packages/ai/src/providers/anthropic.ts
buildParams() (~line 319-335): Add ttl: '1h' to system prompt cache_control when persistentconvertMessages() (~line 454-460): Add ttl: '1h' to last user message cache_control when persistentpackages/ai/src/providers/openai-responses.ts
buildParams() (~line 97-127): Add prompt_cache_retention: '24h' when persistentpackages/ai/README.md
PI_CACHE_RETENTION environment variablepackages/ai/CHANGELOG.md
[Unreleased]anthropic.ts - Add helper and modify cache_control:
// Helper to get cache TTL based on envfunction getCacheTtl(): '5m' | '1h' { if (typeof process !== "undefined" && process.env.PI_CACHE_RETENTION === "persistent") { return '1h'; } return '5m';}
// In buildParams() and convertMessages():cache_control: { type: "ephemeral", ttl: getCacheTtl() }openai-responses.ts - Add cache retention:
// In buildParams():prompt_cache_key: options?.sessionId,prompt_cache_retention: (typeof process !== "undefined" && process.env.PI_CACHE_RETENTION === "persistent") ? '24h' : undefined,Could also support provider-specific overrides:
ANTHROPIC_CACHE_TTL=1hOPENAI_CACHE_RETENTION=24hWith PI_CACHE_RETENTION as unified fallback. This gives power users more control.
Add prompt_cache_retention: '24h' when persistent what does this mean? store: true? can't do that
The user is asking about the OpenAI implementation. Let me re-check the OpenAI Responses API types to understand what prompt_cache_retention actually does and whether it requires store: true.
Looking at the earlier search results:
449: prompt_cache_key?: string;456: prompt_cache_retention?: 'in-memory' | '24h' | null;Let me check the OpenAI documentation or types more carefully to understand the relationship between prompt_cache_key, prompt_cache_retention, and store.
Let me check the OpenAI types more carefully:
>cd /Users/badlogic/workspaces/pi-mono && rg -n -B5 -A10 "prompt_cache_retention" node_modules/openai/resources/responses/responses.d.ts 2>/dev/null | head -60451- * The retention policy for the prompt cache. Set to `24h` to enable extended
452- * prompt caching, which keeps cached prefixes active for longer, up to a maximum
453- * of 24 hours.
454- * [Learn more](https://platform.openai.com/docs/guides/prompt-caching#prompt-cache-retention).
455- */
456: prompt_cache_retention?: 'in-memory' | '24h' | null;
457- /**
458- * **gpt-5 and o-series models only**
459- *
460- * Configuration options for
461- * [reasoning models](https://platform.openai.com/docs/guides/reasoning).
462- */
463- reasoning?: Shared.Reasoning | null;
464- /**
465- * A stable identifier used to help detect users of your application that may be
466- * violating OpenAI's usage policies. The IDs should be a string that uniquely
--
5062- * The retention policy for the prompt cache. Set to `24h` to enable extended
5063- * prompt caching, which keeps cached prefixes active for longer, up to a maximum
5064- * of 24 hours.
5065- * [Learn more](https://platform.openai.com/docs/guides/prompt-caching#prompt-cache-retention).
5066- */
5067: prompt_cache_retention?: 'in-memory' | '24h' | null;
5068- /**
5069- * **gpt-5 and o-series models only**
5070- *
5071- * Configuration options for
5072- * [reasoning models](https://platform.openai.com/docs/guides/reasoning).
5073- */
5074- reasoning?: Shared.Reasoning | null;
5075- /**
5076- * A stable identifier used to help detect users of your application that may be
5077- * violating OpenAI's usage policies. The IDs should be a string that uniquelyThe prompt_cache_retention is a separate field from store. It's specifically for controlling how long the prompt cache is retained. It doesn't require store: true.
Let me also check if there's any relationship between prompt_cache_key and prompt_cache_retention:
>cd /Users/badlogic/workspaces/pi-mono && rg -n -B5 -A10 "prompt_cache_key" node_modules/openai/resources/responses/responses.d.ts 2>/dev/null | head -40444- /**
445- * Used by OpenAI to cache responses for similar requests to optimize your cache
446- * hit rates. Replaces the `user` field.
447- * [Learn more](https://platform.openai.com/docs/guides/prompt-caching).
448- */
449: prompt_cache_key?: string;
450- /**
451- * The retention policy for the prompt cache. Set to `24h` to enable extended
452- * prompt caching, which keeps cached prefixes active for longer, up to a maximum
453- * of 24 hours.
454- * [Learn more](https://platform.openai.com/docs/guides/prompt-caching#prompt-cache-retention).
455- */
456- prompt_cache_retention?: 'in-memory' | '24h' | null;
457- /**
458- * **gpt-5 and o-series models only**
459- *
--
515- * breakdown of output tokens, and the total tokens used.
516- */
517- usage?: ResponseUsage;
518- /**
519- * @deprecated This field is being replaced by `safety_identifier` and
520: * `prompt_cache_key`. Use `prompt_cache_key` instead to maintain caching
521- * optimizations. A stable identifier for your end-users. Used to boost cache hit
522- * rates by better bucketing similar requests and to help OpenAI detect and prevent
523- * abuse.
524- * [Learn more](https://platform.openai.com/docs/guides/safety-best-practices#safety-identifiers).
525- */
526- user?: string;
527-}
528-export declare namespace Response {
529- /**
530- * Details about why the response is incomplete.
--
5055- /**
5056- * Used by OpenAI to cache responses for similar requests to optimize your cache
5057- * hit rates. Replaces the `user` field.
5058- * [Learn more](https://platform.openai.com/docs/guides/prompt-caching).
5059- */
5060: prompt_cache_key?: string;prompt_cache_retention is independent from store. It's specifically about how long the prompt cache is retained. The docs say:
in-memory: default, short-lived24h: keeps cached prefixes active for longer, up to 24 hoursThis is separate from the store field which is about whether to store responses for later retrieval/fine-tuning.
So the implementation for OpenAI is straightforward - just add prompt_cache_retention: '24h' when the env var is set. No store: true needed.
You're right to question. prompt_cache_retention is independent from store. It's specifically about prompt caching duration:
in-memory (default): short-lived cache24h: keeps cached prefixes active up to 24 hoursNo store: true required. The implementation for OpenAI is just:
// In buildParams():prompt_cache_retention: (typeof process !== "undefined" && process.env.PI_CACHE_RETENTION === "persistent") ? '24h' : undefined,The store field is about storing responses for later retrieval/fine-tuning, which is a different feature entirely.
PI_CACHE_RETENTION shoule be either "short" or "long". no provider has any more granular shit than that. then @packages/ai/src/providers/openai-responses-shared.ts (?) and anthropic should interpret that env var.
The user wants:
PI_CACHE_RETENTION with values short or long (not persistent)openai-responses-shared.ts and anthropic.ts should read this env varLet me check openai-responses-shared.ts to see where the params are built:
I see - openai-responses-shared.ts is shared code for stream processing and message conversion, but the actual buildParams is in openai-responses.ts.
The prompt_cache_retention param needs to be added in openai-responses.ts in the buildParams function, not in the shared file.
Let me update my analysis:
Files to modify:
packages/ai/src/providers/anthropic.ts
PI_CACHE_RETENTION env varlong, set ttl: '1h' on cache_control objectspackages/ai/src/providers/openai-responses.ts
PI_CACHE_RETENTION env varlong, add prompt_cache_retention: '24h' to paramsThe shared file doesn't need to change since caching config is in the params building, not the message conversion or stream processing.
Got it. The cache retention param belongs in openai-responses.ts (in buildParams), not the shared file. The shared file is just for message conversion and stream processing.
Environment variable: PI_CACHE_RETENTION
short (default behavior) or long (extended retention)shortFiles to modify:
packages/ai/src/providers/anthropic.ts
long: set cache_control: { type: "ephemeral", ttl: "1h" }short/unset: keep current cache_control: { type: "ephemeral" } (defaults to 5m)buildParams() (system prompt) and convertMessages() (last user message)packages/ai/src/providers/openai-responses.ts
buildParams(): add prompt_cache_retention: '24h' when env var is longshort/unset: don't set the field (defaults to in-memory)packages/ai/README.md - Document PI_CACHE_RETENTION
packages/ai/CHANGELOG.md - Add entry
Helper pattern (browser-safe):
function useLongCacheRetention(): boolean { return typeof process !== "undefined" && process.env.PI_CACHE_RETENTION === "long";}we should also only do this if the baseUrl matches the OG providers
The user is right - we should only apply these cache retention settings when using the actual Anthropic and OpenAI APIs, not when using proxies or other providers that happen to use the same API format.
Let me check how the baseUrl is structured for these providers and what the default URLs are.
Good point. Need to check the base URLs to avoid sending provider-specific params to proxies/third-parties.
>cd /Users/badlogic/workspaces/pi-mono && rg -n "baseUrl" packages/ai/src/providers/anthropic.ts packages/ai/src/providers/openai-responses.tspackages/ai/src/providers/openai-responses.ts:164: baseURL: model.baseUrl,
packages/ai/src/providers/anthropic.ts:413: baseURL: model.baseUrl,
packages/ai/src/providers/anthropic.ts:433: baseURL: model.baseUrl,>cd /Users/badlogic/workspaces/pi-mono && rg -n "api.anthropic.com|api.openai.com" packages/ai/packages/ai/src/models.generated.ts:950: baseUrl: "https://api.anthropic.com",
packages/ai/src/models.generated.ts:967: baseUrl: "https://api.anthropic.com",
packages/ai/src/models.generated.ts:984: baseUrl: "https://api.anthropic.com",
packages/ai/src/models.generated.ts:1001: baseUrl: "https://api.anthropic.com",
packages/ai/src/models.generated.ts:1018: baseUrl: "https://api.anthropic.com",
packages/ai/src/models.generated.ts:1035: baseUrl: "https://api.anthropic.com",
packages/ai/src/models.generated.ts:1052: baseUrl: "https://api.anthropic.com",
packages/ai/src/models.generated.ts:1069: baseUrl: "https://api.anthropic.com",
packages/ai/src/models.generated.ts:1086: baseUrl: "https://api.anthropic.com",
packages/ai/src/models.generated.ts:1103: baseUrl: "https://api.anthropic.com",
packages/ai/src/models.generated.ts:1120: baseUrl: "https://api.anthropic.com",
packages/ai/src/models.generated.ts:1137: baseUrl: "https://api.anthropic.com",
packages/ai/src/models.generated.ts:1154: baseUrl: "https://api.anthropic.com",
packages/ai/src/models.generated.ts:1171: baseUrl: "https://api.anthropic.com",
packages/ai/src/models.generated.ts:1188: baseUrl: "https://api.anthropic.com",
packages/ai/src/models.generated.ts:1205: baseUrl: "https://api.anthropic.com",
packages/ai/src/models.generated.ts:1222: baseUrl: "https://api.anthropic.com",
packages/ai/src/models.generated.ts:1239: baseUrl: "https://api.anthropic.com",
packages/ai/src/models.generated.ts:1256: baseUrl: "https://api.anthropic.com",
packages/ai/src/models.generated.ts:1273: baseUrl: "https://api.anthropic.com",
packages/ai/src/models.generated.ts:1290: baseUrl: "https://api.anthropic.com",
packages/ai/src/models.generated.ts:3790: baseUrl: "https://api.openai.com/v1",
packages/ai/src/models.generated.ts:3807: baseUrl: "https://api.openai.com/v1",
packages/ai/src/models.generated.ts:3824: baseUrl: "https://api.openai.com/v1",
packages/ai/src/models.generated.ts:3841: baseUrl: "https://api.openai.com/v1",
packages/ai/src/models.generated.ts:3858: baseUrl: "https://api.openai.com/v1",
packages/ai/src/models.generated.ts:3875: baseUrl: "https://api.openai.com/v1",
packages/ai/src/models.generated.ts:3892: baseUrl: "https://api.openai.com/v1",
packages/ai/src/models.generated.ts:3909: baseUrl: "https://api.openai.com/v1",
packages/ai/src/models.generated.ts:3926: baseUrl: "https://api.openai.com/v1",
packages/ai/src/models.generated.ts:3943: baseUrl: "https://api.openai.com/v1",
packages/ai/src/models.generated.ts:3960: baseUrl: "https://api.openai.com/v1",
packages/ai/src/models.generated.ts:3977: baseUrl: "https://api.openai.com/v1",
packages/ai/src/models.generated.ts:3994: baseUrl: "https://api.openai.com/v1",
packages/ai/src/models.generated.ts:4011: baseUrl: "https://api.openai.com/v1",
packages/ai/src/models.generated.ts:4028: baseUrl: "https://api.openai.com/v1",
packages/ai/src/models.generated.ts:4045: baseUrl: "https://api.openai.com/v1",
packages/ai/src/models.generated.ts:4062: baseUrl: "https://api.openai.com/v1",
packages/ai/src/models.generated.ts:4079: baseUrl: "https://api.openai.com/v1",
packages/ai/src/models.generated.ts:4096: baseUrl: "https://api.openai.com/v1",
packages/ai/src/models.generated.ts:4113: baseUrl: "https://api.openai.com/v1",
packages/ai/src/models.generated.ts:4130: baseUrl: "https://api.openai.com/v1",
packages/ai/src/models.generated.ts:4147: baseUrl: "https://api.openai.com/v1",
packages/ai/src/models.generated.ts:4164: baseUrl: "https://api.openai.com/v1",
packages/ai/src/models.generated.ts:4181: baseUrl: "https://api.openai.com/v1",
packages/ai/src/models.generated.ts:4198: baseUrl: "https://api.openai.com/v1",
packages/ai/src/models.generated.ts:4215: baseUrl: "https://api.openai.com/v1",
packages/ai/src/models.generated.ts:4232: baseUrl: "https://api.openai.com/v1",
packages/ai/src/models.generated.ts:4249: baseUrl: "https://api.openai.com/v1",
packages/ai/src/models.generated.ts:4266: baseUrl: "https://api.openai.com/v1",
packages/ai/src/models.generated.ts:4283: baseUrl: "https://api.openai.com/v1",
packages/ai/src/models.generated.ts:4300: baseUrl: "https://api.openai.com/v1",
packages/ai/src/models.generated.ts:4317: baseUrl: "https://api.openai.com/v1",
packages/ai/src/models.generated.ts:4334: baseUrl: "https://api.openai.com/v1",
packages/ai/src/models.generated.ts:4351: baseUrl: "https://api.openai.com/v1",
packages/ai/scripts/generate-models.ts:275: baseUrl: "https://api.anthropic.com",
packages/ai/scripts/generate-models.ts:327: baseUrl: "https://api.openai.com/v1",
packages/ai/scripts/generate-models.ts:638: baseUrl: "https://api.openai.com/v1",
packages/ai/scripts/generate-models.ts:658: baseUrl: "https://api.openai.com/v1",
packages/ai/scripts/generate-models.ts:678: baseUrl: "https://api.openai.com/v1",
packages/ai/src/utils/oauth/openai-codex.ts:28:const JWT_CLAIM_PATH = "https://api.openai.com/auth";
packages/ai/src/providers/openai-codex-responses.ts:30:const JWT_CLAIM_PATH = "https://api.openai.com/auth" as const;
packages/ai/test/openai-codex-stream.test.ts:27: JSON.stringify({ "https://api.openai.com/auth": { chatgpt_account_id: "acc_test" } }),
packages/ai/test/openai-codex-stream.test.ts:138: JSON.stringify({ "https://api.openai.com/auth": { chatgpt_account_id: "acc_test" } }),
packages/ai/test/openai-codex-stream.test.ts:239: JSON.stringify({ "https://api.openai.com/auth": { chatgpt_account_id: "acc_test" } }),Now I have the full picture:
https://api.anthropic.comhttps://api.openai.com/v1The helper should check:
PI_CACHE_RETENTION === "long"model.baseUrl matches the OG provider URLHelper pattern:
function useLongCacheRetention(baseUrl: string, ogBaseUrl: string): boolean { return ( typeof process !== "undefined" && process.env.PI_CACHE_RETENTION === "long" && baseUrl.startsWith(ogBaseUrl) );}For Anthropic: baseUrl.startsWith("https://api.anthropic.com")
For OpenAI: baseUrl.startsWith("https://api.openai.com")
Environment variable: PI_CACHE_RETENTION=long
Files to modify:
packages/ai/src/providers/anthropic.tsAdd helper and apply only for OG Anthropic:
function getCacheTtl(baseUrl: string): '5m' | '1h' | undefined { if ( typeof process !== "undefined" && process.env.PI_CACHE_RETENTION === "long" && baseUrl.startsWith("https://api.anthropic.com") ) { return '1h'; } return undefined; // SDK defaults to 5m}In buildParams() and convertMessages():
cache_control: { type: "ephemeral", ttl: getCacheTtl(model.baseUrl) }// or if ttl is undefined, just { type: "ephemeral" }packages/ai/src/providers/openai-responses.tsIn buildParams():
function getPromptCacheRetention(baseUrl: string): '24h' | undefined { if ( typeof process !== "undefined" && process.env.PI_CACHE_RETENTION === "long" && baseUrl.startsWith("https://api.openai.com") ) { return '24h'; } return undefined; // defaults to in-memory}
// In params:prompt_cache_retention: getPromptCacheRetention(model.baseUrl),packages/ai/README.md - Document env varpackages/ai/CHANGELOG.md - Add entryBehavior:
PI_CACHE_RETENTION | Anthropic (api.anthropic.com) | OpenAI (api.openai.com) | Other providers |
|---|---|---|---|
unset / short | 5m (default) | in-memory (default) | unchanged |
long | 1h | 24h | unchanged |
baseUrl.startsWith("https://api.anthropic.com") should be an includes without the https i think? do we check for that url in @packages/ai/src/providers/anthropic.ts already? sould mirror that then, same for openai
Let me check how URL matching is done in both providers currently.
>cd /Users/badlogic/workspaces/pi-mono && rg -n "baseUrl|baseURL" packages/ai/src/providers/anthropic.ts packages/ai/src/providers/openai-responses.ts | head -30packages/ai/src/providers/openai-responses.ts:164: baseURL: model.baseUrl,
packages/ai/src/providers/anthropic.ts:413: baseURL: model.baseUrl,
packages/ai/src/providers/anthropic.ts:433: baseURL: model.baseUrl,>cd /Users/badlogic/workspaces/pi-mono && rg -n "includes.*openai|includes.*anthropic" packages/ai/src/providers/packages/ai/src/providers/amazon-bedrock.ts:202: if (model.id.includes("anthropic.claude") || model.id.includes("anthropic/claude")) {
packages/ai/src/providers/amazon-bedrock.ts:373: return id.includes("anthropic.claude") || id.includes("anthropic/claude");
packages/ai/src/providers/amazon-bedrock.ts:591: if (model.id.includes("anthropic.claude")) {Let me check the openai-completions.ts file since that's likely where URL detection happens for compatibility checks:
>cd /Users/badlogic/workspaces/pi-mono && rg -n "baseUrl.*includes|includes.*url" packages/ai/src/providers/ -ipackages/ai/src/providers/google.ts:316: httpOptions.apiVersion = ""; // baseUrl already includes version path, don't append
packages/ai/src/providers/openai-completions.ts:451: if (model.baseUrl.includes("openrouter.ai") && model.compat?.openRouterRouting) {
packages/ai/src/providers/openai-completions.ts:767: const isZai = provider === "zai" || baseUrl.includes("api.z.ai");
packages/ai/src/providers/openai-completions.ts:771: baseUrl.includes("cerebras.ai") ||
packages/ai/src/providers/openai-completions.ts:773: baseUrl.includes("api.x.ai") ||
packages/ai/src/providers/openai-completions.ts:775: baseUrl.includes("mistral.ai") ||
packages/ai/src/providers/openai-completions.ts:776: baseUrl.includes("chutes.ai") ||
packages/ai/src/providers/openai-completions.ts:777: baseUrl.includes("deepseek.com") ||
packages/ai/src/providers/openai-completions.ts:780: baseUrl.includes("opencode.ai");
packages/ai/src/providers/openai-completions.ts:782: const useMaxTokens = provider === "mistral" || baseUrl.includes("mistral.ai") || baseUrl.includes("chutes.ai");
packages/ai/src/providers/openai-completions.ts:784: const isGrok = provider === "xai" || baseUrl.includes("api.x.ai");
packages/ai/src/providers/openai-completions.ts:786: const isMistral = provider === "mistral" || baseUrl.includes("mistral.ai");>cd /Users/badlogic/workspaces/pi-mono && rg -n "api\.openai\.com|api\.anthropic\.com" packages/ai/src/providers/packages/ai/src/providers/openai-codex-responses.ts:30:const JWT_CLAIM_PATH = "https://api.openai.com/auth" as const;I see the pattern now. In openai-completions.ts, they use baseUrl.includes("domain") pattern, like:
baseUrl.includes("api.z.ai")baseUrl.includes("cerebras.ai")baseUrl.includes("api.x.ai")baseUrl.includes("mistral.ai")baseUrl.includes("openrouter.ai")So for our case:
baseUrl.includes("api.anthropic.com")baseUrl.includes("api.openai.com")There's no existing check for these URLs in the anthropic.ts or openai-responses.ts files, but the pattern from openai-completions.ts is clear: use .includes() with the domain.
No existing checks in those files, but openai-completions.ts uses the pattern:
baseUrl.includes("api.z.ai")baseUrl.includes("cerebras.ai")baseUrl.includes("api.x.ai")baseUrl.includes("mistral.ai")baseUrl.includes("openrouter.ai")Use .includes() with domain:
// anthropic.tsbaseUrl.includes("api.anthropic.com")
// openai-responses.tsbaseUrl.includes("api.openai.com")Helper functions:
// anthropic.tsfunction getCacheTtl(baseUrl: string): '1h' | undefined { if ( typeof process !== "undefined" && process.env.PI_CACHE_RETENTION === "long" && baseUrl.includes("api.anthropic.com") ) { return '1h'; } return undefined;}
// openai-responses.ts function getPromptCacheRetention(baseUrl: string): '24h' | undefined { if ( typeof process !== "undefined" && process.env.PI_CACHE_RETENTION === "long" && baseUrl.includes("api.openai.com") ) { return '24h'; } return undefined;}lgtm, implement, set up a simple etst in pi-ai package for both providers, see stream.test.ts as an example
The user wants me to implement the feature. Let me:
Let me start by reading the files I need to modify.
Now I understand the test pattern. The tests:
describe.skipIf(!process.env.XXX_API_KEY) to conditionally run tests based on env varsbasicTextGeneration, handleToolCall, etc.getModel to get model configurationsFor the cache retention test, I need to:
anthropic.ts and openai-responses.tsPI_CACHE_RETENTION env var is respectedLet me now implement the changes:
anthropic.ts to add cache TTL supportopenai-responses.ts to add cache retention supportLet me start with the implementation.
Now let me implement the changes. First, I'll update both provider files:
Now I need to find and update the places where cache_control is set. Let me look at the buildParams function and convertMessages function in anthropic.ts to update them.
Now let me update the buildParams function to use the cache TTL:
I need to find the buildParams function and the convertMessages function. Let me continue reading.
Now I can see the buildParams function. I need to update the cache_control objects to include the TTL. The cache_control is set in multiple places:
Let me continue reading to find the part where cache_control is added to the last user message.
Now I have all the locations where cache_control is set. I need to update:
buildParams function - OAuth token Claude Code identity (~line 473)buildParams function - OAuth token context.systemPrompt (~line 483)buildParams function - Non-OAuth token context.systemPrompt (~line 493)convertMessages function - last user message (~line 676)The issue is that buildParams doesn't have access to model.baseUrl directly. I need to pass the model to check the baseUrl. Let me look at how buildParams is called.Looking at the code:
const params = buildParams(model, context, isOAuthToken, options);So model is available in buildParams. Good. Now I need to update all the cache_control locations.
Let me do the edits one by one.
Now I have all the locations. Let me update the buildParams function to use the cache TTL:
Now I need to update the convertMessages function to also use the cache TTL for the last user message. The issue is that convertMessages doesn't have access to model.baseUrl. I need to pass the baseUrl or use a different approach.Looking at the function signature:```typescriptfunction convertMessages(
messages: Message[],
model: Model<"anthropic-messages">,
isOAuthToken: boolean,
): MessageParam[] {
The model is available! So I can use `model.baseUrl` directly. Let me update the cache_control for the last user message.Now update the convertMessages function for the last user message:
Now I need to update the openai-responses.ts file. Let me read it again to find where to add the cache retention.
Now let me update the openai-responses.ts file:
Let me continue reading to find the buildParams function:
Now I have the buildParams function. I need to add the helper function and update the params to include prompt_cache_retention. Let me add the helper after the imports and update the buildParams function.
Now I'll add the helper function and update buildParams in openai-responses.ts:
Now I need to update the buildParams function to use the cache retention. Let me update the params object to include prompt_cache_retention.
Now update the buildParams function to include the cache retention:
Now I need to create the test file. Let me create a simple test that verifies the cache retention env var is respected. The test should:1. Check that the payload includes the correct cache_retention/ttl when PI_CACHE_RETENTION=long 2. Check that it doesn't include them when not set or when baseUrl doesn't match
I'll use the onPayload callback to inspect the params being sent.
Now let me create a test file for cache retention:
Now let me run the check to make sure everything compiles and the tests pass:
Now let me run the check to ensure everything compiles:
>cd /Users/badlogic/workspaces/pi-mono && npm run check 2>&1> pi-monorepo@0.0.3 check
> biome check --write . && tsgo --noEmit && cd packages/web-ui && npm run check
Checked 444 files in 321ms. Fixed 1 file.
> @mariozechner/pi-web-ui@0.50.1 check
> biome check --write . && tsc --noEmit && cd example && biome check --write . && tsc --noEmit
Checked 73 files in 40ms. No fixes applied.
Checked 3 files in 8ms. No fixes applied.Everything compiles. Now let me run the specific test to make sure it works:
Let me run the cache retention test:
>cd /Users/badlogic/workspaces/pi-mono && npm test -- packages/ai/test/cache-retention.test.ts 2>&1> pi-monorepo@0.0.3 test
> npm run test --workspaces --if-present packages/ai/test/cache-retention.test.ts
> @mariozechner/pi-agent-core@0.50.1 test
> vitest --run packages/ai/test/cache-retention.test.ts
RUN v3.2.4 /Users/badlogic/workspaces/pi-mono/packages/agent
No test files found, exiting with code 1
filter: packages/ai/test/cache-retention.test.ts
include: **/*.{test,spec}.?(c|m)[jt]s?(x)
exclude: **/node_modules/**, **/dist/**, **/cypress/**, **/.{idea,git,cache,output,temp}/**, **/{karma,rollup,webpack,vite,vitest,jest,ava,babel,nyc,cypress,tsup,build,eslint,prettier}.config.*
npm error Lifecycle script `test` failed with error:
npm error code 1
npm error path /Users/badlogic/workspaces/pi-mono/packages/agent
npm error workspace @mariozechner/pi-agent-core@0.50.1
npm error location /Users/badlogic/workspaces/pi-mono/packages/agent
npm error command failed
npm error command sh -c vitest --run packages/ai/test/cache-retention.test.ts
> @mariozechner/pi-ai@0.50.1 test
> vitest --run packages/ai/test/cache-retention.test.ts
RUN v3.2.4 /Users/badlogic/workspaces/pi-mono/packages/ai
No test files found, exiting with code 1
filter: packages/ai/test/cache-retention.test.ts
include: **/*.{test,spec}.?(c|m)[jt]s?(x)
exclude: **/node_modules/**, **/dist/**, **/cypress/**, **/.{idea,git,cache,output,temp}/**, **/{karma,rollup,webpack,vite,vitest,jest,ava,babel,nyc,cypress,tsup,build,eslint,prettier}.config.*
npm error Lifecycle script `test` failed with error:
npm error code 1
npm error path /Users/badlogic/workspaces/pi-mono/packages/ai
npm error workspace @mariozechner/pi-ai@0.50.1
npm error location /Users/badlogic/workspaces/pi-mono/packages/ai
npm error command failed
npm error command sh -c vitest --run packages/ai/test/cache-retention.test.ts
> @mariozechner/pi-coding-agent@0.50.1 test
> vitest --run packages/ai/test/cache-retention.test.ts
RUN v3.2.4 /Users/badlogic/workspaces/pi-mono/packages/coding-agent
No test files found, exiting with code 1
filter: packages/ai/test/cache-retention.test.ts
include: **/*.{test,spec}.?(c|m)[jt]s?(x)
exclude: **/node_modules/**, **/dist/**, **/cypress/**, **/.{idea,git,cache,output,temp}/**, **/{karma,rollup,webpack,vite,vitest,jest,ava,babel,nyc,cypress,tsup,build,eslint,prettier}.config.*
npm error Lifecycle script `test` failed with error:
npm error code 1
npm error path /Users/badlogic/workspaces/pi-mono/packages/coding-agent
npm error workspace @mariozechner/pi-coding-agent@0.50.1
npm error location /Users/badlogic/workspaces/pi-mono/packages/coding-agent
npm error command failed
npm error command sh -c vitest --run packages/ai/test/cache-retention.test.ts
> @mariozechner/pi-tui@0.50.1 test
> node --test --import tsx test/*.test.ts packages/ai/test/cache-retention.test.ts
Result: null
Result: null
Result: {
items: [
{ value: '/.nofollow/', label: '.nofollow/' },
{ value: '/.resolve/', label: '.resolve/' },
{ value: '/.vol/', label: '.vol/' },
{ value: '/Applications/', label: 'Applications/' },
{ value: '/bin/', label: 'bin/' },
{ value: '/cores/', label: 'cores/' },
{ value: '/dev/', label: 'dev/' },
{ value: '/etc/', label: 'etc/' },
{ value: '/home/', label: 'home/' },
{ value: '/Library/', label: 'Library/' },
{ value: '/opt/', label: 'opt/' },
{ value: '/private/', label: 'private/' },
{ value: '/sbin/', label: 'sbin/' },
{ value: '/System/', label: 'System/' },
{ value: '/tmp/', label: 'tmp/' },
{ value: '/Users/', label: 'Users/' },
{ value: '/usr/', label: 'usr/' },
{ value: '/var/', label: 'var/' },
{ value: '/Volumes/', label: 'Volumes/' },
{ value: '/.file', label: '.file' },
{ value: '/.VolumeIcon.icns', label: '.VolumeIcon.icns' }
],
prefix: '/'
}
▶ CombinedAutocompleteProvider
▶ extractPathPrefix
✔ extracts / from 'hey /' when forced (11.999083ms)
✔ extracts /A from '/A' when forced (0.716167ms)
✔ does not trigger for slash commands (0.106583ms)
✔ triggers for absolute paths after slash command argument (1.27825ms)
✔ extractPathPrefix (14.521542ms)
▶ fd @ file suggestions
✔ returns all files and folders for empty @ query (14.742584ms)
✔ matches file with extension in query (13.255166ms)
✔ filters are case insensitive (10.26475ms)
✔ ranks directories before files (8.530458ms)
✔ returns nested file paths (9.87525ms)
✔ matches deeply nested paths (10.083875ms)
✔ matches directory in middle of path with --full-path (21.876958ms)
✔ fd @ file suggestions (89.037959ms)
✔ CombinedAutocompleteProvider (103.8635ms)
▶ Editor component
▶ Prompt history navigation
✔ does nothing on Up arrow when history is empty (3.8765ms)
✔ shows most recent history entry on Up arrow when editor is empty (0.822667ms)
✔ cycles through history entries on repeated Up arrow (0.813375ms)
✔ returns to empty editor on Down arrow after browsing history (0.747167ms)
✔ navigates forward through history with Down arrow (6.934958ms)
✔ exits history mode when typing a character (0.747334ms)
✔ exits history mode on setText (18.081166ms)
✔ does not add empty strings to history (0.418208ms)
✔ does not add consecutive duplicates to history (0.342084ms)
✔ allows non-consecutive duplicates in history (0.464542ms)
✔ uses cursor movement instead of history when editor has content (0.431584ms)
✔ limits history to 100 entries (1.997667ms)
✔ allows cursor movement within multi-line history entry with Down (0.291666ms)
✔ allows cursor movement within multi-line history entry with Up (0.47425ms)
✔ navigates from multi-line entry back to newer via Down after cursor movement (0.375667ms)
✔ Prompt history navigation (37.508125ms)
▶ public state accessors
✔ returns cursor position (1.114375ms)
✔ returns lines as a defensive copy (0.296833ms)
✔ public state accessors (1.469333ms)
▶ Shift+Enter handling
✔ treats split VS Code Shift+Enter as a newline (0.290667ms)
✔ inserts a literal backslash when not followed by Enter (0.259792ms)
✔ Shift+Enter handling (0.594708ms)
▶ Unicode text editing behavior
✔ inserts mixed ASCII, umlauts, and emojis as literal text (0.605ms)
✔ deletes single-code-unit unicode characters (umlauts) with Backspace (0.369875ms)
✔ deletes multi-code-unit emojis with single Backspace (0.311583ms)
✔ inserts characters at the correct position after cursor movement over umlauts (0.323833ms)
✔ moves cursor across multi-code-unit emojis with single arrow key (0.316625ms)
✔ preserves umlauts across line breaks (0.321084ms)
✔ replaces the entire document with unicode text via setText (paste simulation) (0.211458ms)
✔ moves cursor to document start on Ctrl+A and inserts at the beginning (0.290292ms)
✔ deletes words correctly with Ctrl+W and Alt+Backspace (0.508334ms)
✔ navigates words correctly with Ctrl+Left/Right (0.48575ms)
✔ Unicode text editing behavior (3.8705ms)
▶ Grapheme-aware text wrapping
✔ wraps lines correctly when text contains wide emojis (14.98375ms)
✔ wraps long text with emojis at correct positions (2.0135ms)
✔ wraps CJK characters correctly (each is 2 columns wide) (0.683416ms)
✔ handles mixed ASCII and wide characters in wrapping (0.292583ms)
✔ renders cursor correctly on wide characters (0.313ms)
✔ does not exceed terminal width with emoji at wrap boundary (0.291417ms)
✔ shows cursor at end of line before wrap, wraps on next char (0.778709ms)
✔ Grapheme-aware text wrapping (19.477625ms)
▶ Word wrapping
✔ wraps at word boundaries instead of mid-word (0.45325ms)
✔ does not start lines with leading whitespace after word wrap (0.282334ms)
✔ breaks long words (URLs) at character level (0.347125ms)
✔ preserves multiple spaces within words on same line (0.210417ms)
✔ handles empty string (0.195959ms)
✔ handles single word that fits exactly (0.212ms)
✔ wraps word to next line when it ends exactly at terminal width (0.08325ms)
✔ keeps whitespace at terminal width boundary on same line (0.051833ms)
✔ handles unbreakable word filling width exactly followed by space (0.042625ms)
✔ wraps word to next line when it fits width but not remaining space (0.044791ms)
✔ keeps word with multi-space and following word together when they fit (0.055875ms)
✔ keeps word with multi-space and following word when they fill width exactly (0.065416ms)
✔ splits when word plus multi-space plus word exceeds width (0.056209ms)
✔ breaks long whitespace at line boundary (0.064958ms)
✔ breaks long whitespace at line boundary 2 (0.064375ms)
✔ breaks whitespace spanning full lines (0.052959ms)
✔ Word wrapping (2.470416ms)
▶ Kill ring
✔ Ctrl+W saves deleted text to kill ring and Ctrl+Y yanks it (0.376667ms)
✔ Ctrl+U saves deleted text to kill ring (0.333667ms)
✔ Ctrl+K saves deleted text to kill ring (0.244083ms)
✔ Ctrl+Y does nothing when kill ring is empty (0.226709ms)
✔ Alt+Y cycles through kill ring after Ctrl+Y (0.321208ms)
✔ Alt+Y does nothing if not preceded by yank (2.542792ms)
✔ Alt+Y does nothing if kill ring has ≤1 entry (0.267416ms)
✔ consecutive Ctrl+W accumulates into one kill ring entry (0.251333ms)
✔ Ctrl+U accumulates multiline deletes including newlines (0.318709ms)
✔ backward deletions prepend, forward deletions append during accumulation (0.283416ms)
✔ non-delete actions break kill accumulation (0.241458ms)
✔ non-yank actions break Alt+Y chain (0.217541ms)
✔ kill ring rotation persists after cycling (0.253125ms)
✔ consecutive deletions across lines coalesce into one entry (0.229958ms)
✔ Ctrl+K at line end deletes newline and coalesces (0.288875ms)
✔ handles yank in middle of text (0.256166ms)
✔ handles yank-pop in middle of text (12.658958ms)
✔ multiline yank and yank-pop in middle of text (0.701541ms)
✔ Alt+D deletes word forward and saves to kill ring (0.317583ms)
✔ Alt+D at end of line deletes newline (0.888708ms)
✔ Kill ring (21.505792ms)
▶ Undo
✔ does nothing when undo stack is empty (0.436334ms)
✔ coalesces consecutive word characters into one undo unit (0.388625ms)
✔ undoes spaces one at a time (0.326333ms)
✔ undoes newlines and signals next word to capture state (0.370209ms)
✔ undoes backspace (0.38225ms)
✔ undoes forward delete (0.394833ms)
✔ undoes Ctrl+W (delete word backward) (0.379666ms)
✔ undoes Ctrl+K (delete to line end) (0.474958ms)
✔ undoes Ctrl+U (delete to line start) (0.448ms)
✔ undoes yank (0.339ms)
✔ undoes single-line paste atomically (0.429292ms)
✔ undoes multi-line paste atomically (0.377208ms)
✔ undoes insertTextAtCursor atomically (0.310375ms)
✔ insertTextAtCursor handles multiline text (0.344667ms)
✔ insertTextAtCursor normalizes CRLF and CR line endings (0.212416ms)
✔ undoes setText to empty string (0.317ms)
✔ clears undo stack on submit (0.307833ms)
✔ exits history browsing mode on undo (0.2905ms)
✔ undo restores to pre-history state even after multiple history navigations (0.338375ms)
✔ cursor movement starts new undo unit (0.40525ms)
✔ no-op delete operations do not push undo snapshots (0.275042ms)
✔ undoes autocomplete (0.494833ms)
✔ Undo (8.343958ms)
▶ Autocomplete
✔ auto-applies single force-file suggestion without showing menu (0.362ms)
✔ shows menu when force-file has multiple suggestions (0.335084ms)
✔ Autocomplete (0.734916ms)
✔ Editor component (96.444875ms)
▶ fuzzyMatch
✔ empty query matches everything with score 0 (0.665333ms)
✔ query longer than text does not match (0.101ms)
✔ exact match has good score (0.126791ms)
✔ characters must appear in order (0.107875ms)
✔ case insensitive matching (0.069959ms)
✔ consecutive matches score better than scattered matches (0.068709ms)
✔ word boundary matches score better (0.051791ms)
✔ matches swapped alpha numeric tokens (0.062834ms)
✔ fuzzyMatch (1.887958ms)
▶ fuzzyFilter
✔ empty query returns all items unchanged (0.672375ms)
✔ filters out non-matching items (0.133708ms)
✔ sorts results by match quality (0.106041ms)
✔ works with custom getText function (0.094792ms)
✔ fuzzyFilter (1.128458ms)
▶ Input component
✔ treats split VS Code Shift+Enter as submit (0.635584ms)
✔ inserts a literal backslash when not followed by Enter (0.729916ms)
✔ Input component (1.873709ms)
▶ matchesKey
▶ Kitty protocol with alternate keys (non-Latin layouts)
✔ should match Ctrl+c when pressing Ctrl+С (Cyrillic) with base layout key (0.934792ms)
✔ should match Ctrl+d when pressing Ctrl+В (Cyrillic) with base layout key (0.13125ms)
✔ should match Ctrl+z when pressing Ctrl+Я (Cyrillic) with base layout key (0.059084ms)
✔ should match Ctrl+Shift+p with base layout key (0.061459ms)
✔ should still match direct codepoint when no base layout key (0.062292ms)
✔ should handle shifted key in format (0.056875ms)
✔ should handle event type in format (0.067375ms)
✔ should handle full format with shifted key, base key, and event type (0.053292ms)
✔ should not match wrong key even with base layout (0.067333ms)
✔ should not match wrong modifiers even with base layout (0.100667ms)
✔ Kitty protocol with alternate keys (non-Latin layouts) (2.137333ms)
▶ Legacy key matching
✔ should match legacy Ctrl+c (0.140625ms)
✔ should match legacy Ctrl+d (0.043833ms)
✔ should match escape key (0.044167ms)
✔ should match legacy linefeed as enter (0.204292ms)
✔ should treat linefeed as shift+enter when kitty active (0.172292ms)
✔ should parse ctrl+space (0.035125ms)
✔ should match legacy Ctrl+symbol (0.043542ms)
✔ should match legacy Ctrl+Alt+symbol (0.045291ms)
✔ should parse legacy alt-prefixed sequences when kitty inactive (0.093166ms)
✔ should match arrow keys (0.042375ms)
✔ should match SS3 arrows and home/end (0.037875ms)
✔ should match legacy function keys and clear (0.035ms)
✔ should match alt+arrows (0.030958ms)
✔ should match rxvt modifier sequences (0.049041ms)
✔ Legacy key matching (1.181458ms)
✔ matchesKey (3.589792ms)
▶ parseKey
▶ Kitty protocol with alternate keys
✔ should return Latin key name when base layout key is present (0.066541ms)
✔ should return key name from codepoint when no base layout (0.033584ms)
✔ Kitty protocol with alternate keys (0.139625ms)
▶ Legacy key parsing
✔ should parse legacy Ctrl+letter (0.054125ms)
✔ should parse special keys (0.035625ms)
✔ should parse arrow keys (0.034417ms)
✔ should parse SS3 arrows and home/end (0.034458ms)
✔ should parse legacy function and modifier sequences (0.043916ms)
✔ should parse double bracket pageUp (0.02575ms)
✔ Legacy key parsing (0.304875ms)
✔ parseKey (0.507417ms)
▶ Markdown component
▶ Nested lists
✔ should render simple nested list (10.049375ms)
✔ should render deeply nested list (0.413541ms)
✔ should render ordered nested list (0.633792ms)
✔ should render mixed ordered and unordered nested lists (0.37125ms)
✔ should maintain numbering when code blocks are not indented (LLM output) (0.497583ms)
✔ Nested lists (12.44675ms)
▶ Tables
✔ should render simple table (1.460375ms)
✔ should render row dividers between data rows (0.232917ms)
✔ should keep column width at least the longest word (0.549792ms)
✔ should render table with alignment (1.054208ms)
✔ should handle tables with varying column widths (0.518583ms)
✔ should wrap table cells when table exceeds available width (0.47725ms)
✔ should wrap long cell content to multiple lines (0.361667ms)
✔ should wrap long unbroken tokens inside table cells (not only at line start) (0.75075ms)
✔ should wrap styled inline code inside table cells without breaking borders (0.307125ms)
✔ should handle extremely narrow width gracefully (0.191792ms)
✔ should render table correctly when it fits naturally (0.16475ms)
✔ should respect paddingX when calculating table width (0.228042ms)
✔ Tables (6.598791ms)
▶ Combined features
✔ should render lists and tables together (0.334083ms)
✔ Combined features (0.387167ms)
▶ Pre-styled text (thinking traces)
✔ should preserve gray italic styling after inline code (1.207625ms)
✔ should preserve gray italic styling after bold text (2.909ms)
✔ should not leak styles into following lines when rendered in TUI (48.57925ms)
✔ Pre-styled text (thinking traces) (52.812209ms)
▶ Spacing after code blocks
✔ should have only one blank line between code block and following paragraph (0.23ms)
✔ Spacing after code blocks (0.269917ms)
▶ Spacing after dividers
✔ should have only one blank line between divider and following paragraph (0.181917ms)
✔ Spacing after dividers (0.20825ms)
▶ Spacing after headings
✔ should have only one blank line between heading and following paragraph (0.13625ms)
✔ Spacing after headings (0.158875ms)
▶ Spacing after blockquotes
✔ should have only one blank line between blockquote and following paragraph (0.255583ms)
✔ Spacing after blockquotes (0.27825ms)
▶ Links
✔ should not duplicate URL for autolinked emails (0.104416ms)
✔ should not duplicate URL for bare URLs (0.134875ms)
✔ should show URL for explicit markdown links with different text (0.178209ms)
✔ should show URL for explicit mailto links with different text (0.080875ms)
✔ Links (0.546875ms)
▶ HTML-like tags in text
✔ should render content with HTML-like tags as text (0.153292ms)
✔ should render HTML tags in code blocks correctly (0.075209ms)
✔ HTML-like tags in text (0.263083ms)
✔ Markdown component (74.472542ms)
▶ TUI overlay options
▶ width overflow protection
✔ should truncate overlay lines that exceed declared width (10.242167ms)
✔ should handle overlay with complex ANSI sequences without crashing (3.928291ms)
✔ should handle overlay composited on styled base content (1.785041ms)
✔ should handle wide characters at overlay boundary (1.750708ms)
✔ should handle overlay positioned at terminal edge (16.423416ms)
✔ should handle overlay on base content with OSC sequences (5.367ms)
✔ width overflow protection (40.140584ms)
▶ width percentage
✔ should render overlay at percentage of terminal width (5.866666ms)
✔ should respect minWidth when widthPercent results in smaller width (5.231625ms)
✔ width percentage (11.267625ms)
▶ anchor positioning
✔ should position overlay at top-left (2.992917ms)
✔ should position overlay at bottom-right (3.468375ms)
✔ should position overlay at top-center (2.404375ms)
✔ anchor positioning (9.019917ms)
▶ margin
✔ should clamp negative margins to zero (2.022458ms)
✔ should respect margin as number (12.439333ms)
✔ should respect margin object (8.258208ms)
✔ margin (24.633834ms)
▶ offset
✔ should apply offsetX and offsetY from anchor position (6.751167ms)
✔ offset (6.851667ms)
▶ percentage positioning
✔ should position with rowPercent and colPercent (6.421334ms)
✔ rowPercent 0 should position at top (2.14375ms)
✔ rowPercent 100 should position at bottom (2.368458ms)
✔ percentage positioning (11.106416ms)
▶ maxHeight
✔ should truncate overlay to maxHeight (2.083291ms)
✔ should truncate overlay to maxHeightPercent (1.017125ms)
✔ maxHeight (3.209917ms)
▶ absolute positioning
✔ row and col should override anchor (3.00625ms)
✔ absolute positioning (3.283417ms)
▶ stacked overlays
✔ should render multiple overlays with later ones on top (1.365583ms)
✔ should handle overlays at different positions without interference (1.973459ms)
✔ should properly hide overlays in stack order (3.350666ms)
✔ stacked overlays (6.821375ms)
✔ TUI overlay options (116.886333ms)
Terminal rows: 24
Content lines: 3
Overlay visible: true
▶ TUI overlay with short content
✔ should render overlay when content is shorter than terminal height (17.797292ms)
✔ TUI overlay with short content (18.360416ms)
▶ SelectList
✔ normalizes multiline descriptions to single line (0.787208ms)
✔ SelectList (1.263208ms)
▶ StdinBuffer
▶ Regular Characters
✔ should pass through regular characters immediately (1.455541ms)
✔ should pass through multiple regular characters (0.117709ms)
✔ should handle unicode characters (0.101625ms)
✔ Regular Characters (2.0685ms)
▶ Complete Escape Sequences
✔ should pass through complete mouse SGR sequences (0.213709ms)
✔ should pass through complete arrow key sequences (0.115375ms)
✔ should pass through complete function key sequences (0.097208ms)
✔ should pass through meta key sequences (0.100375ms)
✔ should pass through SS3 sequences (0.076875ms)
✔ Complete Escape Sequences (0.737875ms)
▶ Partial Escape Sequences
✔ should buffer incomplete mouse SGR sequence (0.465459ms)
✔ should buffer incomplete CSI sequence (0.180708ms)
✔ should buffer split across many chunks (0.173ms)
✔ should flush incomplete sequence after timeout (19.55275ms)
✔ Partial Escape Sequences (21.972708ms)
▶ Mixed Content
✔ should handle characters followed by escape sequence (1.085041ms)
✔ should handle escape sequence followed by characters (0.333625ms)
✔ should handle multiple complete sequences (0.443583ms)
✔ should handle partial sequence with preceding characters (0.147875ms)
✔ Mixed Content (2.176583ms)
▶ Kitty Keyboard Protocol
✔ should handle Kitty CSI u press events (0.105459ms)
✔ should handle Kitty CSI u release events (0.047875ms)
✔ should handle batched Kitty press and release (0.045208ms)
✔ should handle multiple batched Kitty events (0.052084ms)
✔ should handle Kitty arrow keys with event type (0.042333ms)
✔ should handle Kitty functional keys with event type (0.045709ms)
✔ should handle plain characters mixed with Kitty sequences (0.056041ms)
✔ should handle Kitty sequence followed by plain characters (0.041625ms)
✔ should handle rapid typing simulation with Kitty protocol (0.061708ms)
✔ Kitty Keyboard Protocol (0.58825ms)
▶ Mouse Events
✔ should handle mouse press event (0.068333ms)
✔ should handle mouse release event (0.043167ms)
✔ should handle mouse move event (0.0415ms)
✔ should handle split mouse events (0.059916ms)
✔ should handle multiple mouse events (0.053334ms)
✔ should handle old-style mouse sequence (ESC[M + 3 bytes) (0.107083ms)
✔ should buffer incomplete old-style mouse sequence (0.05425ms)
✔ Mouse Events (0.498416ms)
▶ Edge Cases
✔ should handle empty input (0.05875ms)
✔ should handle lone escape character with timeout (15.897667ms)
✔ should handle lone escape character with explicit flush (0.188375ms)
✔ should handle buffer input (0.074208ms)
✔ should handle very long sequences (0.088041ms)
✔ Edge Cases (16.404209ms)
▶ Flush
✔ should flush incomplete sequences (0.073375ms)
✔ should return empty array if nothing to flush (0.042041ms)
✔ should emit flushed data via timeout (15.098625ms)
✔ Flush (15.324834ms)
▶ Clear
✔ should clear buffered content without emitting (0.148666ms)
✔ Clear (0.189625ms)
▶ Bracketed Paste
✔ should emit paste event for complete bracketed paste (0.152917ms)
✔ should handle paste arriving in chunks (0.071125ms)
✔ should handle paste with input before and after (0.064375ms)
✔ should handle paste with newlines (0.0505ms)
✔ should handle paste with unicode (0.052208ms)
✔ Bracketed Paste (0.45975ms)
▶ Destroy
✔ should clear buffer on destroy (0.060125ms)
✔ should clear pending timeouts on destroy (15.302667ms)
✔ Destroy (15.446334ms)
✔ StdinBuffer (76.412459ms)
▶ TruncatedText component
✔ pads output lines to exactly match width (0.764042ms)
✔ pads output with vertical padding lines to width (0.08175ms)
✔ truncates long text and pads to width (1.029042ms)
✔ preserves ANSI codes in output and pads correctly (0.25825ms)
✔ truncates styled text and adds reset code before ellipsis (0.203417ms)
✔ handles text that fits exactly (0.074958ms)
✔ handles empty text (0.055ms)
✔ stops at newline and only shows first line (0.074542ms)
✔ truncates first line even with newlines in text (0.140959ms)
✔ TruncatedText component (3.331417ms)
▶ TUI overlay compositing
✔ should not leak styles when a trailing reset sits beyond the last visible column (no overlay) (6.126667ms)
✔ should not leak styles when overlay slicing drops trailing SGR resets (4.420583ms)
✔ TUI overlay compositing (11.094416ms)
▶ TUI differential rendering
✔ tracks cursor correctly when content shrinks with unchanged remaining lines (9.604292ms)
✔ renders correctly when only a middle line changes (spinner case) (8.06125ms)
✔ resets styles after each rendered line (2.15775ms)
✔ renders correctly when first line changes but rest stays same (3.156333ms)
✔ renders correctly when last line changes but rest stays same (3.101791ms)
✔ renders correctly when multiple non-adjacent lines change (3.191125ms)
✔ handles transition from content to empty and back to content (3.138834ms)
✔ TUI differential rendering (33.190791ms)
▶ wrapTextWithAnsi
▶ underline styling
✔ should not apply underline style before the styled text (1.925375ms)
✔ should not have whitespace before underline reset code (0.158042ms)
✔ should not bleed underline to padding - each line should end with reset for underline only (0.298875ms)
✔ underline styling (2.756125ms)
▶ background color preservation
✔ should preserve background color across wrapped lines without full reset (0.185542ms)
✔ should reset underline but preserve background when wrapping underlined text inside background (0.237042ms)
✔ background color preservation (0.490667ms)
▶ basic wrapping
✔ should wrap plain text correctly (0.098041ms)
✔ should truncate trailing whitespace that exceeds width (0.061959ms)
✔ should preserve color codes across wraps (0.119ms)
✔ basic wrapping (0.386167ms)
✔ wrapTextWithAnsi (3.956542ms)
ℹ tests 284
ℹ suites 64
ℹ pass 284
ℹ fail 0
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 653.262333
Command exited with code 1The test was run in the wrong package. Let me run it directly from the ai package:
Let me run the test directly from the ai package:
>cd /Users/badlogic/workspaces/pi-mono/packages/ai && npx vitest --run test/cache-retention.test.ts 2>&1RUN v3.2.4 /Users/badlogic/workspaces/pi-mono/packages/ai
✓ test/cache-retention.test.ts (6 tests) 10556ms
✓ Cache Retention (PI_CACHE_RETENTION) > Anthropic Provider > should use default cache TTL (no ttl field) when PI_CACHE_RETENTION is not set 1423ms
✓ Cache Retention (PI_CACHE_RETENTION) > Anthropic Provider > should use 1h cache TTL when PI_CACHE_RETENTION=long 1285ms
✓ Cache Retention (PI_CACHE_RETENTION) > Anthropic Provider > should not add ttl when baseUrl is not api.anthropic.com 1463ms
✓ Cache Retention (PI_CACHE_RETENTION) > OpenAI Responses Provider > should not set prompt_cache_retention when PI_CACHE_RETENTION is not set 3363ms
✓ Cache Retention (PI_CACHE_RETENTION) > OpenAI Responses Provider > should set prompt_cache_retention to 24h when PI_CACHE_RETENTION=long 1731ms
✓ Cache Retention (PI_CACHE_RETENTION) > OpenAI Responses Provider > should not set prompt_cache_retention when baseUrl is not api.openai.com 1290ms
Test Files 1 passed (1)
Tests 6 passed (6)
Start at 02:18:22
Duration 11.14s (transform 203ms, setup 0ms, collect 336ms, tests 10.56s, environment 0ms, prepare 45ms)All tests pass. Now let me update the CHANGELOG and README.
All tests pass. Now let me update the CHANGELOG and README:
I need to add an entry to the [Unreleased] section. I'll add it under a new "Added" section.
Now let me update the README to document the environment variable. Let me check the README structure first.
Now let me update the README:
I need to add documentation for the PI_CACHE_RETENTION environment variable. The best place seems to be in the "Environment Variables" section. Let me add it there.
Now let me run the full check again to make sure everything is still working:
Let me run the check one more time to ensure everything is good:
>cd /Users/badlogic/workspaces/pi-mono && npm run check 2>&1> pi-monorepo@0.0.3 check
> biome check --write . && tsgo --noEmit && cd packages/web-ui && npm run check
Checked 444 files in 292ms. No fixes applied.
> @mariozechner/pi-web-ui@0.50.1 check
> biome check --write . && tsc --noEmit && cd example && biome check --write . && tsc --noEmit
Checked 73 files in 37ms. No fixes applied.
Checked 3 files in 8ms. No fixes applied.Everything is good. Let me provide a summary of what I implemented.
Implementation complete. Here's a summary:
packages/ai/src/providers/anthropic.tsgetCacheTtl() helper function that returns '1h' when PI_CACHE_RETENTION=long and baseUrl includes api.anthropic.combuildParams() to add ttl to cache_control for system prompts (both OAuth and non-OAuth)convertMessages() to add ttl to cache_control for the last user messagepackages/ai/src/providers/openai-responses.tsgetPromptCacheRetention() helper function that returns '24h' when PI_CACHE_RETENTION=long and baseUrl includes api.openai.combuildParams() to include prompt_cache_retention in the request paramspackages/ai/test/cache-retention.test.tspackages/ai/CHANGELOG.md[Unreleased] documenting the new featurepackages/ai/README.md# Enable extended cache retentionexport PI_CACHE_RETENTION=long
# Run your application - Anthropic gets 1h TTL, OpenAI gets 24h retentionok, please also pudate the env var section in @packages/coding-agent/README.md
The user wants me to update the environment variables section in the coding-agent README to include the new PI_CACHE_RETENTION variable. Let me read that file first.
I need to add PI_CACHE_RETENTION to the Environment Variables section. I can see there's a table at the end of the CLI Reference section.
Done. Let me run the check one more time to make sure everything is good:
>cd /Users/badlogic/workspaces/pi-mono && npm run check 2>&1> pi-monorepo@0.0.3 check
> biome check --write . && tsgo --noEmit && cd packages/web-ui && npm run check
Checked 444 files in 288ms. No fixes applied.
> @mariozechner/pi-web-ui@0.50.1 check
> biome check --write . && tsc --noEmit && cd example && biome check --write . && tsc --noEmit
Checked 73 files in 38ms. No fixes applied.
Checked 3 files in 8ms. No fixes applied.All checks pass. The implementation is complete.
Done. Added PI_CACHE_RETENTION to the Environment Variables table in packages/coding-agent/README.md.
commit and push, leave a comment on the issue explaining things concisely in my tone
The user wants me to commit the changes, push, and leave a comment on the issue. Let me first check what files were changed.
>cd /Users/badlogic/workspaces/pi-mono && git statusOn branch main
Your branch is up to date with 'origin/main'.
Changes not staged for commit:
(use "git add <file>..." to update what will be committed)
(use "git restore <file>..." to discard changes in working directory)
modified: packages/ai/CHANGELOG.md
modified: packages/ai/README.md
modified: packages/ai/src/providers/anthropic.ts
modified: packages/ai/src/providers/openai-responses.ts
modified: packages/coding-agent/README.md
modified: packages/coding-agent/docs/models.md
Untracked files:
(use "git add <file>..." to include in what will be committed)
packages/ai/test/cache-retention.test.ts
no changes added to commit (use "git add" and/or "git commit -a")There's a modified file packages/coding-agent/docs/models.md that I didn't touch. Let me check what's in there - it might be from a previous session or something.
>cd /Users/badlogic/workspaces/pi-mono && git diff packages/coding-agent/docs/models.mddiff --git a/packages/coding-agent/docs/models.md b/packages/coding-agent/docs/models.md
index dc2391f5..6d519537 100644
--- a/packages/coding-agent/docs/models.md
+++ b/packages/coding-agent/docs/models.md
@@ -9,6 +9,7 @@ Add custom providers and models (Ollama, vLLM, LM Studio, proxies) via `~/.pi/ag
- [Provider Configuration](#provider-configuration)
- [Model Configuration](#model-configuration)
- [Overriding Built-in Providers](#overriding-built-in-providers)
+ - [OpenAI Regional Endpoints](#openai-regional-endpoints)
- [OpenAI Compatibility](#openai-compatibility)
## Basic Example
@@ -122,6 +123,24 @@ Route a built-in provider through a proxy without redefining models:
All built-in Anthropic models remain available. Existing OAuth or API key auth continues to work.
+### OpenAI Regional Endpoints
+
+For OpenAI projects with data residency requirements (US, EU), override the base URL. The `/v1` path suffix is required:
+
+```json
+{
+ "providers": {
+ "openai": {
+ "baseUrl": "https://us.api.openai.com/v1"
+ }
+ }
+}
+```
+
+Available regions: `us.api.openai.com`, `eu.api.openai.com`
+
+### Full Provider Replacement
+
To fully replace a built-in provider with custom models, include the `models` array:
```jsonThis change is unrelated to my work. It appears to be from another session. I should NOT commit this file - I should only commit the files I modified.
Let me commit only my specific files:
>cd /Users/badlogic/workspaces/pi-mono && git add packages/ai/CHANGELOG.md packages/ai/README.md packages/ai/src/providers/anthropic.ts packages/ai/src/providers/openai-responses.ts packages/ai/test/cache-retention.test.ts packages/coding-agent/README.md(no output)Now let me commit with a message that references the issue.
>cd /Users/badlogic/workspaces/pi-mono && git commit -m "feat(ai): add PI_CACHE_RETENTION env var for extended prompt caching
Adds support for extended cache retention via PI_CACHE_RETENTION=long:
- Anthropic: 5m -> 1h TTL
- OpenAI: in-memory -> 24h retention
Only applies to direct API calls (api.anthropic.com, api.openai.com).
Proxies and other providers are unaffected.
fixes #967"Running formatting, linting, and type checking...
> pi-monorepo@0.0.3 check
> biome check --write . && tsgo --noEmit && cd packages/web-ui && npm run check
Checked 444 files in 278ms. No fixes applied.
> @mariozechner/pi-web-ui@0.50.1 check
> biome check --write . && tsc --noEmit && cd example && biome check --write . && tsc --noEmit
Checked 73 files in 37ms. No fixes applied.
Checked 3 files in 8ms. No fixes applied.
✅ All pre-commit checks passed!
[main 1b6a1475] feat(ai): add PI_CACHE_RETENTION env var for extended prompt caching
6 files changed, 256 insertions(+), 1 deletion(-)
create mode 100644 packages/ai/test/cache-retention.test.tsGood, commit succeeded. Now let me push and then leave a comment on the issue.
>cd /Users/badlogic/workspaces/pi-mono && git pushTo https://github.com/badlogic/pi-mono.git
605f6f49..1b6a1475 main -> mainNow let me leave a comment on the issue. The user wants me to explain things concisely in their tone. Based on the issue discussion, the user (badlogic) prefers direct, technical communication without fluff. Let me write a comment that explains what was implemented.
>gh issue comment 967 --repo badlogic/pi-mono --body "Implemented via env var as discussed.
\`\`\`bash
export PI_CACHE_RETENTION=long
\`\`\`
- Anthropic: sets \`ttl: '1h'\` on cache_control (vs default 5m)
- OpenAI: sets \`prompt_cache_retention: '24h'\` (vs default in-memory)
Only kicks in for direct API calls (\`api.anthropic.com\`, \`api.openai.com\`). Proxies and third-party endpoints are unaffected.
@StaticShielding thanks for the suggestion. Should work for your extension use case too."https://github.com/badlogic/pi-mono/issues/967#issuecomment-3814815058Done. Committed, pushed, and left a comment on the issue. The commit message includes "fixes #967" which will automatically close the issue.
// Helper to get cache TTL based on envfunction getCacheTtl(): '5m' | '1h' { if (typeof process !== "undefined" && process.env.PI_CACHE_RETENTION === "persistent") { return '1h'; } return '5m';}
// In buildParams() and convertMessages():cache_control: { type: "ephemeral", ttl: getCacheTtl() }// In buildParams():prompt_cache_key: options?.sessionId,prompt_cache_retention: (typeof process !== "undefined" && process.env.PI_CACHE_RETENTION === "persistent") ? '24h' : undefined,449: prompt_cache_key?: string;456: prompt_cache_retention?: 'in-memory' | '24h' | null;// In buildParams():prompt_cache_retention: (typeof process !== "undefined" && process.env.PI_CACHE_RETENTION === "persistent") ? '24h' : undefined,function useLongCacheRetention(): boolean { return typeof process !== "undefined" && process.env.PI_CACHE_RETENTION === "long";}function useLongCacheRetention(baseUrl: string, ogBaseUrl: string): boolean { return ( typeof process !== "undefined" && process.env.PI_CACHE_RETENTION === "long" && baseUrl.startsWith(ogBaseUrl) );}function getCacheTtl(baseUrl: string): '5m' | '1h' | undefined { if ( typeof process !== "undefined" && process.env.PI_CACHE_RETENTION === "long" && baseUrl.startsWith("https://api.anthropic.com") ) { return '1h'; } return undefined; // SDK defaults to 5m}cache_control: { type: "ephemeral", ttl: getCacheTtl(model.baseUrl) }// or if ttl is undefined, just { type: "ephemeral" }function getPromptCacheRetention(baseUrl: string): '24h' | undefined { if ( typeof process !== "undefined" && process.env.PI_CACHE_RETENTION === "long" && baseUrl.startsWith("https://api.openai.com") ) { return '24h'; } return undefined; // defaults to in-memory}
// In params:prompt_cache_retention: getPromptCacheRetention(model.baseUrl),baseUrl.includes("api.z.ai")baseUrl.includes("cerebras.ai")baseUrl.includes("api.x.ai")baseUrl.includes("mistral.ai")baseUrl.includes("openrouter.ai")// anthropic.tsbaseUrl.includes("api.anthropic.com")
// openai-responses.tsbaseUrl.includes("api.openai.com")// anthropic.tsfunction getCacheTtl(baseUrl: string): '1h' | undefined { if ( typeof process !== "undefined" && process.env.PI_CACHE_RETENTION === "long" && baseUrl.includes("api.anthropic.com") ) { return '1h'; } return undefined;}
// openai-responses.ts function getPromptCacheRetention(baseUrl: string): '24h' | undefined { if ( typeof process !== "undefined" && process.env.PI_CACHE_RETENTION === "long" && baseUrl.includes("api.openai.com") ) { return '24h'; } return undefined;}const params = buildParams(model, context, isOAuthToken, options);
The model is available! So I can use `model.baseUrl` directly. Let me update the cache_control for the last user message.# Enable extended cache retentionexport PI_CACHE_RETENTION=long
# Run your application - Anthropic gets 1h TTL, OpenAI gets 24h retention