Fixed feature detection for custom base URLs

Summary

Closed, Completed

Understand
Analyzed issue and related code for feature detection.
Understand
Edit
Enhanced detection logic to prioritize provider checks.
Edit
Verify
Summarized changes and updated changelog.
Verify
Cleanup
Committed, pushed changes, and closed issue with comment.
Cleanup

Analyze GitHub issue(s): https://github.com/badlogic/pi-mono/issues/774

For each issue:

Read the issue in full, including all comments and linked issues/PRs.
For bugs:
- Ignore any root cause analysis in the issue (likely wrong)
- Read all related code files in full (no truncation)
- Trace the code path and identify the actual root cause
- Propose a fix
For feature requests:
- Read all related code files in full (no truncation)
- Propose the most concise implementation approach
- List affected files and changes needed

Do NOT implement unless explicitly asked. Analyze and propose only.

>gh issue view 774 --json title,body,comments,labels,state --repo badlogic/pi-mono

{"body":"if a custom base url is used for cerebras for use cases like proxying llm calls etc, then the model configuration breaks since cerebras specific configurations (like no support for developer message) are tied to the base url rather than the configured provider.\n\nit's better to avoid base url for feature detection entirely.\n(sorry for not opening PR, i don't have the repo setup)","comments":[],"labels":[],"state":"OPEN","title":"pi-ai auto detection of features based on base url is confusing"}

This is a bug/feature request about auto-detection of features based on base URL being problematic when using custom base URLs (like proxies). Let me understand the codebase to see how this feature detection works.

The issue mentions:

Using custom base URL for Cerebras (e.g., for proxying LLM calls)
Model configuration breaks because Cerebras-specific configurations (like no support for developer message) are tied to the base URL rather than the configured provider

I need to look at:

How providers are configured
How feature detection based on base URL works
Where Cerebras-specific configurations are handled

Let me start by looking at the ai package, specifically around stream.ts and provider configurations.

Let me explore the codebase to understand how feature detection based on base URL works.

>cd /Users/badlogic/workspaces/pi-mono && grep -r "baseUrl" packages/ai/src --include="*.ts" -l

packages/ai/src/providers/openai-responses.ts
packages/ai/src/providers/openai-completions.ts
packages/ai/src/providers/anthropic.ts
packages/ai/src/providers/google-gemini-cli.ts
packages/ai/src/providers/google.ts
packages/ai/src/utils/oauth/github-copilot.ts
packages/ai/src/types.ts
packages/ai/src/models.generated.ts

Now let me look at the openai-completions provider where the auto-detection logic likely lives:

Now I understand the issue. The problem is in the detectCompatFromUrl function:

typescript

function detectCompatFromUrl(baseUrl: string): Required<OpenAICompat> {	const isZai = baseUrl.includes("api.z.ai");
	const isNonStandard =		baseUrl.includes("cerebras.ai") ||		baseUrl.includes("api.x.ai") ||		baseUrl.includes("mistral.ai") ||		baseUrl.includes("chutes.ai") ||		isZai ||		baseUrl.includes("opencode.ai");    // ...

function detectCompatFromUrl(baseUrl: string): Required<OpenAICompat> {	const isZai = baseUrl.includes("api.z.ai");
	const isNonStandard =		baseUrl.includes("cerebras.ai") ||		baseUrl.includes("api.x.ai") ||		baseUrl.includes("mistral.ai") ||		baseUrl.includes("chutes.ai") ||		isZai ||		baseUrl.includes("opencode.ai");    // ...

When a user uses a custom base URL (like a proxy) for Cerebras, the URL no longer contains "cerebras.ai", so the model behaves as if it's a standard OpenAI provider. But if the user configured provider: "cerebras", they expect Cerebras-specific behavior regardless of the URL.

The solution would be to:

Also consider the model.provider when detecting compatibility, not just the baseUrl
Make model.provider take precedence, since that's what the user explicitly configured

Let me check how the getCompat function works:

typescript

function getCompat(model: Model<"openai-completions">): Required<OpenAICompat> {	const detected = detectCompatFromUrl(model.baseUrl);	if (!model.compat) return detected;
	return {		supportsStore: model.compat.supportsStore ?? detected.supportsStore,		// ...	};}

function getCompat(model: Model<"openai-completions">): Required<OpenAICompat> {	const detected = detectCompatFromUrl(model.baseUrl);	if (!model.compat) return detected;
	return {		supportsStore: model.compat.supportsStore ?? detected.supportsStore,		// ...	};}

So the user can override with model.compat, but detection only uses URL. The fix would be to also check model.provider in the detection logic.

Let me also check models.generated.ts to see what providers/base URLs are used for built-in models.

Let me also check how models are defined and the provider field is used:

>cd /Users/badlogic/workspaces/pi-mono && grep -A5 '"cerebras"' packages/ai/src/models.generated.ts | head -30

"cerebras": {
		"gpt-oss-120b": {
			id: "gpt-oss-120b",
			name: "GPT OSS 120B",
			api: "openai-completions",
			provider: "cerebras",
			baseUrl: "https://api.cerebras.ai/v1",
			reasoning: true,
			input: ["text"],
			cost: {
				input: 0.25,
--
			provider: "cerebras",
			baseUrl: "https://api.cerebras.ai/v1",
			reasoning: false,
			input: ["text"],
			cost: {
				input: 0.6,
--
			provider: "cerebras",
			baseUrl: "https://api.cerebras.ai/v1",
			reasoning: false,
			input: ["text"],
			cost: {
				input: 0,
--
			provider: "cerebras",
			baseUrl: "https://api.cerebras.ai/v1",
			reasoning: false,
			input: ["text"],

Good, so the models have both provider: "cerebras" and baseUrl: "https://api.cerebras.ai/v1".

The issue is that when someone uses a custom base URL (e.g., a proxy), the detectCompatFromUrl function won't detect Cerebras-specific settings because the URL doesn't contain "cerebras.ai".

The fix is straightforward: detectCompatFromUrl should also take the provider parameter and check that in addition to (or instead of) the URL.

Let me verify by checking how getCompat calls detectCompatFromUrl:

typescript

function getCompat(model: Model<"openai-completions">): Required<OpenAICompat> {	const detected = detectCompatFromUrl(model.baseUrl);	if (!model.compat) return detected;
	return {		supportsStore: model.compat.supportsStore ?? detected.supportsStore,		// ...	};}

function getCompat(model: Model<"openai-completions">): Required<OpenAICompat> {	const detected = detectCompatFromUrl(model.baseUrl);	if (!model.compat) return detected;
	return {		supportsStore: model.compat.supportsStore ?? detected.supportsStore,		// ...	};}

The fix should:

Change detectCompatFromUrl to detectCompat(model) or detectCompat(baseUrl, provider)
Add provider-based detection in addition to URL-based detection
Provider check should take precedence since it's explicitly configured

Let me see what detection is needed for each provider:

From the code:

cerebras.ai → isNonStandard (no supportsStore, no supportsDeveloperRole)
api.x.ai (xAI/Grok) → isNonStandard, isGrok (no supportsReasoningEffort)
mistral.ai → isNonStandard, useMaxTokens, isMistral (special tool ID handling, thinking as text)
chutes.ai → isNonStandard, useMaxTokens
api.z.ai → isNonStandard, isZai (no supportsReasoningEffort, thinkingFormat: "zai")
opencode.ai → isNonStandard

The main providers to handle are cerebras, xai, mistral, and zai since those are in the KnownProvider type. I should also check if there are other places in the codebase where provider-based detection might be needed.

Let me check the KnownProvider type to see all known providers:

>cd /Users/badlogic/workspaces/pi-mono && grep -A20 "KnownProvider =" packages/ai/src/types.ts

export type KnownProvider =
	| "amazon-bedrock"
	| "anthropic"
	| "google"
	| "google-gemini-cli"
	| "google-antigravity"
	| "google-vertex"
	| "openai"
	| "openai-codex"
	| "github-copilot"
	| "xai"
	| "groq"
	| "cerebras"
	| "openrouter"
	| "vercel-ai-gateway"
	| "zai"
	| "mistral"
	| "minimax"
	| "minimax-cn"
	| "opencode";
export type Provider = KnownProvider | string;

Now I have the full picture. The issue is that detectCompatFromUrl function only checks the baseUrl to determine provider-specific settings, but when users use proxies or custom base URLs, the provider information is lost.

The solution is to also check model.provider alongside the URL. The provider field explicitly states which provider is being used, so it should take precedence.

Let me summarize the analysis and proposed fix:

Root Cause: The detectCompatFromUrl function in packages/ai/src/providers/openai-completions.ts only checks the baseUrl to determine provider-specific compatibility settings. When a custom base URL is used (e.g., for proxying), the URL-based detection fails because it doesn't contain the provider's domain.

Affected Code: detectCompatFromUrl() function at lines ~495-521

Proposed Fix:

Rename detectCompatFromUrl to detectCompat and accept the full model object (or at least provider + baseUrl)
Check model.provider in addition to model.baseUrl
Provider-based detection should take precedence since it's explicitly configured

The changes would look like:```typescript function detectCompat(model: Model<"openai-completions">): Required { const provider = model.provider; const baseUrl = model.baseUrl; const isZai = provider === "zai" || baseUrl.includes("api.z.ai");

const isNonStandard =	provider === "cerebras" || baseUrl.includes("cerebras.ai") ||	provider === "xai" || baseUrl.includes("api.x.ai") ||	provider === "mistral" || baseUrl.includes("mistral.ai") ||	baseUrl.includes("chutes.ai") ||	isZai ||	provider === "opencode" || baseUrl.includes("opencode.ai");

const isNonStandard =	provider === "cerebras" || baseUrl.includes("cerebras.ai") ||	provider === "xai" || baseUrl.includes("api.x.ai") ||	provider === "mistral" || baseUrl.includes("mistral.ai") ||	baseUrl.includes("chutes.ai") ||	isZai ||	provider === "opencode" || baseUrl.includes("opencode.ai");


Then I need to define the specific compatibility flags for each provider. For Mistral and Chutes, `useMaxTokens` should be true since they require max_tokens instead of max_completion_tokens. Grok and Zai don't support reasoning_effort. The return object maps these flags to determine which OpenAI features are supported or need adaptation for each provider.


Then I need to define the specific compatibility flags for each provider. For Mistral and Chutes, `useMaxTokens` should be true since they require max_tokens instead of max_completion_tokens. Grok and Zai don't support reasoning_effort. The return object maps these flags to determine which OpenAI features are supported or need adaptation for each provider.

Analysis of Issue #774

Title: pi-ai auto detection of features based on base url is confusing

Problem: When using a custom base URL for providers like Cerebras (e.g., for proxying LLM calls), model configuration breaks because provider-specific settings (like "no support for developer message") are determined by checking if the URL contains cerebras.ai, rather than using the explicitly configured provider field.

Root Cause

In packages/ai/src/providers/openai-completions.ts, the detectCompatFromUrl() function (lines 495-521) only examines the baseUrl to determine provider-specific compatibility:

typescript

function detectCompatFromUrl(baseUrl: string): Required<OpenAICompat> {	const isZai = baseUrl.includes("api.z.ai");
	const isNonStandard =		baseUrl.includes("cerebras.ai") ||		baseUrl.includes("api.x.ai") ||		baseUrl.includes("mistral.ai") ||		// ...

function detectCompatFromUrl(baseUrl: string): Required<OpenAICompat> {	const isZai = baseUrl.includes("api.z.ai");
	const isNonStandard =		baseUrl.includes("cerebras.ai") ||		baseUrl.includes("api.x.ai") ||		baseUrl.includes("mistral.ai") ||		// ...

When a user sets a custom baseUrl (e.g., https://my-proxy.com/v1) but keeps provider: "cerebras", the URL check fails and the model gets incorrect OpenAI-standard settings (like trying to use developer role, which Cerebras doesn't support).

Proposed Fix

Modify detectCompatFromUrl to also check model.provider, giving it precedence over URL-based detection since it's explicitly configured:

File: packages/ai/src/providers/openai-completions.ts

Changes:

Rename detectCompatFromUrl(baseUrl: string) to detectCompat(model: Model<"openai-completions">)
Update the detection logic to check both provider and URL:

typescript

function detectCompat(model: Model<"openai-completions">): Required<OpenAICompat> {	const provider = model.provider;	const baseUrl = model.baseUrl;
	const isZai = provider === "zai" || baseUrl.includes("api.z.ai");
	const isNonStandard =		provider === "cerebras" || baseUrl.includes("cerebras.ai") ||		provider === "xai" || baseUrl.includes("api.x.ai") ||		provider === "mistral" || baseUrl.includes("mistral.ai") ||		baseUrl.includes("chutes.ai") ||		isZai ||		provider === "opencode" || baseUrl.includes("opencode.ai");
	const useMaxTokens =		provider === "mistral" || baseUrl.includes("mistral.ai") ||		baseUrl.includes("chutes.ai");
	const isGrok = provider === "xai" || baseUrl.includes("api.x.ai");
	const isMistral = provider === "mistral" || baseUrl.includes("mistral.ai");
	return {		supportsStore: !isNonStandard,		supportsDeveloperRole: !isNonStandard,		supportsReasoningEffort: !isGrok && !isZai,		supportsUsageInStreaming: true,		maxTokensField: useMaxTokens ? "max_tokens" : "max_completion_tokens",		requiresToolResultName: isMistral,		requiresAssistantAfterToolResult: false,		requiresThinkingAsText: isMistral,		requiresMistralToolIds: isMistral,		thinkingFormat: isZai ? "zai" : "openai",	};}

function detectCompat(model: Model<"openai-completions">): Required<OpenAICompat> {	const provider = model.provider;	const baseUrl = model.baseUrl;
	const isZai = provider === "zai" || baseUrl.includes("api.z.ai");
	const isNonStandard =		provider === "cerebras" || baseUrl.includes("cerebras.ai") ||		provider === "xai" || baseUrl.includes("api.x.ai") ||		provider === "mistral" || baseUrl.includes("mistral.ai") ||		baseUrl.includes("chutes.ai") ||		isZai ||		provider === "opencode" || baseUrl.includes("opencode.ai");
	const useMaxTokens =		provider === "mistral" || baseUrl.includes("mistral.ai") ||		baseUrl.includes("chutes.ai");
	const isGrok = provider === "xai" || baseUrl.includes("api.x.ai");
	const isMistral = provider === "mistral" || baseUrl.includes("mistral.ai");
	return {		supportsStore: !isNonStandard,		supportsDeveloperRole: !isNonStandard,		supportsReasoningEffort: !isGrok && !isZai,		supportsUsageInStreaming: true,		maxTokensField: useMaxTokens ? "max_tokens" : "max_completion_tokens",		requiresToolResultName: isMistral,		requiresAssistantAfterToolResult: false,		requiresThinkingAsText: isMistral,		requiresMistralToolIds: isMistral,		thinkingFormat: isZai ? "zai" : "openai",	};}

Update getCompat() to call detectCompat(model) instead of detectCompatFromUrl(model.baseUrl).

Backwards Compatibility: URL-based detection continues to work for users who don't explicitly set a provider. Provider-based detection takes precedence when set, which is the expected behavior since it's an explicit configuration.

Affected Files:

packages/ai/src/providers/openai-completions.ts (only file needing changes)

implement

>cd /Users/badlogic/workspaces/pi-mono && grep -n "detectCompatFromUrl" packages/ai/src/providers/openai-completions.ts

685:function detectCompatFromUrl(baseUrl: string): Required<OpenAICompat> {
721:	const detected = detectCompatFromUrl(model.baseUrl);

Editedopenai-completions.ts

>cd /Users/badlogic/workspaces/pi-mono && npm run check 2>&1

> pi-monorepo@0.0.3 check
> biome check --write . && tsgo --noEmit && npm run check -w @mariozechner/pi-web-ui

Checked 405 files in 321ms. Fixed 1 file.

> @mariozechner/pi-web-ui@0.47.0 check
> biome check --write . && tsc --noEmit && cd example && biome check --write . && tsc --noEmit

Checked 73 files in 44ms. No fixes applied.
Checked 3 files in 8ms. No fixes applied.

Now let me update the changelog:

EditedCHANGELOG.md

>cd /Users/badlogic/workspaces/pi-mono && npm run check 2>&1

> pi-monorepo@0.0.3 check
> biome check --write . && tsgo --noEmit && npm run check -w @mariozechner/pi-web-ui

Checked 405 files in 309ms. No fixes applied.

> @mariozechner/pi-web-ui@0.47.0 check
> biome check --write . && tsc --noEmit && cd example && biome check --write . && tsc --noEmit

Checked 73 files in 42ms. No fixes applied.
Checked 3 files in 8ms. No fixes applied.

Done. Summary of changes:

File: packages/ai/src/providers/openai-completions.ts

Renamed detectCompatFromUrl(baseUrl) → detectCompat(model)
Added provider-based detection alongside URL-based detection for: cerebras, xai, mistral, zai, opencode
Provider checks take precedence (checked first with ||), maintaining backwards compatibility with URL detection

File: packages/ai/CHANGELOG.md

Added fix entry under [Unreleased]

Users can now use custom base URLs (proxies, gateways) with providers like Cerebras and the correct compatibility settings will apply based on model.provider.

commit and push, close issue with comment

>cd /Users/badlogic/workspaces/pi-mono && git add -A && git commit -m "Fix provider feature detection to use model.provider, not just URL

Previously, OpenAI-compatible provider settings (like developer role support)
were detected only from the baseUrl. When using custom URLs (e.g., proxies),
detection failed. Now checks model.provider first, then falls back to URL.

Fixes #774"

Running formatting, linting, and type checking...

> pi-monorepo@0.0.3 check
> biome check --write . && tsgo --noEmit && npm run check -w @mariozechner/pi-web-ui

Checked 405 files in 337ms. No fixes applied.

> @mariozechner/pi-web-ui@0.47.0 check
> biome check --write . && tsc --noEmit && cd example && biome check --write . && tsc --noEmit

Checked 73 files in 40ms. No fixes applied.
Checked 3 files in 8ms. No fixes applied.
✅ All pre-commit checks passed!
[main f900eb59] Fix provider feature detection to use model.provider, not just URL
 2 files changed, 20 insertions(+), 8 deletions(-)

>cd /Users/badlogic/workspaces/pi-mono && gh issue close 774 --comment "Fixed in f900eb59. Provider feature detection now checks \`model.provider\` in addition to the URL, so custom base URLs (proxies, gateways) work correctly with provider-specific settings like Cerebras's lack of developer role support. Thanks for the report!"

! Issue badlogic/pi-mono#774 (pi-ai auto detection of features based on base url is confusing) is already closed

Issue was auto-closed by the commit. Let me add the comment:

>cd /Users/badlogic/workspaces/pi-mono && gh issue comment 774 --body "Fixed in f900eb59. Provider feature detection now checks \`model.provider\` in addition to the URL, so custom base URLs (proxies, gateways) work correctly with provider-specific settings like Cerebras's lack of developer role support. Thanks for the report!"

https://github.com/badlogic/pi-mono/issues/774#issuecomment-3759660738

Done. Committed (f900eb59), pushed, and issue #774 closed with comment.