Fixed feature detection for custom base URLs

Summary

Closed, Completed

Understand
Analyzed issue and related code for feature detection.
Understand
Edit
Enhanced detection logic to prioritize provider checks.
Edit
Verify
Summarized changes and updated changelog.
Verify
Cleanup
Committed, pushed changes, and closed issue with comment.
Cleanup

Analyze GitHub issue(s): https://github.com/badlogic/pi-mono/issues/774

For each issue:

Read the issue in full, including all comments and linked issues/PRs.
For bugs:
- Ignore any root cause analysis in the issue (likely wrong)
- Read all related code files in full (no truncation)
- Trace the code path and identify the actual root cause
- Propose a fix
For feature requests:
- Read all related code files in full (no truncation)
- Propose the most concise implementation approach
- List affected files and changes needed

Do NOT implement unless explicitly asked. Analyze and propose only.

This is a bug/feature request about auto-detection of features based on base URL being problematic when using custom base URLs (like proxies). Let me understand the codebase to see how this feature detection works.

The issue mentions:

Using custom base URL for Cerebras (e.g., for proxying LLM calls)
Model configuration breaks because Cerebras-specific configurations (like no support for developer message) are tied to the base URL rather than the configured provider

I need to look at:

How providers are configured
How feature detection based on base URL works
Where Cerebras-specific configurations are handled

Let me start by looking at the ai package, specifically around stream.ts and provider configurations.

Let me explore the codebase to understand how feature detection based on base URL works.

Readtypes.ts

Now let me look at the openai-completions provider where the auto-detection logic likely lives:

Readopenai-completions.ts

Now I understand the issue. The problem is in the detectCompatFromUrl function:

typescript

function detectCompatFromUrl(baseUrl: string): Required<OpenAICompat> {	const isZai = baseUrl.includes("api.z.ai");
	const isNonStandard =		baseUrl.includes("cerebras.ai") ||		baseUrl.includes("api.x.ai") ||		baseUrl.includes("mistral.ai") ||		baseUrl.includes("chutes.ai") ||		isZai ||		baseUrl.includes("opencode.ai");    // ...

function detectCompatFromUrl(baseUrl: string): Required<OpenAICompat> {	const isZai = baseUrl.includes("api.z.ai");
	const isNonStandard =		baseUrl.includes("cerebras.ai") ||		baseUrl.includes("api.x.ai") ||		baseUrl.includes("mistral.ai") ||		baseUrl.includes("chutes.ai") ||		isZai ||		baseUrl.includes("opencode.ai");    // ...

When a user uses a custom base URL (like a proxy) for Cerebras, the URL no longer contains "cerebras.ai", so the model behaves as if it's a standard OpenAI provider. But if the user configured provider: "cerebras", they expect Cerebras-specific behavior regardless of the URL.

The solution would be to:

Also consider the model.provider when detecting compatibility, not just the baseUrl
Make model.provider take precedence, since that's what the user explicitly configured

Let me check how the getCompat function works:

typescript

function getCompat(model: Model<"openai-completions">): Required<OpenAICompat> {	const detected = detectCompatFromUrl(model.baseUrl);	if (!model.compat) return detected;
	return {		supportsStore: model.compat.supportsStore ?? detected.supportsStore,		// ...	};}

function getCompat(model: Model<"openai-completions">): Required<OpenAICompat> {	const detected = detectCompatFromUrl(model.baseUrl);	if (!model.compat) return detected;
	return {		supportsStore: model.compat.supportsStore ?? detected.supportsStore,		// ...	};}

So the user can override with model.compat, but detection only uses URL. The fix would be to also check model.provider in the detection logic.

Let me also check models.generated.ts to see what providers/base URLs are used for built-in models.

Let me also check how models are defined and the provider field is used:

Good, so the models have both provider: "cerebras" and baseUrl: "https://api.cerebras.ai/v1".

The issue is that when someone uses a custom base URL (e.g., a proxy), the detectCompatFromUrl function won't detect Cerebras-specific settings because the URL doesn't contain "cerebras.ai".

The fix is straightforward: detectCompatFromUrl should also take the provider parameter and check that in addition to (or instead of) the URL.

Let me verify by checking how getCompat calls detectCompatFromUrl:

typescript

function getCompat(model: Model<"openai-completions">): Required<OpenAICompat> {	const detected = detectCompatFromUrl(model.baseUrl);	if (!model.compat) return detected;
	return {		supportsStore: model.compat.supportsStore ?? detected.supportsStore,		// ...	};}

function getCompat(model: Model<"openai-completions">): Required<OpenAICompat> {	const detected = detectCompatFromUrl(model.baseUrl);	if (!model.compat) return detected;
	return {		supportsStore: model.compat.supportsStore ?? detected.supportsStore,		// ...	};}

The fix should:

Change detectCompatFromUrl to detectCompat(model) or detectCompat(baseUrl, provider)
Add provider-based detection in addition to URL-based detection
Provider check should take precedence since it's explicitly configured

Let me see what detection is needed for each provider:

From the code:

cerebras.ai → isNonStandard (no supportsStore, no supportsDeveloperRole)
api.x.ai (xAI/Grok) → isNonStandard, isGrok (no supportsReasoningEffort)
mistral.ai → isNonStandard, useMaxTokens, isMistral (special tool ID handling, thinking as text)
chutes.ai → isNonStandard, useMaxTokens
api.z.ai → isNonStandard, isZai (no supportsReasoningEffort, thinkingFormat: "zai")
opencode.ai → isNonStandard

The main providers to handle are cerebras, xai, mistral, and zai since those are in the KnownProvider type. I should also check if there are other places in the codebase where provider-based detection might be needed.

Let me check the KnownProvider type to see all known providers:

Now I have the full picture. The issue is that detectCompatFromUrl function only checks the baseUrl to determine provider-specific settings, but when users use proxies or custom base URLs, the provider information is lost.

The solution is to also check model.provider alongside the URL. The provider field explicitly states which provider is being used, so it should take precedence.

Let me summarize the analysis and proposed fix:

Root Cause: The detectCompatFromUrl function in packages/ai/src/providers/openai-completions.ts only checks the baseUrl to determine provider-specific compatibility settings. When a custom base URL is used (e.g., for proxying), the URL-based detection fails because it doesn't contain the provider's domain.

Affected Code: detectCompatFromUrl() function at lines ~495-521

Proposed Fix:

Rename detectCompatFromUrl to detectCompat and accept the full model object (or at least provider + baseUrl)
Check model.provider in addition to model.baseUrl
Provider-based detection should take precedence since it's explicitly configured

The changes would look like:```typescript function detectCompat(model: Model<"openai-completions">): Required { const provider = model.provider; const baseUrl = model.baseUrl; const isZai = provider === "zai" || baseUrl.includes("api.z.ai");

const isNonStandard =	provider === "cerebras" || baseUrl.includes("cerebras.ai") ||	provider === "xai" || baseUrl.includes("api.x.ai") ||	provider === "mistral" || baseUrl.includes("mistral.ai") ||	baseUrl.includes("chutes.ai") ||	isZai ||	provider === "opencode" || baseUrl.includes("opencode.ai");

const isNonStandard =	provider === "cerebras" || baseUrl.includes("cerebras.ai") ||	provider === "xai" || baseUrl.includes("api.x.ai") ||	provider === "mistral" || baseUrl.includes("mistral.ai") ||	baseUrl.includes("chutes.ai") ||	isZai ||	provider === "opencode" || baseUrl.includes("opencode.ai");


Then I need to define the specific compatibility flags for each provider. For Mistral and Chutes, `useMaxTokens` should be true since they require max_tokens instead of max_completion_tokens. Grok and Zai don't support reasoning_effort. The return object maps these flags to determine which OpenAI features are supported or need adaptation for each provider.


Then I need to define the specific compatibility flags for each provider. For Mistral and Chutes, `useMaxTokens` should be true since they require max_tokens instead of max_completion_tokens. Grok and Zai don't support reasoning_effort. The return object maps these flags to determine which OpenAI features are supported or need adaptation for each provider.

Analysis of Issue #774

Title: pi-ai auto detection of features based on base url is confusing

Problem: When using a custom base URL for providers like Cerebras (e.g., for proxying LLM calls), model configuration breaks because provider-specific settings (like "no support for developer message") are determined by checking if the URL contains cerebras.ai, rather than using the explicitly configured provider field.

Root Cause

In packages/ai/src/providers/openai-completions.ts, the detectCompatFromUrl() function (lines 495-521) only examines the baseUrl to determine provider-specific compatibility:

typescript

function detectCompatFromUrl(baseUrl: string): Required<OpenAICompat> {	const isZai = baseUrl.includes("api.z.ai");
	const isNonStandard =		baseUrl.includes("cerebras.ai") ||		baseUrl.includes("api.x.ai") ||		baseUrl.includes("mistral.ai") ||		// ...

function detectCompatFromUrl(baseUrl: string): Required<OpenAICompat> {	const isZai = baseUrl.includes("api.z.ai");
	const isNonStandard =		baseUrl.includes("cerebras.ai") ||		baseUrl.includes("api.x.ai") ||		baseUrl.includes("mistral.ai") ||		// ...

When a user sets a custom baseUrl (e.g., https://my-proxy.com/v1) but keeps provider: "cerebras", the URL check fails and the model gets incorrect OpenAI-standard settings (like trying to use developer role, which Cerebras doesn't support).

Proposed Fix

Modify detectCompatFromUrl to also check model.provider, giving it precedence over URL-based detection since it's explicitly configured:

File: packages/ai/src/providers/openai-completions.ts

Changes:

Rename detectCompatFromUrl(baseUrl: string) to detectCompat(model: Model<"openai-completions">)
Update the detection logic to check both provider and URL:

typescript

function detectCompat(model: Model<"openai-completions">): Required<OpenAICompat> {	const provider = model.provider;	const baseUrl = model.baseUrl;
	const isZai = provider === "zai" || baseUrl.includes("api.z.ai");
	const isNonStandard =		provider === "cerebras" || baseUrl.includes("cerebras.ai") ||		provider === "xai" || baseUrl.includes("api.x.ai") ||		provider === "mistral" || baseUrl.includes("mistral.ai") ||		baseUrl.includes("chutes.ai") ||		isZai ||		provider === "opencode" || baseUrl.includes("opencode.ai");
	const useMaxTokens =		provider === "mistral" || baseUrl.includes("mistral.ai") ||		baseUrl.includes("chutes.ai");
	const isGrok = provider === "xai" || baseUrl.includes("api.x.ai");
	const isMistral = provider === "mistral" || baseUrl.includes("mistral.ai");
	return {		supportsStore: !isNonStandard,		supportsDeveloperRole: !isNonStandard,		supportsReasoningEffort: !isGrok && !isZai,		supportsUsageInStreaming: true,		maxTokensField: useMaxTokens ? "max_tokens" : "max_completion_tokens",		requiresToolResultName: isMistral,		requiresAssistantAfterToolResult: false,		requiresThinkingAsText: isMistral,		requiresMistralToolIds: isMistral,		thinkingFormat: isZai ? "zai" : "openai",	};}

function detectCompat(model: Model<"openai-completions">): Required<OpenAICompat> {	const provider = model.provider;	const baseUrl = model.baseUrl;
	const isZai = provider === "zai" || baseUrl.includes("api.z.ai");
	const isNonStandard =		provider === "cerebras" || baseUrl.includes("cerebras.ai") ||		provider === "xai" || baseUrl.includes("api.x.ai") ||		provider === "mistral" || baseUrl.includes("mistral.ai") ||		baseUrl.includes("chutes.ai") ||		isZai ||		provider === "opencode" || baseUrl.includes("opencode.ai");
	const useMaxTokens =		provider === "mistral" || baseUrl.includes("mistral.ai") ||		baseUrl.includes("chutes.ai");
	const isGrok = provider === "xai" || baseUrl.includes("api.x.ai");
	const isMistral = provider === "mistral" || baseUrl.includes("mistral.ai");
	return {		supportsStore: !isNonStandard,		supportsDeveloperRole: !isNonStandard,		supportsReasoningEffort: !isGrok && !isZai,		supportsUsageInStreaming: true,		maxTokensField: useMaxTokens ? "max_tokens" : "max_completion_tokens",		requiresToolResultName: isMistral,		requiresAssistantAfterToolResult: false,		requiresThinkingAsText: isMistral,		requiresMistralToolIds: isMistral,		thinkingFormat: isZai ? "zai" : "openai",	};}

Update getCompat() to call detectCompat(model) instead of detectCompatFromUrl(model.baseUrl).

Backwards Compatibility: URL-based detection continues to work for users who don't explicitly set a provider. Provider-based detection takes precedence when set, which is the expected behavior since it's an explicit configuration.

Affected Files:

packages/ai/src/providers/openai-completions.ts (only file needing changes)

implement

Readopenai-completions.ts60 lines

Editedopenai-completions.ts

Now let me update the changelog:

ReadCHANGELOG.md40 lines

EditedCHANGELOG.md

Done. Summary of changes:

File: packages/ai/src/providers/openai-completions.ts

Renamed detectCompatFromUrl(baseUrl) → detectCompat(model)
Added provider-based detection alongside URL-based detection for: cerebras, xai, mistral, zai, opencode
Provider checks take precedence (checked first with ||), maintaining backwards compatibility with URL detection

File: packages/ai/CHANGELOG.md

Added fix entry under [Unreleased]

Users can now use custom base URLs (proxies, gateways) with providers like Cerebras and the correct compatibility settings will apply based on model.provider.

commit and push, close issue with comment

Issue was auto-closed by the commit. Let me add the comment:

Done. Committed (f900eb59), pushed, and issue #774 closed with comment.