Fixed test.sh to skip LLM endpoint tests
Public
Fixed test.sh to skip LLM endpoint tests
Agent
Pi
Model
GPT-5.2 Codex
Started
Jan 20, 2:11 PM
Messages36
Downloads0
Updated test.sh to unset LLM-related environment variables.
EditRan checks to confirm tests skipped LLM endpoints.
Verify
test.sh is supposed not to run any tests that require an llm endpoint, however, it does run some of them. please fix
Planning file inspection
I'm deciding to locate and inspect the test.sh file by listing directory contents with bash before reading it.
>lsReviewing test script
Investigating LLM authentication detection
I'm checking how tests detect local LLM keys, focusing on auth.json or environment variables like AWS, Bedrock, or Azure; I'll examine the auth detection logic in packages/ai/src/stream.ts and related files.
Reviewing test environment requirements
I found that test.sh unsets many API keys but leaves AWS, Google ADC, AI_GATEWAY_API_KEY, and MINIMAX set, indicating tests might run for Bedrock, Vertex, or similar. I'll check if tests requiring LLM keys default to skipping when keys are missing, particularly in packages/ai/test and the test runner configuration.
>rg "getEnvApiKey|apiKey|skip" packages/ai/testNoticing incomplete env cleanup
I see test.sh unsets only some environment variables while others related to AI gateways, AWS, Google ADC, and Google Cloud remain set, which might cause tests relying on bedrock or vertex credentials to misbehave. Next, I'll check tests for hasBedrockCredentials to confirm which env vars need unsetting or disabling in test.sh.
>rg "hasBedrockCredentials|isVertexConfigured" -n packages/ai/testLocating environment details in utilities
Verifying environment variable usage in tests
I found that test.sh doesn't unset environment variables, and stream tests use env vars for project and location plus Google application credentials; I'll examine stream.test.ts to confirm their usage.
Identifying environment conflicts in tests
I'm noticing that various environment variables linked to cloud/auth services can cause tests to run unexpectedly; updating test.sh to explicitly unset all related envs will help isolate tests and prevent unintended external calls. Next, I'll verify token storage locations and ensure relevant env vars like API keys and AWS credentials are cleared before testing.
Checking environment variables in tests
I'm verifying which environment variables tests set or rely on, especially tokens like those for the Gemini CLI, to ensure all authentication env vars are unset during testing for clean, isolated runs.
>rg "process\.env" packages/ai/testPlanning environment variable unsets in test.sh
I'm identifying all relevant API keys and cloud credentials to unset in test.sh to fully disable cross-provider tests and avoid unintended usage. Next, I'll edit test.sh to batch-add these unset commands after the existing ones, ensuring no duplicates and clear grouping.
Expanding AWS and cloud unsets in test.sh
I'm including all relevant AWS credential and region variables plus additional Google cloud project/location environment variables to the unset block, ensuring comprehensive disabling of cloud-dependent tests. I'll replace the existing unset block with the expanded one for clarity.
Confirming need to run npm check
Despite concerns about runtime cost, must run npm run check after code changes as per instructions.
>npm run checkConfirming tests run
The tests are run using the command npm run check, verifying code correctness and quality.
Summary:
Tests: