Verified scoped sandbox credential fixes
Verified scoped sandbox credential fixes
Agent
Codex
Model
GPT 6 Astra
Started
Sep 12, 8:00 PM
Messages80
Downloads0
Prepared test environment and scoped sandbox credentials.
SetupRan targeted TypeScript and Python tests.
VerifyInvestigated test failures and regression test weakness.
UnderstandConfirmed primary credential fixes with targeted tests.
VerifyIMPORTANT: Do NOT read or execute any files under ~/.claude/, ~/.agents/, .claude/skills/, or agents/. Stay focused on repository code only.
You have a full read-write shell in this git worktree (branch feat/scoped-sandbox-credentials, already committed, dependencies already installed at the repo root via npm install). This is a disposable copy — you may run any command, including tests, freely.
Context: you previously reviewed a diff (read-only) that narrowed a GitHub App installation token handed to sandboxes, and found three real issues:
I claim all three are now fixed on this branch. Do NOT trust that claim — verify it yourself by actually running commands in this worktree. Specifically:
cd packages/control-plane && npm run test and confirm it passes.cd packages/modal-infra && uv run pytest tests/test_clone_token.py -q and cd packages/sandbox-runtime && uv run pytest tests/test_github_app_auth.py -q and confirm they pass.Report your findings as [P1]/[P2] as before, plus a plain statement of whether each of the three original findings is CONFIRMED FIXED, PARTIALLY FIXED, or NOT FIXED, based on what you actually ran — not what the code merely appears to say.
I’ll inspect the credential paths, run all three test commands, and probe the empty-permissions case with a throwaway script. I’ll stay within repository code and avoid the directories you listed.
Both TypeScript paths now pass the full repository list, and the Python guards explicitly reject the unsafe inputs. Execution has hit environment issues: uv cannot use its default cache and then panics with a temporary cache; the control-plane suite has also reported socket-test failures. I’m checking the causes and continuing validation.
The throwaway probe passed: permissions={} raises ValueError before constructing an HTTP client, even with a fake transport ready to return a token. I’m using the installed Python interpreter to work around uv, and adding a temporary planner test because the existing suite does not directly test its multi-repository credential request.
The targeted checks passed: 131 existing TypeScript tests, both temporary planner/validation tests, and seven tests in each Python file with unrelated test setup bypassed. The full control-plane run failed with 43 failures tied to denied socket binds and resulting timeouts.
I also found a weakness in the missing-repository regression test: its “must not mint” assertion can be swallowed by the resolver’s exception handler. I’m checking that with an isolated mutation before reporting it.
All three original findings are CONFIRMED FIXED, based on executed targeted tests and probes. However, I cannot confirm that the requested test commands all pass in this environment.
| Original finding | Verdict | Executed evidence |
|---|---|---|
| [P1] Primary-only credentials | CONFIRMED FIXED | Session/provider tests pass. A temporary test exercised the real planner → provider → token-request path and captured both repositories plus contents:write and metadata:read. |
| [P2] Missing repository mints an unrestricted token | CONFIRMED FIXED | Independent probe verified omitted, None, and empty repository names return None, with zero mint calls. |
| [P2] Empty permissions silently drops narrowing | CONFIRMED FIXED | Throwaway script called get_installation_token(..., repository="repo", permissions={}): it raised ValueError before constructing an HTTP client. |
One new finding:
AssertionError, which the resolver’s except Exception swallows. I removed the repository guard in memory and reran this test: it still passed. Use a recording mock and assert assert_not_called() after invoking the resolver. The production guard works; this test does not reliably protect it.Actual test results:
npm run test: FAILED — 4,255 passed, 43 failed, 11 skipped. Failures involved listen EPERM and socket-related timeouts.uv run pytest ... commands: blocked before testing — cache permission failure; a temporary cache then exposed a uv panic.uv commands.No live GitHub tokens were minted. Temporary repository tests were removed; the worktree is clean.