Verified fixes with adversarial testing

Summary

In Progress

Understand
Reviewed code diff and prior failure reports.
Understand
Reproduce
Ran tests to reproduce failures against HEAD.
Reproduce
Verify
Confirmed most failures fixed with regression tests.
Verify
Understand
Identified remaining issue with error deduplication.
Understand
Understand
Found additional minor issues during hourly review.
Understand

You are an independent reviewer with no stake in prior conclusions. In this repository, branch feat/l5-meta-improvement vs origin/main (run git diff origin/main...HEAD), an earlier review reported the findings listed below and the author claims all of them are now fixed at HEAD. Your job is adversarial VERIFICATION: for each numbered finding, reproduce the original failure scenario against HEAD and state whether it is CLOSED (the failure no longer reproduces, and a test pins it) or STILL OPEN (with the reproduction). Then look for any NEW regression the fixes introduced, and any finding you consider more serious than the ones listed. Run python3 -m pytest scripts/ -q -p no:cacheprovider and ruff check scripts/ and targeted python3 - <<'PY' ... PY reproductions. Never claim a result you did not observe. Do not modify tracked files.

Findings claimed fixed: A1. Policy metadata (origin/parent) not validated against history; editing origin alone from "revision" to "init" disabled the wait gate (revise-improvement-policy.py main; improvement_policy.py). A2. A topic name containing @ was skipped by candidate_anchor() and allowed acceptance of a validity regression. A3. mine-trace-failures.py matched topics only on the shortened command+excerpt; a keyword past the excerpt (e.g. ValueError: secret exposed after a long frame line) produced zero evidence with truncated=[]. A4. Rolling back to v1 skipped the waiting period (judged_from() returned None). A5. Ancestor comparison checked coverage only against the immediate parent; a coverage regression inherited from a grandparent (v1 knew "quartz", v2 dropped it, v3 added something unrelated) never rolled back. A6. Hard-linked outputs bypassed input protection (a hard link named report.json to the archive, passed as --out-json, overwrote the archive). B1. The codex-review.yml posting step sanitised codex-review-status but not codex-review-sha; archive-and-recommend.yml accepted the SHA marker anywhere in a bot comment. B2. Two archive proposals opened before either merged got the same round number; measurement merged them by number and rounds_under() counted them once. C1. render-rsi-dashboard.py clamped negative validity to zero on the chart. C2. docs/plans/recursive-meta-improvement.md presented unreproducible first-run numbers (0.95 with verifier sessions; 96/3/69/28) as if reproducible. R36a. measure-policy-validity.py treated a complete scan with only unclassified failures as an empty anchor (evidence_trace_ids counted only matching sessions). R36b. (same as A4). Also review the new hourly path: mine-trace-failures.py --namespace @slug --traces-key KEY (list/sync/show through the Traces CLI) and the new evidence-refresh PR branch logic in revise-improvement-policy.yml — look for ways the hourly job could spam PRs, leak the key, write outside docs/rsi, or accept a policy revision on evidence it should not.

Output: one line per finding A1: CLOSED|STILL OPEN — <one sentence>; then "## New findings" (numbered, with [P1]/[P2]/[P3], file:line, reproduction) or "none"; then "## Verification" with the commands and observed results; then "## Verdict".

I’ll inspect the diff, reproduce each reported failure against HEAD, and review the hourly path. I won’t modify tracked files.

>git status --short; git branch --show-current; git diff origin/main...HEAD --stat

{"chunk_id":"c74939","wall_time_seconds":0.000133125,"exit_code":0,"original_token_count":431,"output":"?? .claude/settings.json\nfeat/l5-meta-improvement\n .github/workflows/archive-and-recommend.yml     |   17 +-\n .github/workflows/codex-review.yml              |   96 +-\n .github/workflows/revise-improvement-policy.yml |  241 ++++\n .prettierignore                                 |    3 +\n docs/improvement-policy-history.jsonl           |    1 +\n docs/improvement-policy.json                    |   68 ++\n docs/plans/recursive-meta-improvement.md        |  166 +++\n docs/production-hardening-backlog.md            |   52 +-\n docs/rsi/dashboard.html                         |  113 ++\n docs/rsi/measurement.json                       |  431 ++++++++\n docs/rsi/trace-evidence-verifier.json           |   26 +\n docs/rsi/trace-evidence.json                    |   60 +\n scripts/archive-round.py                        |   58 +-\n scripts/archive_round_test.py                   |   90 +-\n scripts/detect-recurring-pattern.py             |   96 +-\n scripts/detect_recurring_pattern_test.py        |   13 +\n scripts/improvement_policy.py                   |  370 +++++++\n scripts/improvement_policy_test.py              |  162 +++\n scripts/measure-policy-validity.py              |  492 +++++++++\n scripts/measure_policy_validity_test.py         |  417 +++++++\n scripts/mine-trace-failures.py                  |  490 +++++++++\n scripts/mine_trace_failures_test.py             |  466 ++++++++\n scripts/render-rsi-dashboard.py                 |  583 ++++++++++\n scripts/render_rsi_dashboard_test.py            |  286 +++++\n scripts/revise-improvement-policy.py            |  987 +++++++++++++++++\n scripts/revise_improvement_policy_test.py       | 1333 +++++++++++++++++++++++\n 26 files changed, 7032 insertions(+), 85 deletions(-)\n"}
{"chunk_id":"e71f35","wall_time_seconds":0.000005166,"exit_code":0,"original_token_count":425,"output":".github/workflows/analyze-pr-traces.yml\n.github/workflows/archive-and-recommend.yml\n.github/workflows/ci-python.yml\n.github/workflows/ci.yml\n.github/workflows/codex-review.yml\n.github/workflows/compose-smoke.yml\n.github/workflows/deploy-web.yml\n.github/workflows/revise-improvement-policy.yml\n.github/workflows/sandbox-images.yml\n.github/workflows/terraform.yml\nscripts/analyze-latest-review.py\nscripts/analyze-traces.py\nscripts/analyze_latest_review_test.py\nscripts/analyze_traces_test.py\nscripts/archive-round.py\nscripts/archive_round_test.py\nscripts/bootstrap-workspace-owner.test.ts\nscripts/bootstrap-workspace-owner.ts\nscripts/cf-logs.test.ts\nscripts/cf-logs.ts\nscripts/check-aws-stack.sh\nscripts/compose-smoke-paths.test.mjs\nscripts/compose-smoke.sh\nscripts/d1-migrate.sh\nscripts/detect-recurring-pattern.py\nscripts/detect_recurring_pattern_test.py\nscripts/improvement_policy.py\nscripts/improvement_policy_test.py\nscripts/lint-complexity-message.mjs\nscripts/lint-complexity-message.test.mjs\nscripts/lint-complexity.mjs\nscripts/lint-sql-portability.mjs\nscripts/lint-sql-portability.test.mjs\nscripts/measure-policy-validity.py\nscripts/measure_policy_validity_test.py\nscripts/merge-split-users.test.ts\nscripts/merge-split-users.ts\nscripts/mine-trace-failures.py\nscripts/mine_trace_failures_test.py\nscripts/parse-review-findings.py\nscripts/parse_review_findings_test.py\nscripts/redact-secrets.py\nscripts/redact_secrets_test.py\nscripts/render-rsi-dashboard.py\nscripts/render_rsi_dashboard_test.py\nscripts/revise-improvement-policy.py\nscripts/revise_improvement_policy_test.py\nscripts/sql-portability-baseline.json\nscripts/sync-pr-traces.py\nscripts/sync_pr_traces_test.py\nscripts/wrangler-secrets.sh\n"}

>rg --files scripts .github/workflows | sort

{"chunk_id":"c74939","wall_time_seconds":0.000133125,"exit_code":0,"original_token_count":431,"output":"?? .claude/settings.json\nfeat/l5-meta-improvement\n .github/workflows/archive-and-recommend.yml     |   17 +-\n .github/workflows/codex-review.yml              |   96 +-\n .github/workflows/revise-improvement-policy.yml |  241 ++++\n .prettierignore                                 |    3 +\n docs/improvement-policy-history.jsonl           |    1 +\n docs/improvement-policy.json                    |   68 ++\n docs/plans/recursive-meta-improvement.md        |  166 +++\n docs/production-hardening-backlog.md            |   52 +-\n docs/rsi/dashboard.html                         |  113 ++\n docs/rsi/measurement.json                       |  431 ++++++++\n docs/rsi/trace-evidence-verifier.json           |   26 +\n docs/rsi/trace-evidence.json                    |   60 +\n scripts/archive-round.py                        |   58 +-\n scripts/archive_round_test.py                   |   90 +-\n scripts/detect-recurring-pattern.py             |   96 +-\n scripts/detect_recurring_pattern_test.py        |   13 +\n scripts/improvement_policy.py                   |  370 +++++++\n scripts/improvement_policy_test.py              |  162 +++\n scripts/measure-policy-validity.py              |  492 +++++++++\n scripts/measure_policy_validity_test.py         |  417 +++++++\n scripts/mine-trace-failures.py                  |  490 +++++++++\n scripts/mine_trace_failures_test.py             |  466 ++++++++\n scripts/render-rsi-dashboard.py                 |  583 ++++++++++\n scripts/render_rsi_dashboard_test.py            |  286 +++++\n scripts/revise-improvement-policy.py            |  987 +++++++++++++++++\n scripts/revise_improvement_policy_test.py       | 1333 +++++++++++++++++++++++\n 26 files changed, 7032 insertions(+), 85 deletions(-)\n"}
{"chunk_id":"e71f35","wall_time_seconds":0.000005166,"exit_code":0,"original_token_count":425,"output":".github/workflows/analyze-pr-traces.yml\n.github/workflows/archive-and-recommend.yml\n.github/workflows/ci-python.yml\n.github/workflows/ci.yml\n.github/workflows/codex-review.yml\n.github/workflows/compose-smoke.yml\n.github/workflows/deploy-web.yml\n.github/workflows/revise-improvement-policy.yml\n.github/workflows/sandbox-images.yml\n.github/workflows/terraform.yml\nscripts/analyze-latest-review.py\nscripts/analyze-traces.py\nscripts/analyze_latest_review_test.py\nscripts/analyze_traces_test.py\nscripts/archive-round.py\nscripts/archive_round_test.py\nscripts/bootstrap-workspace-owner.test.ts\nscripts/bootstrap-workspace-owner.ts\nscripts/cf-logs.test.ts\nscripts/cf-logs.ts\nscripts/check-aws-stack.sh\nscripts/compose-smoke-paths.test.mjs\nscripts/compose-smoke.sh\nscripts/d1-migrate.sh\nscripts/detect-recurring-pattern.py\nscripts/detect_recurring_pattern_test.py\nscripts/improvement_policy.py\nscripts/improvement_policy_test.py\nscripts/lint-complexity-message.mjs\nscripts/lint-complexity-message.test.mjs\nscripts/lint-complexity.mjs\nscripts/lint-sql-portability.mjs\nscripts/lint-sql-portability.test.mjs\nscripts/measure-policy-validity.py\nscripts/measure_policy_validity_test.py\nscripts/merge-split-users.test.ts\nscripts/merge-split-users.ts\nscripts/mine-trace-failures.py\nscripts/mine_trace_failures_test.py\nscripts/parse-review-findings.py\nscripts/parse_review_findings_test.py\nscripts/redact-secrets.py\nscripts/redact_secrets_test.py\nscripts/render-rsi-dashboard.py\nscripts/render_rsi_dashboard_test.py\nscripts/revise-improvement-policy.py\nscripts/revise_improvement_policy_test.py\nscripts/sql-portability-baseline.json\nscripts/sync-pr-traces.py\nscripts/sync_pr_traces_test.py\nscripts/wrangler-secrets.sh\n"}

>git diff origin/main...HEAD

{"status":"fulfilled","value":{"chunk_id":"bd8162","wall_time_seconds":0.000005584,"exit_code":0,"original_token_count":87368,"output":"Warning: truncated output (original token count: 87368)\nTotal output lines: 7554\n\ndiff --git a/.github/workflows/archive-and-recommend.yml b/.github/workflows/archive-and-recommend.yml\nindex fc306370..ce5adaa1 100644\n--- a/.github/workflows/archive-and-recommend.yml\n+++ b/.github/workflows/archive-and-recommend.yml\n@@ -59,7 +59,12 @@ on:\n     types: [completed]\n   push:\n     branches: [main]\n-    paths: [\"docs/self-improvement-archive.jsonl\"]\n+    paths:\n+      - \"docs/self-improvement-archive.jsonl\"\n+      # A revised policy (new topic, restored weight) can cross the\n+      # mechanism-fix threshold on the existing archive; re-evaluate then\n+      # too (Codex review of PR #10, round 31).\n+      - \"docs/improvement-policy.json\"\n \n permissions:\n   contents: write\n@@ -122,14 +127,18 @@ jobs:\n               per_page: 100,\n             });\n \n-            // Both checks matter: the author check stops a PR commenter\n+            // All three checks matter: the author check stops a PR commenter\n             // from forging a review-shaped comment; the SHA marker stops\n             // an old (correctly bot-authored) review comment from a prior\n-            // commit being mistaken for this run's review.\n+            // commit being mistaken for this run's review; and the marker\n+            // must be the comment's LAST line — the footer the workflow\n+            // itself appends — so a marker quoted inside another commit's\n+            // review text cannot claim that review for this SHA (Codex\n+            // full-branch review, workflows finding 1).\n             const reviewComments = comments.filter(\n               (c) =>\n                 (c.body || \"\").startsWith(\"### Codex independent review\") &&\n-                (c.body || \"\").includes(marker) &&\n+                (c.body || \"\").trimEnd().endsWith(marker) &&\n                 c.user?.type === \"Bot\" &&\n                 c.user?.login === \"github-actions[bot]\"\n             );\ndiff --git a/.github/workflows/codex-review.yml b/.github/workflows/codex-review.yml\nindex 6ed2be84..0e63e4ed 100644\n--- a…86368 tokens truncated…uiltin_policy()\n+    v2 = policy_mod.new_version(\n+        v1, topics=dict(v1[\"topics\"]), threshold=v1[\"threshold\"], origin=\"revision\", rationale=\"x\"\n+    )\n+    archive = tmp_path / \"archive.jsonl\"\n+    archive.write_text(\"\\n\".join(json.dumps(e) for e in _archive()) + \"\\n\")\n+    history = tmp_path / \"history.jsonl\"\n+    history.write_text(json.dumps({\"version\": 2, \"policy\": v2}) + \"\\n\")\n+    edited = {**v2, \"origin\": \"init\", \"parent\": None}\n+    policy_path = tmp_path / \"policy.json\"\n+    policy_path.write_text(json.dumps(edited))\n+    m_path = tmp_path / \"m.json\"\n+    m_path.write_text(json.dumps(measure.measure(_archive(), edited, None)))\n+    code = revise.main(\n+        [\n+            \"r\",\n+            str(archive),\n+            \"--measurement\",\n+            str(m_path),\n+            \"--policy\",\n+            str(policy_path),\n+            \"--history\",\n+            str(history),\n+            \"--dry-run\",\n+            \"--now\",\n+            NOW,\n+        ]\n+    )\n+    assert code == 1\n+    assert \"lineage metadata\" in capsys.readouterr().err\n+    with pytest.raises(ValueError, match=\"history records no versions\"):\n+        policy_mod.assert_policy_matches_history(v2, [])\n+    policy_mod.assert_policy_matches_history(v2, [{\"version\": 2, \"policy\": v2}])\n+\n+\n+def test_evidence_window_filters_by_round_identity_not_number():\n+    # Two reviews share round 5; one predates the snapshot and one does not.\n+    entries = [\n+        {\n+            \"round\": 5,\n+            \"source_sha\": \"old\",\n+            \"findings\": [\"**[P1]** a.\"],\n+            \"occurred_at\": \"2026-09-14T15:00:00Z\",\n+        },\n+        {\n+            \"round\": 5,\n+            \"source_sha\": \"new\",\n+            \"findings\": [\"**[P1]** b.\"],\n+            \"occurred_at\": \"2026-09-14T17:00:00Z\",\n+        },\n+    ]\n+    measurement = {\"anchor\": {\"collected_at\": \"2026-09-14T16:00:00Z\"}}\n+    covered = revise.entries_covered_by_evidence(entries, measurement)\n+    assert [e[\"source_sha\"] for e in covered] == [\"old\"]\n"}}
{"status":"fulfilled","value":{"chunk_id":"20363e","wall_time_seconds":1.002056208,"session_id":95306,"original_token_count":3,"output":"........."}}
{"status":"fulfilled","value":{"chunk_id":"52f194","wall_time_seconds":0.000001625,"exit_code":0,"original_token_count":5,"output":"All checks passed!\n"}}
{"status":"fulfilled","value":{"chunk_id":"7506da","wall_time_seconds":0.000005333,"exit_code":0,"original_token_count":7413,"output":"\"\"\"The improvement mechanism's own policy, as versioned data instead of code.\n\nBefore this module, the rule that decides whether a recurring finding\ndeserves a target-level patch or a mechanism-level fix lived as constants in\nscripts/detect-recurring-pattern.py: a keyword taxonomy and a recurrence\nthreshold, written by hand once and never revisited. That is an L4 loop in\nthe paper's terms (docs/plans/recursive-meta-improvement.md): the system\nadapts its deployed state, but the mechanism governing what counts as an\nimprovement stays fixed human infrastructure.\n\nL5 requires that mechanism to be something the system can revise from\nevidence, with the same safeguards it applies to every other change. So the\npolicy becomes a JSON document with a version, a parent, and an origin, and\nevery revision is appended to a history file with the evidence that\njustified it. The pieces that must NOT be revisable by the meta-improver\n(the archive, the external anchor, the independent verifier, the acceptance\nthresholds, and the promotion path) are enumerated in FIXED_INFRASTRUCTURE,\nand `assert_ai_may_write` refuses any write outside AI_OWNED_COMPONENTS.\n\"\"\"\n\nfrom __future__ import annotations\n\nimport hashlib\nimport json\nfrom datetime import UTC, datetime\nfrom pathlib import Path\n\nREPO_ROOT = Path(__file__).resolve().parent.parent\nPOLICY_PATH = REPO_ROOT / \"docs\" / \"improvement-policy.json\"\nHISTORY_PATH = REPO_ROOT / \"docs\" / \"improvement-policy-history.jsonl\"\n\n# The v1 taxonomy and threshold, kept in code only as a fallback so every\n# existing tool still runs in a checkout that predates the policy file.\nBUILTIN_THRESHOLD = 3\nBUILTIN_TOPIC_KEYWORDS: dict[str, list[str]] = {\n    \"credential-redaction\": [\"redact\", \"credential\", \"secret\", \"token\", \"leak\", \"expos\"],\n    \"shell-semantics\": [\"errexit\", \"bash -e\", \"exit code\", \"-e\", \"pipefail\", \"shell\"],\n    \"env-var-precedence\": [\"precedence\", \"env var\", \"environment variable\", \"unconditionally\"],\n    \"fork-pr-permissions\": [\"fork\", \"github_token\", \"persist-credentials\"],\n    \"auth-lifecycle\": [\"refresh token\", \"rotat\", \"expir\", \"auth.json\", \"stale\"],\n}\n\n# Autonomy attribution (paper failure mode 2): the meta-improver may rewrite\n# exactly these files, and nothing else. Paths are repo-relative.\nAI_OWNED_COMPONENTS: dict[str, str] = {\n    \"improvement-policy\": \"docs/improvement-policy.json\",\n    \"improvement-policy-history\": \"docs/improvement-policy-history.jsonl\",\n}\n\n# Everything the loop depends on that stays human-owned infrastructure. The\n# dashboard renders this list verbatim so the boundary is visible, not implied.\nFIXED_INFRASTRUCTURE: dict[str, str] = {\n    \"archive\": \"docs/self-improvement-archive.jsonl — append-only, SHA-idempotent (archive-round.py)\",\n    \"verifier\": \".github/workflows/codex-review.yml — independent second-model review of every PR\",\n    \"anchor\": \"Traces evidence from working sessions — never consulted when a round is decided\",\n    \"meta-acceptance-rule\": \"MIN_COVERAGE / MIN_VALIDITY / MIN_ROUNDS_TO_JUDGE in revise-improvement-policy.py\",\n    \"promotion\": \"pull requests only; a human merges every policy revision and every rollback\",\n}\n\n\ndef utc_now_iso() -> str:\n    return datetime.now(UTC).replace(microsecond=0).isoformat().replace(\"+00:00\", \"Z\")\n\n\ndef builtin_policy() -> dict:\n    return {\n        \"version\": 1,\n        \"parent\": None,\n        \"origin\": \"init\",\n        \"created_at\": \"2026-09-14T18:00:00Z\",\n        \"threshold\": BUILTIN_THRESHOLD,\n        \"topics\": {\n            topic: {\"keywords\": list(keywords), \"weight\": 1.0}\n            for topic, keywords in BUILTIN_TOPIC_KEYWORDS.items()\n        },\n        \"rationale\": \"Built-in fallback identical to policy version 1.\",\n    }\n\n\ndef load_policy(path: Path | str = POLICY_PATH) -> dict:\n    with open(path) as f:\n        policy = json.load(f)\n    validate_policy(policy)\n    return policy\n\n\ndef load_policy_or_builtin(path: Path | str = POLICY_PATH) -> dict:\n    if Path(path).exists():\n        return load_policy(path)\n    return builtin_policy()\n\n\nVALID_ORIGINS = frozenset({\"init\", \"revision\", \"rollback\"})\n\n\ndef round_key(entry: dict) -> str:\n    \"\"\"Identity of an archived round, shared by every tool that counts\n    rounds (the detector that opens issues and the measurer the policy is\n    judged on must count the same signal — Codex review of PR #10, round\n    37). Automated rounds are identified by the commit they reviewed: two\n    archive proposals opened before either merged both computed the same\n    next round number, and merging by number alone collapsed two reviews\n    into one round (full-branch review, workflows finding 2). Legacy rounds\n    without a source_sha keep their number (placeholder + result pairs).\"\"\"\n    sha = entry.get(\"source_sha\")\n    if isinstance(sha, str) and sha:\n        return f\"sha:{sha}\"\n    return f\"round:{entry.get('round')}\"\n\n\ndef assert_policy_matches_history(policy: dict, history: list[dict]) -> None:\n    \"\"\"The policy's lineage metadata (version, parent, origin,\n    restored_version) is not part of its content hash, yet the wait gate and\n    rollback logic depend on it. So a policy in force must be exactly the\n    snapshot its history recorded for that version, metadata included; a\n    root policy must not claim to be a revision or rollback. Otherwise an\n    edit to `origin` alone would switch the evaluation gates off (Codex\n    full-branch review, finding 1).\"\"\"\n    if not history:\n        if policy.get(\"origin\") != \"init\" or policy.get(\"parent\") is not None:\n            raise ValueError(\n                \"policy claims a revision/rollback lineage but the history records no versions\"\n            )\n        return\n    latest = history[-1]\n    snapshot = latest.get(\"policy\") or {}\n    if latest.get(\"version\") != policy.get(\"version\"):\n        raise ValueError(\n            f\"policy is v{policy.get('version')} but the history's latest entry is \"\n            f\"v{latest.get('version')}; the policy and its history must be written together\"\n        )\n    for key in (\"version\", \"parent\", \"origin\", \"restored_version\"):\n        if snapshot.get(key) != policy.get(key):\n            raise ValueError(\n                f\"policy.{key}={policy.get(key)!r} differs from the recorded v{policy.get('version')} \"\n                f\"snapshot ({snapshot.get(key)!r}); lineage metadata may not be edited in place\"\n            )\n    if policy_hash(snapshot) != policy_hash(policy):\n        raise ValueError(\n            f\"policy content hashes to {policy_hash(policy)} but the recorded v{policy.get('version')} \"\n            f\"snapshot hashes to {policy_hash(snapshot)}\"\n        )\n\n\ndef validate_policy(policy: dict) -> None:\n    if not isinstance(policy.get(\"version\"), int) or policy[\"version\"] < 1:\n        raise ValueError(\"policy.version must be a positive integer\")\n    if not isinstance(policy.get(\"threshold\"), int) or policy[\"threshold\"] < 1:\n        raise ValueError(\"policy.threshold must be a positive integer\")\n    if policy.get(\"origin\") not in VALID_ORIGINS:\n        raise ValueError(f\"policy.origin must be one of {sorted(VALID_ORIGINS)}\")\n    topics = policy.get(\"topics\")\n    if not isinstance(topics, dict) or not topics:\n        raise ValueError(\"policy.topics must be a non-empty object\")\n    for name, spec in topics.items():\n        if not isinstance(name, str) or not name or \"@\" in name or name != name.strip():\n            # `name@tag` keys are reserved for older definitions of a name in\n            # the evidence; a topic named that way would be skipped by the\n            # candidate anchor and escape the validity comparison (Codex\n            # full-branch review, finding 2).\n            raise ValueError(\n                f\"topic name {name!r} is invalid (non-empty, no '@', no surrounding whitespace)\"\n            )\n        keywords = spec.get(\"keywords\")\n        if (\n            not isinstance(keywords, list)\n            or not keywords\n            or not all(isinstance(k, str) and k for k in keywords)\n        ):\n            # An empty list would classify nothing while matching every trace\n            # (Codex review of PR #10, round 16).\n            raise ValueError(f\"topic {name!r} needs a non-empty list of keyword strings\")\n        weight = spec.get(\"weight\", 1.0)\n        if not isinstance(weight, int | float) or weight <= 0:\n            raise ValueError(f\"topic {name!r} weight must be a positive number\")\n\n\ndef policy_hash(policy: dict) -> str:\n    \"\"\"Content hash of the decision-relevant fields. Two policies with the\n    same taxonomy, weights, and threshold decide identically, whatever their\n    version metadata says — this is what the dashboard pins per epoch to\n    show the evaluator was frozen while a round was decided.\n\n    Topic ORDER is part of the hash: classification takes the first topic\n    whose keyword matches, so reordering overlapping topics changes\n    decisions and must not pass the stale-measurement guard (Codex review\n    of PR #10, finding 3).\"\"\"\n    canonical = json.dumps(\n        {\n            \"threshold\": policy[\"threshold\"],\n            \"topics\": [\n                [name, spec[\"keywords\"], float(spec.get(\"weight\", 1.0))]\n                for name, spec in policy[\"topics\"].items()\n            ],\n        },\n        separators=(\",\", \":\"),\n    )\n    return hashlib.sha256(canonical.encode()).hexdigest()[:12]\n\n\ndef topic_keywords(policy: dict) -> dict[str, list[str]]:\n    return {name: list(spec[\"keywords\"]) for name, spec in policy[\"topics\"].items()}\n\n\ndef topic_weights(policy: dict) -> dict[str, float]:\n    return {name: float(spec.get(\"weight\", 1.0)) for name, spec in policy[\"topics\"].items()}\n\n\ndef classify_finding(text: str, keywords: dict[str, list[str]]) -> str | None:\n    \"\"\"First topic (in policy order) with any keyword present. Same rule the\n    detector has always applied; it lives here so every tool classifies\n    identically under the same policy version.\"\"\"\n    lowered = text.lower()\n    for topic, words in keywords.items():\n        if any(word in lowered for word in words):\n            return topic\n    return None\n\n\ndef new_version(\n    parent: dict,\n    *,\n    topics: dict,\n    threshold: int,\n    origin: str,\n    rationale: str,\n    created_at: str | None = None,\n    restored_version: int | None = None,\n) -> dict:\n    if origin not in {\"revision\", \"rollback\"}:\n        raise ValueError(\"origin must be 'revision' or 'rollback'\")\n    policy = {\n        \"version\": parent[\"version\"] + 1,\n        \"parent\": parent[\"version\"],\n        \"origin\": origin,\n        \"created_at\": created_at or utc_now_iso(),\n        \"threshold\": threshold,\n        \"topics\": topics,\n        \"rationale\": rationale,\n    }\n    if origin == \"rollback\":\n        # Which version's configuration this restores, so ancestry checks can\n        # continue through it (Codex review of PR #10, round 17).\n        policy[\"restored_version\"] = restored_version\n    validate_policy(policy)\n    return policy\n\n\ndef relative_to_repo(path: Path | str) -> str:\n    resolved = Path(path).resolve()\n    try:\n        return resolved.relative_to(REPO_ROOT).as_posix()\n    except ValueError:\n        return resolved.as_posix()\n\n\ndef component_paths(role: str | None, allowed: dict[str, str] | None = None) -> set[str]:\n    \"\"\"Paths the meta-improver may write for one role ('policy' or\n    'history'), or for any role when role is None.\"\"\"\n    components = allowed or AI_OWNED_COMPONENTS\n    if role is None:\n        return set(components.values())\n    return {p for name, p in components.items() if name == role or name.endswith(f\"-{role}\")}\n\n\ndef assert_ai_may_write(\n    path: Path | str, *, allowed: dict[str, str] | None = None, role: str | None = None\n) -> None:\n    \"\"\"Attribution guard: the meta-improver only ever writes the files it owns,\n    and each file only in its own role: the policy destination must be the\n    policy component and the history destination the history component, so\n    swapped arguments cannot append a policy to the history or overwrite the\n    policy with a history line (Codex review of PR #10, rounds 20 and 35).\n    Raises PermissionError otherwise, so a bug that tries to 'fix' the archive\n    or the verifier fails loudly instead of silently widening autonomy.\"\"\"\n    allowed_paths = component_paths(role, allowed)\n    rel = relative_to_repo(path)\n    if rel not in allowed_paths:\n        what = f\"the {role} component\" if role else \"a file it owns\"\n        raise PermissionError(\n            f\"{rel} is fixed infrastructure or not {what}; \"\n            f\"the meta-improver may only write {sorted(allowed_paths)} here\"\n        )\n\n\nPROTECTED_OUTPUT_PREFIXES: tuple[str, ...] = (\".github/\", \"scripts/\", \"packages/\", \"terraform/\")\nPROTECTED_OUTPUT_FILES: tuple[str, ...] = (\n    \"docs/self-improvement-archive.jsonl\",\n    \"docs/improvement-policy.json\",\n    \"docs/improvement-policy-history.jsonl\",\n)\n# The committed field anchors: only a deliberate evidence refresh may write\n# them, never a report or decision output (Codex review of PR #10, round 15).\nCANONICAL_EVIDENCE_FILES: tuple[str, ...] = (\n    \"docs/rsi/trace-evidence.json\",\n    \"docs/rsi/trace-evidence-verifier.json\",\n)\n\n\ndef assert_safe_output(\n    path: Path | str, *, inputs: list[str | Path] = (), kind: str = \"report\"\n) -> None:\n    \"\"\"Side outputs may go anywhere EXCEPT the loop's own records, its code,\n    the files the invocation is reading, and (for anything but an evidence\n    refresh) the canonical evidence snapshots (Codex review of PR #10,\n    rounds 11 and 15).\"\"\"\n    rel = relative_to_repo(path)\n    if rel in PROTECTED_OUTPUT_FILES or any(rel.startswith(p) for p in PROTECTED_OUTPUT_PREFIXES):\n        raise PermissionError(f\"{rel} is a protected file; choose another output path\")\n    if kind != \"evidence\" and rel in CANONICAL_EVIDENCE_FILES:\n        raise PermissionError(\n            f\"{rel} is a canonical evidence snapshot; only --save-evidence may write it\"\n        )\n    # Identity is by path AND by inode: a hard link to the archive named\n    # report.json resolves to a different path but is the same file (Codex\n    # full-branch review, finding 6).\n    for protected in (\n        *PROTECTED_OUTPUT_FILES,\n        *(() if kind == \"evidence\" else CANONICAL_EVIDENCE_FILES),\n    ):\n        if same_file(REPO_ROOT / protected, path):\n            raise PermissionError(f\"{rel} is the same file as protected {protected}\")\n    for source in inputs:\n        if source and (Path(source).resolve() == Path(path).resolve() or same_file(source, path)):\n            raise PermissionError(f\"{rel} is an input of this run; choose another output path\")\n\n\ndef same_file(a: Path | str, b: Path | str) -> bool:\n    try:\n        return Path(a).samefile(b)\n    except OSError:\n        return False\n\n\ndef save_policy(\n    policy: dict, path: Path | str = POLICY_PATH, *, allowed: dict[str, str] | None = None\n) -> None:\n    assert_ai_may_write(path, allowed=allowed, role=\"policy\")\n    validate_policy(policy)\n    Path(path).write_text(json.dumps(policy, indent=2) + \"\\n\")\n\n\ndef load_history(path: Path | str = HISTORY_PATH) -> list[dict]:\n    if not Path(path).exists():\n        return []\n    entries = []\n    with open(path) as f:\n        for line in f:\n            line = line.strip()\n            if line:\n                entries.append(json.loads(line))\n    return entries\n\n\ndef append_history(\n    entry: dict, path: Path | str = HISTORY_PATH, *, allowed: dict[str, str] | None = None\n) -> None:\n    assert_ai_may_write(path, allowed=allowed, role=\"history\")\n    # No sort_keys: a snapshot's topic order is its classification\n    # precedence, and restoring an alphabetized snapshot would silently\n    # reclassify findings (Codex review of PR #10, finding 2).\n    with open(path, \"a\") as f:\n        f.write(json.dumps(entry) + \"\\n\")\n\n\n# The checked-in policy, resolved once so every tool stamps and decides with\n# the same version and hash in one process.\n_CURRENT = load_policy_or_builtin()\nPOLICY_VERSION: int = _CURRENT[\"version\"]\nPOLICY_HASH: str = policy_hash(_CURRENT)\nname: Revise Improvement Policy\n\n# The L5 step of docs/plans/recursive-meta-improvement.md, run automatically\n# but bounded exactly like archive-and-recommend.yml: it never pushes to the\n# default branch, never merges, never deploys, and requests no repository\n# repository secrets at all.\n#\n# After every change to the review archive on main (an archive-round PR\n# merging), this workflow:\n#   1. measures whether docs/improvement-policy.json's signal still predicts\n#      the field (scripts/measure-policy-validity.py) — coverage of archived\n#      findings, and agreement with Traces evidence when a key is present;\n#   2. lets scripts/revise-improvement-policy.py apply its fixed acceptance\n#      rule: propose a bounded policy revision, propose a rollback of a\n#      revision that made things worse, or do nothing;\n#   3. re-renders docs/rsi/dashboard.html from the resulting state;\n#   4. opens ONE pull request carrying the policy, its history entry, the\n#      measurement, and the dashboard. A human merges it, or closes it.\n#\n# The field anchor (Traces evidence) is refreshed on every run from the\n# shared traces.com namespace when a read-only TRACES_API_KEY is present\n# (hourly on the schedule above); otherwise the committed snapshot is used.\n# Without any snapshot the anchor is absent: coverage is still measured and\n# can still trigger a revision, validity is reported as null, and the\n# workflow says so rather than pretending it was checked.\n\non:\n  push:\n    branches: [main]\n    paths:\n      - docs/self-improvement-archive.jsonl\n      # A refreshed field snapshot is new evidence too (Codex review of PR #10, round 10).\n      - docs/rsi/trace-evidence.json\n  schedule:\n    # Hourly: refresh the field anchor from the shared Traces namespace,\n    # re-measure, and let the fixed acceptance rule decide. Off the hour so\n    # it does not queue behind every other project's :00 cron.\n    - cron: \"17 * * * *\"\n  workflow_dispatch:\n\npermissions:\n  contents: write\n  pull-requests: write\n\nconcurrency:\n  group: revise-improvement-policy\n  cancel-in-progress: false\n\njobs:\n  revise:\n    name: Measure, revise, and propose\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    steps:\n      - name: Checkout (default branch — trusted)\n        uses: actions/checkout@v4\n        with:\n          # Always the default branch, whatever ref a manual dispatch was\n          # started from, so a proposal never carries an unrelated feature\n          # branch's commits (Codex review of PR #10, round 5).\n          ref: ${{ github.event.repository.default_branch }}\n          fetch-depth: 0\n\n      - name: Set up Python\n        uses: actions/setup-python@v5\n        with:\n          python-version: \"3.12\"\n\n      - name: Record the commit actually checked out\n        # The default branch may have advanced past the triggering commit, and\n        # a manual dispatch may come from another ref; label everything with\n        # what this run measured (Codex review of PR #10, round 16).\n        id: source\n        run: echo \"sha=$(git rev-parse HEAD)\" >> \"$GITHUB_OUTPUT\"\n\n      - name: Check for a Traces API key\n        id: traces-key\n        env:\n          HAS_KEY: ${{ secrets.TRACES_API_KEY != '' }}\n        run: echo \"present=$HAS_KEY\" >> \"$GITHUB_OUTPUT\"\n\n      - name: Install Traces CLI\n        if: steps.traces-key.outputs.present == 'true'\n        run: |\n          curl -fsSL https://www.traces.com/install | bash\n          echo \"$HOME/.traces/bin\" >> \"$GITHUB_PATH\"\n\n      - name: Refresh the field anchor from the shared Traces namespace\n        # Working sessions live on developers' machines, but every session\n        # they share lands in the traces.com namespace, which a read-only key\n        # can list and sync. The miner keeps only tool results the tool\n        # itself reported as errors, so the anchor is failures, never\n        # narration. Without a key the committed snapshot is used and its\n        # age is reported instead of silently reused (Codex review of\n        # PR #10, round 3).\n        if: steps.traces-key.outputs.present == 'true'\n        env:\n          TRACES_API_KEY: ${{ secrets.TRACES_API_KEY }}\n          TRACES_NAMESPACE: ${{ vars.TRACES_NAMESPACE || '@gagan114' }}\n        run: |\n          set -euo pipefail\n          mkdir -p docs/rsi\n          if python3 scripts/mine-trace-failures.py \\\n              --namespace \"$TRACES_NAMESPACE\" --traces-key \"$TRACES_API_KEY\" \\\n              --save-evidence docs/rsi/trace-evidence.json \\\n              --out-json \"$RUNNER_TEMP/field-failures.json\" > \"$RUNNER_TEMP/mine.txt\"; then\n            head -n 30 \"$RUNNER_TEMP/mine.txt\"\n          else\n            echo \"::warning::Traces refresh failed; using the committed evidence snapshot instead.\"\n            sed -n '1,40p' \"$RUNNER_TEMP/mine.txt\" || true\n            git checkout -- docs/rsi/trace-evidence.json 2>/dev/null || true\n            rm -f \"$RUNNER_TEMP/field-failures.json\"\n          fi\n\n      - name: Measure policy validity\n        # This step reports the snapshot's age so a stale anchor is visible\n        # rather than silently reused (Codex review of PR #10, round 3).\n        run: |\n          set -euo pipefail\n          mkdir -p docs/rsi\n          if [ -f docs/rsi/trace-evidence.json ]; then\n            collected=$(python3 -c 'import json; print(json.load(open(\"docs/rsi/trace-evidence.json\")).get(\"collected_at\", \"unknown\"))')\n            echo \"::notice::Using the committed Traces evidence snapshot collected at ${collected}. Refresh it locally to update the field anchor.\"\n            python3 scripts/measure-policy-validity.py docs/self-improvement-archive.jsonl \\\n              --trace-evidence docs/rsi/trace-evidence.json --out-json docs/rsi/measurement.json > measurement.txt\n          else\n            echo \"::notice::No Traces evidence snapshot committed; validity will be null and only coverage can trigger a revision.\"\n            python3 scripts/measure-policy-validity.py docs/self-improvement-archive.jsonl \\\n              --out-json docs/rsi/measurement.json > measurement.txt\n          fi\n          sed -n '1,/^---/p' measurement.txt\n\n      - name: Apply the fixed acceptance rule\n        id: revise\n        run: |\n          set -euo pipefail\n          args=(docs/self-improvement-archive.jsonl --measurement docs/rsi/measurement.json --out-json decision.json)\n          # Fresh field blind spots (from this run's Traces refresh) feed\n          # topic mining the same way unclassified review findings do.\n          [ -f \"$RUNNER_TEMP/field-failures.json\" ] && args+=(--field-failures \"$RUNNER_TEMP/field-failures.json\")\n          python3 scripts/revise-improvement-policy.py \"${args[@]}\" > revise.txt\n          sed -n '1,/^---/p' revise.txt\n          action=$(python3 -c 'import json; print(json.load(open(\"decision.json\"))[\"action\"])')\n          echo \"action=$action\" >> \"$GITHUB_OUTPUT\"\n          if [ \"$action\" != \"none\" ]; then\n            # The measurement that justified the decision is evidence; keep it.\n            # The live measurement must describe the policy now in force, or the\n            # next run's hash check would refuse it (Codex review of PR #10, round 4).\n            cp docs/rsi/measurement.json docs/rsi/measurement-decision.json\n            if [ -f docs/rsi/trace-evidence.json ]; then\n              python3 scripts/measure-policy-validity.py docs/self-improvement-archive.jsonl \\\n                --trace-evidence docs/rsi/trace-evidence.json --out-json docs/rsi/measurement.json > remeasure.txt\n            else\n              python3 scripts/measure-policy-validity.py docs/self-improvement-archive.jsonl \\\n                --out-json docs/rsi/measurement.json > remeasure.txt\n            fi\n          fi\n\n      - name: Re-render the dashboard\n        env:\n          SOURCE_SHA: ${{ steps.source.outputs.sha }}\n        run: |\n          set -euo pipefail\n          args=(docs/self-improvement-archive.jsonl --head \"${SOURCE_SHA:0:8}\" --out docs/rsi/dashboard.html)\n          [ -f docs/rsi/trace-evidence.json ] && args+=(--trace-evidence docs/rsi/trace-evidence.json)\n          [ -f docs/rsi/trace-evidence-verifier.json ] && args+=(--verifier-evidence docs/rsi/trace-evidence-verifier.json)\n          python3 scripts/render-rsi-dashboard.py \"${args[@]}\"\n\n      - name: Propose the result as a pull request\n        env:\n          GH_TOKEN: ${{ github.token }}\n          ACTION: ${{ steps.revise.outputs.action }}\n          SOURCE_SHA: ${{ steps.source.outputs.sha }}\n        run: |\n          set -euo pipefail\n          if git diff --quiet -- docs/improvement-policy.json docs/improvement-policy-history.jsonl docs/rsi/; then\n            echo \"Nothing changed (action: $ACTION).\"\n            exit 0\n          fi\n          version=$(python3 -c 'import json; print(json.load(open(\"docs/improvement-policy.json\"))[\"version\"])')\n          default_branch=$(gh repo view --json defaultBranchRef --jq .defaultBranchRef.name)\n          git config user.name \"github-actions[bot]\"\n          git config user.email \"41898282+github-actions[bot]@users.noreply.github.com\"\n          if git diff --quiet -- docs/improvement-policy.json docs/improvement-policy-history.jsonl; then\n            # Only the evidence, measurement and dashboard moved (the hourly\n            # refresh with no decision). One standing evidence branch, updated\n            # in place, so the hourly run never opens a new PR per hour; a\n            # human merges it whenever, and merging it re-triggers this\n            # workflow through the trace-evidence.json path filter.\n            # A standing policy proposal that the current decision no longer\n            # supports is withdrawn, not left open to merge (Codex review of\n            # PR #10, round 37): new rounds can legitimately turn a\n            # `revise` into `none`.\n            stale_pr=$(gh pr list --head \"improvement-policy-proposal\" --base \"$default_branch\" --state open \\\n              --json number,isCrossRepository --jq '[.[] | select(.isCrossRepository == false)][0].number // empty')\n            if [ -n \"$stale_pr\" ]; then\n              gh pr close \"$stale_pr\" --delete-branch \\\n                --comment \"Withdrawn: re-evaluated at \\`${SOURCE_SHA}\\` the fixed acceptance rule proposes no policy change (action: **${ACTION}**), so this proposal no longer satisfies it. A new proposal will be opened if a later round supports one.\"\n              echo \"Closed the stale policy proposal PR #$stale_pr.\"\n            fi\n            branch=\"improvement-evidence-refresh\"\n            title=\"chore(rsi): refresh the Traces field anchor\"\n            body=\"Automatically refreshed by [\\`revise-improvement-policy.yml\\`](../blob/main/.github/workflows/revise-improvement-policy.yml) from the shared Traces namespace at \\`${SOURCE_SHA}\\`. The fixed acceptance rule proposed no policy change on this evidence (action: **${ACTION}**). This changes only the evidence snapshot, the measurement and the dashboard; the policy is untouched.\"\n            git checkout -b \"$branch\"\n            git add docs/rsi/\n            git commit -m \"$title\"\n            git push --force origin \"$branch\"\n            existing_pr=$(gh pr list --head \"$branch\" --base \"$default_branch\" --state open \\\n              --json number,isCrossRepository --jq '[.[] | select(.isCrossRepository == false)][0].number // empty')\n            if [ -n \"$existing_pr\" ]; then\n              echo \"Updated the open evidence PR #$existing_pr in place.\"\n              exit 0\n            fi\n            gh pr create --title \"$title\" --body \"$body\" --label \"self-improvement-archive\" \\\n              --base \"$default_branch\" --head \"$branch\"\n            exit 0\n          fi\n          # One standing proposal branch: a newer archive round supersedes an\n          # open proposal instead of opening a competing one with the same\n          # version and parent (Codex review of PR #10, round 5).\n          branch=\"improvement-policy-proposal\"\n          git checkout -b \"$branch\"\n          git add docs/improvement-policy.json docs/improvement-policy-history.jsonl docs/rsi/\n          git commit -m \"chore(rsi): ${ACTION} of the improvement policy (v${version})\"\n          git push --force origin \"$branch\"\n          # Same-repository PRs only: `--head` matches by branch name alone, so\n          # a fork PR using this branch name must not be mistaken for the\n          # bot's own proposal (Codex review of PR #10, round 19).\n          existing_pr=$(gh pr list --head \"$branch\" --base \"$default_branch\" --state open \\\n            --json number,isCrossRepository --jq '[.[] | select(.isCrossRepository == false)][0].number // empty')\n          if [ -n \"$existing_pr\" ]; then\n            gh pr comment \"$existing_pr\" --body \"Superseded by a newer archive round at \\`${SOURCE_SHA}\\`: this proposal was regenerated from the current archive (action: **${ACTION}**, policy v${version}).\"\n            echo \"Updated the open proposal PR #$existing_pr in place.\"\n            exit 0\n          fi\n          gh pr create \\\n            --title \"chore(rsi): ${ACTION} of the improvement policy (v${version})\" \\\n            --body \"Automatically proposed by [\\`revise-improvement-policy.yml\\`](../blob/main/.github/workflows/revise-improvement-policy.yml) after the archive changed at \\`${SOURCE_SHA}\\`. Action: **${ACTION}**. See \\`docs/improvement-policy-history.jsonl\\` for the evidence and \\`docs/rsi/dashboard.html\\` for the rendered state. This changes only the AI-owned policy files; a human merges it, same as every other change in this repo.\" \\\n            --label \"self-improvement-archive\" \\\n            --base \"$default_branch\" \\\n            --head \"$branch\"\n"}}

>python3 -m pytest scripts/ -q -p no:cacheprovider

{"status":"fulfilled","value":{"chunk_id":"bd8162","wall_time_seconds":0.000005584,"exit_code":0,"original_token_count":87368,"output":"Warning: truncated output (original token count: 87368)\nTotal output lines: 7554\n\ndiff --git a/.github/workflows/archive-and-recommend.yml b/.github/workflows/archive-and-recommend.yml\nindex fc306370..ce5adaa1 100644\n--- a/.github/workflows/archive-and-recommend.yml\n+++ b/.github/workflows/archive-and-recommend.yml\n@@ -59,7 +59,12 @@ on:\n     types: [completed]\n   push:\n     branches: [main]\n-    paths: [\"docs/self-improvement-archive.jsonl\"]\n+    paths:\n+      - \"docs/self-improvement-archive.jsonl\"\n+      # A revised policy (new topic, restored weight) can cross the\n+      # mechanism-fix threshold on the existing archive; re-evaluate then\n+      # too (Codex review of PR #10, round 31).\n+      - \"docs/improvement-policy.json\"\n \n permissions:\n   contents: write\n@@ -122,14 +127,18 @@ jobs:\n               per_page: 100,\n             });\n \n-            // Both checks matter: the author check stops a PR commenter\n+            // All three checks matter: the author check stops a PR commenter\n             // from forging a review-shaped comment; the SHA marker stops\n             // an old (correctly bot-authored) review comment from a prior\n-            // commit being mistaken for this run's review.\n+            // commit being mistaken for this run's review; and the marker\n+            // must be the comment's LAST line — the footer the workflow\n+            // itself appends — so a marker quoted inside another commit's\n+            // review text cannot claim that review for this SHA (Codex\n+            // full-branch review, workflows finding 1).\n             const reviewComments = comments.filter(\n               (c) =>\n                 (c.body || \"\").startsWith(\"### Codex independent review\") &&\n-                (c.body || \"\").includes(marker) &&\n+                (c.body || \"\").trimEnd().endsWith(marker) &&\n                 c.user?.type === \"Bot\" &&\n                 c.user?.login === \"github-actions[bot]\"\n             );\ndiff --git a/.github/workflows/codex-review.yml b/.github/workflows/codex-review.yml\nindex 6ed2be84..0e63e4ed 100644\n--- a…86368 tokens truncated…uiltin_policy()\n+    v2 = policy_mod.new_version(\n+        v1, topics=dict(v1[\"topics\"]), threshold=v1[\"threshold\"], origin=\"revision\", rationale=\"x\"\n+    )\n+    archive = tmp_path / \"archive.jsonl\"\n+    archive.write_text(\"\\n\".join(json.dumps(e) for e in _archive()) + \"\\n\")\n+    history = tmp_path / \"history.jsonl\"\n+    history.write_text(json.dumps({\"version\": 2, \"policy\": v2}) + \"\\n\")\n+    edited = {**v2, \"origin\": \"init\", \"parent\": None}\n+    policy_path = tmp_path / \"policy.json\"\n+    policy_path.write_text(json.dumps(edited))\n+    m_path = tmp_path / \"m.json\"\n+    m_path.write_text(json.dumps(measure.measure(_archive(), edited, None)))\n+    code = revise.main(\n+        [\n+            \"r\",\n+            str(archive),\n+            \"--measurement\",\n+            str(m_path),\n+            \"--policy\",\n+            str(policy_path),\n+            \"--history\",\n+            str(history),\n+            \"--dry-run\",\n+            \"--now\",\n+            NOW,\n+        ]\n+    )\n+    assert code == 1\n+    assert \"lineage metadata\" in capsys.readouterr().err\n+    with pytest.raises(ValueError, match=\"history records no versions\"):\n+        policy_mod.assert_policy_matches_history(v2, [])\n+    policy_mod.assert_policy_matches_history(v2, [{\"version\": 2, \"policy\": v2}])\n+\n+\n+def test_evidence_window_filters_by_round_identity_not_number():\n+    # Two reviews share round 5; one predates the snapshot and one does not.\n+    entries = [\n+        {\n+            \"round\": 5,\n+            \"source_sha\": \"old\",\n+            \"findings\": [\"**[P1]** a.\"],\n+            \"occurred_at\": \"2026-09-14T15:00:00Z\",\n+        },\n+        {\n+            \"round\": 5,\n+            \"source_sha\": \"new\",\n+            \"findings\": [\"**[P1]** b.\"],\n+            \"occurred_at\": \"2026-09-14T17:00:00Z\",\n+        },\n+    ]\n+    measurement = {\"anchor\": {\"collected_at\": \"2026-09-14T16:00:00Z\"}}\n+    covered = revise.entries_covered_by_evidence(entries, measurement)\n+    assert [e[\"source_sha\"] for e in covered] == [\"old\"]\n"}}
{"status":"fulfilled","value":{"chunk_id":"20363e","wall_time_seconds":1.002056208,"session_id":95306,"original_token_count":3,"output":"........."}}
{"status":"fulfilled","value":{"chunk_id":"52f194","wall_time_seconds":0.000001625,"exit_code":0,"original_token_count":5,"output":"All checks passed!\n"}}
{"status":"fulfilled","value":{"chunk_id":"7506da","wall_time_seconds":0.000005333,"exit_code":0,"original_token_count":7413,"output":"\"\"\"The improvement mechanism's own policy, as versioned data instead of code.\n\nBefore this module, the rule that decides whether a recurring finding\ndeserves a target-level patch or a mechanism-level fix lived as constants in\nscripts/detect-recurring-pattern.py: a keyword taxonomy and a recurrence\nthreshold, written by hand once and never revisited. That is an L4 loop in\nthe paper's terms (docs/plans/recursive-meta-improvement.md): the system\nadapts its deployed state, but the mechanism governing what counts as an\nimprovement stays fixed human infrastructure.\n\nL5 requires that mechanism to be something the system can revise from\nevidence, with the same safeguards it applies to every other change. So the\npolicy becomes a JSON document with a version, a parent, and an origin, and\nevery revision is appended to a history file with the evidence that\njustified it. The pieces that must NOT be revisable by the meta-improver\n(the archive, the external anchor, the independent verifier, the acceptance\nthresholds, and the promotion path) are enumerated in FIXED_INFRASTRUCTURE,\nand `assert_ai_may_write` refuses any write outside AI_OWNED_COMPONENTS.\n\"\"\"\n\nfrom __future__ import annotations\n\nimport hashlib\nimport json\nfrom datetime import UTC, datetime\nfrom pathlib import Path\n\nREPO_ROOT = Path(__file__).resolve().parent.parent\nPOLICY_PATH = REPO_ROOT / \"docs\" / \"improvement-policy.json\"\nHISTORY_PATH = REPO_ROOT / \"docs\" / \"improvement-policy-history.jsonl\"\n\n# The v1 taxonomy and threshold, kept in code only as a fallback so every\n# existing tool still runs in a checkout that predates the policy file.\nBUILTIN_THRESHOLD = 3\nBUILTIN_TOPIC_KEYWORDS: dict[str, list[str]] = {\n    \"credential-redaction\": [\"redact\", \"credential\", \"secret\", \"token\", \"leak\", \"expos\"],\n    \"shell-semantics\": [\"errexit\", \"bash -e\", \"exit code\", \"-e\", \"pipefail\", \"shell\"],\n    \"env-var-precedence\": [\"precedence\", \"env var\", \"environment variable\", \"unconditionally\"],\n    \"fork-pr-permissions\": [\"fork\", \"github_token\", \"persist-credentials\"],\n    \"auth-lifecycle\": [\"refresh token\", \"rotat\", \"expir\", \"auth.json\", \"stale\"],\n}\n\n# Autonomy attribution (paper failure mode 2): the meta-improver may rewrite\n# exactly these files, and nothing else. Paths are repo-relative.\nAI_OWNED_COMPONENTS: dict[str, str] = {\n    \"improvement-policy\": \"docs/improvement-policy.json\",\n    \"improvement-policy-history\": \"docs/improvement-policy-history.jsonl\",\n}\n\n# Everything the loop depends on that stays human-owned infrastructure. The\n# dashboard renders this list verbatim so the boundary is visible, not implied.\nFIXED_INFRASTRUCTURE: dict[str, str] = {\n    \"archive\": \"docs/self-improvement-archive.jsonl — append-only, SHA-idempotent (archive-round.py)\",\n    \"verifier\": \".github/workflows/codex-review.yml — independent second-model review of every PR\",\n    \"anchor\": \"Traces evidence from working sessions — never consulted when a round is decided\",\n    \"meta-acceptance-rule\": \"MIN_COVERAGE / MIN_VALIDITY / MIN_ROUNDS_TO_JUDGE in revise-improvement-policy.py\",\n    \"promotion\": \"pull requests only; a human merges every policy revision and every rollback\",\n}\n\n\ndef utc_now_iso() -> str:\n    return datetime.now(UTC).replace(microsecond=0).isoformat().replace(\"+00:00\", \"Z\")\n\n\ndef builtin_policy() -> dict:\n    return {\n        \"version\": 1,\n        \"parent\": None,\n        \"origin\": \"init\",\n        \"created_at\": \"2026-09-14T18:00:00Z\",\n        \"threshold\": BUILTIN_THRESHOLD,\n        \"topics\": {\n            topic: {\"keywords\": list(keywords), \"weight\": 1.0}\n            for topic, keywords in BUILTIN_TOPIC_KEYWORDS.items()\n        },\n        \"rationale\": \"Built-in fallback identical to policy version 1.\",\n    }\n\n\ndef load_policy(path: Path | str = POLICY_PATH) -> dict:\n    with open(path) as f:\n        policy = json.load(f)\n    validate_policy(policy)\n    return policy\n\n\ndef load_policy_or_builtin(path: Path | str = POLICY_PATH) -> dict:\n    if Path(path).exists():\n        return load_policy(path)\n    return builtin_policy()\n\n\nVALID_ORIGINS = frozenset({\"init\", \"revision\", \"rollback\"})\n\n\ndef round_key(entry: dict) -> str:\n    \"\"\"Identity of an archived round, shared by every tool that counts\n    rounds (the detector that opens issues and the measurer the policy is\n    judged on must count the same signal — Codex review of PR #10, round\n    37). Automated rounds are identified by the commit they reviewed: two\n    archive proposals opened before either merged both computed the same\n    next round number, and merging by number alone collapsed two reviews\n    into one round (full-branch review, workflows finding 2). Legacy rounds\n    without a source_sha keep their number (placeholder + result pairs).\"\"\"\n    sha = entry.get(\"source_sha\")\n    if isinstance(sha, str) and sha:\n        return f\"sha:{sha}\"\n    return f\"round:{entry.get('round')}\"\n\n\ndef assert_policy_matches_history(policy: dict, history: list[dict]) -> None:\n    \"\"\"The policy's lineage metadata (version, parent, origin,\n    restored_version) is not part of its content hash, yet the wait gate and\n    rollback logic depend on it. So a policy in force must be exactly the\n    snapshot its history recorded for that version, metadata included; a\n    root policy must not claim to be a revision or rollback. Otherwise an\n    edit to `origin` alone would switch the evaluation gates off (Codex\n    full-branch review, finding 1).\"\"\"\n    if not history:\n        if policy.get(\"origin\") != \"init\" or policy.get(\"parent\") is not None:\n            raise ValueError(\n                \"policy claims a revision/rollback lineage but the history records no versions\"\n            )\n        return\n    latest = history[-1]\n    snapshot = latest.get(\"policy\") or {}\n    if latest.get(\"version\") != policy.get(\"version\"):\n        raise ValueError(\n            f\"policy is v{policy.get('version')} but the history's latest entry is \"\n            f\"v{latest.get('version')}; the policy and its history must be written together\"\n        )\n    for key in (\"version\", \"parent\", \"origin\", \"restored_version\"):\n        if snapshot.get(key) != policy.get(key):\n            raise ValueError(\n                f\"policy.{key}={policy.get(key)!r} differs from the recorded v{policy.get('version')} \"\n                f\"snapshot ({snapshot.get(key)!r}); lineage metadata may not be edited in place\"\n            )\n    if policy_hash(snapshot) != policy_hash(policy):\n        raise ValueError(\n            f\"policy content hashes to {policy_hash(policy)} but the recorded v{policy.get('version')} \"\n            f\"snapshot hashes to {policy_hash(snapshot)}\"\n        )\n\n\ndef validate_policy(policy: dict) -> None:\n    if not isinstance(policy.get(\"version\"), int) or policy[\"version\"] < 1:\n        raise ValueError(\"policy.version must be a positive integer\")\n    if not isinstance(policy.get(\"threshold\"), int) or policy[\"threshold\"] < 1:\n        raise ValueError(\"policy.threshold must be a positive integer\")\n    if policy.get(\"origin\") not in VALID_ORIGINS:\n        raise ValueError(f\"policy.origin must be one of {sorted(VALID_ORIGINS)}\")\n    topics = policy.get(\"topics\")\n    if not isinstance(topics, dict) or not topics:\n        raise ValueError(\"policy.topics must be a non-empty object\")\n    for name, spec in topics.items():\n        if not isinstance(name, str) or not name or \"@\" in name or name != name.strip():\n            # `name@tag` keys are reserved for older definitions of a name in\n            # the evidence; a topic named that way would be skipped by the\n            # candidate anchor and escape the validity comparison (Codex\n            # full-branch review, finding 2).\n            raise ValueError(\n                f\"topic name {name!r} is invalid (non-empty, no '@', no surrounding whitespace)\"\n            )\n        keywords = spec.get(\"keywords\")\n        if (\n            not isinstance(keywords, list)\n            or not keywords\n            or not all(isinstance(k, str) and k for k in keywords)\n        ):\n            # An empty list would classify nothing while matching every trace\n            # (Codex review of PR #10, round 16).\n            raise ValueError(f\"topic {name!r} needs a non-empty list of keyword strings\")\n        weight = spec.get(\"weight\", 1.0)\n        if not isinstance(weight, int | float) or weight <= 0:\n            raise ValueError(f\"topic {name!r} weight must be a positive number\")\n\n\ndef policy_hash(policy: dict) -> str:\n    \"\"\"Content hash of the decision-relevant fields. Two policies with the\n    same taxonomy, weights, and threshold decide identically, whatever their\n    version metadata says — this is what the dashboard pins per epoch to\n    show the evaluator was frozen while a round was decided.\n\n    Topic ORDER is part of the hash: classification takes the first topic\n    whose keyword matches, so reordering overlapping topics changes\n    decisions and must not pass the stale-measurement guard (Codex review\n    of PR #10, finding 3).\"\"\"\n    canonical = json.dumps(\n        {\n            \"threshold\": policy[\"threshold\"],\n            \"topics\": [\n                [name, spec[\"keywords\"], float(spec.get(\"weight\", 1.0))]\n                for name, spec in policy[\"topics\"].items()\n            ],\n        },\n        separators=(\",\", \":\"),\n    )\n    return hashlib.sha256(canonical.encode()).hexdigest()[:12]\n\n\ndef topic_keywords(policy: dict) -> dict[str, list[str]]:\n    return {name: list(spec[\"keywords\"]) for name, spec in policy[\"topics\"].items()}\n\n\ndef topic_weights(policy: dict) -> dict[str, float]:\n    return {name: float(spec.get(\"weight\", 1.0)) for name, spec in policy[\"topics\"].items()}\n\n\ndef classify_finding(text: str, keywords: dict[str, list[str]]) -> str | None:\n    \"\"\"First topic (in policy order) with any keyword present. Same rule the\n    detector has always applied; it lives here so every tool classifies\n    identically under the same policy version.\"\"\"\n    lowered = text.lower()\n    for topic, words in keywords.items():\n        if any(word in lowered for word in words):\n            return topic\n    return None\n\n\ndef new_version(\n    parent: dict,\n    *,\n    topics: dict,\n    threshold: int,\n    origin: str,\n    rationale: str,\n    created_at: str | None = None,\n    restored_version: int | None = None,\n) -> dict:\n    if origin not in {\"revision\", \"rollback\"}:\n        raise ValueError(\"origin must be 'revision' or 'rollback'\")\n    policy = {\n        \"version\": parent[\"version\"] + 1,\n        \"parent\": parent[\"version\"],\n        \"origin\": origin,\n        \"created_at\": created_at or utc_now_iso(),\n        \"threshold\": threshold,\n        \"topics\": topics,\n        \"rationale\": rationale,\n    }\n    if origin == \"rollback\":\n        # Which version's configuration this restores, so ancestry checks can\n        # continue through it (Codex review of PR #10, round 17).\n        policy[\"restored_version\"] = restored_version\n    validate_policy(policy)\n    return policy\n\n\ndef relative_to_repo(path: Path | str) -> str:\n    resolved = Path(path).resolve()\n    try:\n        return resolved.relative_to(REPO_ROOT).as_posix()\n    except ValueError:\n        return resolved.as_posix()\n\n\ndef component_paths(role: str | None, allowed: dict[str, str] | None = None) -> set[str]:\n    \"\"\"Paths the meta-improver may write for one role ('policy' or\n    'history'), or for any role when role is None.\"\"\"\n    components = allowed or AI_OWNED_COMPONENTS\n    if role is None:\n        return set(components.values())\n    return {p for name, p in components.items() if name == role or name.endswith(f\"-{role}\")}\n\n\ndef assert_ai_may_write(\n    path: Path | str, *, allowed: dict[str, str] | None = None, role: str | None = None\n) -> None:\n    \"\"\"Attribution guard: the meta-improver only ever writes the files it owns,\n    and each file only in its own role: the policy destination must be the\n    policy component and the history destination the history component, so\n    swapped arguments cannot append a policy to the history or overwrite the\n    policy with a history line (Codex review of PR #10, rounds 20 and 35).\n    Raises PermissionError otherwise, so a bug that tries to 'fix' the archive\n    or the verifier fails loudly instead of silently widening autonomy.\"\"\"\n    allowed_paths = component_paths(role, allowed)\n    rel = relative_to_repo(path)\n    if rel not in allowed_paths:\n        what = f\"the {role} component\" if role else \"a file it owns\"\n        raise PermissionError(\n            f\"{rel} is fixed infrastructure or not {what}; \"\n            f\"the meta-improver may only write {sorted(allowed_paths)} here\"\n        )\n\n\nPROTECTED_OUTPUT_PREFIXES: tuple[str, ...] = (\".github/\", \"scripts/\", \"packages/\", \"terraform/\")\nPROTECTED_OUTPUT_FILES: tuple[str, ...] = (\n    \"docs/self-improvement-archive.jsonl\",\n    \"docs/improvement-policy.json\",\n    \"docs/improvement-policy-history.jsonl\",\n)\n# The committed field anchors: only a deliberate evidence refresh may write\n# them, never a report or decision output (Codex review of PR #10, round 15).\nCANONICAL_EVIDENCE_FILES: tuple[str, ...] = (\n    \"docs/rsi/trace-evidence.json\",\n    \"docs/rsi/trace-evidence-verifier.json\",\n)\n\n\ndef assert_safe_output(\n    path: Path | str, *, inputs: list[str | Path] = (), kind: str = \"report\"\n) -> None:\n    \"\"\"Side outputs may go anywhere EXCEPT the loop's own records, its code,\n    the files the invocation is reading, and (for anything but an evidence\n    refresh) the canonical evidence snapshots (Codex review of PR #10,\n    rounds 11 and 15).\"\"\"\n    rel = relative_to_repo(path)\n    if rel in PROTECTED_OUTPUT_FILES or any(rel.startswith(p) for p in PROTECTED_OUTPUT_PREFIXES):\n        raise PermissionError(f\"{rel} is a protected file; choose another output path\")\n    if kind != \"evidence\" and rel in CANONICAL_EVIDENCE_FILES:\n        raise PermissionError(\n            f\"{rel} is a canonical evidence snapshot; only --save-evidence may write it\"\n        )\n    # Identity is by path AND by inode: a hard link to the archive named\n    # report.json resolves to a different path but is the same file (Codex\n    # full-branch review, finding 6).\n    for protected in (\n        *PROTECTED_OUTPUT_FILES,\n        *(() if kind == \"evidence\" else CANONICAL_EVIDENCE_FILES),\n    ):\n        if same_file(REPO_ROOT / protected, path):\n            raise PermissionError(f\"{rel} is the same file as protected {protected}\")\n    for source in inputs:\n        if source and (Path(source).resolve() == Path(path).resolve() or same_file(source, path)):\n            raise PermissionError(f\"{rel} is an input of this run; choose another output path\")\n\n\ndef same_file(a: Path | str, b: Path | str) -> bool:\n    try:\n        return Path(a).samefile(b)\n    except OSError:\n        return False\n\n\ndef save_policy(\n    policy: dict, path: Path | str = POLICY_PATH, *, allowed: dict[str, str] | None = None\n) -> None:\n    assert_ai_may_write(path, allowed=allowed, role=\"policy\")\n    validate_policy(policy)\n    Path(path).write_text(json.dumps(policy, indent=2) + \"\\n\")\n\n\ndef load_history(path: Path | str = HISTORY_PATH) -> list[dict]:\n    if not Path(path).exists():\n        return []\n    entries = []\n    with open(path) as f:\n        for line in f:\n            line = line.strip()\n            if line:\n                entries.append(json.loads(line))\n    return entries\n\n\ndef append_history(\n    entry: dict, path: Path | str = HISTORY_PATH, *, allowed: dict[str, str] | None = None\n) -> None:\n    assert_ai_may_write(path, allowed=allowed, role=\"history\")\n    # No sort_keys: a snapshot's topic order is its classification\n    # precedence, and restoring an alphabetized snapshot would silently\n    # reclassify findings (Codex review of PR #10, finding 2).\n    with open(path, \"a\") as f:\n        f.write(json.dumps(entry) + \"\\n\")\n\n\n# The checked-in policy, resolved once so every tool stamps and decides with\n# the same version and hash in one process.\n_CURRENT = load_policy_or_builtin()\nPOLICY_VERSION: int = _CURRENT[\"version\"]\nPOLICY_HASH: str = policy_hash(_CURRENT)\nname: Revise Improvement Policy\n\n# The L5 step of docs/plans/recursive-meta-improvement.md, run automatically\n# but bounded exactly like archive-and-recommend.yml: it never pushes to the\n# default branch, never merges, never deploys, and requests no repository\n# repository secrets at all.\n#\n# After every change to the review archive on main (an archive-round PR\n# merging), this workflow:\n#   1. measures whether docs/improvement-policy.json's signal still predicts\n#      the field (scripts/measure-policy-validity.py) — coverage of archived\n#      findings, and agreement with Traces evidence when a key is present;\n#   2. lets scripts/revise-improvement-policy.py apply its fixed acceptance\n#      rule: propose a bounded policy revision, propose a rollback of a\n#      revision that made things worse, or do nothing;\n#   3. re-renders docs/rsi/dashboard.html from the resulting state;\n#   4. opens ONE pull request carrying the policy, its history entry, the\n#      measurement, and the dashboard. A human merges it, or closes it.\n#\n# The field anchor (Traces evidence) is refreshed on every run from the\n# shared traces.com namespace when a read-only TRACES_API_KEY is present\n# (hourly on the schedule above); otherwise the committed snapshot is used.\n# Without any snapshot the anchor is absent: coverage is still measured and\n# can still trigger a revision, validity is reported as null, and the\n# workflow says so rather than pretending it was checked.\n\non:\n  push:\n    branches: [main]\n    paths:\n      - docs/self-improvement-archive.jsonl\n      # A refreshed field snapshot is new evidence too (Codex review of PR #10, round 10).\n      - docs/rsi/trace-evidence.json\n  schedule:\n    # Hourly: refresh the field anchor from the shared Traces namespace,\n    # re-measure, and let the fixed acceptance rule decide. Off the hour so\n    # it does not queue behind every other project's :00 cron.\n    - cron: \"17 * * * *\"\n  workflow_dispatch:\n\npermissions:\n  contents: write\n  pull-requests: write\n\nconcurrency:\n  group: revise-improvement-policy\n  cancel-in-progress: false\n\njobs:\n  revise:\n    name: Measure, revise, and propose\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    steps:\n      - name: Checkout (default branch — trusted)\n        uses: actions/checkout@v4\n        with:\n          # Always the default branch, whatever ref a manual dispatch was\n          # started from, so a proposal never carries an unrelated feature\n          # branch's commits (Codex review of PR #10, round 5).\n          ref: ${{ github.event.repository.default_branch }}\n          fetch-depth: 0\n\n      - name: Set up Python\n        uses: actions/setup-python@v5\n        with:\n          python-version: \"3.12\"\n\n      - name: Record the commit actually checked out\n        # The default branch may have advanced past the triggering commit, and\n        # a manual dispatch may come from another ref; label everything with\n        # what this run measured (Codex review of PR #10, round 16).\n        id: source\n        run: echo \"sha=$(git rev-parse HEAD)\" >> \"$GITHUB_OUTPUT\"\n\n      - name: Check for a Traces API key\n        id: traces-key\n        env:\n          HAS_KEY: ${{ secrets.TRACES_API_KEY != '' }}\n        run: echo \"present=$HAS_KEY\" >> \"$GITHUB_OUTPUT\"\n\n      - name: Install Traces CLI\n        if: steps.traces-key.outputs.present == 'true'\n        run: |\n          curl -fsSL https://www.traces.com/install | bash\n          echo \"$HOME/.traces/bin\" >> \"$GITHUB_PATH\"\n\n      - name: Refresh the field anchor from the shared Traces namespace\n        # Working sessions live on developers' machines, but every session\n        # they share lands in the traces.com namespace, which a read-only key\n        # can list and sync. The miner keeps only tool results the tool\n        # itself reported as errors, so the anchor is failures, never\n        # narration. Without a key the committed snapshot is used and its\n        # age is reported instead of silently reused (Codex review of\n        # PR #10, round 3).\n        if: steps.traces-key.outputs.present == 'true'\n        env:\n          TRACES_API_KEY: ${{ secrets.TRACES_API_KEY }}\n          TRACES_NAMESPACE: ${{ vars.TRACES_NAMESPACE || '@gagan114' }}\n        run: |\n          set -euo pipefail\n          mkdir -p docs/rsi\n          if python3 scripts/mine-trace-failures.py \\\n              --namespace \"$TRACES_NAMESPACE\" --traces-key \"$TRACES_API_KEY\" \\\n              --save-evidence docs/rsi/trace-evidence.json \\\n              --out-json \"$RUNNER_TEMP/field-failures.json\" > \"$RUNNER_TEMP/mine.txt\"; then\n            head -n 30 \"$RUNNER_TEMP/mine.txt\"\n          else\n            echo \"::warning::Traces refresh failed; using the committed evidence snapshot instead.\"\n            sed -n '1,40p' \"$RUNNER_TEMP/mine.txt\" || true\n            git checkout -- docs/rsi/trace-evidence.json 2>/dev/null || true\n            rm -f \"$RUNNER_TEMP/field-failures.json\"\n          fi\n\n      - name: Measure policy validity\n        # This step reports the snapshot's age so a stale anchor is visible\n        # rather than silently reused (Codex review of PR #10, round 3).\n        run: |\n          set -euo pipefail\n          mkdir -p docs/rsi\n          if [ -f docs/rsi/trace-evidence.json ]; then\n            collected=$(python3 -c 'import json; print(json.load(open(\"docs/rsi/trace-evidence.json\")).get(\"collected_at\", \"unknown\"))')\n            echo \"::notice::Using the committed Traces evidence snapshot collected at ${collected}. Refresh it locally to update the field anchor.\"\n            python3 scripts/measure-policy-validity.py docs/self-improvement-archive.jsonl \\\n              --trace-evidence docs/rsi/trace-evidence.json --out-json docs/rsi/measurement.json > measurement.txt\n          else\n            echo \"::notice::No Traces evidence snapshot committed; validity will be null and only coverage can trigger a revision.\"\n            python3 scripts/measure-policy-validity.py docs/self-improvement-archive.jsonl \\\n              --out-json docs/rsi/measurement.json > measurement.txt\n          fi\n          sed -n '1,/^---/p' measurement.txt\n\n      - name: Apply the fixed acceptance rule\n        id: revise\n        run: |\n          set -euo pipefail\n          args=(docs/self-improvement-archive.jsonl --measurement docs/rsi/measurement.json --out-json decision.json)\n          # Fresh field blind spots (from this run's Traces refresh) feed\n          # topic mining the same way unclassified review findings do.\n          [ -f \"$RUNNER_TEMP/field-failures.json\" ] && args+=(--field-failures \"$RUNNER_TEMP/field-failures.json\")\n          python3 scripts/revise-improvement-policy.py \"${args[@]}\" > revise.txt\n          sed -n '1,/^---/p' revise.txt\n          action=$(python3 -c 'import json; print(json.load(open(\"decision.json\"))[\"action\"])')\n          echo \"action=$action\" >> \"$GITHUB_OUTPUT\"\n          if [ \"$action\" != \"none\" ]; then\n            # The measurement that justified the decision is evidence; keep it.\n            # The live measurement must describe the policy now in force, or the\n            # next run's hash check would refuse it (Codex review of PR #10, round 4).\n            cp docs/rsi/measurement.json docs/rsi/measurement-decision.json\n            if [ -f docs/rsi/trace-evidence.json ]; then\n              python3 scripts/measure-policy-validity.py docs/self-improvement-archive.jsonl \\\n                --trace-evidence docs/rsi/trace-evidence.json --out-json docs/rsi/measurement.json > remeasure.txt\n            else\n              python3 scripts/measure-policy-validity.py docs/self-improvement-archive.jsonl \\\n                --out-json docs/rsi/measurement.json > remeasure.txt\n            fi\n          fi\n\n      - name: Re-render the dashboard\n        env:\n          SOURCE_SHA: ${{ steps.source.outputs.sha }}\n        run: |\n          set -euo pipefail\n          args=(docs/self-improvement-archive.jsonl --head \"${SOURCE_SHA:0:8}\" --out docs/rsi/dashboard.html)\n          [ -f docs/rsi/trace-evidence.json ] && args+=(--trace-evidence docs/rsi/trace-evidence.json)\n          [ -f docs/rsi/trace-evidence-verifier.json ] && args+=(--verifier-evidence docs/rsi/trace-evidence-verifier.json)\n          python3 scripts/render-rsi-dashboard.py \"${args[@]}\"\n\n      - name: Propose the result as a pull request\n        env:\n          GH_TOKEN: ${{ github.token }}\n          ACTION: ${{ steps.revise.outputs.action }}\n          SOURCE_SHA: ${{ steps.source.outputs.sha }}\n        run: |\n          set -euo pipefail\n          if git diff --quiet -- docs/improvement-policy.json docs/improvement-policy-history.jsonl docs/rsi/; then\n            echo \"Nothing changed (action: $ACTION).\"\n            exit 0\n          fi\n          version=$(python3 -c 'import json; print(json.load(open(\"docs/improvement-policy.json\"))[\"version\"])')\n          default_branch=$(gh repo view --json defaultBranchRef --jq .defaultBranchRef.name)\n          git config user.name \"github-actions[bot]\"\n          git config user.email \"41898282+github-actions[bot]@users.noreply.github.com\"\n          if git diff --quiet -- docs/improvement-policy.json docs/improvement-policy-history.jsonl; then\n            # Only the evidence, measurement and dashboard moved (the hourly\n            # refresh with no decision). One standing evidence branch, updated\n            # in place, so the hourly run never opens a new PR per hour; a\n            # human merges it whenever, and merging it re-triggers this\n            # workflow through the trace-evidence.json path filter.\n            # A standing policy proposal that the current decision no longer\n            # supports is withdrawn, not left open to merge (Codex review of\n            # PR #10, round 37): new rounds can legitimately turn a\n            # `revise` into `none`.\n            stale_pr=$(gh pr list --head \"improvement-policy-proposal\" --base \"$default_branch\" --state open \\\n              --json number,isCrossRepository --jq '[.[] | select(.isCrossRepository == false)][0].number // empty')\n            if [ -n \"$stale_pr\" ]; then\n              gh pr close \"$stale_pr\" --delete-branch \\\n                --comment \"Withdrawn: re-evaluated at \\`${SOURCE_SHA}\\` the fixed acceptance rule proposes no policy change (action: **${ACTION}**), so this proposal no longer satisfies it. A new proposal will be opened if a later round supports one.\"\n              echo \"Closed the stale policy proposal PR #$stale_pr.\"\n            fi\n            branch=\"improvement-evidence-refresh\"\n            title=\"chore(rsi): refresh the Traces field anchor\"\n            body=\"Automatically refreshed by [\\`revise-improvement-policy.yml\\`](../blob/main/.github/workflows/revise-improvement-policy.yml) from the shared Traces namespace at \\`${SOURCE_SHA}\\`. The fixed acceptance rule proposed no policy change on this evidence (action: **${ACTION}**). This changes only the evidence snapshot, the measurement and the dashboard; the policy is untouched.\"\n            git checkout -b \"$branch\"\n            git add docs/rsi/\n            git commit -m \"$title\"\n            git push --force origin \"$branch\"\n            existing_pr=$(gh pr list --head \"$branch\" --base \"$default_branch\" --state open \\\n              --json number,isCrossRepository --jq '[.[] | select(.isCrossRepository == false)][0].number // empty')\n            if [ -n \"$existing_pr\" ]; then\n              echo \"Updated the open evidence PR #$existing_pr in place.\"\n              exit 0\n            fi\n            gh pr create --title \"$title\" --body \"$body\" --label \"self-improvement-archive\" \\\n              --base \"$default_branch\" --head \"$branch\"\n            exit 0\n          fi\n          # One standing proposal branch: a newer archive round supersedes an\n          # open proposal instead of opening a competing one with the same\n          # version and parent (Codex review of PR #10, round 5).\n          branch=\"improvement-policy-proposal\"\n          git checkout -b \"$branch\"\n          git add docs/improvement-policy.json docs/improvement-policy-history.jsonl docs/rsi/\n          git commit -m \"chore(rsi): ${ACTION} of the improvement policy (v${version})\"\n          git push --force origin \"$branch\"\n          # Same-repository PRs only: `--head` matches by branch name alone, so\n          # a fork PR using this branch name must not be mistaken for the\n          # bot's own proposal (Codex review of PR #10, round 19).\n          existing_pr=$(gh pr list --head \"$branch\" --base \"$default_branch\" --state open \\\n            --json number,isCrossRepository --jq '[.[] | select(.isCrossRepository == false)][0].number // empty')\n          if [ -n \"$existing_pr\" ]; then\n            gh pr comment \"$existing_pr\" --body \"Superseded by a newer archive round at \\`${SOURCE_SHA}\\`: this proposal was regenerated from the current archive (action: **${ACTION}**, policy v${version}).\"\n            echo \"Updated the open proposal PR #$existing_pr in place.\"\n            exit 0\n          fi\n          gh pr create \\\n            --title \"chore(rsi): ${ACTION} of the improvement policy (v${version})\" \\\n            --body \"Automatically proposed by [\\`revise-improvement-policy.yml\\`](../blob/main/.github/workflows/revise-improvement-policy.yml) after the archive changed at \\`${SOURCE_SHA}\\`. Action: **${ACTION}**. See \\`docs/improvement-policy-history.jsonl\\` for the evidence and \\`docs/rsi/dashboard.html\\` for the rendered state. This changes only the AI-owned policy files; a human merges it, same as every other change in this repo.\" \\\n            --label \"self-improvement-archive\" \\\n            --base \"$default_branch\" \\\n            --head \"$branch\"\n"}}

>ruff check scripts/

{"status":"fulfilled","value":{"chunk_id":"bd8162","wall_time_seconds":0.000005584,"exit_code":0,"original_token_count":87368,"output":"Warning: truncated output (original token count: 87368)\nTotal output lines: 7554\n\ndiff --git a/.github/workflows/archive-and-recommend.yml b/.github/workflows/archive-and-recommend.yml\nindex fc306370..ce5adaa1 100644\n--- a/.github/workflows/archive-and-recommend.yml\n+++ b/.github/workflows/archive-and-recommend.yml\n@@ -59,7 +59,12 @@ on:\n     types: [completed]\n   push:\n     branches: [main]\n-    paths: [\"docs/self-improvement-archive.jsonl\"]\n+    paths:\n+      - \"docs/self-improvement-archive.jsonl\"\n+      # A revised policy (new topic, restored weight) can cross the\n+      # mechanism-fix threshold on the existing archive; re-evaluate then\n+      # too (Codex review of PR #10, round 31).\n+      - \"docs/improvement-policy.json\"\n \n permissions:\n   contents: write\n@@ -122,14 +127,18 @@ jobs:\n               per_page: 100,\n             });\n \n-            // Both checks matter: the author check stops a PR commenter\n+            // All three checks matter: the author check stops a PR commenter\n             // from forging a review-shaped comment; the SHA marker stops\n             // an old (correctly bot-authored) review comment from a prior\n-            // commit being mistaken for this run's review.\n+            // commit being mistaken for this run's review; and the marker\n+            // must be the comment's LAST line — the footer the workflow\n+            // itself appends — so a marker quoted inside another commit's\n+            // review text cannot claim that review for this SHA (Codex\n+            // full-branch review, workflows finding 1).\n             const reviewComments = comments.filter(\n               (c) =>\n                 (c.body || \"\").startsWith(\"### Codex independent review\") &&\n-                (c.body || \"\").includes(marker) &&\n+                (c.body || \"\").trimEnd().endsWith(marker) &&\n                 c.user?.type === \"Bot\" &&\n                 c.user?.login === \"github-actions[bot]\"\n             );\ndiff --git a/.github/workflows/codex-review.yml b/.github/workflows/codex-review.yml\nindex 6ed2be84..0e63e4ed 100644\n--- a…86368 tokens truncated…uiltin_policy()\n+    v2 = policy_mod.new_version(\n+        v1, topics=dict(v1[\"topics\"]), threshold=v1[\"threshold\"], origin=\"revision\", rationale=\"x\"\n+    )\n+    archive = tmp_path / \"archive.jsonl\"\n+    archive.write_text(\"\\n\".join(json.dumps(e) for e in _archive()) + \"\\n\")\n+    history = tmp_path / \"history.jsonl\"\n+    history.write_text(json.dumps({\"version\": 2, \"policy\": v2}) + \"\\n\")\n+    edited = {**v2, \"origin\": \"init\", \"parent\": None}\n+    policy_path = tmp_path / \"policy.json\"\n+    policy_path.write_text(json.dumps(edited))\n+    m_path = tmp_path / \"m.json\"\n+    m_path.write_text(json.dumps(measure.measure(_archive(), edited, None)))\n+    code = revise.main(\n+        [\n+            \"r\",\n+            str(archive),\n+            \"--measurement\",\n+            str(m_path),\n+            \"--policy\",\n+            str(policy_path),\n+            \"--history\",\n+            str(history),\n+            \"--dry-run\",\n+            \"--now\",\n+            NOW,\n+        ]\n+    )\n+    assert code == 1\n+    assert \"lineage metadata\" in capsys.readouterr().err\n+    with pytest.raises(ValueError, match=\"history records no versions\"):\n+        policy_mod.assert_policy_matches_history(v2, [])\n+    policy_mod.assert_policy_matches_history(v2, [{\"version\": 2, \"policy\": v2}])\n+\n+\n+def test_evidence_window_filters_by_round_identity_not_number():\n+    # Two reviews share round 5; one predates the snapshot and one does not.\n+    entries = [\n+        {\n+            \"round\": 5,\n+            \"source_sha\": \"old\",\n+            \"findings\": [\"**[P1]** a.\"],\n+            \"occurred_at\": \"2026-09-14T15:00:00Z\",\n+        },\n+        {\n+            \"round\": 5,\n+            \"source_sha\": \"new\",\n+            \"findings\": [\"**[P1]** b.\"],\n+            \"occurred_at\": \"2026-09-14T17:00:00Z\",\n+        },\n+    ]\n+    measurement = {\"anchor\": {\"collected_at\": \"2026-09-14T16:00:00Z\"}}\n+    covered = revise.entries_covered_by_evidence(entries, measurement)\n+    assert [e[\"source_sha\"] for e in covered] == [\"old\"]\n"}}
{"status":"fulfilled","value":{"chunk_id":"20363e","wall_time_seconds":1.002056208,"session_id":95306,"original_token_count":3,"output":"........."}}
{"status":"fulfilled","value":{"chunk_id":"52f194","wall_time_seconds":0.000001625,"exit_code":0,"original_token_count":5,"output":"All checks passed!\n"}}
{"status":"fulfilled","value":{"chunk_id":"7506da","wall_time_seconds":0.000005333,"exit_code":0,"original_token_count":7413,"output":"\"\"\"The improvement mechanism's own policy, as versioned data instead of code.\n\nBefore this module, the rule that decides whether a recurring finding\ndeserves a target-level patch or a mechanism-level fix lived as constants in\nscripts/detect-recurring-pattern.py: a keyword taxonomy and a recurrence\nthreshold, written by hand once and never revisited. That is an L4 loop in\nthe paper's terms (docs/plans/recursive-meta-improvement.md): the system\nadapts its deployed state, but the mechanism governing what counts as an\nimprovement stays fixed human infrastructure.\n\nL5 requires that mechanism to be something the system can revise from\nevidence, with the same safeguards it applies to every other change. So the\npolicy becomes a JSON document with a version, a parent, and an origin, and\nevery revision is appended to a history file with the evidence that\njustified it. The pieces that must NOT be revisable by the meta-improver\n(the archive, the external anchor, the independent verifier, the acceptance\nthresholds, and the promotion path) are enumerated in FIXED_INFRASTRUCTURE,\nand `assert_ai_may_write` refuses any write outside AI_OWNED_COMPONENTS.\n\"\"\"\n\nfrom __future__ import annotations\n\nimport hashlib\nimport json\nfrom datetime import UTC, datetime\nfrom pathlib import Path\n\nREPO_ROOT = Path(__file__).resolve().parent.parent\nPOLICY_PATH = REPO_ROOT / \"docs\" / \"improvement-policy.json\"\nHISTORY_PATH = REPO_ROOT / \"docs\" / \"improvement-policy-history.jsonl\"\n\n# The v1 taxonomy and threshold, kept in code only as a fallback so every\n# existing tool still runs in a checkout that predates the policy file.\nBUILTIN_THRESHOLD = 3\nBUILTIN_TOPIC_KEYWORDS: dict[str, list[str]] = {\n    \"credential-redaction\": [\"redact\", \"credential\", \"secret\", \"token\", \"leak\", \"expos\"],\n    \"shell-semantics\": [\"errexit\", \"bash -e\", \"exit code\", \"-e\", \"pipefail\", \"shell\"],\n    \"env-var-precedence\": [\"precedence\", \"env var\", \"environment variable\", \"unconditionally\"],\n    \"fork-pr-permissions\": [\"fork\", \"github_token\", \"persist-credentials\"],\n    \"auth-lifecycle\": [\"refresh token\", \"rotat\", \"expir\", \"auth.json\", \"stale\"],\n}\n\n# Autonomy attribution (paper failure mode 2): the meta-improver may rewrite\n# exactly these files, and nothing else. Paths are repo-relative.\nAI_OWNED_COMPONENTS: dict[str, str] = {\n    \"improvement-policy\": \"docs/improvement-policy.json\",\n    \"improvement-policy-history\": \"docs/improvement-policy-history.jsonl\",\n}\n\n# Everything the loop depends on that stays human-owned infrastructure. The\n# dashboard renders this list verbatim so the boundary is visible, not implied.\nFIXED_INFRASTRUCTURE: dict[str, str] = {\n    \"archive\": \"docs/self-improvement-archive.jsonl — append-only, SHA-idempotent (archive-round.py)\",\n    \"verifier\": \".github/workflows/codex-review.yml — independent second-model review of every PR\",\n    \"anchor\": \"Traces evidence from working sessions — never consulted when a round is decided\",\n    \"meta-acceptance-rule\": \"MIN_COVERAGE / MIN_VALIDITY / MIN_ROUNDS_TO_JUDGE in revise-improvement-policy.py\",\n    \"promotion\": \"pull requests only; a human merges every policy revision and every rollback\",\n}\n\n\ndef utc_now_iso() -> str:\n    return datetime.now(UTC).replace(microsecond=0).isoformat().replace(\"+00:00\", \"Z\")\n\n\ndef builtin_policy() -> dict:\n    return {\n        \"version\": 1,\n        \"parent\": None,\n        \"origin\": \"init\",\n        \"created_at\": \"2026-09-14T18:00:00Z\",\n        \"threshold\": BUILTIN_THRESHOLD,\n        \"topics\": {\n            topic: {\"keywords\": list(keywords), \"weight\": 1.0}\n            for topic, keywords in BUILTIN_TOPIC_KEYWORDS.items()\n        },\n        \"rationale\": \"Built-in fallback identical to policy version 1.\",\n    }\n\n\ndef load_policy(path: Path | str = POLICY_PATH) -> dict:\n    with open(path) as f:\n        policy = json.load(f)\n    validate_policy(policy)\n    return policy\n\n\ndef load_policy_or_builtin(path: Path | str = POLICY_PATH) -> dict:\n    if Path(path).exists():\n        return load_policy(path)\n    return builtin_policy()\n\n\nVALID_ORIGINS = frozenset({\"init\", \"revision\", \"rollback\"})\n\n\ndef round_key(entry: dict) -> str:\n    \"\"\"Identity of an archived round, shared by every tool that counts\n    rounds (the detector that opens issues and the measurer the policy is\n    judged on must count the same signal — Codex review of PR #10, round\n    37). Automated rounds are identified by the commit they reviewed: two\n    archive proposals opened before either merged both computed the same\n    next round number, and merging by number alone collapsed two reviews\n    into one round (full-branch review, workflows finding 2). Legacy rounds\n    without a source_sha keep their number (placeholder + result pairs).\"\"\"\n    sha = entry.get(\"source_sha\")\n    if isinstance(sha, str) and sha:\n        return f\"sha:{sha}\"\n    return f\"round:{entry.get('round')}\"\n\n\ndef assert_policy_matches_history(policy: dict, history: list[dict]) -> None:\n    \"\"\"The policy's lineage metadata (version, parent, origin,\n    restored_version) is not part of its content hash, yet the wait gate and\n    rollback logic depend on it. So a policy in force must be exactly the\n    snapshot its history recorded for that version, metadata included; a\n    root policy must not claim to be a revision or rollback. Otherwise an\n    edit to `origin` alone would switch the evaluation gates off (Codex\n    full-branch review, finding 1).\"\"\"\n    if not history:\n        if policy.get(\"origin\") != \"init\" or policy.get(\"parent\") is not None:\n            raise ValueError(\n                \"policy claims a revision/rollback lineage but the history records no versions\"\n            )\n        return\n    latest = history[-1]\n    snapshot = latest.get(\"policy\") or {}\n    if latest.get(\"version\") != policy.get(\"version\"):\n        raise ValueError(\n            f\"policy is v{policy.get('version')} but the history's latest entry is \"\n            f\"v{latest.get('version')}; the policy and its history must be written together\"\n        )\n    for key in (\"version\", \"parent\", \"origin\", \"restored_version\"):\n        if snapshot.get(key) != policy.get(key):\n            raise ValueError(\n                f\"policy.{key}={policy.get(key)!r} differs from the recorded v{policy.get('version')} \"\n                f\"snapshot ({snapshot.get(key)!r}); lineage metadata may not be edited in place\"\n            )\n    if policy_hash(snapshot) != policy_hash(policy):\n        raise ValueError(\n            f\"policy content hashes to {policy_hash(policy)} but the recorded v{policy.get('version')} \"\n            f\"snapshot hashes to {policy_hash(snapshot)}\"\n        )\n\n\ndef validate_policy(policy: dict) -> None:\n    if not isinstance(policy.get(\"version\"), int) or policy[\"version\"] < 1:\n        raise ValueError(\"policy.version must be a positive integer\")\n    if not isinstance(policy.get(\"threshold\"), int) or policy[\"threshold\"] < 1:\n        raise ValueError(\"policy.threshold must be a positive integer\")\n    if policy.get(\"origin\") not in VALID_ORIGINS:\n        raise ValueError(f\"policy.origin must be one of {sorted(VALID_ORIGINS)}\")\n    topics = policy.get(\"topics\")\n    if not isinstance(topics, dict) or not topics:\n        raise ValueError(\"policy.topics must be a non-empty object\")\n    for name, spec in topics.items():\n        if not isinstance(name, str) or not name or \"@\" in name or name != name.strip():\n            # `name@tag` keys are reserved for older definitions of a name in\n            # the evidence; a topic named that way would be skipped by the\n            # candidate anchor and escape the validity comparison (Codex\n            # full-branch review, finding 2).\n            raise ValueError(\n                f\"topic name {name!r} is invalid (non-empty, no '@', no surrounding whitespace)\"\n            )\n        keywords = spec.get(\"keywords\")\n        if (\n            not isinstance(keywords, list)\n            or not keywords\n            or not all(isinstance(k, str) and k for k in keywords)\n        ):\n            # An empty list would classify nothing while matching every trace\n            # (Codex review of PR #10, round 16).\n            raise ValueError(f\"topic {name!r} needs a non-empty list of keyword strings\")\n        weight = spec.get(\"weight\", 1.0)\n        if not isinstance(weight, int | float) or weight <= 0:\n            raise ValueError(f\"topic {name!r} weight must be a positive number\")\n\n\ndef policy_hash(policy: dict) -> str:\n    \"\"\"Content hash of the decision-relevant fields. Two policies with the\n    same taxonomy, weights, and threshold decide identically, whatever their\n    version metadata says — this is what the dashboard pins per epoch to\n    show the evaluator was frozen while a round was decided.\n\n    Topic ORDER is part of the hash: classification takes the first topic\n    whose keyword matches, so reordering overlapping topics changes\n    decisions and must not pass the stale-measurement guard (Codex review\n    of PR #10, finding 3).\"\"\"\n    canonical = json.dumps(\n        {\n            \"threshold\": policy[\"threshold\"],\n            \"topics\": [\n                [name, spec[\"keywords\"], float(spec.get(\"weight\", 1.0))]\n                for name, spec in policy[\"topics\"].items()\n            ],\n        },\n        separators=(\",\", \":\"),\n    )\n    return hashlib.sha256(canonical.encode()).hexdigest()[:12]\n\n\ndef topic_keywords(policy: dict) -> dict[str, list[str]]:\n    return {name: list(spec[\"keywords\"]) for name, spec in policy[\"topics\"].items()}\n\n\ndef topic_weights(policy: dict) -> dict[str, float]:\n    return {name: float(spec.get(\"weight\", 1.0)) for name, spec in policy[\"topics\"].items()}\n\n\ndef classify_finding(text: str, keywords: dict[str, list[str]]) -> str | None:\n    \"\"\"First topic (in policy order) with any keyword present. Same rule the\n    detector has always applied; it lives here so every tool classifies\n    identically under the same policy version.\"\"\"\n    lowered = text.lower()\n    for topic, words in keywords.items():\n        if any(word in lowered for word in words):\n            return topic\n    return None\n\n\ndef new_version(\n    parent: dict,\n    *,\n    topics: dict,\n    threshold: int,\n    origin: str,\n    rationale: str,\n    created_at: str | None = None,\n    restored_version: int | None = None,\n) -> dict:\n    if origin not in {\"revision\", \"rollback\"}:\n        raise ValueError(\"origin must be 'revision' or 'rollback'\")\n    policy = {\n        \"version\": parent[\"version\"] + 1,\n        \"parent\": parent[\"version\"],\n        \"origin\": origin,\n        \"created_at\": created_at or utc_now_iso(),\n        \"threshold\": threshold,\n        \"topics\": topics,\n        \"rationale\": rationale,\n    }\n    if origin == \"rollback\":\n        # Which version's configuration this restores, so ancestry checks can\n        # continue through it (Codex review of PR #10, round 17).\n        policy[\"restored_version\"] = restored_version\n    validate_policy(policy)\n    return policy\n\n\ndef relative_to_repo(path: Path | str) -> str:\n    resolved = Path(path).resolve()\n    try:\n        return resolved.relative_to(REPO_ROOT).as_posix()\n    except ValueError:\n        return resolved.as_posix()\n\n\ndef component_paths(role: str | None, allowed: dict[str, str] | None = None) -> set[str]:\n    \"\"\"Paths the meta-improver may write for one role ('policy' or\n    'history'), or for any role when role is None.\"\"\"\n    components = allowed or AI_OWNED_COMPONENTS\n    if role is None:\n        return set(components.values())\n    return {p for name, p in components.items() if name == role or name.endswith(f\"-{role}\")}\n\n\ndef assert_ai_may_write(\n    path: Path | str, *, allowed: dict[str, str] | None = None, role: str | None = None\n) -> None:\n    \"\"\"Attribution guard: the meta-improver only ever writes the files it owns,\n    and each file only in its own role: the policy destination must be the\n    policy component and the history destination the history component, so\n    swapped arguments cannot append a policy to the history or overwrite the\n    policy with a history line (Codex review of PR #10, rounds 20 and 35).\n    Raises PermissionError otherwise, so a bug that tries to 'fix' the archive\n    or the verifier fails loudly instead of silently widening autonomy.\"\"\"\n    allowed_paths = component_paths(role, allowed)\n    rel = relative_to_repo(path)\n    if rel not in allowed_paths:\n        what = f\"the {role} component\" if role else \"a file it owns\"\n        raise PermissionError(\n            f\"{rel} is fixed infrastructure or not {what}; \"\n            f\"the meta-improver may only write {sorted(allowed_paths)} here\"\n        )\n\n\nPROTECTED_OUTPUT_PREFIXES: tuple[str, ...] = (\".github/\", \"scripts/\", \"packages/\", \"terraform/\")\nPROTECTED_OUTPUT_FILES: tuple[str, ...] = (\n    \"docs/self-improvement-archive.jsonl\",\n    \"docs/improvement-policy.json\",\n    \"docs/improvement-policy-history.jsonl\",\n)\n# The committed field anchors: only a deliberate evidence refresh may write\n# them, never a report or decision output (Codex review of PR #10, round 15).\nCANONICAL_EVIDENCE_FILES: tuple[str, ...] = (\n    \"docs/rsi/trace-evidence.json\",\n    \"docs/rsi/trace-evidence-verifier.json\",\n)\n\n\ndef assert_safe_output(\n    path: Path | str, *, inputs: list[str | Path] = (), kind: str = \"report\"\n) -> None:\n    \"\"\"Side outputs may go anywhere EXCEPT the loop's own records, its code,\n    the files the invocation is reading, and (for anything but an evidence\n    refresh) the canonical evidence snapshots (Codex review of PR #10,\n    rounds 11 and 15).\"\"\"\n    rel = relative_to_repo(path)\n    if rel in PROTECTED_OUTPUT_FILES or any(rel.startswith(p) for p in PROTECTED_OUTPUT_PREFIXES):\n        raise PermissionError(f\"{rel} is a protected file; choose another output path\")\n    if kind != \"evidence\" and rel in CANONICAL_EVIDENCE_FILES:\n        raise PermissionError(\n            f\"{rel} is a canonical evidence snapshot; only --save-evidence may write it\"\n        )\n    # Identity is by path AND by inode: a hard link to the archive named\n    # report.json resolves to a different path but is the same file (Codex\n    # full-branch review, finding 6).\n    for protected in (\n        *PROTECTED_OUTPUT_FILES,\n        *(() if kind == \"evidence\" else CANONICAL_EVIDENCE_FILES),\n    ):\n        if same_file(REPO_ROOT / protected, path):\n            raise PermissionError(f\"{rel} is the same file as protected {protected}\")\n    for source in inputs:\n        if source and (Path(source).resolve() == Path(path).resolve() or same_file(source, path)):\n            raise PermissionError(f\"{rel} is an input of this run; choose another output path\")\n\n\ndef same_file(a: Path | str, b: Path | str) -> bool:\n    try:\n        return Path(a).samefile(b)\n    except OSError:\n        return False\n\n\ndef save_policy(\n    policy: dict, path: Path | str = POLICY_PATH, *, allowed: dict[str, str] | None = None\n) -> None:\n    assert_ai_may_write(path, allowed=allowed, role=\"policy\")\n    validate_policy(policy)\n    Path(path).write_text(json.dumps(policy, indent=2) + \"\\n\")\n\n\ndef load_history(path: Path | str = HISTORY_PATH) -> list[dict]:\n    if not Path(path).exists():\n        return []\n    entries = []\n    with open(path) as f:\n        for line in f:\n            line = line.strip()\n            if line:\n                entries.append(json.loads(line))\n    return entries\n\n\ndef append_history(\n    entry: dict, path: Path | str = HISTORY_PATH, *, allowed: dict[str, str] | None = None\n) -> None:\n    assert_ai_may_write(path, allowed=allowed, role=\"history\")\n    # No sort_keys: a snapshot's topic order is its classification\n    # precedence, and restoring an alphabetized snapshot would silently\n    # reclassify findings (Codex review of PR #10, finding 2).\n    with open(path, \"a\") as f:\n        f.write(json.dumps(entry) + \"\\n\")\n\n\n# The checked-in policy, resolved once so every tool stamps and decides with\n# the same version and hash in one process.\n_CURRENT = load_policy_or_builtin()\nPOLICY_VERSION: int = _CURRENT[\"version\"]\nPOLICY_HASH: str = policy_hash(_CURRENT)\nname: Revise Improvement Policy\n\n# The L5 step of docs/plans/recursive-meta-improvement.md, run automatically\n# but bounded exactly like archive-and-recommend.yml: it never pushes to the\n# default branch, never merges, never deploys, and requests no repository\n# repository secrets at all.\n#\n# After every change to the review archive on main (an archive-round PR\n# merging), this workflow:\n#   1. measures whether docs/improvement-policy.json's signal still predicts\n#      the field (scripts/measure-policy-validity.py) — coverage of archived\n#      findings, and agreement with Traces evidence when a key is present;\n#   2. lets scripts/revise-improvement-policy.py apply its fixed acceptance\n#      rule: propose a bounded policy revision, propose a rollback of a\n#      revision that made things worse, or do nothing;\n#   3. re-renders docs/rsi/dashboard.html from the resulting state;\n#   4. opens ONE pull request carrying the policy, its history entry, the\n#      measurement, and the dashboard. A human merges it, or closes it.\n#\n# The field anchor (Traces evidence) is refreshed on every run from the\n# shared traces.com namespace when a read-only TRACES_API_KEY is present\n# (hourly on the schedule above); otherwise the committed snapshot is used.\n# Without any snapshot the anchor is absent: coverage is still measured and\n# can still trigger a revision, validity is reported as null, and the\n# workflow says so rather than pretending it was checked.\n\non:\n  push:\n    branches: [main]\n    paths:\n      - docs/self-improvement-archive.jsonl\n      # A refreshed field snapshot is new evidence too (Codex review of PR #10, round 10).\n      - docs/rsi/trace-evidence.json\n  schedule:\n    # Hourly: refresh the field anchor from the shared Traces namespace,\n    # re-measure, and let the fixed acceptance rule decide. Off the hour so\n    # it does not queue behind every other project's :00 cron.\n    - cron: \"17 * * * *\"\n  workflow_dispatch:\n\npermissions:\n  contents: write\n  pull-requests: write\n\nconcurrency:\n  group: revise-improvement-policy\n  cancel-in-progress: false\n\njobs:\n  revise:\n    name: Measure, revise, and propose\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    steps:\n      - name: Checkout (default branch — trusted)\n        uses: actions/checkout@v4\n        with:\n          # Always the default branch, whatever ref a manual dispatch was\n          # started from, so a proposal never carries an unrelated feature\n          # branch's commits (Codex review of PR #10, round 5).\n          ref: ${{ github.event.repository.default_branch }}\n          fetch-depth: 0\n\n      - name: Set up Python\n        uses: actions/setup-python@v5\n        with:\n          python-version: \"3.12\"\n\n      - name: Record the commit actually checked out\n        # The default branch may have advanced past the triggering commit, and\n        # a manual dispatch may come from another ref; label everything with\n        # what this run measured (Codex review of PR #10, round 16).\n        id: source\n        run: echo \"sha=$(git rev-parse HEAD)\" >> \"$GITHUB_OUTPUT\"\n\n      - name: Check for a Traces API key\n        id: traces-key\n        env:\n          HAS_KEY: ${{ secrets.TRACES_API_KEY != '' }}\n        run: echo \"present=$HAS_KEY\" >> \"$GITHUB_OUTPUT\"\n\n      - name: Install Traces CLI\n        if: steps.traces-key.outputs.present == 'true'\n        run: |\n          curl -fsSL https://www.traces.com/install | bash\n          echo \"$HOME/.traces/bin\" >> \"$GITHUB_PATH\"\n\n      - name: Refresh the field anchor from the shared Traces namespace\n        # Working sessions live on developers' machines, but every session\n        # they share lands in the traces.com namespace, which a read-only key\n        # can list and sync. The miner keeps only tool results the tool\n        # itself reported as errors, so the anchor is failures, never\n        # narration. Without a key the committed snapshot is used and its\n        # age is reported instead of silently reused (Codex review of\n        # PR #10, round 3).\n        if: steps.traces-key.outputs.present == 'true'\n        env:\n          TRACES_API_KEY: ${{ secrets.TRACES_API_KEY }}\n          TRACES_NAMESPACE: ${{ vars.TRACES_NAMESPACE || '@gagan114' }}\n        run: |\n          set -euo pipefail\n          mkdir -p docs/rsi\n          if python3 scripts/mine-trace-failures.py \\\n              --namespace \"$TRACES_NAMESPACE\" --traces-key \"$TRACES_API_KEY\" \\\n              --save-evidence docs/rsi/trace-evidence.json \\\n              --out-json \"$RUNNER_TEMP/field-failures.json\" > \"$RUNNER_TEMP/mine.txt\"; then\n            head -n 30 \"$RUNNER_TEMP/mine.txt\"\n          else\n            echo \"::warning::Traces refresh failed; using the committed evidence snapshot instead.\"\n            sed -n '1,40p' \"$RUNNER_TEMP/mine.txt\" || true\n            git checkout -- docs/rsi/trace-evidence.json 2>/dev/null || true\n            rm -f \"$RUNNER_TEMP/field-failures.json\"\n          fi\n\n      - name: Measure policy validity\n        # This step reports the snapshot's age so a stale anchor is visible\n        # rather than silently reused (Codex review of PR #10, round 3).\n        run: |\n          set -euo pipefail\n          mkdir -p docs/rsi\n          if [ -f docs/rsi/trace-evidence.json ]; then\n            collected=$(python3 -c 'import json; print(json.load(open(\"docs/rsi/trace-evidence.json\")).get(\"collected_at\", \"unknown\"))')\n            echo \"::notice::Using the committed Traces evidence snapshot collected at ${collected}. Refresh it locally to update the field anchor.\"\n            python3 scripts/measure-policy-validity.py docs/self-improvement-archive.jsonl \\\n              --trace-evidence docs/rsi/trace-evidence.json --out-json docs/rsi/measurement.json > measurement.txt\n          else\n            echo \"::notice::No Traces evidence snapshot committed; validity will be null and only coverage can trigger a revision.\"\n            python3 scripts/measure-policy-validity.py docs/self-improvement-archive.jsonl \\\n              --out-json docs/rsi/measurement.json > measurement.txt\n          fi\n          sed -n '1,/^---/p' measurement.txt\n\n      - name: Apply the fixed acceptance rule\n        id: revise\n        run: |\n          set -euo pipefail\n          args=(docs/self-improvement-archive.jsonl --measurement docs/rsi/measurement.json --out-json decision.json)\n          # Fresh field blind spots (from this run's Traces refresh) feed\n          # topic mining the same way unclassified review findings do.\n          [ -f \"$RUNNER_TEMP/field-failures.json\" ] && args+=(--field-failures \"$RUNNER_TEMP/field-failures.json\")\n          python3 scripts/revise-improvement-policy.py \"${args[@]}\" > revise.txt\n          sed -n '1,/^---/p' revise.txt\n          action=$(python3 -c 'import json; print(json.load(open(\"decision.json\"))[\"action\"])')\n          echo \"action=$action\" >> \"$GITHUB_OUTPUT\"\n          if [ \"$action\" != \"none\" ]; then\n            # The measurement that justified the decision is evidence; keep it.\n            # The live measurement must describe the policy now in force, or the\n            # next run's hash check would refuse it (Codex review of PR #10, round 4).\n            cp docs/rsi/measurement.json docs/rsi/measurement-decision.json\n            if [ -f docs/rsi/trace-evidence.json ]; then\n              python3 scripts/measure-policy-validity.py docs/self-improvement-archive.jsonl \\\n                --trace-evidence docs/rsi/trace-evidence.json --out-json docs/rsi/measurement.json > remeasure.txt\n            else\n              python3 scripts/measure-policy-validity.py docs/self-improvement-archive.jsonl \\\n                --out-json docs/rsi/measurement.json > remeasure.txt\n            fi\n          fi\n\n      - name: Re-render the dashboard\n        env:\n          SOURCE_SHA: ${{ steps.source.outputs.sha }}\n        run: |\n          set -euo pipefail\n          args=(docs/self-improvement-archive.jsonl --head \"${SOURCE_SHA:0:8}\" --out docs/rsi/dashboard.html)\n          [ -f docs/rsi/trace-evidence.json ] && args+=(--trace-evidence docs/rsi/trace-evidence.json)\n          [ -f docs/rsi/trace-evidence-verifier.json ] && args+=(--verifier-evidence docs/rsi/trace-evidence-verifier.json)\n          python3 scripts/render-rsi-dashboard.py \"${args[@]}\"\n\n      - name: Propose the result as a pull request\n        env:\n          GH_TOKEN: ${{ github.token }}\n          ACTION: ${{ steps.revise.outputs.action }}\n          SOURCE_SHA: ${{ steps.source.outputs.sha }}\n        run: |\n          set -euo pipefail\n          if git diff --quiet -- docs/improvement-policy.json docs/improvement-policy-history.jsonl docs/rsi/; then\n            echo \"Nothing changed (action: $ACTION).\"\n            exit 0\n          fi\n          version=$(python3 -c 'import json; print(json.load(open(\"docs/improvement-policy.json\"))[\"version\"])')\n          default_branch=$(gh repo view --json defaultBranchRef --jq .defaultBranchRef.name)\n          git config user.name \"github-actions[bot]\"\n          git config user.email \"41898282+github-actions[bot]@users.noreply.github.com\"\n          if git diff --quiet -- docs/improvement-policy.json docs/improvement-policy-history.jsonl; then\n            # Only the evidence, measurement and dashboard moved (the hourly\n            # refresh with no decision). One standing evidence branch, updated\n            # in place, so the hourly run never opens a new PR per hour; a\n            # human merges it whenever, and merging it re-triggers this\n            # workflow through the trace-evidence.json path filter.\n            # A standing policy proposal that the current decision no longer\n            # supports is withdrawn, not left open to merge (Codex review of\n            # PR #10, round 37): new rounds can legitimately turn a\n            # `revise` into `none`.\n            stale_pr=$(gh pr list --head \"improvement-policy-proposal\" --base \"$default_branch\" --state open \\\n              --json number,isCrossRepository --jq '[.[] | select(.isCrossRepository == false)][0].number // empty')\n            if [ -n \"$stale_pr\" ]; then\n              gh pr close \"$stale_pr\" --delete-branch \\\n                --comment \"Withdrawn: re-evaluated at \\`${SOURCE_SHA}\\` the fixed acceptance rule proposes no policy change (action: **${ACTION}**), so this proposal no longer satisfies it. A new proposal will be opened if a later round supports one.\"\n              echo \"Closed the stale policy proposal PR #$stale_pr.\"\n            fi\n            branch=\"improvement-evidence-refresh\"\n            title=\"chore(rsi): refresh the Traces field anchor\"\n            body=\"Automatically refreshed by [\\`revise-improvement-policy.yml\\`](../blob/main/.github/workflows/revise-improvement-policy.yml) from the shared Traces namespace at \\`${SOURCE_SHA}\\`. The fixed acceptance rule proposed no policy change on this evidence (action: **${ACTION}**). This changes only the evidence snapshot, the measurement and the dashboard; the policy is untouched.\"\n            git checkout -b \"$branch\"\n            git add docs/rsi/\n            git commit -m \"$title\"\n            git push --force origin \"$branch\"\n            existing_pr=$(gh pr list --head \"$branch\" --base \"$default_branch\" --state open \\\n              --json number,isCrossRepository --jq '[.[] | select(.isCrossRepository == false)][0].number // empty')\n            if [ -n \"$existing_pr\" ]; then\n              echo \"Updated the open evidence PR #$existing_pr in place.\"\n              exit 0\n            fi\n            gh pr create --title \"$title\" --body \"$body\" --label \"self-improvement-archive\" \\\n              --base \"$default_branch\" --head \"$branch\"\n            exit 0\n          fi\n          # One standing proposal branch: a newer archive round supersedes an\n          # open proposal instead of opening a competing one with the same\n          # version and parent (Codex review of PR #10, round 5).\n          branch=\"improvement-policy-proposal\"\n          git checkout -b \"$branch\"\n          git add docs/improvement-policy.json docs/improvement-policy-history.jsonl docs/rsi/\n          git commit -m \"chore(rsi): ${ACTION} of the improvement policy (v${version})\"\n          git push --force origin \"$branch\"\n          # Same-repository PRs only: `--head` matches by branch name alone, so\n          # a fork PR using this branch name must not be mistaken for the\n          # bot's own proposal (Codex review of PR #10, round 19).\n          existing_pr=$(gh pr list --head \"$branch\" --base \"$default_branch\" --state open \\\n            --json number,isCrossRepository --jq '[.[] | select(.isCrossRepository == false)][0].number // empty')\n          if [ -n \"$existing_pr\" ]; then\n            gh pr comment \"$existing_pr\" --body \"Superseded by a newer archive round at \\`${SOURCE_SHA}\\`: this proposal was regenerated from the current archive (action: **${ACTION}**, policy v${version}).\"\n            echo \"Updated the open proposal PR #$existing_pr in place.\"\n            exit 0\n          fi\n          gh pr create \\\n            --title \"chore(rsi): ${ACTION} of the improvement policy (v${version})\" \\\n            --body \"Automatically proposed by [\\`revise-improvement-policy.yml\\`](../blob/main/.github/workflows/revise-improvement-policy.yml) after the archive changed at \\`${SOURCE_SHA}\\`. Action: **${ACTION}**. See \\`docs/improvement-policy-history.jsonl\\` for the evidence and \\`docs/rsi/dashboard.html\\` for the rendered state. This changes only the AI-owned policy files; a human merges it, same as every other change in this repo.\" \\\n            --label \"self-improvement-archive\" \\\n            --base \"$default_branch\" \\\n            --head \"$branch\"\n"}}

>cat scripts/improvement_policy.py; cat .github/workflows/revise-improvement-policy.yml

{"status":"fulfilled","value":{"chunk_id":"bd8162","wall_time_seconds":0.000005584,"exit_code":0,"original_token_count":87368,"output":"Warning: truncated output (original token count: 87368)\nTotal output lines: 7554\n\ndiff --git a/.github/workflows/archive-and-recommend.yml b/.github/workflows/archive-and-recommend.yml\nindex fc306370..ce5adaa1 100644\n--- a/.github/workflows/archive-and-recommend.yml\n+++ b/.github/workflows/archive-and-recommend.yml\n@@ -59,7 +59,12 @@ on:\n     types: [completed]\n   push:\n     branches: [main]\n-    paths: [\"docs/self-improvement-archive.jsonl\"]\n+    paths:\n+      - \"docs/self-improvement-archive.jsonl\"\n+      # A revised policy (new topic, restored weight) can cross the\n+      # mechanism-fix threshold on the existing archive; re-evaluate then\n+      # too (Codex review of PR #10, round 31).\n+      - \"docs/improvement-policy.json\"\n \n permissions:\n   contents: write\n@@ -122,14 +127,18 @@ jobs:\n               per_page: 100,\n             });\n \n-            // Both checks matter: the author check stops a PR commenter\n+            // All three checks matter: the author check stops a PR commenter\n             // from forging a review-shaped comment; the SHA marker stops\n             // an old (correctly bot-authored) review comment from a prior\n-            // commit being mistaken for this run's review.\n+            // commit being mistaken for this run's review; and the marker\n+            // must be the comment's LAST line — the footer the workflow\n+            // itself appends — so a marker quoted inside another commit's\n+            // review text cannot claim that review for this SHA (Codex\n+            // full-branch review, workflows finding 1).\n             const reviewComments = comments.filter(\n               (c) =>\n                 (c.body || \"\").startsWith(\"### Codex independent review\") &&\n-                (c.body || \"\").includes(marker) &&\n+                (c.body || \"\").trimEnd().endsWith(marker) &&\n                 c.user?.type === \"Bot\" &&\n                 c.user?.login === \"github-actions[bot]\"\n             );\ndiff --git a/.github/workflows/codex-review.yml b/.github/workflows/codex-review.yml\nindex 6ed2be84..0e63e4ed 100644\n--- a…86368 tokens truncated…uiltin_policy()\n+    v2 = policy_mod.new_version(\n+        v1, topics=dict(v1[\"topics\"]), threshold=v1[\"threshold\"], origin=\"revision\", rationale=\"x\"\n+    )\n+    archive = tmp_path / \"archive.jsonl\"\n+    archive.write_text(\"\\n\".join(json.dumps(e) for e in _archive()) + \"\\n\")\n+    history = tmp_path / \"history.jsonl\"\n+    history.write_text(json.dumps({\"version\": 2, \"policy\": v2}) + \"\\n\")\n+    edited = {**v2, \"origin\": \"init\", \"parent\": None}\n+    policy_path = tmp_path / \"policy.json\"\n+    policy_path.write_text(json.dumps(edited))\n+    m_path = tmp_path / \"m.json\"\n+    m_path.write_text(json.dumps(measure.measure(_archive(), edited, None)))\n+    code = revise.main(\n+        [\n+            \"r\",\n+            str(archive),\n+            \"--measurement\",\n+            str(m_path),\n+            \"--policy\",\n+            str(policy_path),\n+            \"--history\",\n+            str(history),\n+            \"--dry-run\",\n+            \"--now\",\n+            NOW,\n+        ]\n+    )\n+    assert code == 1\n+    assert \"lineage metadata\" in capsys.readouterr().err\n+    with pytest.raises(ValueError, match=\"history records no versions\"):\n+        policy_mod.assert_policy_matches_history(v2, [])\n+    policy_mod.assert_policy_matches_history(v2, [{\"version\": 2, \"policy\": v2}])\n+\n+\n+def test_evidence_window_filters_by_round_identity_not_number():\n+    # Two reviews share round 5; one predates the snapshot and one does not.\n+    entries = [\n+        {\n+            \"round\": 5,\n+            \"source_sha\": \"old\",\n+            \"findings\": [\"**[P1]** a.\"],\n+            \"occurred_at\": \"2026-09-14T15:00:00Z\",\n+        },\n+        {\n+            \"round\": 5,\n+            \"source_sha\": \"new\",\n+            \"findings\": [\"**[P1]** b.\"],\n+            \"occurred_at\": \"2026-09-14T17:00:00Z\",\n+        },\n+    ]\n+    measurement = {\"anchor\": {\"collected_at\": \"2026-09-14T16:00:00Z\"}}\n+    covered = revise.entries_covered_by_evidence(entries, measurement)\n+    assert [e[\"source_sha\"] for e in covered] == [\"old\"]\n"}}
{"status":"fulfilled","value":{"chunk_id":"20363e","wall_time_seconds":1.002056208,"session_id":95306,"original_token_count":3,"output":"........."}}
{"status":"fulfilled","value":{"chunk_id":"52f194","wall_time_seconds":0.000001625,"exit_code":0,"original_token_count":5,"output":"All checks passed!\n"}}
{"status":"fulfilled","value":{"chunk_id":"7506da","wall_time_seconds":0.000005333,"exit_code":0,"original_token_count":7413,"output":"\"\"\"The improvement mechanism's own policy, as versioned data instead of code.\n\nBefore this module, the rule that decides whether a recurring finding\ndeserves a target-level patch or a mechanism-level fix lived as constants in\nscripts/detect-recurring-pattern.py: a keyword taxonomy and a recurrence\nthreshold, written by hand once and never revisited. That is an L4 loop in\nthe paper's terms (docs/plans/recursive-meta-improvement.md): the system\nadapts its deployed state, but the mechanism governing what counts as an\nimprovement stays fixed human infrastructure.\n\nL5 requires that mechanism to be something the system can revise from\nevidence, with the same safeguards it applies to every other change. So the\npolicy becomes a JSON document with a version, a parent, and an origin, and\nevery revision is appended to a history file with the evidence that\njustified it. The pieces that must NOT be revisable by the meta-improver\n(the archive, the external anchor, the independent verifier, the acceptance\nthresholds, and the promotion path) are enumerated in FIXED_INFRASTRUCTURE,\nand `assert_ai_may_write` refuses any write outside AI_OWNED_COMPONENTS.\n\"\"\"\n\nfrom __future__ import annotations\n\nimport hashlib\nimport json\nfrom datetime import UTC, datetime\nfrom pathlib import Path\n\nREPO_ROOT = Path(__file__).resolve().parent.parent\nPOLICY_PATH = REPO_ROOT / \"docs\" / \"improvement-policy.json\"\nHISTORY_PATH = REPO_ROOT / \"docs\" / \"improvement-policy-history.jsonl\"\n\n# The v1 taxonomy and threshold, kept in code only as a fallback so every\n# existing tool still runs in a checkout that predates the policy file.\nBUILTIN_THRESHOLD = 3\nBUILTIN_TOPIC_KEYWORDS: dict[str, list[str]] = {\n    \"credential-redaction\": [\"redact\", \"credential\", \"secret\", \"token\", \"leak\", \"expos\"],\n    \"shell-semantics\": [\"errexit\", \"bash -e\", \"exit code\", \"-e\", \"pipefail\", \"shell\"],\n    \"env-var-precedence\": [\"precedence\", \"env var\", \"environment variable\", \"unconditionally\"],\n    \"fork-pr-permissions\": [\"fork\", \"github_token\", \"persist-credentials\"],\n    \"auth-lifecycle\": [\"refresh token\", \"rotat\", \"expir\", \"auth.json\", \"stale\"],\n}\n\n# Autonomy attribution (paper failure mode 2): the meta-improver may rewrite\n# exactly these files, and nothing else. Paths are repo-relative.\nAI_OWNED_COMPONENTS: dict[str, str] = {\n    \"improvement-policy\": \"docs/improvement-policy.json\",\n    \"improvement-policy-history\": \"docs/improvement-policy-history.jsonl\",\n}\n\n# Everything the loop depends on that stays human-owned infrastructure. The\n# dashboard renders this list verbatim so the boundary is visible, not implied.\nFIXED_INFRASTRUCTURE: dict[str, str] = {\n    \"archive\": \"docs/self-improvement-archive.jsonl — append-only, SHA-idempotent (archive-round.py)\",\n    \"verifier\": \".github/workflows/codex-review.yml — independent second-model review of every PR\",\n    \"anchor\": \"Traces evidence from working sessions — never consulted when a round is decided\",\n    \"meta-acceptance-rule\": \"MIN_COVERAGE / MIN_VALIDITY / MIN_ROUNDS_TO_JUDGE in revise-improvement-policy.py\",\n    \"promotion\": \"pull requests only; a human merges every policy revision and every rollback\",\n}\n\n\ndef utc_now_iso() -> str:\n    return datetime.now(UTC).replace(microsecond=0).isoformat().replace(\"+00:00\", \"Z\")\n\n\ndef builtin_policy() -> dict:\n    return {\n        \"version\": 1,\n        \"parent\": None,\n        \"origin\": \"init\",\n        \"created_at\": \"2026-09-14T18:00:00Z\",\n        \"threshold\": BUILTIN_THRESHOLD,\n        \"topics\": {\n            topic: {\"keywords\": list(keywords), \"weight\": 1.0}\n            for topic, keywords in BUILTIN_TOPIC_KEYWORDS.items()\n        },\n        \"rationale\": \"Built-in fallback identical to policy version 1.\",\n    }\n\n\ndef load_policy(path: Path | str = POLICY_PATH) -> dict:\n    with open(path) as f:\n        policy = json.load(f)\n    validate_policy(policy)\n    return policy\n\n\ndef load_policy_or_builtin(path: Path | str = POLICY_PATH) -> dict:\n    if Path(path).exists():\n        return load_policy(path)\n    return builtin_policy()\n\n\nVALID_ORIGINS = frozenset({\"init\", \"revision\", \"rollback\"})\n\n\ndef round_key(entry: dict) -> str:\n    \"\"\"Identity of an archived round, shared by every tool that counts\n    rounds (the detector that opens issues and the measurer the policy is\n    judged on must count the same signal — Codex review of PR #10, round\n    37). Automated rounds are identified by the commit they reviewed: two\n    archive proposals opened before either merged both computed the same\n    next round number, and merging by number alone collapsed two reviews\n    into one round (full-branch review, workflows finding 2). Legacy rounds\n    without a source_sha keep their number (placeholder + result pairs).\"\"\"\n    sha = entry.get(\"source_sha\")\n    if isinstance(sha, str) and sha:\n        return f\"sha:{sha}\"\n    return f\"round:{entry.get('round')}\"\n\n\ndef assert_policy_matches_history(policy: dict, history: list[dict]) -> None:\n    \"\"\"The policy's lineage metadata (version, parent, origin,\n    restored_version) is not part of its content hash, yet the wait gate and\n    rollback logic depend on it. So a policy in force must be exactly the\n    snapshot its history recorded for that version, metadata included; a\n    root policy must not claim to be a revision or rollback. Otherwise an\n    edit to `origin` alone would switch the evaluation gates off (Codex\n    full-branch review, finding 1).\"\"\"\n    if not history:\n        if policy.get(\"origin\") != \"init\" or policy.get(\"parent\") is not None:\n            raise ValueError(\n                \"policy claims a revision/rollback lineage but the history records no versions\"\n            )\n        return\n    latest = history[-1]\n    snapshot = latest.get(\"policy\") or {}\n    if latest.get(\"version\") != policy.get(\"version\"):\n        raise ValueError(\n            f\"policy is v{policy.get('version')} but the history's latest entry is \"\n            f\"v{latest.get('version')}; the policy and its history must be written together\"\n        )\n    for key in (\"version\", \"parent\", \"origin\", \"restored_version\"):\n        if snapshot.get(key) != policy.get(key):\n            raise ValueError(\n                f\"policy.{key}={policy.get(key)!r} differs from the recorded v{policy.get('version')} \"\n                f\"snapshot ({snapshot.get(key)!r}); lineage metadata may not be edited in place\"\n            )\n    if policy_hash(snapshot) != policy_hash(policy):\n        raise ValueError(\n            f\"policy content hashes to {policy_hash(policy)} but the recorded v{policy.get('version')} \"\n            f\"snapshot hashes to {policy_hash(snapshot)}\"\n        )\n\n\ndef validate_policy(policy: dict) -> None:\n    if not isinstance(policy.get(\"version\"), int) or policy[\"version\"] < 1:\n        raise ValueError(\"policy.version must be a positive integer\")\n    if not isinstance(policy.get(\"threshold\"), int) or policy[\"threshold\"] < 1:\n        raise ValueError(\"policy.threshold must be a positive integer\")\n    if policy.get(\"origin\") not in VALID_ORIGINS:\n        raise ValueError(f\"policy.origin must be one of {sorted(VALID_ORIGINS)}\")\n    topics = policy.get(\"topics\")\n    if not isinstance(topics, dict) or not topics:\n        raise ValueError(\"policy.topics must be a non-empty object\")\n    for name, spec in topics.items():\n        if not isinstance(name, str) or not name or \"@\" in name or name != name.strip():\n            # `name@tag` keys are reserved for older definitions of a name in\n            # the evidence; a topic named that way would be skipped by the\n            # candidate anchor and escape the validity comparison (Codex\n            # full-branch review, finding 2).\n            raise ValueError(\n                f\"topic name {name!r} is invalid (non-empty, no '@', no surrounding whitespace)\"\n            )\n        keywords = spec.get(\"keywords\")\n        if (\n            not isinstance(keywords, list)\n            or not keywords\n            or not all(isinstance(k, str) and k for k in keywords)\n        ):\n            # An empty list would classify nothing while matching every trace\n            # (Codex review of PR #10, round 16).\n            raise ValueError(f\"topic {name!r} needs a non-empty list of keyword strings\")\n        weight = spec.get(\"weight\", 1.0)\n        if not isinstance(weight, int | float) or weight <= 0:\n            raise ValueError(f\"topic {name!r} weight must be a positive number\")\n\n\ndef policy_hash(policy: dict) -> str:\n    \"\"\"Content hash of the decision-relevant fields. Two policies with the\n    same taxonomy, weights, and threshold decide identically, whatever their\n    version metadata says — this is what the dashboard pins per epoch to\n    show the evaluator was frozen while a round was decided.\n\n    Topic ORDER is part of the hash: classification takes the first topic\n    whose keyword matches, so reordering overlapping topics changes\n    decisions and must not pass the stale-measurement guard (Codex review\n    of PR #10, finding 3).\"\"\"\n    canonical = json.dumps(\n        {\n            \"threshold\": policy[\"threshold\"],\n            \"topics\": [\n                [name, spec[\"keywords\"], float(spec.get(\"weight\", 1.0))]\n                for name, spec in policy[\"topics\"].items()\n            ],\n        },\n        separators=(\",\", \":\"),\n    )\n    return hashlib.sha256(canonical.encode()).hexdigest()[:12]\n\n\ndef topic_keywords(policy: dict) -> dict[str, list[str]]:\n    return {name: list(spec[\"keywords\"]) for name, spec in policy[\"topics\"].items()}\n\n\ndef topic_weights(policy: dict) -> dict[str, float]:\n    return {name: float(spec.get(\"weight\", 1.0)) for name, spec in policy[\"topics\"].items()}\n\n\ndef classify_finding(text: str, keywords: dict[str, list[str]]) -> str | None:\n    \"\"\"First topic (in policy order) with any keyword present. Same rule the\n    detector has always applied; it lives here so every tool classifies\n    identically under the same policy version.\"\"\"\n    lowered = text.lower()\n    for topic, words in keywords.items():\n        if any(word in lowered for word in words):\n            return topic\n    return None\n\n\ndef new_version(\n    parent: dict,\n    *,\n    topics: dict,\n    threshold: int,\n    origin: str,\n    rationale: str,\n    created_at: str | None = None,\n    restored_version: int | None = None,\n) -> dict:\n    if origin not in {\"revision\", \"rollback\"}:\n        raise ValueError(\"origin must be 'revision' or 'rollback'\")\n    policy = {\n        \"version\": parent[\"version\"] + 1,\n        \"parent\": parent[\"version\"],\n        \"origin\": origin,\n        \"created_at\": created_at or utc_now_iso(),\n        \"threshold\": threshold,\n        \"topics\": topics,\n        \"rationale\": rationale,\n    }\n    if origin == \"rollback\":\n        # Which version's configuration this restores, so ancestry checks can\n        # continue through it (Codex review of PR #10, round 17).\n        policy[\"restored_version\"] = restored_version\n    validate_policy(policy)\n    return policy\n\n\ndef relative_to_repo(path: Path | str) -> str:\n    resolved = Path(path).resolve()\n    try:\n        return resolved.relative_to(REPO_ROOT).as_posix()\n    except ValueError:\n        return resolved.as_posix()\n\n\ndef component_paths(role: str | None, allowed: dict[str, str] | None = None) -> set[str]:\n    \"\"\"Paths the meta-improver may write for one role ('policy' or\n    'history'), or for any role when role is None.\"\"\"\n    components = allowed or AI_OWNED_COMPONENTS\n    if role is None:\n        return set(components.values())\n    return {p for name, p in components.items() if name == role or name.endswith(f\"-{role}\")}\n\n\ndef assert_ai_may_write(\n    path: Path | str, *, allowed: dict[str, str] | None = None, role: str | None = None\n) -> None:\n    \"\"\"Attribution guard: the meta-improver only ever writes the files it owns,\n    and each file only in its own role: the policy destination must be the\n    policy component and the history destination the history component, so\n    swapped arguments cannot append a policy to the history or overwrite the\n    policy with a history line (Codex review of PR #10, rounds 20 and 35).\n    Raises PermissionError otherwise, so a bug that tries to 'fix' the archive\n    or the verifier fails loudly instead of silently widening autonomy.\"\"\"\n    allowed_paths = component_paths(role, allowed)\n    rel = relative_to_repo(path)\n    if rel not in allowed_paths:\n        what = f\"the {role} component\" if role else \"a file it owns\"\n        raise PermissionError(\n            f\"{rel} is fixed infrastructure or not {what}; \"\n            f\"the meta-improver may only write {sorted(allowed_paths)} here\"\n        )\n\n\nPROTECTED_OUTPUT_PREFIXES: tuple[str, ...] = (\".github/\", \"scripts/\", \"packages/\", \"terraform/\")\nPROTECTED_OUTPUT_FILES: tuple[str, ...] = (\n    \"docs/self-improvement-archive.jsonl\",\n    \"docs/improvement-policy.json\",\n    \"docs/improvement-policy-history.jsonl\",\n)\n# The committed field anchors: only a deliberate evidence refresh may write\n# them, never a report or decision output (Codex review of PR #10, round 15).\nCANONICAL_EVIDENCE_FILES: tuple[str, ...] = (\n    \"docs/rsi/trace-evidence.json\",\n    \"docs/rsi/trace-evidence-verifier.json\",\n)\n\n\ndef assert_safe_output(\n    path: Path | str, *, inputs: list[str | Path] = (), kind: str = \"report\"\n) -> None:\n    \"\"\"Side outputs may go anywhere EXCEPT the loop's own records, its code,\n    the files the invocation is reading, and (for anything but an evidence\n    refresh) the canonical evidence snapshots (Codex review of PR #10,\n    rounds 11 and 15).\"\"\"\n    rel = relative_to_repo(path)\n    if rel in PROTECTED_OUTPUT_FILES or any(rel.startswith(p) for p in PROTECTED_OUTPUT_PREFIXES):\n        raise PermissionError(f\"{rel} is a protected file; choose another output path\")\n    if kind != \"evidence\" and rel in CANONICAL_EVIDENCE_FILES:\n        raise PermissionError(\n            f\"{rel} is a canonical evidence snapshot; only --save-evidence may write it\"\n        )\n    # Identity is by path AND by inode: a hard link to the archive named\n    # report.json resolves to a different path but is the same file (Codex\n    # full-branch review, finding 6).\n    for protected in (\n        *PROTECTED_OUTPUT_FILES,\n        *(() if kind == \"evidence\" else CANONICAL_EVIDENCE_FILES),\n    ):\n        if same_file(REPO_ROOT / protected, path):\n            raise PermissionError(f\"{rel} is the same file as protected {protected}\")\n    for source in inputs:\n        if source and (Path(source).resolve() == Path(path).resolve() or same_file(source, path)):\n            raise PermissionError(f\"{rel} is an input of this run; choose another output path\")\n\n\ndef same_file(a: Path | str, b: Path | str) -> bool:\n    try:\n        return Path(a).samefile(b)\n    except OSError:\n        return False\n\n\ndef save_policy(\n    policy: dict, path: Path | str = POLICY_PATH, *, allowed: dict[str, str] | None = None\n) -> None:\n    assert_ai_may_write(path, allowed=allowed, role=\"policy\")\n    validate_policy(policy)\n    Path(path).write_text(json.dumps(policy, indent=2) + \"\\n\")\n\n\ndef load_history(path: Path | str = HISTORY_PATH) -> list[dict]:\n    if not Path(path).exists():\n        return []\n    entries = []\n    with open(path) as f:\n        for line in f:\n            line = line.strip()\n            if line:\n                entries.append(json.loads(line))\n    return entries\n\n\ndef append_history(\n    entry: dict, path: Path | str = HISTORY_PATH, *, allowed: dict[str, str] | None = None\n) -> None:\n    assert_ai_may_write(path, allowed=allowed, role=\"history\")\n    # No sort_keys: a snapshot's topic order is its classification\n    # precedence, and restoring an alphabetized snapshot would silently\n    # reclassify findings (Codex review of PR #10, finding 2).\n    with open(path, \"a\") as f:\n        f.write(json.dumps(entry) + \"\\n\")\n\n\n# The checked-in policy, resolved once so every tool stamps and decides with\n# the same version and hash in one process.\n_CURRENT = load_policy_or_builtin()\nPOLICY_VERSION: int = _CURRENT[\"version\"]\nPOLICY_HASH: str = policy_hash(_CURRENT)\nname: Revise Improvement Policy\n\n# The L5 step of docs/plans/recursive-meta-improvement.md, run automatically\n# but bounded exactly like archive-and-recommend.yml: it never pushes to the\n# default branch, never merges, never deploys, and requests no repository\n# repository secrets at all.\n#\n# After every change to the review archive on main (an archive-round PR\n# merging), this workflow:\n#   1. measures whether docs/improvement-policy.json's signal still predicts\n#      the field (scripts/measure-policy-validity.py) — coverage of archived\n#      findings, and agreement with Traces evidence when a key is present;\n#   2. lets scripts/revise-improvement-policy.py apply its fixed acceptance\n#      rule: propose a bounded policy revision, propose a rollback of a\n#      revision that made things worse, or do nothing;\n#   3. re-renders docs/rsi/dashboard.html from the resulting state;\n#   4. opens ONE pull request carrying the policy, its history entry, the\n#      measurement, and the dashboard. A human merges it, or closes it.\n#\n# The field anchor (Traces evidence) is refreshed on every run from the\n# shared traces.com namespace when a read-only TRACES_API_KEY is present\n# (hourly on the schedule above); otherwise the committed snapshot is used.\n# Without any snapshot the anchor is absent: coverage is still measured and\n# can still trigger a revision, validity is reported as null, and the\n# workflow says so rather than pretending it was checked.\n\non:\n  push:\n    branches: [main]\n    paths:\n      - docs/self-improvement-archive.jsonl\n      # A refreshed field snapshot is new evidence too (Codex review of PR #10, round 10).\n      - docs/rsi/trace-evidence.json\n  schedule:\n    # Hourly: refresh the field anchor from the shared Traces namespace,\n    # re-measure, and let the fixed acceptance rule decide. Off the hour so\n    # it does not queue behind every other project's :00 cron.\n    - cron: \"17 * * * *\"\n  workflow_dispatch:\n\npermissions:\n  contents: write\n  pull-requests: write\n\nconcurrency:\n  group: revise-improvement-policy\n  cancel-in-progress: false\n\njobs:\n  revise:\n    name: Measure, revise, and propose\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    steps:\n      - name: Checkout (default branch — trusted)\n        uses: actions/checkout@v4\n        with:\n          # Always the default branch, whatever ref a manual dispatch was\n          # started from, so a proposal never carries an unrelated feature\n          # branch's commits (Codex review of PR #10, round 5).\n          ref: ${{ github.event.repository.default_branch }}\n          fetch-depth: 0\n\n      - name: Set up Python\n        uses: actions/setup-python@v5\n        with:\n          python-version: \"3.12\"\n\n      - name: Record the commit actually checked out\n        # The default branch may have advanced past the triggering commit, and\n        # a manual dispatch may come from another ref; label everything with\n        # what this run measured (Codex review of PR #10, round 16).\n        id: source\n        run: echo \"sha=$(git rev-parse HEAD)\" >> \"$GITHUB_OUTPUT\"\n\n      - name: Check for a Traces API key\n        id: traces-key\n        env:\n          HAS_KEY: ${{ secrets.TRACES_API_KEY != '' }}\n        run: echo \"present=$HAS_KEY\" >> \"$GITHUB_OUTPUT\"\n\n      - name: Install Traces CLI\n        if: steps.traces-key.outputs.present == 'true'\n        run: |\n          curl -fsSL https://www.traces.com/install | bash\n          echo \"$HOME/.traces/bin\" >> \"$GITHUB_PATH\"\n\n      - name: Refresh the field anchor from the shared Traces namespace\n        # Working sessions live on developers' machines, but every session\n        # they share lands in the traces.com namespace, which a read-only key\n        # can list and sync. The miner keeps only tool results the tool\n        # itself reported as errors, so the anchor is failures, never\n        # narration. Without a key the committed snapshot is used and its\n        # age is reported instead of silently reused (Codex review of\n        # PR #10, round 3).\n        if: steps.traces-key.outputs.present == 'true'\n        env:\n          TRACES_API_KEY: ${{ secrets.TRACES_API_KEY }}\n          TRACES_NAMESPACE: ${{ vars.TRACES_NAMESPACE || '@gagan114' }}\n        run: |\n          set -euo pipefail\n          mkdir -p docs/rsi\n          if python3 scripts/mine-trace-failures.py \\\n              --namespace \"$TRACES_NAMESPACE\" --traces-key \"$TRACES_API_KEY\" \\\n              --save-evidence docs/rsi/trace-evidence.json \\\n              --out-json \"$RUNNER_TEMP/field-failures.json\" > \"$RUNNER_TEMP/mine.txt\"; then\n            head -n 30 \"$RUNNER_TEMP/mine.txt\"\n          else\n            echo \"::warning::Traces refresh failed; using the committed evidence snapshot instead.\"\n            sed -n '1,40p' \"$RUNNER_TEMP/mine.txt\" || true\n            git checkout -- docs/rsi/trace-evidence.json 2>/dev/null || true\n            rm -f \"$RUNNER_TEMP/field-failures.json\"\n          fi\n\n      - name: Measure policy validity\n        # This step reports the snapshot's age so a stale anchor is visible\n        # rather than silently reused (Codex review of PR #10, round 3).\n        run: |\n          set -euo pipefail\n          mkdir -p docs/rsi\n          if [ -f docs/rsi/trace-evidence.json ]; then\n            collected=$(python3 -c 'import json; print(json.load(open(\"docs/rsi/trace-evidence.json\")).get(\"collected_at\", \"unknown\"))')\n            echo \"::notice::Using the committed Traces evidence snapshot collected at ${collected}. Refresh it locally to update the field anchor.\"\n            python3 scripts/measure-policy-validity.py docs/self-improvement-archive.jsonl \\\n              --trace-evidence docs/rsi/trace-evidence.json --out-json docs/rsi/measurement.json > measurement.txt\n          else\n            echo \"::notice::No Traces evidence snapshot committed; validity will be null and only coverage can trigger a revision.\"\n            python3 scripts/measure-policy-validity.py docs/self-improvement-archive.jsonl \\\n              --out-json docs/rsi/measurement.json > measurement.txt\n          fi\n          sed -n '1,/^---/p' measurement.txt\n\n      - name: Apply the fixed acceptance rule\n        id: revise\n        run: |\n          set -euo pipefail\n          args=(docs/self-improvement-archive.jsonl --measurement docs/rsi/measurement.json --out-json decision.json)\n          # Fresh field blind spots (from this run's Traces refresh) feed\n          # topic mining the same way unclassified review findings do.\n          [ -f \"$RUNNER_TEMP/field-failures.json\" ] && args+=(--field-failures \"$RUNNER_TEMP/field-failures.json\")\n          python3 scripts/revise-improvement-policy.py \"${args[@]}\" > revise.txt\n          sed -n '1,/^---/p' revise.txt\n          action=$(python3 -c 'import json; print(json.load(open(\"decision.json\"))[\"action\"])')\n          echo \"action=$action\" >> \"$GITHUB_OUTPUT\"\n          if [ \"$action\" != \"none\" ]; then\n            # The measurement that justified the decision is evidence; keep it.\n            # The live measurement must describe the policy now in force, or the\n            # next run's hash check would refuse it (Codex review of PR #10, round 4).\n            cp docs/rsi/measurement.json docs/rsi/measurement-decision.json\n            if [ -f docs/rsi/trace-evidence.json ]; then\n              python3 scripts/measure-policy-validity.py docs/self-improvement-archive.jsonl \\\n                --trace-evidence docs/rsi/trace-evidence.json --out-json docs/rsi/measurement.json > remeasure.txt\n            else\n              python3 scripts/measure-policy-validity.py docs/self-improvement-archive.jsonl \\\n                --out-json docs/rsi/measurement.json > remeasure.txt\n            fi\n          fi\n\n      - name: Re-render the dashboard\n        env:\n          SOURCE_SHA: ${{ steps.source.outputs.sha }}\n        run: |\n          set -euo pipefail\n          args=(docs/self-improvement-archive.jsonl --head \"${SOURCE_SHA:0:8}\" --out docs/rsi/dashboard.html)\n          [ -f docs/rsi/trace-evidence.json ] && args+=(--trace-evidence docs/rsi/trace-evidence.json)\n          [ -f docs/rsi/trace-evidence-verifier.json ] && args+=(--verifier-evidence docs/rsi/trace-evidence-verifier.json)\n          python3 scripts/render-rsi-dashboard.py \"${args[@]}\"\n\n      - name: Propose the result as a pull request\n        env:\n          GH_TOKEN: ${{ github.token }}\n          ACTION: ${{ steps.revise.outputs.action }}\n          SOURCE_SHA: ${{ steps.source.outputs.sha }}\n        run: |\n          set -euo pipefail\n          if git diff --quiet -- docs/improvement-policy.json docs/improvement-policy-history.jsonl docs/rsi/; then\n            echo \"Nothing changed (action: $ACTION).\"\n            exit 0\n          fi\n          version=$(python3 -c 'import json; print(json.load(open(\"docs/improvement-policy.json\"))[\"version\"])')\n          default_branch=$(gh repo view --json defaultBranchRef --jq .defaultBranchRef.name)\n          git config user.name \"github-actions[bot]\"\n          git config user.email \"41898282+github-actions[bot]@users.noreply.github.com\"\n          if git diff --quiet -- docs/improvement-policy.json docs/improvement-policy-history.jsonl; then\n            # Only the evidence, measurement and dashboard moved (the hourly\n            # refresh with no decision). One standing evidence branch, updated\n            # in place, so the hourly run never opens a new PR per hour; a\n            # human merges it whenever, and merging it re-triggers this\n            # workflow through the trace-evidence.json path filter.\n            # A standing policy proposal that the current decision no longer\n            # supports is withdrawn, not left open to merge (Codex review of\n            # PR #10, round 37): new rounds can legitimately turn a\n            # `revise` into `none`.\n            stale_pr=$(gh pr list --head \"improvement-policy-proposal\" --base \"$default_branch\" --state open \\\n              --json number,isCrossRepository --jq '[.[] | select(.isCrossRepository == false)][0].number // empty')\n            if [ -n \"$stale_pr\" ]; then\n              gh pr close \"$stale_pr\" --delete-branch \\\n                --comment \"Withdrawn: re-evaluated at \\`${SOURCE_SHA}\\` the fixed acceptance rule proposes no policy change (action: **${ACTION}**), so this proposal no longer satisfies it. A new proposal will be opened if a later round supports one.\"\n              echo \"Closed the stale policy proposal PR #$stale_pr.\"\n            fi\n            branch=\"improvement-evidence-refresh\"\n            title=\"chore(rsi): refresh the Traces field anchor\"\n            body=\"Automatically refreshed by [\\`revise-improvement-policy.yml\\`](../blob/main/.github/workflows/revise-improvement-policy.yml) from the shared Traces namespace at \\`${SOURCE_SHA}\\`. The fixed acceptance rule proposed no policy change on this evidence (action: **${ACTION}**). This changes only the evidence snapshot, the measurement and the dashboard; the policy is untouched.\"\n            git checkout -b \"$branch\"\n            git add docs/rsi/\n            git commit -m \"$title\"\n            git push --force origin \"$branch\"\n            existing_pr=$(gh pr list --head \"$branch\" --base \"$default_branch\" --state open \\\n              --json number,isCrossRepository --jq '[.[] | select(.isCrossRepository == false)][0].number // empty')\n            if [ -n \"$existing_pr\" ]; then\n              echo \"Updated the open evidence PR #$existing_pr in place.\"\n              exit 0\n            fi\n            gh pr create --title \"$title\" --body \"$body\" --label \"self-improvement-archive\" \\\n              --base \"$default_branch\" --head \"$branch\"\n            exit 0\n          fi\n          # One standing proposal branch: a newer archive round supersedes an\n          # open proposal instead of opening a competing one with the same\n          # version and parent (Codex review of PR #10, round 5).\n          branch=\"improvement-policy-proposal\"\n          git checkout -b \"$branch\"\n          git add docs/improvement-policy.json docs/improvement-policy-history.jsonl docs/rsi/\n          git commit -m \"chore(rsi): ${ACTION} of the improvement policy (v${version})\"\n          git push --force origin \"$branch\"\n          # Same-repository PRs only: `--head` matches by branch name alone, so\n          # a fork PR using this branch name must not be mistaken for the\n          # bot's own proposal (Codex review of PR #10, round 19).\n          existing_pr=$(gh pr list --head \"$branch\" --base \"$default_branch\" --state open \\\n            --json number,isCrossRepository --jq '[.[] | select(.isCrossRepository == false)][0].number // empty')\n          if [ -n \"$existing_pr\" ]; then\n            gh pr comment \"$existing_pr\" --body \"Superseded by a newer archive round at \\`${SOURCE_SHA}\\`: this proposal was regenerated from the current archive (action: **${ACTION}**, policy v${version}).\"\n            echo \"Updated the open proposal PR #$existing_pr in place.\"\n            exit 0\n          fi\n          gh pr create \\\n            --title \"chore(rsi): ${ACTION} of the improvement policy (v${version})\" \\\n            --body \"Automatically proposed by [\\`revise-improvement-policy.yml\\`](../blob/main/.github/workflows/revise-improvement-policy.yml) after the archive changed at \\`${SOURCE_SHA}\\`. Action: **${ACTION}**. See \\`docs/improvement-policy-history.jsonl\\` for the evidence and \\`docs/rsi/dashboard.html\\` for the rendered state. This changes only the AI-owned policy files; a human merges it, same as every other change in this repo.\" \\\n            --label \"self-improvement-archive\" \\\n            --base \"$default_branch\" \\\n            --head \"$branch\"\n"}}

>cat scripts/mine-trace-failures.py

{"chunk_id":"429266","wall_time_seconds":0.000001541,"exit_code":0,"original_token_count":5069,"output":"#!/usr/bin/env python3\n\"\"\"Mine actual failures out of agent session traces, so the field anchor\ncounts things that went wrong, never things people said.\n\nThe first field anchor searched transcript text for the taxonomy's\nkeywords. Every hit turned out to be narration — the assistant summarising\nreview findings — so the \"field\" merely echoed the reviews it was supposed\nto check. This tool reads the events of each working session directly\nthrough `traces show --json` and keeps only:\n\n  * tool results Traces itself marked `status: \"error\"` — nothing else.\n    The failure shape of the output (traceback, non-zero exit, test\n    failure, permission or auth error, git rejection, timeout) only names\n    the kind; displayed text never turns a successful execution into a\n    failure.\n\nEach failure is paired with the command that produced it, deduplicated per\nsession, classified with the current improvement policy, and written out\ntwo ways: a human report plus JSON (`--out-json`), and an evidence file in\nthe shape scripts/measure-policy-validity.py consumes (`--save-evidence`),\nwhere a topic's evidence is the set of sessions in which a failure matching\nthat topic's keywords actually occurred. Failures no topic claims are the\nfield's blind spots — the same kind of signal the meta-improver mines from\nunclassified review findings.\n\nUsage:\n    python3 mine-trace-failures.py --repo-dir DIR [--agents a,b|all]\n        [--policy PATH] [--history PATH] [--out-json PATH] [--save-evidence PATH]\n        [--traces-bin PATH] [--limit N]\n\"\"\"\n\nfrom __future__ import annotations\n\nimport argparse\nimport importlib.util\nimport json\nimport re\nimport subprocess\nimport sys\nimport tempfile\nfrom collections import Counter, defaultdict\nfrom pathlib import Path\n\n\ndef _load_sibling_module(name: str, filename: str):\n    if name in sys.modules:\n        return sys.modules[name]\n    path = Path(__file__).parent / filename\n    spec = importlib.util.spec_from_file_location(name, path)\n    assert spec is not None and spec.loader is not None\n    module = importlib.util.module_from_spec(spec)\n    sys.modules[name] = module\n    spec.loader.exec_module(module)\n    return module\n\n\npolicy_mod = _load_sibling_module(\"improvement_policy\", \"improvement_policy.py\")\nmeasure_mod = _load_sibling_module(\"measure_policy_validity\", \"measure-policy-validity.py\")\n\nVERIFIER_AGENT = \"codex\"\nPAGE_SIZE = 200\nEXCERPT_CHARS = 160\n\n# Failure shapes, each named so a report can say what kind of thing broke.\nFAILURE_PATTERNS: dict[str, re.Pattern[str]] = {\n    \"traceback\": re.compile(r\"Traceback \\(most recent call last\\)\"),\n    \"nonzero-exit\": re.compile(r\"^Exit code [1-9]\\d*\", re.M),\n    # \"0 failed\" is a pass; only a positive count is a failure.\n    \"test-failure\": re.compile(r\"\\b[1-9]\\d* failed\\b|^FAILED \", re.M),\n    \"permission\": re.compile(\n        r\"Permission denied|EACCES|denied by the .* classifier|Operation not permitted\"\n    ),\n    # HTTP-shaped only: a bare \"401\" is far more often a line number in a\n    # file read than an auth failure.\n    \"auth\": re.compile(\n        r\"HTTP/?[\\d.]* ?40[13]\\b|\\b40[13] (?:Unauthorized|Forbidden)|status(?: code)?[:=]? ?40[13]\\b\"\n        r\"|\\bUnauthorized\\b|\\bForbidden\\b|token (?:expired|invalid)|authentication failed\",\n        re.I,\n    ),\n    \"git-rejected\": re.compile(r\"non-fast-forward|rejected\\]|^fatal: |merge conflict\", re.I | re.M),\n    \"timeout\": re.compile(r\"timed out|timeout of \\d+|ETIMEDOUT|TLS handshake timeout\", re.I),\n    \"not-found\": re.compile(r\"No such file or directory|command not found|ENOENT\", re.I),\n}\n\n\nclass TracesCliError(RuntimeError):\n    pass\n\n\ndef parse_cli_json(stdout: str) -> dict | None:\n    \"\"\"The CLI may print a hydration notice before the JSON document the\n    first time a session's events are loaded; parse from the first brace.\"\"\"\n    for candidate in (stdout, stdout[stdout.find(\"{\") :] if \"{\" in stdout else \"\"):\n        if not candidate:\n            continue\n        try:\n            return json.loads(candidate)\n        except json.JSONDecodeError:\n            continue\n    return None\n\n\n# Extra arguments appended to every CLI call (an API key on a runner that is\n# not logged in). Set once by main(); never printed.\nEXTRA_CLI_ARGS: list[str] = []\n\n\ndef run_traces_json(traces_bin: str, args: list[str], *, retries: int = 1) -> dict:\n    last_error = \"no output\"\n    for attempt in range(retries + 1):\n        # stdout goes to a file, not a pipe: the CLI truncates piped output at\n        # 64 KiB (observed: 65519 bytes of an 80 KB document), while a file\n        # redirect receives everything.\n        try:\n            with tempfile.TemporaryFile(mode=\"w+\", encoding=\"utf-8\") as out:\n                result = subprocess.run(\n                    [traces_bin, *args, *EXTRA_CLI_ARGS, \"--json\"],\n                    stdout=out,\n                    stderr=subprocess.PIPE,\n                    text=True,\n                    timeout=300,\n                )\n                out.seek(0)\n                stdout = out.read()\n        except OSError as exc:\n            raise TracesCliError(f\"Could not run `{traces_bin}`: {exc}\") from exc\n        if result.returncode != 0:\n            raise TracesCliError(f\"`{traces_bin} {' '.join(args)}` failed: {result.stderr.strip()}\")\n        payload = parse_cli_json(stdout)\n        if payload is not None:\n            if not payload.get(\"ok\"):\n                raise TracesCliError(f\"`{traces_bin} {' '.join(args)}` reported failure: {payload}\")\n            return payload[\"data\"]\n        last_error = stdout[:120].replace(\"\\n\", \" \")\n        if attempt < retries:\n            continue\n    raise TracesCliError(f\"Non-JSON output from `{traces_bin} {' '.join(args)}`: {last_error}\")\n\n\ndef list_traces(\n    traces_bin: str, repo_dir: str, agents: list[str] | None, limit: int\n) -> tuple[list[dict], bool]:\n    \"\"\"Sessions recorded in the folder, and whether the listing was complete.\n    A listing that fills its limit may have missed sessions, and evidence\n    built from it must say so rather than report confirmed zero counts\n    (Codex review of PR #10, round 27).\"\"\"\n    found: dict[str, dict] = {}\n    complete = True\n    for agent_args in [[\"--agent\", a] for a in agents] if agents else [[]]:\n        data = run_traces_json(\n            traces_bin, [\"list\", \"--dir\", repo_dir, *agent_args, \"--limit\", str(limit)]\n        )\n        traces = data.get(\"traces\", [])\n        if len(traces) >= limit:\n            complete = False\n        for trace in traces:\n            found[trace[\"id\"]] = trace\n    return sorted(found.values(), key=lambda t: (t.get(\"timestamp\") or 0, t[\"id\"])), complete\n\n\ndef list_namespace_traces(\n    traces_bin: str, namespace: str, agents: list[str] | None\n) -> tuple[list[dict], bool]:\n    \"\"\"Sessions shared to a traces.com namespace (for a runner that has no\n    local sessions of its own). `--all` returns the whole namespace, so the\n    listing is complete by construction; agents are filtered client-side.\"\"\"\n    if not namespace.startswith(\"@\"):\n        namespace = f\"@{namespace}\"\n    data = run_traces_json(traces_bin, [\"list\", namespace, \"--all\"])\n    traces = [\n        t\n        for t in data.get(\"traces\", [])\n        if isinstance(t.get(\"id\"), str) and (agents is None or t.get(\"agentId\") in agents)\n    ]\n    return sorted(traces, key=lambda t: (t.get(\"timestamp\") or 0, t[\"id\"])), True\n\n\ndef sync_trace(traces_bin: str, trace_id: str) -> None:\n    \"\"\"Pull a shared trace's events into the local database so `show` can\n    page through them (a fresh runner has none).\"\"\"\n    run_traces_json(traces_bin, [\"sync\", trace_id])\n\n\ndef iter_events(traces_bin: str, trace_id: str):\n    offset = 1\n    while True:\n        data = run_traces_json(\n            traces_bin, [\"show\", trace_id, \"--offset\", str(offset), \"--limit\", str(PAGE_SIZE)]\n        )\n        events = data.get(\"events\") or []\n        yield from events\n        if len(events) < PAGE_SIZE:\n            return\n        offset += len(events)\n\n\ndef failure_kind(event: dict) -> str | None:\n    \"\"\"A failure is an execution the tool itself reported as an error\n    (`status: \"error\"`). Output text never decides whether something\n    failed — a displayed transcript can contain \"Exit code 1\" or \"HTTP 401\"\n    verbatim (Codex review of PR #10, rounds 27-30). The failure shape only\n    names the kind once the status says it failed.\"\"\"\n    if event.get(\"status\") != \"error\":\n        return None\n    output = str(event.get(\"output\") or event.get(\"text\") or \"\")\n    # Most specific shape first; a bare non-zero exit is the fallback name.\n    for name, pattern in FAILURE_PATTERNS.items():\n        if name != \"nonzero-exit\" and pattern.search(output):\n            return name\n    if FAILURE_PATTERNS[\"nonzero-exit\"].search(output):\n        return \"nonzero-exit\"\n    return \"tool-error\"\n\n\ndef excerpt_for(kind: str, output: str) -> str:\n    pattern = FAILURE_PATTERNS.get(kind)\n    if pattern is not None:\n        match = pattern.search(output)\n        if match:\n            line_start = output.rfind(\"\\n\", 0, match.start()) + 1\n            line_end = output.find(\"\\n\", match.end())\n            line = output[line_start : line_end if line_end != -1 else None]\n            rest = output[line_end + 1 :] if line_end != -1 else \"\"\n            # The matched line plus the next non-empty line: \"Exit code 1\"\n            # alone would collapse every distinct failure into one.\n            follow = next((ln for ln in rest.splitlines() if ln.strip()), \"\")\n            return \" \".join(f\"{line} {follow}\".split())[:EXCERPT_CHARS]\n    return \" \".join(output.split())[:EXCERPT_CHARS]\n\n\ndef mine_trace(traces_bin: str, trace: dict) -> list[dict]:\n    \"\"\"Failures in one session, each paired with the command that caused it\n    and deduplicated by (tool, excerpt) with an occurrence count.\"\"\"\n    calls: dict[str, dict] = {}\n    failures: dict[tuple[str, str], dict] = {}\n    for event in iter_events(traces_bin, trace[\"id\"]):\n        etype = event.get(\"type\")\n        if etype == \"tool_call\":\n            calls[str(event.get(\"callId\"))] = event\n            continue\n        if etype not in {\"tool_result\", \"error\"}:\n            continue\n        kind = failure_kind(event)\n        if kind is None:\n            continue\n        output = str(event.get(\"output\") or event.get(\"text\") or \"\")\n        tool = str(event.get(\"toolName\") or \"\")\n        call = calls.get(str(event.get(\"callId\")), {})\n        args = call.get(\"args\") or {}\n        command = str(args.get(\"command\") or args.get(\"file_path\") or args.get(\"pattern\") or \"\")\n        excerpt = excerpt_for(kind, output)\n        # The command is part of identity: two commands with the same output\n        # are two failures, and classification reads the command\n        # (Codex review of PR #10, round 28).\n        key = (tool, \" \".join(command.split())[:EXCERPT_CHARS], excerpt)\n        if key in failures:\n            failures[key][\"count\"] += 1\n            continue\n        failures[key] = {\n            \"_text\": f\"{command} {output}\".lower(),\n            \"trace_id\": trace[\"id\"],\n            \"agent\": trace.get(\"agentId\"),\n            \"event_number\": event.get(\"eventNumber\"),\n            \"timestamp\": event.get(\"timestamp\"),\n            \"tool\": tool,\n            \"kind\": kind,\n            \"command\": \" \".join(command.split())[:EXCERPT_CHARS],\n            \"excerpt\": excerpt,\n            \"count\": 1,\n        }\n    return sorted(failures.values(), key=lambda f: (f[\"event_number\"] or 0))\n\n\ndef failure_text(failure: dict) -> str:\n    \"\"\"The text a topic is matched against: the full command and the full\n    tool output while mining (kept in memory under a private key and never\n    written out), so a keyword past the excerpt still counts; the stored\n    command and excerpt for records read back from disk (Codex full-branch\n    review, finding 3).\"\"\"\n    full = failure.get(\"_text\")\n    if isinstance(full, str):\n        return full\n    return f\"{failure['command']} {failure['excerpt']}\".lower()\n\n\ndef public_failure(failure: dict) -> dict:\n    return {k: v for k, v in failure.items() if not k.startswith(\"_\")}\n\n\ndef matching_topics(failure: dict, keywords: dict[str, list[str]]) -> list[str]:\n    \"\"\"Every topic whose keywords appear in the failure, independently of\n    taxonomy order, so a stored count never depends on which other topics\n    existed when it was collected (Codex review of PR #10, round 27).\"\"\"\n    text = failure_text(failure)\n    return [topic for topic, words in keywords.items() if any(w.lower() in text for w in words)]\n\n\ndef classify(failure: dict, keywords: dict[str, list[str]]) -> str | None:\n    text = f\"{failure['command']} {failure['excerpt']}\"\n    return policy_mod.classify_finding(text, keywords)\n\n\ndef build_evidence(\n    failures: list[dict],\n    keywords: dict[str, list[str]],\n    repo_dir: str,\n    agents: list[str] | None,\n    complete: bool = True,\n    scanned: list[str] | None = None,\n    namespace: str | None = None,\n) -> dict:\n    per_topic: dict[str, dict[str, dict]] = defaultdict(dict)\n    for failure in failures:\n        for topic in matching_topics(failure, keywords):\n            per_topic[topic].setdefault(\n                failure[\"trace_id\"],\n                {\n                    \"id\": failure[\"trace_id\"],\n                    \"agentId\": failure[\"agent\"],\n                    \"timestamp\": failure[\"timestamp\"],\n                },\n            )\n    return {\n        \"source\": \"trace-failures\",\n        \"collected_at\": policy_mod.utc_now_iso(),\n        \"repo_dir\": repo_dir,\n        \"agents\": agents or [\"all\"],\n        \"event_types\": \"tool_result(status=error) or failure-shaped output\",\n        \"definitions\": {topic: list(words) for topic, words in keywords.items()},\n        \"topics\": {\n            topic: sorted(\n                per_topic.get(topic, {}).values(), key=lambda t: (t[\"timestamp\"] or 0, t[\"id\"])\n            )\n            for topic in keywords\n        },\n        \"truncated\": [] if complete else list(keywords),\n        \"listing_complete\": complete,\n        \"failure_count\": len(failures),\n        # Every session scanned, matched or not: a complete scan with only\n        # unclassified failures is still an observed field (round 36).\n        \"sessions\": sorted(set(scanned or []) | {f[\"trace_id\"] for f in failures}),\n        \"namespace\": namespace,\n    }\n\n\ndef report(failures: list[dict], keywords: dict[str, list[str]]) -> tuple[list[str], dict]:\n    lines: list[str] = []\n    by_kind = Counter(f[\"kind\"] for f in failures)\n    by_topic: Counter[str] = Counter()\n    blind: list[dict] = []\n    for failure in failures:\n        topics = matching_topics(failure, keywords)\n        if not topics:\n            blind.append(failure)\n        for topic in topics:\n            by_topic[topic] += 1\n    sessions = {f[\"trace_id\"] for f in failures}\n    lines.append(f\"{len(failures)} distinct failure(s) across {len(sessions)} session(s)\")\n    for kind, n in by_kind.most_common():\n        lines.append(f\"  {kind}: {n}\")\n    lines.append(\"by policy topic (failures whose command or output matches the topic's keywords):\")\n    for topic in keywords:\n        lines.append(f\"  [{topic}] {by_topic.get(topic, 0)}\")\n    lines.append(f\"unclassified failures (field blind spots): {len(blind)}\")\n    for failure in blind[:12]:\n        lines.append(\n            f\"  {failure['trace_id'][:8]} #{failure['event_number']} {failure['tool']} ({failure['kind']}, x{failure['count']}): {failure['excerpt'][:110]}\"\n        )\n    return lines, {\n        \"failures\": [public_failure(f) for f in failures],\n        \"by_kind\": dict(by_kind),\n        \"by_topic\": {t: by_topic.get(t, 0) for t in keywords},\n        \"blind_spots\": [public_failure(f) for f in blind],\n        \"sessions\": sorted(sessions),\n    }\n\n\ndef main(argv: list[str]) -> int:\n    parser = argparse.ArgumentParser(\n        description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter\n    )\n    where = parser.add_mutually_exclusive_group(required=True)\n    where.add_argument(\"--repo-dir\", help=\"mine the local sessions recorded in this folder\")\n    where.add_argument(\n        \"--namespace\",\n        help=\"mine every session shared to this traces.com namespace (e.g. @gagan114); \"\n        \"for runners with no local sessions\",\n    )\n    parser.add_argument(\n        \"--traces-key\",\n        default=None,\n        help=\"API key passed as --key to every traces call (CI); omit when logged in locally\",\n    )\n    parser.add_argument(\"--agents\", default=\",\".join(measure_mod.DEFAULT_ANCHOR_AGENTS))\n    parser.add_argument(\"--policy\", default=None)\n    parser.add_argument(\"--history\", default=str(policy_mod.HISTORY_PATH))\n    parser.add_argument(\"--out-json\", default=None)\n    parser.add_argument(\"--save-evidence\", default=None)\n    parser.add_argument(\"--traces-bin\", default=\"traces\")\n    parser.add_argument(\"--limit\", type=int, default=200)\n    args = parser.parse_args(argv[1:])\n\n    policy = (\n        policy_mod.load_policy(args.policy) if args.policy else policy_mod.load_policy_or_builtin()\n    )\n    inputs = [args.policy, args.history]\n    if args.out_json:\n        policy_mod.assert_safe_output(args.out_json, inputs=inputs)\n    if args.save_evidence:\n        policy_mod.assert_safe_output(args.save_evidence, inputs=inputs, kind=\"evidence\")\n    if (\n        args.out_json\n        and args.save_evidence\n        and Path(args.out_json).resolve() == Path(args.save_evidence).resolve()\n    ):\n        print(\"::error::--out-json and --save-evidence must be different files\", file=sys.stderr)\n        return 1\n\n    # Two keyword sets with two jobs: blind spots are what the CURRENT policy\n    # cannot classify (a retired topic must not keep hiding fresh failures\n    # from mining), while evidence keeps counting every definition any\n    # policy version ever had, so a rolled-back topic retains the adverse\n    # evidence that blocks re-mining it (Codex review of PR #10, round 34).\n    current_keywords = dict(policy_mod.topic_keywords(policy))\n    search_keywords = dict(current_keywords)\n    search_keywords.update(\n        measure_mod.historical_definitions(policy_mod.load_history(args.history), current_keywords)\n    )\n    agents = (\n        None\n        if args.agents.strip() == \"all\"\n        else [a.strip() for a in args.agents.split(\",\") if a.strip()]\n    )\n\n    if args.traces_key:\n        EXTRA_CLI_ARGS[:] = [\"--key\", args.traces_key]\n\n    try:\n        if args.namespace:\n            traces, complete = list_namespace_traces(args.traces_bin, args.namespace, agents)\n        else:\n            traces, complete = list_traces(args.traces_bin, args.repo_dir, agents, args.limit)\n        if not complete:\n            print(\n                f\"::warning::session listing hit --limit {args.limit}; evidence counts are marked unknown, raise --limit\"\n            )\n        failures: list[dict] = []\n        for trace in traces:\n            if agents is None and trace.get(\"agentId\") == VERIFIER_AGENT:\n                pass  # \"all\" deliberately includes the verifier's own sessions\n            if args.n…1037 tokens truncated…ies: list[dict], anchor: dict | None) -> float | None:\nscripts/revise-improvement-policy.py:343:def entries_covered_by_evidence(entries: list[dict], measurement: dict) -> list[dict]:\nscripts/revise-improvement-policy.py:361:def validity_regressed(before: float | None, after: float | None) -> bool:\nscripts/revise-improvement-policy.py:371:def weight_repair(\nscripts/revise-improvement-policy.py:384:    anchor = candidate_anchor(current, policy)\nscripts/revise-improvement-policy.py:415:    after_v = validity_under(candidate, entries, candidate_anchor(current, candidate))\nscripts/revise-improvement-policy.py:423:def candidate_anchor(current: dict, policy: dict) -> dict | None:\nscripts/revise-improvement-policy.py:463:def rolled_back_hashes(history: list[dict]) -> set[str]:\nscripts/revise-improvement-policy.py:466:    restored v1 would otherwise be judged worse than v2 on coverage two\nscripts/revise-improvement-policy.py:475:def rejected_configuration(candidate: dict, history: list[dict], measurement: dict) -> dict | None:\nscripts/revise-improvement-policy.py:491:def unjudged_ancestors(policy: dict, history: list[dict]) -> list[dict]:\nscripts/revise-improvement-policy.py:494:    and including the first non-revision ancestor. Later evidence must be\nscripts/revise-improvement-policy.py:495:    able to expose a harmful ancestor that a newer revision or a rollback\nscripts/revise-improvement-policy.py:500:    version = judged_from(policy, history)\nscripts/revise-improvement-policy.py:508:            version = judged_from(snapshot, history)\nscripts/revise-improvement-policy.py:516:def judged_from(policy: dict, history: list[dict]) -> int | None:\nscripts/revise-improvement-policy.py:528:def entries_under(entries: list[dict], policy: dict) -> list[dict]:\nscripts/revise-improvement-policy.py:531:    of PR #10, round 25: older rounds decided by an ancestor must not enter\nscripts/revise-improvement-policy.py:540:def rounds_under(entries: list[dict], policy: dict) -> int:\nscripts/revise-improvement-policy.py:561:def snapshot_for_version(version: int, history: list[dict]) -> dict | None:\nscripts/revise-improvement-policy.py:570:def adoption_entry(version: int, history: list[dict]) -> dict | None:\nscripts/revise-improvement-policy.py:577:def field_blind_spots(field_failures: dict | None, keywords: dict[str, list[str]]) -> list[dict]:\nscripts/revise-improvement-policy.py:609:def decide(\nscripts/revise-improvement-policy.py:620:    coverage = current.get(\"coverage\")\nscripts/revise-improvement-policy.py:626:    #    re-measured on the SAME findings: comparing today's coverage with\nscripts/revise-improvement-policy.py:630:    base_version = judged_from(policy, history)\nscripts/revise-improvement-policy.py:635:            if parent is not None and coverage is not None:\nscripts/revise-improvement-policy.py:642:                # full-archive `coverage` that the trigger and candidate\nscripts/revise-improvement-policy.py:645:                parent_now = measure_mod.measure(own_rounds, parent, None)[\"current\"][\"coverage\"]\nscripts/revise-improvement-policy.py:646:                own_coverage = measure_mod.measure(own_rounds, policy, None)[\"current\"][\"coverage\"]\nscripts/revise-improvement-policy.py:647:                child_validity = validity_under(policy, covered, candidate_anchor(current, policy))\nscripts/revise-improvement-policy.py:649:                worse_coverage = (\nscripts/revise-improvement-policy.py:651:                    and own_coverage is not None\nscripts/revise-improvement-policy.py:652:                    and own_coverage < parent_now\nscripts/revise-improvement-policy.py:655:                # Compare against every ancestor in the unjudged chain, not\nscripts/revise-improvement-policy.py:656:                # only the parent: the best-scoring ancestor is the rollback\nscripts/revise-improvement-policy.py:660:                # Coverage is compared with every ancestor too: a topic a\nscripts/revise-improvement-policy.py:666:                for ancestor in unjudged_ancestors(policy, history):\nscripts/revise-improvement-policy.py:667:                    if policy_mod.policy_hash(ancestor) in rejected:\nscripts/revise-improvement-policy.py:669:                    v = validity_under(ancestor, covered, candidate_anchor(current, ancestor))\nscripts/revise-improvement-policy.py:671:                        best, best_validity = ancestor, v\nscripts/revise-improvement-policy.py:672:                    c = measure_mod.measure(own_rounds, ancestor, None)[\"current\"][\"coverage\"]\nscripts/revise-improvement-policy.py:674:                        best_cov, best_cov_value = ancestor, c\nscripts/revise-improvement-policy.py:677:                    and own_coverage is not None\nscripts/revise-improvement-policy.py:678:                    and best_cov_value > own_coverage\nscripts/revise-improvement-policy.py:680:                    worse_coverage = True\nscripts/revise-improvement-policy.py:685:                if worse_coverage or worse_validity:\nscripts/revise-improvement-policy.py:686:                    target = parent if worse_coverage else best\nscripts/revise-improvement-policy.py:690:                    target_coverage = measure_mod.measure(entries, target, None)[\"current\"][\nscripts/revise-improvement-policy.py:691:                        \"coverage\"\nscripts/revise-improvement-policy.py:694:                        f\"coverage {own_coverage} vs {parent_now}\"\nscripts/revise-improvement-policy.py:695:                        if worse_coverage\nscripts/revise-improvement-policy.py:713:                        \"coverage_before\": own_coverage,\nscripts/revise-improvement-policy.py:714:                        \"coverage_after\": target_coverage,\nscripts/revise-improvement-policy.py:716:                        \"validity_after\": best_validity if not worse_coverage else None,\nscripts/revise-improvement-policy.py:727:    # The wait does not depend on having an ancestor to compare with: a\nscripts/revise-improvement-policy.py:742:    if coverage is not None and coverage < MIN_COVERAGE:\nscripts/revise-improvement-policy.py:743:        triggers.append(f\"coverage {coverage} < {MIN_COVERAGE}\")\nscripts/revise-improvement-policy.py:747:    coverage_trigger = coverage is not None and coverage < MIN_COVERAGE\nscripts/revise-improvement-policy.py:759:    if field_trigger or coverage_trigger:\nscripts/revise-improvement-policy.py:761:    mined = mine_topics(mining_input, keywords) if (coverage_trigger or field_trigger) else []\nscripts/revise-improvement-policy.py:817:    v_before = validity_under(policy, covered, candidate_anchor(current, policy))\nscripts/revise-improvement-policy.py:818:    v_after = validity_under(revised, covered, candidate_anchor(current, revised))\nscripts/revise-improvement-policy.py:838:    if coverage is not None and after[\"coverage\"] is not None and after[\"coverage\"] < coverage:\nscripts/revise-improvement-policy.py:841:            \"reason\": f\"candidate revision would lower coverage {coverage} -> {after['coverage']}; refused\",\nscripts/revise-improvement-policy.py:851:        \"coverage_before\": coverage,\nscripts/revise-improvement-policy.py:852:        \"coverage_after\": after[\"coverage\"],\nscripts/revise-improvement-policy.py:860:def history_entry(decision: dict, policy: dict, measurement: dict, now: str) -> dict:\nscripts/revise-improvement-policy.py:869:        \"coverage_before\": decision.get(\"coverage_before\"),\nscripts/revise-improvement-policy.py:870:        \"coverage_after\": decision.get(\"coverage_after\"),\nscripts/revise-improvement-policy.py:883:def main(argv: list[str]) -> int:\nscripts/revise-improvement-policy.py:956:        if decision.get(\"coverage_after\") is not None:\nscripts/revise-improvement-policy.py:957:            print(f\"  coverage {decision['coverage_before']} -> {decision['coverage_after']}\")\nscripts/render_rsi_dashboard_test.py:39:def test_renders_every_section_from_real_shapes(tmp_path):\nscripts/render_rsi_dashboard_test.py:130:def test_revision_markers_sit_at_the_epoch_they_were_created_after():\nscripts/render_rsi_dashboard_test.py:143:def test_evidence_strings_are_escaped_in_the_echo_note(tmp_path):\nscripts/render_rsi_dashboard_test.py:181:def test_trigger_chart_plots_current_policy_validity():\nscripts/render_rsi_dashboard_test.py:193:def test_head_label_is_escaped_in_the_loop_diagram():\nscripts/render_rsi_dashboard_test.py:209:def test_out_may_not_overwrite_protected_or_input_files(tmp_path):\nscripts/render_rsi_dashboard_test.py:236:def test_history_version_labels_are_escaped_in_the_chart():\nscripts/render_rsi_dashboard_test.py:257:def test_reproduce_commands_only_use_flags_their_scripts_define():\nscripts/render_rsi_dashboard_test.py:272:def test_negative_validity_is_drawn_below_zero_not_clamped_onto_it():\nscripts/archive_round_test.py:26:def test_appends_new_round_and_tags_it_with_source_sha(tmp_path):\nscripts/archive_round_test.py:46:def test_target_reflects_the_passed_argument_not_a_hardcoded_file(tmp_path):\nscripts/archive_round_test.py:74:def test_target_defaults_to_something_generic_when_not_passed(tmp_path):\nscripts/archive_round_test.py:86:def test_rerunning_with_same_source_sha_does_not_duplicate(tmp_path):\nscripts/archive_round_test.py:108:def test_cross_pr_accumulation_crosses_threshold_on_the_third_contributing_round(tmp_path, capsys):\nscripts/archive_round_test.py:145:def test_clean_review_is_persisted_as_a_stamped_round(tmp_path, capsys):\nscripts/archive_round_test.py:187:def test_reviews_that_did_not_complete_are_not_archived_as_clean_rounds(tmp_path, capsys, body):\nscripts/archive_round_test.py:215:def test_a_failed_review_that_still_contains_numbered_lines_is_not_archived(tmp_path):\nscripts/mine_trace_failures_test.py:114:def test_mines_failed_and_failure_shaped_tool_results_only(monkeypatch):\nscripts/mine_trace_failures_test.py:127:def test_pagination_walks_every_page(monkeypatch):\nscripts/mine_trace_failures_test.py:146:def test_evidence_counts_sessions_per_topic_and_records_definitions(monkeypatch):\nscripts/mine_trace_failures_test.py:164:def test_main_writes_report_and_evidence(tmp_path, monkeypatch, capsys):\nscripts/mine_trace_failures_test.py:191:def test_cli_json_is_parsed_past_a_leading_notice():\nscripts/mine_trace_failures_test.py:199:def test_line_numbers_and_file_contents_are_not_failures():\nscripts/mine_trace_failures_test.py:246:def test_capped_listing_marks_every_topic_unknown(monkeypatch):\nscripts/mine_trace_failures_test.py:255:def test_evidence_matches_every_topic_independently_of_order():\nscripts/mine_trace_failures_test.py:275:def test_same_output_from_different_commands_are_distinct_failures(monkeypatch):\nscripts/mine_trace_failures_test.py:320:def test_zero_failed_is_not_a_test_failure():\nscripts/mine_trace_failures_test.py:337:def test_retired_topics_count_as_evidence_but_do_not_hide_blind_spots(\nscripts/mine_trace_failures_test.py:383:def test_evidence_records_every_scanned_session_even_without_matches(monkeypatch):\nscripts/mine_trace_failures_test.py:417:def test_namespace_mode_syncs_each_shared_session_before_reading_it(tmp_path, monkeypatch, capsys):\nscripts/revise_improvement_policy_test.py:55:def test_no_revision_when_coverage_and_validity_hold():\nscripts/revise_improvement_policy_test.py:62:def test_low_coverage_triggers_a_bounded_mined_revision():\nscripts/revise_improvement_policy_test.py:81:def test_low_validity_discounts_topics_the_field_never_shows():\nscripts/revise_improvement_policy_test.py:110:def test_rollback_when_an_adopted_revision_is_worse_than_its_parent():\nscripts/revise_improvement_policy_test.py:140:def test_no_rollback_when_unfamiliar_findings_lower_both_policies():\nscripts/revise_improvement_policy_test.py:175:def test_mined_topics_never_share_a_supporting_finding():\nscripts/revise_improvement_policy_test.py:240:def test_weight_repair_is_dropped_when_it_would_lower_validity():\nscripts/revise_improvement_policy_test.py:256:def test_reweighting_that_undefines_validity_is_refused():\nscripts/revise_improvement_policy_test.py:275:def test_corroborated_topic_regains_weight_even_when_scores_are_healthy():\nscripts/revise_improvement_policy_test.py:296:def test_rollback_on_validity_regression_with_same_coverage():\nscripts/revise_improvement_policy_test.py:321:def test_main_refuses_a_measurement_from_a_different_archive(tmp_path, capsys):\nscripts/revise_improvement_policy_test.py:349:def test_rollback_waits_for_rounds_decided_under_the_revision():\nscripts/revise_improvement_policy_test.py:389:def test_no_new_revision_until_the_current_one_has_been_judged():\nscripts/revise_improvement_policy_test.py:420:def test_discounts_wait_for_fresh_evidence():\nscripts/revise_improvement_policy_test.py:431:def test_rounds_under_requires_matching_version_not_just_contents():\nscripts/revise_improvement_policy_test.py:450:def test_validity_rollback_ignores_rounds_newer_than_the_evidence():\nscripts/revise_improvement_policy_test.py:477:def test_candidate_acceptance_and_rollback_use_the_same_evidence_window():\nscripts/revise_improvement_policy_test.py:488:def test_rolled_back_configuration_is_not_retried_on_the_same_evidence():\nscripts/revise_improvement_policy_test.py:543:def test_evidence_against_a_removed_topic_survives_for_candidates():\nscripts/revise_improvement_policy_test.py:562:def test_main_writes_nothing_when_the_history_path_is_refused(tmp_path, monkeypatch):\nscripts/revise_improvement_policy_test.py:591:def test_out_json_may_not_target_a_protected_or_input_file(tmp_path):\nscripts/revise_improvement_policy_test.py:617:def test_mined_topic_names_never_collide_with_existing_topics():\nscripts/revise_improvement_policy_test.py:632:def test_late_evidence_rolls_back_past_an_unjudged_parent_to_the_better_ancestor():\nscripts/revise_improvement_policy_test.py:685:def test_report_outputs_may_not_overwrite_canonical_evidence(tmp_path):\nscripts/revise_improvement_policy_test.py:713:def test_ancestry_continues_through_a_rollback():\nscripts/revise_improvement_policy_test.py:765:def test_current_definition_mismatch_does_not_erase_an_ancestors_evidence():\nscripts/revise_improvement_policy_test.py:779:def test_out_policy_may_not_be_the_history_file(tmp_path, monkeypatch):\nscripts/revise_improvement_policy_test.py:815:def test_rollback_judges_a_revision_only_on_its_own_rounds():\nscripts/revise_improvement_policy_test.py:841:def test_rollback_check_does_not_clobber_full_archive_coverage():\nscripts/revise_improvement_policy_test.py:896:def test_out_json_may_not_overwrite_the_field_failures_input(tmp_path):\nscripts/revise_improvement_policy_test.py:926:def test_accepted_revision_never_regresses_validity():\nscripts/revise_improvement_policy_test.py:936:def test_main_refuses_a_measurement_taken_under_a_different_policy(tmp_path, capsys):\nscripts/revise_improvement_policy_test.py:964:def test_main_writes_only_ai_owned_files(tmp_path, capsys, monkeypatch):\nscripts/revise_improvement_policy_test.py:1020:def test_failure_kind_labels_do_not_hide_field_blind_spots():\nscripts/revise_improvement_policy_test.py:1035:def test_candidate_anchor_reaches_an_older_definition_of_a_reused_name():\nscripts/revise_improvement_policy_test.py:1052:def test_candidate_anchor_never_borrows_a_count_from_another_definition():\nscripts/revise_improvement_policy_test.py:1082:def test_failure_kind_labels_cannot_become_a_mined_topic():\nscripts/revise_improvement_policy_test.py:1102:def test_swapped_policy_and_history_destinations_are_refused_before_any_write(\nscripts/revise_improvement_policy_test.py:1158:def test_rollback_to_the_root_version_still_waits_before_being_judged():\nscripts/revise_improvement_policy_test.py:1198:def test_a_rolled_back_configuration_is_never_a_rollback_target():\nscripts/revise_improvement_policy_test.py:1236:def test_coverage_lost_at_a_grandparent_still_triggers_rollback():\nscripts/revise_improvement_policy_test.py:1277:def test_main_refuses_a_policy_whose_lineage_metadata_was_edited(tmp_path, monkeypatch, capsys):\nscripts/revise_improvement_policy_test.py:1315:def test_evidence_window_filters_by_round_identity_not_number():\nscripts/measure_policy_validity_test.py:72:def test_rounds_merge_pending_placeholders_and_carry_timestamps():\nscripts/measure_policy_validity_test.py:79:def test_coverage_counts_unclassified_findings():\nscripts/measure_policy_validity_test.py:90:def test_anchor_is_replayed_per_epoch_by_timestamp():\nscripts/measure_policy_validity_test.py:100:def test_validity_is_rank_agreement_between_review_signal_and_anchor():\nscripts/measure_policy_validity_test.py:113:def test_dev_only_topics_flag_review_credit_the_field_never_corroborates():\nscripts/measure_policy_validity_test.py:120:def test_unsearched_topics_are_unknown_not_zero():\nscripts/measure_policy_validity_test.py:130:def test_validity_uses_the_weighted_signal_the_detector_decides_on():\nscripts/measure_policy_validity_test.py:140:def test_replay_orders_rounds_by_time_not_round_number():\nscripts/measure_policy_validity_test.py:153:def test_measurement_is_bound_to_the_archive_contents():\nscripts/measure_policy_validity_test.py:160:def test_truncated_searches_are_unknown_not_absolute():\nscripts/measure_policy_validity_test.py:169:def test_findings_newer_than_the_evidence_do_not_mark_topics_dev_only():\nscripts/measure_policy_validity_test.py:187:def test_empty_anchor_is_treated_as_no_anchor():\nscripts/measure_policy_validity_test.py:200:def test_main_reports_and_emits_json(tmp_path, capsys):\nscripts/measure_policy_validity_test.py:220:def test_out_json_and_newline_safe_report(tmp_path, capsys):\nscripts/measure_policy_validity_test.py:235:def test_evidence_searched_under_a_different_definition_is_unknown():\nscripts/measure_policy_validity_test.py:250:def test_side_outputs_may_not_overwrite_protected_files(tmp_path):\nscripts/measure_policy_validity_test.py:264:def test_epoch_without_a_timestamp_has_unknown_evidence_not_all_of_it():\nscripts/measure_policy_validity_test.py:275:def test_evidence_refresh_keeps_searching_topics_from_earlier_policy_versions():\nscripts/measure_policy_validity_test.py:288:def test_undated_traces_make_historical_counts_unknown_but_not_current_ones():\nscripts/measure_policy_validity_test.py:299:def test_history_path_counts_as_an_input_for_output_guards(tmp_path):\nscripts/measure_policy_validity_test.py:323:def test_reused_topic_name_keeps_every_historical_definition():\nscripts/measure_policy_validity_test.py:354:def test_anchor_counts_use_the_evidence_searched_under_the_topics_own_definition():\nscripts/measure_policy_validity_test.py:376:def test_a_scan_that_matched_nothing_is_still_an_observed_field():\nscripts/measure_policy_validity_test.py:390:def test_two_reviews_that_collided_on_a_round_number_stay_two_rounds():\nscripts/improvement_policy_test.py:21:def test_checked_in_policy_validates_and_descends_from_the_builtin_v1():\nscripts/improvement_policy_test.py:36:def test_policy_hash_ignores_metadata_but_tracks_decision_fields():\nscripts/improvement_policy_test.py:45:def test_policy_hash_is_sensitive_to_topic_order():\nscripts/improvement_policy_test.py:51:def test_history_snapshots_preserve_topic_order(tmp_path):\nscripts/improvement_policy_test.py:62:def test_classify_uses_policy_order_and_returns_none_when_uncovered():\nscripts/improvement_policy_test.py:71:def test_new_version_links_to_parent_and_validates():\nscripts/improvement_policy_test.py:97:def test_attribution_guard_refuses_writes_outside_ai_owned_files(tmp_path):\nscripts/improvement_policy_test.py:115:def test_history_round_trips(tmp_path):\nscripts/improvement_policy_test.py:128:def test_validate_policy_rejects_empty_keyword_lists():\nscripts/improvement_policy_test.py:135:def test_topic_names_may_not_contain_at_signs_and_origin_is_checked():\nscripts/improvement_policy_test.py:146:def test_hard_linked_outputs_are_refused(tmp_path, monkeypatch):\n"}

>rg -n '^def |candidate_anchor|judged_from|ancestor|coverage|complete|evidence_trace_ids' scripts/revise-improvement-policy.py scripts/measure-policy-validity.py; rg -n '^def test_' scripts/*policy*test.py scripts/mine_trace_failures_test.py scripts/render_rsi_dashboard_test.py scripts/archive_round_test.py

{"chunk_id":"429266","wall_time_seconds":0.000001541,"exit_code":0,"original_token_count":5069,"output":"#!/usr/bin/env python3\n\"\"\"Mine actual failures out of agent session traces, so the field anchor\ncounts things that went wrong, never things people said.\n\nThe first field anchor searched transcript text for the taxonomy's\nkeywords. Every hit turned out to be narration — the assistant summarising\nreview findings — so the \"field\" merely echoed the reviews it was supposed\nto check. This tool reads the events of each working session directly\nthrough `traces show --json` and keeps only:\n\n  * tool results Traces itself marked `status: \"error\"` — nothing else.\n    The failure shape of the output (traceback, non-zero exit, test\n    failure, permission or auth error, git rejection, timeout) only names\n    the kind; displayed text never turns a successful execution into a\n    failure.\n\nEach failure is paired with the command that produced it, deduplicated per\nsession, classified with the current improvement policy, and written out\ntwo ways: a human report plus JSON (`--out-json`), and an evidence file in\nthe shape scripts/measure-policy-validity.py consumes (`--save-evidence`),\nwhere a topic's evidence is the set of sessions in which a failure matching\nthat topic's keywords actually occurred. Failures no topic claims are the\nfield's blind spots — the same kind of signal the meta-improver mines from\nunclassified review findings.\n\nUsage:\n    python3 mine-trace-failures.py --repo-dir DIR [--agents a,b|all]\n        [--policy PATH] [--history PATH] [--out-json PATH] [--save-evidence PATH]\n        [--traces-bin PATH] [--limit N]\n\"\"\"\n\nfrom __future__ import annotations\n\nimport argparse\nimport importlib.util\nimport json\nimport re\nimport subprocess\nimport sys\nimport tempfile\nfrom collections import Counter, defaultdict\nfrom pathlib import Path\n\n\ndef _load_sibling_module(name: str, filename: str):\n    if name in sys.modules:\n        return sys.modules[name]\n    path = Path(__file__).parent / filename\n    spec = importlib.util.spec_from_file_location(name, path)\n    assert spec is not None and spec.loader is not None\n    module = importlib.util.module_from_spec(spec)\n    sys.modules[name] = module\n    spec.loader.exec_module(module)\n    return module\n\n\npolicy_mod = _load_sibling_module(\"improvement_policy\", \"improvement_policy.py\")\nmeasure_mod = _load_sibling_module(\"measure_policy_validity\", \"measure-policy-validity.py\")\n\nVERIFIER_AGENT = \"codex\"\nPAGE_SIZE = 200\nEXCERPT_CHARS = 160\n\n# Failure shapes, each named so a report can say what kind of thing broke.\nFAILURE_PATTERNS: dict[str, re.Pattern[str]] = {\n    \"traceback\": re.compile(r\"Traceback \\(most recent call last\\)\"),\n    \"nonzero-exit\": re.compile(r\"^Exit code [1-9]\\d*\", re.M),\n    # \"0 failed\" is a pass; only a positive count is a failure.\n    \"test-failure\": re.compile(r\"\\b[1-9]\\d* failed\\b|^FAILED \", re.M),\n    \"permission\": re.compile(\n        r\"Permission denied|EACCES|denied by the .* classifier|Operation not permitted\"\n    ),\n    # HTTP-shaped only: a bare \"401\" is far more often a line number in a\n    # file read than an auth failure.\n    \"auth\": re.compile(\n        r\"HTTP/?[\\d.]* ?40[13]\\b|\\b40[13] (?:Unauthorized|Forbidden)|status(?: code)?[:=]? ?40[13]\\b\"\n        r\"|\\bUnauthorized\\b|\\bForbidden\\b|token (?:expired|invalid)|authentication failed\",\n        re.I,\n    ),\n    \"git-rejected\": re.compile(r\"non-fast-forward|rejected\\]|^fatal: |merge conflict\", re.I | re.M),\n    \"timeout\": re.compile(r\"timed out|timeout of \\d+|ETIMEDOUT|TLS handshake timeout\", re.I),\n    \"not-found\": re.compile(r\"No such file or directory|command not found|ENOENT\", re.I),\n}\n\n\nclass TracesCliError(RuntimeError):\n    pass\n\n\ndef parse_cli_json(stdout: str) -> dict | None:\n    \"\"\"The CLI may print a hydration notice before the JSON document the\n    first time a session's events are loaded; parse from the first brace.\"\"\"\n    for candidate in (stdout, stdout[stdout.find(\"{\") :] if \"{\" in stdout else \"\"):\n        if not candidate:\n            continue\n        try:\n            return json.loads(candidate)\n        except json.JSONDecodeError:\n            continue\n    return None\n\n\n# Extra arguments appended to every CLI call (an API key on a runner that is\n# not logged in). Set once by main(); never printed.\nEXTRA_CLI_ARGS: list[str] = []\n\n\ndef run_traces_json(traces_bin: str, args: list[str], *, retries: int = 1) -> dict:\n    last_error = \"no output\"\n    for attempt in range(retries + 1):\n        # stdout goes to a file, not a pipe: the CLI truncates piped output at\n        # 64 KiB (observed: 65519 bytes of an 80 KB document), while a file\n        # redirect receives everything.\n        try:\n            with tempfile.TemporaryFile(mode=\"w+\", encoding=\"utf-8\") as out:\n                result = subprocess.run(\n                    [traces_bin, *args, *EXTRA_CLI_ARGS, \"--json\"],\n                    stdout=out,\n                    stderr=subprocess.PIPE,\n                    text=True,\n                    timeout=300,\n                )\n                out.seek(0)\n                stdout = out.read()\n        except OSError as exc:\n            raise TracesCliError(f\"Could not run `{traces_bin}`: {exc}\") from exc\n        if result.returncode != 0:\n            raise TracesCliError(f\"`{traces_bin} {' '.join(args)}` failed: {result.stderr.strip()}\")\n        payload = parse_cli_json(stdout)\n        if payload is not None:\n            if not payload.get(\"ok\"):\n                raise TracesCliError(f\"`{traces_bin} {' '.join(args)}` reported failure: {payload}\")\n            return payload[\"data\"]\n        last_error = stdout[:120].replace(\"\\n\", \" \")\n        if attempt < retries:\n            continue\n    raise TracesCliError(f\"Non-JSON output from `{traces_bin} {' '.join(args)}`: {last_error}\")\n\n\ndef list_traces(\n    traces_bin: str, repo_dir: str, agents: list[str] | None, limit: int\n) -> tuple[list[dict], bool]:\n    \"\"\"Sessions recorded in the folder, and whether the listing was complete.\n    A listing that fills its limit may have missed sessions, and evidence\n    built from it must say so rather than report confirmed zero counts\n    (Codex review of PR #10, round 27).\"\"\"\n    found: dict[str, dict] = {}\n    complete = True\n    for agent_args in [[\"--agent\", a] for a in agents] if agents else [[]]:\n        data = run_traces_json(\n            traces_bin, [\"list\", \"--dir\", repo_dir, *agent_args, \"--limit\", str(limit)]\n        )\n        traces = data.get(\"traces\", [])\n        if len(traces) >= limit:\n            complete = False\n        for trace in traces:\n            found[trace[\"id\"]] = trace\n    return sorted(found.values(), key=lambda t: (t.get(\"timestamp\") or 0, t[\"id\"])), complete\n\n\ndef list_namespace_traces(\n    traces_bin: str, namespace: str, agents: list[str] | None\n) -> tuple[list[dict], bool]:\n    \"\"\"Sessions shared to a traces.com namespace (for a runner that has no\n    local sessions of its own). `--all` returns the whole namespace, so the\n    listing is complete by construction; agents are filtered client-side.\"\"\"\n    if not namespace.startswith(\"@\"):\n        namespace = f\"@{namespace}\"\n    data = run_traces_json(traces_bin, [\"list\", namespace, \"--all\"])\n    traces = [\n        t\n        for t in data.get(\"traces\", [])\n        if isinstance(t.get(\"id\"), str) and (agents is None or t.get(\"agentId\") in agents)\n    ]\n    return sorted(traces, key=lambda t: (t.get(\"timestamp\") or 0, t[\"id\"])), True\n\n\ndef sync_trace(traces_bin: str, trace_id: str) -> None:\n    \"\"\"Pull a shared trace's events into the local database so `show` can\n    page through them (a fresh runner has none).\"\"\"\n    run_traces_json(traces_bin, [\"sync\", trace_id])\n\n\ndef iter_events(traces_bin: str, trace_id: str):\n    offset = 1\n    while True:\n        data = run_traces_json(\n            traces_bin, [\"show\", trace_id, \"--offset\", str(offset), \"--limit\", str(PAGE_SIZE)]\n        )\n        events = data.get(\"events\") or []\n        yield from events\n        if len(events) < PAGE_SIZE:\n            return\n        offset += len(events)\n\n\ndef failure_kind(event: dict) -> str | None:\n    \"\"\"A failure is an execution the tool itself reported as an error\n    (`status: \"error\"`). Output text never decides whether something\n    failed — a displayed transcript can contain \"Exit code 1\" or \"HTTP 401\"\n    verbatim (Codex review of PR #10, rounds 27-30). The failure shape only\n    names the kind once the status says it failed.\"\"\"\n    if event.get(\"status\") != \"error\":\n        return None\n    output = str(event.get(\"output\") or event.get(\"text\") or \"\")\n    # Most specific shape first; a bare non-zero exit is the fallback name.\n    for name, pattern in FAILURE_PATTERNS.items():\n        if name != \"nonzero-exit\" and pattern.search(output):\n            return name\n    if FAILURE_PATTERNS[\"nonzero-exit\"].search(output):\n        return \"nonzero-exit\"\n    return \"tool-error\"\n\n\ndef excerpt_for(kind: str, output: str) -> str:\n    pattern = FAILURE_PATTERNS.get(kind)\n    if pattern is not None:\n        match = pattern.search(output)\n        if match:\n            line_start = output.rfind(\"\\n\", 0, match.start()) + 1\n            line_end = output.find(\"\\n\", match.end())\n            line = output[line_start : line_end if line_end != -1 else None]\n            rest = output[line_end + 1 :] if line_end != -1 else \"\"\n            # The matched line plus the next non-empty line: \"Exit code 1\"\n            # alone would collapse every distinct failure into one.\n            follow = next((ln for ln in rest.splitlines() if ln.strip()), \"\")\n            return \" \".join(f\"{line} {follow}\".split())[:EXCERPT_CHARS]\n    return \" \".join(output.split())[:EXCERPT_CHARS]\n\n\ndef mine_trace(traces_bin: str, trace: dict) -> list[dict]:\n    \"\"\"Failures in one session, each paired with the command that caused it\n    and deduplicated by (tool, excerpt) with an occurrence count.\"\"\"\n    calls: dict[str, dict] = {}\n    failures: dict[tuple[str, str], dict] = {}\n    for event in iter_events(traces_bin, trace[\"id\"]):\n        etype = event.get(\"type\")\n        if etype == \"tool_call\":\n            calls[str(event.get(\"callId\"))] = event\n            continue\n        if etype not in {\"tool_result\", \"error\"}:\n            continue\n        kind = failure_kind(event)\n        if kind is None:\n            continue\n        output = str(event.get(\"output\") or event.get(\"text\") or \"\")\n        tool = str(event.get(\"toolName\") or \"\")\n        call = calls.get(str(event.get(\"callId\")), {})\n        args = call.get(\"args\") or {}\n        command = str(args.get(\"command\") or args.get(\"file_path\") or args.get(\"pattern\") or \"\")\n        excerpt = excerpt_for(kind, output)\n        # The command is part of identity: two commands with the same output\n        # are two failures, and classification reads the command\n        # (Codex review of PR #10, round 28).\n        key = (tool, \" \".join(command.split())[:EXCERPT_CHARS], excerpt)\n        if key in failures:\n            failures[key][\"count\"] += 1\n            continue\n        failures[key] = {\n            \"_text\": f\"{command} {output}\".lower(),\n            \"trace_id\": trace[\"id\"],\n            \"agent\": trace.get(\"agentId\"),\n            \"event_number\": event.get(\"eventNumber\"),\n            \"timestamp\": event.get(\"timestamp\"),\n            \"tool\": tool,\n            \"kind\": kind,\n            \"command\": \" \".join(command.split())[:EXCERPT_CHARS],\n            \"excerpt\": excerpt,\n            \"count\": 1,\n        }\n    return sorted(failures.values(), key=lambda f: (f[\"event_number\"] or 0))\n\n\ndef failure_text(failure: dict) -> str:\n    \"\"\"The text a topic is matched against: the full command and the full\n    tool output while mining (kept in memory under a private key and never\n    written out), so a keyword past the excerpt still counts; the stored\n    command and excerpt for records read back from disk (Codex full-branch\n    review, finding 3).\"\"\"\n    full = failure.get(\"_text\")\n    if isinstance(full, str):\n        return full\n    return f\"{failure['command']} {failure['excerpt']}\".lower()\n\n\ndef public_failure(failure: dict) -> dict:\n    return {k: v for k, v in failure.items() if not k.startswith(\"_\")}\n\n\ndef matching_topics(failure: dict, keywords: dict[str, list[str]]) -> list[str]:\n    \"\"\"Every topic whose keywords appear in the failure, independently of\n    taxonomy order, so a stored count never depends on which other topics\n    existed when it was collected (Codex review of PR #10, round 27).\"\"\"\n    text = failure_text(failure)\n    return [topic for topic, words in keywords.items() if any(w.lower() in text for w in words)]\n\n\ndef classify(failure: dict, keywords: dict[str, list[str]]) -> str | None:\n    text = f\"{failure['command']} {failure['excerpt']}\"\n    return policy_mod.classify_finding(text, keywords)\n\n\ndef build_evidence(\n    failures: list[dict],\n    keywords: dict[str, list[str]],\n    repo_dir: str,\n    agents: list[str] | None,\n    complete: bool = True,\n    scanned: list[str] | None = None,\n    namespace: str | None = None,\n) -> dict:\n    per_topic: dict[str, dict[str, dict]] = defaultdict(dict)\n    for failure in failures:\n        for topic in matching_topics(failure, keywords):\n            per_topic[topic].setdefault(\n                failure[\"trace_id\"],\n                {\n                    \"id\": failure[\"trace_id\"],\n                    \"agentId\": failure[\"agent\"],\n                    \"timestamp\": failure[\"timestamp\"],\n                },\n            )\n    return {\n        \"source\": \"trace-failures\",\n        \"collected_at\": policy_mod.utc_now_iso(),\n        \"repo_dir\": repo_dir,\n        \"agents\": agents or [\"all\"],\n        \"event_types\": \"tool_result(status=error) or failure-shaped output\",\n        \"definitions\": {topic: list(words) for topic, words in keywords.items()},\n        \"topics\": {\n            topic: sorted(\n                per_topic.get(topic, {}).values(), key=lambda t: (t[\"timestamp\"] or 0, t[\"id\"])\n            )\n            for topic in keywords\n        },\n        \"truncated\": [] if complete else list(keywords),\n        \"listing_complete\": complete,\n        \"failure_count\": len(failures),\n        # Every session scanned, matched or not: a complete scan with only\n        # unclassified failures is still an observed field (round 36).\n        \"sessions\": sorted(set(scanned or []) | {f[\"trace_id\"] for f in failures}),\n        \"namespace\": namespace,\n    }\n\n\ndef report(failures: list[dict], keywords: dict[str, list[str]]) -> tuple[list[str], dict]:\n    lines: list[str] = []\n    by_kind = Counter(f[\"kind\"] for f in failures)\n    by_topic: Counter[str] = Counter()\n    blind: list[dict] = []\n    for failure in failures:\n        topics = matching_topics(failure, keywords)\n        if not topics:\n            blind.append(failure)\n        for topic in topics:\n            by_topic[topic] += 1\n    sessions = {f[\"trace_id\"] for f in failures}\n    lines.append(f\"{len(failures)} distinct failure(s) across {len(sessions)} session(s)\")\n    for kind, n in by_kind.most_common():\n        lines.append(f\"  {kind}: {n}\")\n    lines.append(\"by policy topic (failures whose command or output matches the topic's keywords):\")\n    for topic in keywords:\n        lines.append(f\"  [{topic}] {by_topic.get(topic, 0)}\")\n    lines.append(f\"unclassified failures (field blind spots): {len(blind)}\")\n    for failure in blind[:12]:\n        lines.append(\n            f\"  {failure['trace_id'][:8]} #{failure['event_number']} {failure['tool']} ({failure['kind']}, x{failure['count']}): {failure['excerpt'][:110]}\"\n        )\n    return lines, {\n        \"failures\": [public_failure(f) for f in failures],\n        \"by_kind\": dict(by_kind),\n        \"by_topic\": {t: by_topic.get(t, 0) for t in keywords},\n        \"blind_spots\": [public_failure(f) for f in blind],\n        \"sessions\": sorted(sessions),\n    }\n\n\ndef main(argv: list[str]) -> int:\n    parser = argparse.ArgumentParser(\n        description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter\n    )\n    where = parser.add_mutually_exclusive_group(required=True)\n    where.add_argument(\"--repo-dir\", help=\"mine the local sessions recorded in this folder\")\n    where.add_argument(\n        \"--namespace\",\n        help=\"mine every session shared to this traces.com namespace (e.g. @gagan114); \"\n        \"for runners with no local sessions\",\n    )\n    parser.add_argument(\n        \"--traces-key\",\n        default=None,\n        help=\"API key passed as --key to every traces call (CI); omit when logged in locally\",\n    )\n    parser.add_argument(\"--agents\", default=\",\".join(measure_mod.DEFAULT_ANCHOR_AGENTS))\n    parser.add_argument(\"--policy\", default=None)\n    parser.add_argument(\"--history\", default=str(policy_mod.HISTORY_PATH))\n    parser.add_argument(\"--out-json\", default=None)\n    parser.add_argument(\"--save-evidence\", default=None)\n    parser.add_argument(\"--traces-bin\", default=\"traces\")\n    parser.add_argument(\"--limit\", type=int, default=200)\n    args = parser.parse_args(argv[1:])\n\n    policy = (\n        policy_mod.load_policy(args.policy) if args.policy else policy_mod.load_policy_or_builtin()\n    )\n    inputs = [args.policy, args.history]\n    if args.out_json:\n        policy_mod.assert_safe_output(args.out_json, inputs=inputs)\n    if args.save_evidence:\n        policy_mod.assert_safe_output(args.save_evidence, inputs=inputs, kind=\"evidence\")\n    if (\n        args.out_json\n        and args.save_evidence\n        and Path(args.out_json).resolve() == Path(args.save_evidence).resolve()\n    ):\n        print(\"::error::--out-json and --save-evidence must be different files\", file=sys.stderr)\n        return 1\n\n    # Two keyword sets with two jobs: blind spots are what the CURRENT policy\n    # cannot classify (a retired topic must not keep hiding fresh failures\n    # from mining), while evidence keeps counting every definition any\n    # policy version ever had, so a rolled-back topic retains the adverse\n    # evidence that blocks re-mining it (Codex review of PR #10, round 34).\n    current_keywords = dict(policy_mod.topic_keywords(policy))\n    search_keywords = dict(current_keywords)\n    search_keywords.update(\n        measure_mod.historical_definitions(policy_mod.load_history(args.history), current_keywords)\n    )\n    agents = (\n        None\n        if args.agents.strip() == \"all\"\n        else [a.strip() for a in args.agents.split(\",\") if a.strip()]\n    )\n\n    if args.traces_key:\n        EXTRA_CLI_ARGS[:] = [\"--key\", args.traces_key]\n\n    try:\n        if args.namespace:\n            traces, complete = list_namespace_traces(args.traces_bin, args.namespace, agents)\n        else:\n            traces, complete = list_traces(args.traces_bin, args.repo_dir, agents, args.limit)\n        if not complete:\n            print(\n                f\"::warning::session listing hit --limit {args.limit}; evidence counts are marked unknown, raise --limit\"\n            )\n        failures: list[dict] = []\n        for trace in traces:\n            if agents is None and trace.get(\"agentId\") == VERIFIER_AGENT:\n                pass  # \"all\" deliberately includes the verifier's own sessions\n            if args.n…1037 tokens truncated…ies: list[dict], anchor: dict | None) -> float | None:\nscripts/revise-improvement-policy.py:343:def entries_covered_by_evidence(entries: list[dict], measurement: dict) -> list[dict]:\nscripts/revise-improvement-policy.py:361:def validity_regressed(before: float | None, after: float | None) -> bool:\nscripts/revise-improvement-policy.py:371:def weight_repair(\nscripts/revise-improvement-policy.py:384:    anchor = candidate_anchor(current, policy)\nscripts/revise-improvement-policy.py:415:    after_v = validity_under(candidate, entries, candidate_anchor(current, candidate))\nscripts/revise-improvement-policy.py:423:def candidate_anchor(current: dict, policy: dict) -> dict | None:\nscripts/revise-improvement-policy.py:463:def rolled_back_hashes(history: list[dict]) -> set[str]:\nscripts/revise-improvement-policy.py:466:    restored v1 would otherwise be judged worse than v2 on coverage two\nscripts/revise-improvement-policy.py:475:def rejected_configuration(candidate: dict, history: list[dict], measurement: dict) -> dict | None:\nscripts/revise-improvement-policy.py:491:def unjudged_ancestors(policy: dict, history: list[dict]) -> list[dict]:\nscripts/revise-improvement-policy.py:494:    and including the first non-revision ancestor. Later evidence must be\nscripts/revise-improvement-policy.py:495:    able to expose a harmful ancestor that a newer revision or a rollback\nscripts/revise-improvement-policy.py:500:    version = judged_from(policy, history)\nscripts/revise-improvement-policy.py:508:            version = judged_from(snapshot, history)\nscripts/revise-improvement-policy.py:516:def judged_from(policy: dict, history: list[dict]) -> int | None:\nscripts/revise-improvement-policy.py:528:def entries_under(entries: list[dict], policy: dict) -> list[dict]:\nscripts/revise-improvement-policy.py:531:    of PR #10, round 25: older rounds decided by an ancestor must not enter\nscripts/revise-improvement-policy.py:540:def rounds_under(entries: list[dict], policy: dict) -> int:\nscripts/revise-improvement-policy.py:561:def snapshot_for_version(version: int, history: list[dict]) -> dict | None:\nscripts/revise-improvement-policy.py:570:def adoption_entry(version: int, history: list[dict]) -> dict | None:\nscripts/revise-improvement-policy.py:577:def field_blind_spots(field_failures: dict | None, keywords: dict[str, list[str]]) -> list[dict]:\nscripts/revise-improvement-policy.py:609:def decide(\nscripts/revise-improvement-policy.py:620:    coverage = current.get(\"coverage\")\nscripts/revise-improvement-policy.py:626:    #    re-measured on the SAME findings: comparing today's coverage with\nscripts/revise-improvement-policy.py:630:    base_version = judged_from(policy, history)\nscripts/revise-improvement-policy.py:635:            if parent is not None and coverage is not None:\nscripts/revise-improvement-policy.py:642:                # full-archive `coverage` that the trigger and candidate\nscripts/revise-improvement-policy.py:645:                parent_now = measure_mod.measure(own_rounds, parent, None)[\"current\"][\"coverage\"]\nscripts/revise-improvement-policy.py:646:                own_coverage = measure_mod.measure(own_rounds, policy, None)[\"current\"][\"coverage\"]\nscripts/revise-improvement-policy.py:647:                child_validity = validity_under(policy, covered, candidate_anchor(current, policy))\nscripts/revise-improvement-policy.py:649:                worse_coverage = (\nscripts/revise-improvement-policy.py:651:                    and own_coverage is not None\nscripts/revise-improvement-policy.py:652:                    and own_coverage < parent_now\nscripts/revise-improvement-policy.py:655:                # Compare against every ancestor in the unjudged chain, not\nscripts/revise-improvement-policy.py:656:                # only the parent: the best-scoring ancestor is the rollback\nscripts/revise-improvement-policy.py:660:                # Coverage is compared with every ancestor too: a topic a\nscripts/revise-improvement-policy.py:666:                for ancestor in unjudged_ancestors(policy, history):\nscripts/revise-improvement-policy.py:667:                    if policy_mod.policy_hash(ancestor) in rejected:\nscripts/revise-improvement-policy.py:669:                    v = validity_under(ancestor, covered, candidate_anchor(current, ancestor))\nscripts/revise-improvement-policy.py:671:                        best, best_validity = ancestor, v\nscripts/revise-improvement-policy.py:672:                    c = measure_mod.measure(own_rounds, ancestor, None)[\"current\"][\"coverage\"]\nscripts/revise-improvement-policy.py:674:                        best_cov, best_cov_value = ancestor, c\nscripts/revise-improvement-policy.py:677:                    and own_coverage is not None\nscripts/revise-improvement-policy.py:678:                    and best_cov_value > own_coverage\nscripts/revise-improvement-policy.py:680:                    worse_coverage = True\nscripts/revise-improvement-policy.py:685:                if worse_coverage or worse_validity:\nscripts/revise-improvement-policy.py:686:                    target = parent if worse_coverage else best\nscripts/revise-improvement-policy.py:690:                    target_coverage = measure_mod.measure(entries, target, None)[\"current\"][\nscripts/revise-improvement-policy.py:691:                        \"coverage\"\nscripts/revise-improvement-policy.py:694:                        f\"coverage {own_coverage} vs {parent_now}\"\nscripts/revise-improvement-policy.py:695:                        if worse_coverage\nscripts/revise-improvement-policy.py:713:                        \"coverage_before\": own_coverage,\nscripts/revise-improvement-policy.py:714:                        \"coverage_after\": target_coverage,\nscripts/revise-improvement-policy.py:716:                        \"validity_after\": best_validity if not worse_coverage else None,\nscripts/revise-improvement-policy.py:727:    # The wait does not depend on having an ancestor to compare with: a\nscripts/revise-improvement-policy.py:742:    if coverage is not None and coverage < MIN_COVERAGE:\nscripts/revise-improvement-policy.py:743:        triggers.append(f\"coverage {coverage} < {MIN_COVERAGE}\")\nscripts/revise-improvement-policy.py:747:    coverage_trigger = coverage is not None and coverage < MIN_COVERAGE\nscripts/revise-improvement-policy.py:759:    if field_trigger or coverage_trigger:\nscripts/revise-improvement-policy.py:761:    mined = mine_topics(mining_input, keywords) if (coverage_trigger or field_trigger) else []\nscripts/revise-improvement-policy.py:817:    v_before = validity_under(policy, covered, candidate_anchor(current, policy))\nscripts/revise-improvement-policy.py:818:    v_after = validity_under(revised, covered, candidate_anchor(current, revised))\nscripts/revise-improvement-policy.py:838:    if coverage is not None and after[\"coverage\"] is not None and after[\"coverage\"] < coverage:\nscripts/revise-improvement-policy.py:841:            \"reason\": f\"candidate revision would lower coverage {coverage} -> {after['coverage']}; refused\",\nscripts/revise-improvement-policy.py:851:        \"coverage_before\": coverage,\nscripts/revise-improvement-policy.py:852:        \"coverage_after\": after[\"coverage\"],\nscripts/revise-improvement-policy.py:860:def history_entry(decision: dict, policy: dict, measurement: dict, now: str) -> dict:\nscripts/revise-improvement-policy.py:869:        \"coverage_before\": decision.get(\"coverage_before\"),\nscripts/revise-improvement-policy.py:870:        \"coverage_after\": decision.get(\"coverage_after\"),\nscripts/revise-improvement-policy.py:883:def main(argv: list[str]) -> int:\nscripts/revise-improvement-policy.py:956:        if decision.get(\"coverage_after\") is not None:\nscripts/revise-improvement-policy.py:957:            print(f\"  coverage {decision['coverage_before']} -> {decision['coverage_after']}\")\nscripts/render_rsi_dashboard_test.py:39:def test_renders_every_section_from_real_shapes(tmp_path):\nscripts/render_rsi_dashboard_test.py:130:def test_revision_markers_sit_at_the_epoch_they_were_created_after():\nscripts/render_rsi_dashboard_test.py:143:def test_evidence_strings_are_escaped_in_the_echo_note(tmp_path):\nscripts/render_rsi_dashboard_test.py:181:def test_trigger_chart_plots_current_policy_validity():\nscripts/render_rsi_dashboard_test.py:193:def test_head_label_is_escaped_in_the_loop_diagram():\nscripts/render_rsi_dashboard_test.py:209:def test_out_may_not_overwrite_protected_or_input_files(tmp_path):\nscripts/render_rsi_dashboard_test.py:236:def test_history_version_labels_are_escaped_in_the_chart():\nscripts/render_rsi_dashboard_test.py:257:def test_reproduce_commands_only_use_flags_their_scripts_define():\nscripts/render_rsi_dashboard_test.py:272:def test_negative_validity_is_drawn_below_zero_not_clamped_onto_it():\nscripts/archive_round_test.py:26:def test_appends_new_round_and_tags_it_with_source_sha(tmp_path):\nscripts/archive_round_test.py:46:def test_target_reflects_the_passed_argument_not_a_hardcoded_file(tmp_path):\nscripts/archive_round_test.py:74:def test_target_defaults_to_something_generic_when_not_passed(tmp_path):\nscripts/archive_round_test.py:86:def test_rerunning_with_same_source_sha_does_not_duplicate(tmp_path):\nscripts/archive_round_test.py:108:def test_cross_pr_accumulation_crosses_threshold_on_the_third_contributing_round(tmp_path, capsys):\nscripts/archive_round_test.py:145:def test_clean_review_is_persisted_as_a_stamped_round(tmp_path, capsys):\nscripts/archive_round_test.py:187:def test_reviews_that_did_not_complete_are_not_archived_as_clean_rounds(tmp_path, capsys, body):\nscripts/archive_round_test.py:215:def test_a_failed_review_that_still_contains_numbered_lines_is_not_archived(tmp_path):\nscripts/mine_trace_failures_test.py:114:def test_mines_failed_and_failure_shaped_tool_results_only(monkeypatch):\nscripts/mine_trace_failures_test.py:127:def test_pagination_walks_every_page(monkeypatch):\nscripts/mine_trace_failures_test.py:146:def test_evidence_counts_sessions_per_topic_and_records_definitions(monkeypatch):\nscripts/mine_trace_failures_test.py:164:def test_main_writes_report_and_evidence(tmp_path, monkeypatch, capsys):\nscripts/mine_trace_failures_test.py:191:def test_cli_json_is_parsed_past_a_leading_notice():\nscripts/mine_trace_failures_test.py:199:def test_line_numbers_and_file_contents_are_not_failures():\nscripts/mine_trace_failures_test.py:246:def test_capped_listing_marks_every_topic_unknown(monkeypatch):\nscripts/mine_trace_failures_test.py:255:def test_evidence_matches_every_topic_independently_of_order():\nscripts/mine_trace_failures_test.py:275:def test_same_output_from_different_commands_are_distinct_failures(monkeypatch):\nscripts/mine_trace_failures_test.py:320:def test_zero_failed_is_not_a_test_failure():\nscripts/mine_trace_failures_test.py:337:def test_retired_topics_count_as_evidence_but_do_not_hide_blind_spots(\nscripts/mine_trace_failures_test.py:383:def test_evidence_records_every_scanned_session_even_without_matches(monkeypatch):\nscripts/mine_trace_failures_test.py:417:def test_namespace_mode_syncs_each_shared_session_before_reading_it(tmp_path, monkeypatch, capsys):\nscripts/revise_improvement_policy_test.py:55:def test_no_revision_when_coverage_and_validity_hold():\nscripts/revise_improvement_policy_test.py:62:def test_low_coverage_triggers_a_bounded_mined_revision():\nscripts/revise_improvement_policy_test.py:81:def test_low_validity_discounts_topics_the_field_never_shows():\nscripts/revise_improvement_policy_test.py:110:def test_rollback_when_an_adopted_revision_is_worse_than_its_parent():\nscripts/revise_improvement_policy_test.py:140:def test_no_rollback_when_unfamiliar_findings_lower_both_policies():\nscripts/revise_improvement_policy_test.py:175:def test_mined_topics_never_share_a_supporting_finding():\nscripts/revise_improvement_policy_test.py:240:def test_weight_repair_is_dropped_when_it_would_lower_validity():\nscripts/revise_improvement_policy_test.py:256:def test_reweighting_that_undefines_validity_is_refused():\nscripts/revise_improvement_policy_test.py:275:def test_corroborated_topic_regains_weight_even_when_scores_are_healthy():\nscripts/revise_improvement_policy_test.py:296:def test_rollback_on_validity_regression_with_same_coverage():\nscripts/revise_improvement_policy_test.py:321:def test_main_refuses_a_measurement_from_a_different_archive(tmp_path, capsys):\nscripts/revise_improvement_policy_test.py:349:def test_rollback_waits_for_rounds_decided_under_the_revision():\nscripts/revise_improvement_policy_test.py:389:def test_no_new_revision_until_the_current_one_has_been_judged():\nscripts/revise_improvement_policy_test.py:420:def test_discounts_wait_for_fresh_evidence():\nscripts/revise_improvement_policy_test.py:431:def test_rounds_under_requires_matching_version_not_just_contents():\nscripts/revise_improvement_policy_test.py:450:def test_validity_rollback_ignores_rounds_newer_than_the_evidence():\nscripts/revise_improvement_policy_test.py:477:def test_candidate_acceptance_and_rollback_use_the_same_evidence_window():\nscripts/revise_improvement_policy_test.py:488:def test_rolled_back_configuration_is_not_retried_on_the_same_evidence():\nscripts/revise_improvement_policy_test.py:543:def test_evidence_against_a_removed_topic_survives_for_candidates():\nscripts/revise_improvement_policy_test.py:562:def test_main_writes_nothing_when_the_history_path_is_refused(tmp_path, monkeypatch):\nscripts/revise_improvement_policy_test.py:591:def test_out_json_may_not_target_a_protected_or_input_file(tmp_path):\nscripts/revise_improvement_policy_test.py:617:def test_mined_topic_names_never_collide_with_existing_topics():\nscripts/revise_improvement_policy_test.py:632:def test_late_evidence_rolls_back_past_an_unjudged_parent_to_the_better_ancestor():\nscripts/revise_improvement_policy_test.py:685:def test_report_outputs_may_not_overwrite_canonical_evidence(tmp_path):\nscripts/revise_improvement_policy_test.py:713:def test_ancestry_continues_through_a_rollback():\nscripts/revise_improvement_policy_test.py:765:def test_current_definition_mismatch_does_not_erase_an_ancestors_evidence():\nscripts/revise_improvement_policy_test.py:779:def test_out_policy_may_not_be_the_history_file(tmp_path, monkeypatch):\nscripts/revise_improvement_policy_test.py:815:def test_rollback_judges_a_revision_only_on_its_own_rounds():\nscripts/revise_improvement_policy_test.py:841:def test_rollback_check_does_not_clobber_full_archive_coverage():\nscripts/revise_improvement_policy_test.py:896:def test_out_json_may_not_overwrite_the_field_failures_input(tmp_path):\nscripts/revise_improvement_policy_test.py:926:def test_accepted_revision_never_regresses_validity():\nscripts/revise_improvement_policy_test.py:936:def test_main_refuses_a_measurement_taken_under_a_different_policy(tmp_path, capsys):\nscripts/revise_improvement_policy_test.py:964:def test_main_writes_only_ai_owned_files(tmp_path, capsys, monkeypatch):\nscripts/revise_improvement_policy_test.py:1020:def test_failure_kind_labels_do_not_hide_field_blind_spots():\nscripts/revise_improvement_policy_test.py:1035:def test_candidate_anchor_reaches_an_older_definition_of_a_reused_name():\nscripts/revise_improvement_policy_test.py:1052:def test_candidate_anchor_never_borrows_a_count_from_another_definition():\nscripts/revise_improvement_policy_test.py:1082:def test_failure_kind_labels_cannot_become_a_mined_topic():\nscripts/revise_improvement_policy_test.py:1102:def test_swapped_policy_and_history_destinations_are_refused_before_any_write(\nscripts/revise_improvement_policy_test.py:1158:def test_rollback_to_the_root_version_still_waits_before_being_judged():\nscripts/revise_improvement_policy_test.py:1198:def test_a_rolled_back_configuration_is_never_a_rollback_target():\nscripts/revise_improvement_policy_test.py:1236:def test_coverage_lost_at_a_grandparent_still_triggers_rollback():\nscripts/revise_improvement_policy_test.py:1277:def test_main_refuses_a_policy_whose_lineage_metadata_was_edited(tmp_path, monkeypatch, capsys):\nscripts/revise_improvement_policy_test.py:1315:def test_evidence_window_filters_by_round_identity_not_number():\nscripts/measure_policy_validity_test.py:72:def test_rounds_merge_pending_placeholders_and_carry_timestamps():\nscripts/measure_policy_validity_test.py:79:def test_coverage_counts_unclassified_findings():\nscripts/measure_policy_validity_test.py:90:def test_anchor_is_replayed_per_epoch_by_timestamp():\nscripts/measure_policy_validity_test.py:100:def test_validity_is_rank_agreement_between_review_signal_and_anchor():\nscripts/measure_policy_validity_test.py:113:def test_dev_only_topics_flag_review_credit_the_field_never_corroborates():\nscripts/measure_policy_validity_test.py:120:def test_unsearched_topics_are_unknown_not_zero():\nscripts/measure_policy_validity_test.py:130:def test_validity_uses_the_weighted_signal_the_detector_decides_on():\nscripts/measure_policy_validity_test.py:140:def test_replay_orders_rounds_by_time_not_round_number():\nscripts/measure_policy_validity_test.py:153:def test_measurement_is_bound_to_the_archive_contents():\nscripts/measure_policy_validity_test.py:160:def test_truncated_searches_are_unknown_not_absolute():\nscripts/measure_policy_validity_test.py:169:def test_findings_newer_than_the_evidence_do_not_mark_topics_dev_only():\nscripts/measure_policy_validity_test.py:187:def test_empty_anchor_is_treated_as_no_anchor():\nscripts/measure_policy_validity_test.py:200:def test_main_reports_and_emits_json(tmp_path, capsys):\nscripts/measure_policy_validity_test.py:220:def test_out_json_and_newline_safe_report(tmp_path, capsys):\nscripts/measure_policy_validity_test.py:235:def test_evidence_searched_under_a_different_definition_is_unknown():\nscripts/measure_policy_validity_test.py:250:def test_side_outputs_may_not_overwrite_protected_files(tmp_path):\nscripts/measure_policy_validity_test.py:264:def test_epoch_without_a_timestamp_has_unknown_evidence_not_all_of_it():\nscripts/measure_policy_validity_test.py:275:def test_evidence_refresh_keeps_searching_topics_from_earlier_policy_versions():\nscripts/measure_policy_validity_test.py:288:def test_undated_traces_make_historical_counts_unknown_but_not_current_ones():\nscripts/measure_policy_validity_test.py:299:def test_history_path_counts_as_an_input_for_output_guards(tmp_path):\nscripts/measure_policy_validity_test.py:323:def test_reused_topic_name_keeps_every_historical_definition():\nscripts/measure_policy_validity_test.py:354:def test_anchor_counts_use_the_evidence_searched_under_the_topics_own_definition():\nscripts/measure_policy_validity_test.py:376:def test_a_scan_that_matched_nothing_is_still_an_observed_field():\nscripts/measure_policy_validity_test.py:390:def test_two_reviews_that_collided_on_a_round_number_stay_two_rounds():\nscripts/improvement_policy_test.py:21:def test_checked_in_policy_validates_and_descends_from_the_builtin_v1():\nscripts/improvement_policy_test.py:36:def test_policy_hash_ignores_metadata_but_tracks_decision_fields():\nscripts/improvement_policy_test.py:45:def test_policy_hash_is_sensitive_to_topic_order():\nscripts/improvement_policy_test.py:51:def test_history_snapshots_preserve_topic_order(tmp_path):\nscripts/improvement_policy_test.py:62:def test_classify_uses_policy_order_and_returns_none_when_uncovered():\nscripts/improvement_policy_test.py:71:def test_new_version_links_to_parent_and_validates():\nscripts/improvement_policy_test.py:97:def test_attribution_guard_refuses_writes_outside_ai_owned_files(tmp_path):\nscripts/improvement_policy_test.py:115:def test_history_round_trips(tmp_path):\nscripts/improvement_policy_test.py:128:def test_validate_policy_rejects_empty_keyword_lists():\nscripts/improvement_policy_test.py:135:def test_topic_names_may_not_contain_at_signs_and_origin_is_checked():\nscripts/improvement_policy_test.py:146:def test_hard_linked_outputs_are_refused(tmp_path, monkeypatch):\n"}

The fixes and regression tests are present. I also found a possible gap in A3: matching now uses full output, but deduplication still uses the shortened excerpt. I’ll test whether it discards a later failure with different keywords.

>sed -n '310,365p' scripts/revise-improvement-policy.py; sed -n '423,560p' scripts/revise-improvement-policy.py; sed -n '609,855p' scripts/revise-improvement-policy.py; sed -n '883,955p' scripts/revise-improvement-policy.py

{"chunk_id":"23beae","wall_time_seconds":0.000006667,"exit_code":0,"original_token_count":5952,"output":"            # its classifications (Codex review of PR #10, round 14).\n            suffix += 1\n            name = f\"{base}-{suffix}\"\n        mined.append(\n            {\n                \"name\": name,\n                \"keywords\": topic_keywords,\n                \"evidence\": [\n                    {\"round\": item[\"round\"], \"finding\": item[\"finding\"][:160]} for item in covering\n                ],\n            }\n        )\n        taken.update(topic_keywords)\n        covered_ids = {id(item) for item in covering}\n        remaining = [item for item in remaining if id(item) not in covered_ids]\n        for item in remaining:\n            item[\"tokens\"] -= set(topic_keywords)\n    return mined\n\n\ndef validity_under(policy: dict, entries: list[dict], anchor: dict | None) -> float | None:\n    \"\"\"Validity a policy would score on the same findings and the same\n    anchor counts the measurement carried. Topics without a known anchor\n    count are excluded, exactly as the measurement excludes them.\"\"\"\n    if anchor is None:\n        return None\n    current = measure_mod.measure(entries, policy, None)[\"current\"]\n    known = [t for t in policy[\"topics\"] if anchor.get(t) is not None]\n    return measure_mod.spearman(\n        [float(current[\"dev_weighted\"][t]) for t in known], [float(anchor[t]) for t in known]\n    )\n\n\ndef entries_covered_by_evidence(entries: list[dict], measurement: dict) -> list[dict]:\n    \"\"\"Archive entries from rounds no later than the evidence snapshot's\n    collection time. Without a collection time, all entries.\"\"\"\n    collected_ms = measure_mod.parse_timestamp_ms(\n        (measurement.get(\"anchor\") or {}).get(\"collected_at\")\n    )\n    if collected_ms is None:\n        return entries\n    # By round identity, not number: two reviews sharing a number can\n    # straddle the snapshot time (Codex review of PR #10, round 37).\n    covered = {\n        r[\"key\"]\n        for r in measure_mod.rounds_in_order(entries)\n        if r[\"timestamp_ms\"] is not None and r[\"timestamp_ms\"] <= collected_ms\n    }\n    return [e for e in entries if policy_mod.round_key(e) in covered]\n\n\ndef validity_regressed(before: float | None, after: float | None) -> bool:\n    \"\"\"A candidate may not lower a defined validity, and may not turn a\n    defined validity into an undefined one (a constant weighted signal has\n    no rank agreement to measure, which would silence every later trigger —\n    Codex review of PR #10, round 3, finding 1).\"\"\"\ndef candidate_anchor(current: dict, policy: dict) -> dict | None:\n    \"\"\"Anchor counts for judging a specific policy: every topic the evidence\n    searched, with each of this policy's topics resolved to the evidence\n    searched under exactly its own keyword definition (plain name or a\n    `name@tag` variant kept for an older definition). A topic whose\n    definition was never searched is unknown, never borrowed from another\n    definition of the same name (Codex review of PR #10, rounds 9, 12, 18,\n    32 and 33).\"\"\"\n    anchor = current.get(\"anchor\")\n    if anchor is None:\n        return None\n    evidence = current.get(\"anchor_evidence\")\n    definitions = current.get(\"anchor_definitions\") or {}\n    keywords = policy_mod.topic_keywords(policy)\n    if evidence is None:\n        # A measurement without evidence-wide counts (older format): only\n        # the measured topics are known, and only where definitions match.\n        return {\n            topic: (\n                count\n                if topic not in keywords\n                or list(definitions.get(topic, [])) == list(keywords[topic])\n                else None\n            )\n            for topic, count in anchor.items()\n        }\n    result: dict[str, int | None] = {}\n    for topic, count in evidence.items():\n        if \"@\" in topic or topic in keywords:\n            continue\n        result[topic] = count  # a topic this policy does not define keeps its evidence\n    searched_names = {key.split(\"@\")[0] for key in evidence}\n    for topic, words in keywords.items():\n        if topic not in searched_names:\n            continue  # never searched under any definition: not part of the anchor\n        key = measure_mod.resolve_evidence_key(definitions, topic, list(words))\n        result[topic] = evidence.get(key) if key is not None else None\n    return result\n\n\ndef rolled_back_hashes(history: list[dict]) -> set[str]:\n    \"\"\"Configurations a rollback has already rejected. They are never\n    rollback targets themselves: after v2 is rolled back for validity, the\n    restored v1 would otherwise be judged worse than v2 on coverage two\n    rounds later and 'rolled back' to v2, and so on forever.\"\"\"\n    return {\n        e[\"replaced_policy_hash\"]\n        for e in history\n        if e.get(\"origin\") == \"rollback\" and isinstance(e.get(\"replaced_policy_hash\"), str)\n    }\n\n\ndef rejected_configuration(candidate: dict, history: list[dict], measurement: dict) -> dict | None:\n    \"\"\"A configuration rolled back on the same archive and evidence is not\n    retried: the rollback's history entry records the rejected hash and\n    what it was judged on.\"\"\"\n    wanted = policy_mod.policy_hash(candidate)\n    collected = (measurement.get(\"anchor\") or {}).get(\"collected_at\")\n    for entry in history:\n        if entry.get(\"origin\") != \"rollback\" or entry.get(\"replaced_policy_hash\") != wanted:\n            continue\n        if entry.get(\"archive_digest\") == measurement.get(\"archive_digest\") and (\n            entry.get(\"evidence_collected_at\") == collected\n        ):\n            return entry\n    return None\n\n\ndef unjudged_ancestors(policy: dict, history: list[dict]) -> list[dict]:\n    \"\"\"Snapshots this policy descends from, nearest first, following a\n    rollback through to the ancestry of the configuration it restored, up to\n    and including the first non-revision ancestor. Later evidence must be\n    able to expose a harmful ancestor that a newer revision or a rollback\n    was layered on before validity could be measured (Codex review of\n    PR #10, rounds 14 and 17).\"\"\"\n    chain: list[dict] = []\n    seen: set[int] = set()\n    version = judged_from(policy, history)\n    while isinstance(version, int) and version not in seen:\n        seen.add(version)\n        snapshot = snapshot_for_version(version, history)\n        if snapshot is None:\n            break\n        chain.append(snapshot)\n        if snapshot.get(\"origin\") == \"rollback\":\n            version = judged_from(snapshot, history)\n            continue\n        if snapshot.get(\"origin\") != \"revision\":\n            break\n        version = snapshot.get(\"parent\")\n    return chain\n\n\ndef judged_from(policy: dict, history: list[dict]) -> int | None:\n    \"\"\"Version whose ancestry a policy continues: the parent for a revision;\n    for a rollback, the parent of the restored version.\"\"\"\n    if policy.get(\"origin\") == \"rollback\":\n        restored = policy.get(\"restored_version\")\n        snapshot = snapshot_for_version(restored, history) if isinstance(restored, int) else None\n        if snapshot is None:\n            return None\n        return snapshot.get(\"parent\")\n    return policy.get(\"parent\")\n\n\ndef entries_under(entries: list[dict], policy: dict) -> list[dict]:\n    \"\"\"Archive entries from rounds stamped with this policy's version and\n    hash: the only rounds on which the policy can be judged (Codex review\n    of PR #10, round 25: older rounds decided by an ancestor must not enter\n    the comparison, or they can drown out the revision's own signal).\"\"\"\n    wanted = policy_mod.policy_hash(policy)\n    version = policy[\"version\"]\n    return [\n        e for e in entries if e.get(\"policy_hash\") == wanted and e.get(\"policy_version\") == version\n    ]\n\n\ndef rounds_under(entries: list[dict], policy: dict) -> int:\n    \"\"\"Rounds decided under this exact policy: archive-round.py stamps each\n    round with the policy hash in force when it was archived. Rounds from\n    before a revision was merged never count toward judging it, however\n    long its pull request sat open (Codex review of PR #10, round 4).\"\"\"\n    wanted = policy_mod.policy_hash(policy)\n    version = policy[\"version\"]\n    # Version AND hash: a later revision that recreates an earlier\n    # configuration must not inherit that configuration's old rounds\n    # (Codex review of PR #10, round 7).\n    return len(\n        {\n            measure_mod.round_key(e)\n            for e in entries\n            if e.get(\"policy_hash\") == wanted\n            and e.get(\"policy_version\") == version\n            and isinstance(e.get(\"round\"), int)\n        }\n    )\n\n\ndef decide(\n    entries: list[dict],\n    policy: dict,\n    history: list[dict],\n    measurement: dict,\n    now: str,\n    field_failures: dict | None = None,\n) -> dict:\n    \"\"\"Pure decision: returns {\"action\": \"none\"|\"revise\"|\"rollback\", ...}\n    without touching disk, so it can be tested and dry-run.\"\"\"\n    current = measurement[\"current\"]\n    coverage = current.get(\"coverage\")\n    validity = current.get(\"validity\")\n    keywords = policy_mod.topic_keywords(policy)\n\n    # 1. Safe inheritance: a revision that made things worse gets rolled back\n    #    before any new revision is layered on top of it. Both policies are\n    #    re-measured on the SAME findings: comparing today's coverage with\n    #    the parent's historical number would punish a revision merely for\n    #    being alive when unfamiliar findings arrived (Codex review of\n    #    PR #10, finding 1).\n    base_version = judged_from(policy, history)\n    if policy.get(\"origin\") in {\"revision\", \"rollback\"} and base_version is not None:\n        adopted = adoption_entry(policy[\"version\"], history)\n        if adopted is not None and rounds_under(entries, policy) >= MIN_ROUNDS_TO_JUDGE:\n            parent = snapshot_for_version(base_version, history)\n            if parent is not None and coverage is not None:\n                # Validity is judged only on rounds the evidence snapshot could\n                # have seen; rounds archived after collection would make an\n                # unchanged field look like a regression (Codex review of\n                # PR #10, round 7).\n                covered = entries_covered_by_evidence(entries_under(entries, policy), measurement)\n                # Coverage on the revision's own rounds, kept apart from the\n                # full-archive `coverage` that the trigger and candidate\n                # acceptance use (Codex review of PR #10, round 26).\n                own_rounds = entries_under(entries, policy)\n                parent_now = measure_mod.measure(own_rounds, parent, None)[\"current\"][\"coverage\"]\n                own_coverage = measure_mod.measure(own_rounds, policy, None)[\"current\"][\"coverage\"]\n                child_validity = validity_under(policy, covered, candidate_anchor(current, policy))\n                rejected = rolled_back_hashes(history)\n                worse_coverage = (\n                    parent_now is not None\n                    and own_coverage is not None\n                    and own_coverage < parent_now\n                    and policy_mod.policy_hash(parent) not in rejected\n                )\n                # Compare against every ancestor in the unjudged chain, not\n                # only the parent: the best-scoring ancestor is the rollback\n                # target when the current policy is worse than any of them.\n                best: dict | None = None\n                best_validity: float | None = None\n                # Coverage is compared with every ancestor too: a topic a\n                # grandparent recognised and the parent dropped is invisible\n                # in a parent-only comparison (Codex full-branch review,\n                # finding 5).\n                best_cov: dict | None = None\n                best_cov_value: float | None = None\n                for ancestor in unjudged_ancestors(policy, history):\n                    if policy_mod.policy_hash(ancestor) in rejected:\n                        continue\n                    v = validity_under(ancestor, covered, candidate_anchor(current, ancestor))\n                    if v is not None and (best_validity is None or v > best_validity):\n                        best, best_validity = ancestor, v\n                    c = measure_mod.measure(own_rounds, ancestor, None)[\"current\"][\"coverage\"]\n                    if c is not None and (best_cov_value is None or c > best_cov_value):\n                        best_cov, best_cov_value = ancestor, c\n                if (\n                    best_cov_value is not None\n                    and own_coverage is not None\n                    and best_cov_value > own_coverage\n                ):\n                    worse_coverage = True\n                    parent, parent_now = best_cov, best_cov_value\n                worse_validity = best_validity is not None and (\n                    child_validity is None or child_validity < best_validity\n                )\n                if worse_coverage or worse_validity:\n                    target = parent if worse_coverage else best\n                    assert target is not None\n                    # Record what the restored policy actually scores, not the\n                    # current one's number (Codex review of PR #10, round 15).\n                    target_coverage = measure_mod.measure(entries, target, None)[\"current\"][\n                        \"coverage\"\n                    ]\n                    what = (\n                        f\"coverage {own_coverage} vs {parent_now}\"\n                        if worse_coverage\n                        else f\"validity {child_validity} vs {best_validity}\"\n                    )\n                    return {\n                        \"action\": \"rollback\",\n                        \"reason\": (\n                            f\"on the same {current['findings_total']} findings and anchor, \"\n                            f\"v{policy['version']} scores {what} against v{target['version']}\"\n                        ),\n                        \"policy\": policy_mod.new_version(\n                            policy,\n                            topics=target[\"topics\"],\n                            threshold=target[\"threshold\"],\n                            origin=\"rollback\",\n                            rationale=f\"Rollback to v{target['version']}: v{policy['version']} scored worse ({what}).\",\n                            created_at=now,\n                            restored_version=target[\"version\"],\n                        ),\n                        \"coverage_before\": own_coverage,\n                        \"coverage_after\": target_coverage,\n                        \"validity_before\": child_validity,\n                        \"validity_after\": best_validity if not worse_coverage else None,\n                        \"changes\": [\n                            f\"restored taxonomy, weights and threshold of v{target['version']}\"\n                        ],\n                    }\n\n    # A revision that has not yet been judged must not be built on: a\n    # successor would only ever be compared with it, so a regression it\n    # introduced against ITS parent could never be rolled back (Codex review\n    # of PR #10, round 6). Wait until MIN_ROUNDS_TO_JUDGE rounds have run\n    # under it; the rollback check above already covered the judged case.\n    # The wait does not depend on having an ancestor to compare with: a\n    # rollback to the root version has none, and skipping the wait there let\n    # a rejected configuration be re-proposed after one round (round 36).\n    if policy.get(\"origin\") in {\"revision\", \"rollback\"}:\n        under = rounds_under(entries, policy)\n        if under < MIN_ROUNDS_TO_JUDGE:\n            return {\n                \"action\": \"none\",\n                \"reason\": (\n                    f\"v{policy['version']} has run under {under} round(s); waiting for \"\n                    f\"{MIN_ROUNDS_TO_JUDGE} before judging it or layering another revision\"\n                ),\n            }\n\n    triggers = []\n    if coverage is not None and coverage < MIN_COVERAGE:\n        triggers.append(f\"coverage {coverage} < {MIN_COVERAGE}\")\n    if validity is not None and validity < MIN_VALIDITY:\n        triggers.append(f\"validity {validity} < {MIN_VALIDITY}\")\n    blind = field_blind_spots(field_failures, keywords)\n    coverage_trigger = coverage is not None and coverage < MIN_COVERAGE\n    field_trigger = len(blind) >= MIN_FIELD_BLIND_SPOTS\n    if field_trigger:\n        triggers.append(f\"field: {len(blind)} failures match no topic\")\n\n    changes: list[str] = []\n    new_topics = dict(policy[\"topics\"])\n\n    # 2. Coverage repair: mine the blind spots.\n    # Explicit booleans, not string prefixes, decide what gets mined\n    # (Codex review of PR #10, round 28).\n    mining_input = list(current.get(\"unclassified_findings\", []))\n    if field_trigger or coverage_trigger:\n        mining_input.extend(blind)\n    mined = mine_topics(mining_input, keywords) if (coverage_trigger or field_trigger) else []\n    for topic in mined:\n        new_topics[topic[\"name\"]] = {\n            \"keywords\": topic[\"keywords\"],\n            \"weight\": 1.0,\n            \"mined_from\": topic[\"evidence\"],\n        }\n        changes.append(\n            f\"added topic {topic['name']} (keywords {topic['keywords']}) covering {len(topic['evidence'])} unclassified finding(s)\"\n        )\n\n    # 3. Weight repair: restorations are always evaluated; discounts only\n    #    when validity actually failed.\n    # Every validity comparison in this decision uses the rounds the evidence\n    # snapshot covers, exactly as the rollback check does, so a candidate\n    # cannot pass on later findings and then be rolled back on the snapshot\n    # (Codex review of PR #10, round 8).\n    covered = entries_covered_by_evidence(entries, measurement)\n    weighted_topics, weight_changes = weight_repair(\n        {**policy, \"topics\": new_topics},\n        current,\n        covered,\n        discount=any(t.startswith(\"validity\") for t in triggers),\n    )\n    new_topics = weighted_topics\n    changes.extend(weight_changes)\n\n    if not triggers and not any(c.startswith(\"restored\") for c in changes):\n        return {\n            \"action\": \"none\",\n            \"reason\": \"policy signal still predicts the field within thresholds\",\n        }\n    if not triggers:\n        triggers.append(\"field evidence corroborates a discounted topic\")\n\n    if not changes or all(\n        c.startswith(\"kept weights unchanged\") or c.startswith(\"skipped discounts\") for c in changes\n    ):\n        return {\n            \"action\": \"none\",\n            \"reason\": \"triggered (\"\n            + \"; \".join(triggers)\n            + \") but no bounded, evidence-backed change was available\",\n            \"triggers\": triggers,\n        }\n\n    revised = policy_mod.new_version(\n        policy,\n        topics=new_topics,\n        threshold=policy[\"threshold\"],\n        origin=\"revision\",\n        rationale=\"Revised because \" + \"; \".join(triggers) + \". \" + \" \".join(changes),\n        created_at=now,\n    )\n    # The whole candidate, not just its weight changes, must not regress\n    # validity against the policy it replaces (Codex review of PR #10, round 4).\n    v_before = validity_under(policy, covered, candidate_anchor(current, policy))\n    v_after = validity_under(revised, covered, can…706 tokens truncated…[\n                args.archive_path,\n                args.measurement,\n                args.policy,\n                args.history,\n                args.field_failures,\n            ],\n        )\n    entries = measure_mod.load_archive(args.archive_path)\n    policy = policy_mod.load_policy(args.policy)\n    history = policy_mod.load_history(args.history)\n    try:\n        policy_mod.assert_policy_matches_history(policy, history)\n    except ValueError as exc:\n        print(f\"::error::{exc}\", file=sys.stderr)\n        return 1\n    with open(args.measurement) as f:\n        measurement = json.load(f)\n    if measurement.get(\"policy_hash\") != policy_mod.policy_hash(policy):\n        print(\n            f\"::error::measurement was taken under policy hash {measurement.get('policy_hash')}, \"\n            f\"but {args.policy} hashes to {policy_mod.policy_hash(policy)}; re-measure first\",\n            file=sys.stderr,\n        )\n        return 1\n    digest = measure_mod.archive_digest(entries)\n    if measurement.get(\"archive_digest\") != digest:\n        print(\n            f\"::error::measurement was taken against archive digest {measurement.get('archive_digest')}, \"\n            f\"but {args.archive_path} now digests to {digest}; re-measure first\",\n            file=sys.stderr,\n        )\n        return 1\n    now = args.now or policy_mod.utc_now_iso()\n\n    field_failures = None\n    if args.field_failures:\n        with open(args.field_failures) as f:\n            field_failures = json.load(f)\n    decision = decide(entries, policy, history, measurement, now, field_failures)\n    if decision[\"action\"] == \"none\":\n        print(f\"no revision: {decision['reason']}\")\n    else:\n        verb = \"ROLLBACK\" if decision[\"action\"] == \"rollback\" else \"REVISION\"\n        print(\n            f\"{verb} -> policy v{decision['policy']['version']} (parent v{decision['policy']['parent']}): {decision['reason']}\"\n        )\n        for change in decision[\"changes\"]:\n            print(f\"  - {change}\")\n"}
{"chunk_id":"a69644","wall_time_seconds":0.000003291,"exit_code":0,"original_token_count":4116,"output":"def test_a_scan_that_matched_nothing_is_still_an_observed_field():\n    evidence = _evidence()\n    for topic in evidence[\"topics\"]:\n        evidence[\"topics\"][topic] = []\n    evidence[\"sessions\"] = [\"s1\", \"s2\", \"s3\"]\n    result = measure.measure(_archive(), policy_mod.builtin_policy(), evidence)\n    assert result[\"anchor\"][\"traces_considered\"] == 3\n    assert all(v == 0 for v in result[\"current\"][\"anchor\"].values())\n    # Without a session list, no matches really is no evidence.\n    del evidence[\"sessions\"]\n    current = measure.measure(_archive(), policy_mod.builtin_policy(), evidence)[\"current\"]\n    assert current[\"anchor\"] is None\n\n\ndef test_two_reviews_that_collided_on_a_round_number_stay_two_rounds():\n    # Two archive proposals opened before either merged both got round 11;\n    # their source SHAs differ, so they are two rounds (Codex full-branch\n    # review, workflows finding 2). Legacy placeholder+result pairs without a\n    # SHA still merge by number.\n    entries = [\n        {\n            \"round\": 11,\n            \"source_sha\": \"aaa\",\n            \"findings\": [\"**[P1]** one.\"],\n            \"occurred_at\": \"2026-09-14T15:00:00Z\",\n        },\n        {\n            \"round\": 11,\n            \"source_sha\": \"bbb\",\n            \"findings\": [\"**[P1]** two.\"],\n            \"occurred_at\": \"2026-09-14T15:05:00Z\",\n        },\n        {\"round\": 12, \"findings\": [], \"occurred_at\": \"2026-09-14T15:10:00Z\"},\n        {\"round\": 12, \"findings\": [\"**[P2]** result.\"], \"occurred_at\": \"2026-09-14T15:11:00Z\"},\n    ]\n    rounds = measure.rounds_in_order(entries)\n    assert [(r[\"round\"], r[\"key\"]) for r in rounds] == [\n        (11, \"sha:aaa\"),\n        (11, \"sha:bbb\"),\n        (12, \"round:12\"),\n    ]\n    assert measure.round_key({\"round\": 3}) == \"round:3\"\ndef test_evidence_records_every_scanned_session_even_without_matches(monkeypatch):\n    # A complete scan whose failures are all unclassified is an observed field\n    # with confirmed zero counts, not an empty anchor (Codex, round 36).\n    monkeypatch.setattr(mine, \"run_traces_json\", _fake_runner({\"t1\": _events(), \"t2\": []}))\n    keywords = {\"nothing-matches\": [\"zzzz-never\"]}\n    traces, complete = mine.list_traces(\"traces\", \"/repo\", [\"claude-code\"], 50)\n    failures = [f for t in traces for f in mine.mine_trace(\"traces\", t)]\n    evidence = mine.build_evidence(\n        failures, keywords, \"/repo\", [\"claude-code\"], complete, scanned=[t[\"id\"] for t in traces]\n    )\n    assert evidence[\"topics\"] == {\"nothing-matches\": []}\n    assert evidence[\"sessions\"] == [\"t1\", \"t2\"]\n    assert sys.modules[\"measure_policy_validity\"].evidence_trace_ids(evidence) == {\"t1\", \"t2\"}\n    result = sys.modules[\"measure_policy_validity\"].measure(\n        [\n            {\n                \"round\": 1,\n                \"findings\": [\"**[P1]** zzzz-never happened.\"],\n                \"occurred_at\": \"2026-09-14T15:00:00Z\",\n            }\n        ],\n        {\n            \"version\": 1,\n            \"parent\": None,\n            \"origin\": \"init\",\n            \"threshold\": 3,\n            \"topics\": {\"nothing-matches\": {\"keywords\": [\"zzzz-never\"], \"weight\": 1.0}},\n        },\n        evidence,\n    )\n    assert result[\"current\"][\"anchor\"] == {\"nothing-matches\": 0}\n    assert result[\"anchor\"][\"traces_considered\"] == 2\n\n\ndef test_namespace_mode_syncs_each_shared_session_before_reading_it(tmp_path, monkeypatch, capsys):\n    calls: list[list[str]] = []\n\n    def run(_bin, args):\n        calls.append(list(args))\n        if args[0] == \"list\":\n            assert args[1] == \"@gagan114\" and \"--all\" in args\n            return {\n                \"traces\": [\n                    {\"id\": \"remote-1\", \"agentId\": \"claude-code\", \"timestamp\": 5},\n                    {\"id\": \"remote-2\", \"agentId\": \"codex\", \"timestamp\": 6},\n                ]\n            }\n        if args[0] == \"sync\":\n            return {\"traceId\": args[1]}\n        if args[0] == \"show\":\n            return {\"events\": _events() if args[1] == \"remote-1\" else []}\n        raise AssertionError(args)\n\n    monkeypatch.setattr(mine, \"run_traces_json\", run)\n    monkeypatch.setattr(mine, \"EXTRA_CLI_ARGS\", [])\n    policy = tmp_path / \"policy.json\"\n    policy.write_text(json.dumps(policy_mod.builtin_policy()))\n    evidence = tmp_path / \"evidence.json\"\n    code = mine.main(\n        [\n            \"m\",\n            \"--namespace\",\n            \"gagan114\",\n            \"--agents\",\n            \"claude-code\",\n            \"--traces-key\",\n            \"tr_secret\",\n            \"--policy\",\n            str(policy),\n            \"--history\",\n            str(tmp_path / \"h.jsonl\"),\n            \"--save-evidence\",\n            str(evidence),\n        ]\n    )\n    assert code == 0\n    assert mine.EXTRA_CLI_ARGS == [\"--key\", \"tr_secret\"]\n    assert [\"sync\", \"remote-1\"] in calls\n    assert not any(c[:2] == [\"sync\", \"remote-2\"] for c in calls), \"codex sessions were filtered out\"\n    saved = json.loads(evidence.read_text())\n    assert saved[\"namespace\"] == \"gagan114\"\n    assert saved[\"sessions\"] == [\"remote-1\"]\n    assert saved[\"listing_complete\"] is True\n    assert \"tr_secret\" not in capsys.readouterr().out\ndef test_rollback_to_the_root_version_still_waits_before_being_judged():\n    # judged_from() is None for a rollback to v1 (no parent), which used to\n    # skip the waiting period and let the rejected configuration be\n    # re-proposed after a single clean round (Codex, round 36).\n    v1 = policy_mod.builtin_policy()\n    v2 = policy_mod.new_version(\n        v1, topics=dict(v1[\"topics\"]), threshold=v1[\"threshold\"], origin=\"revision\", rationale=\"x\"\n    )\n    v3 = policy_mod.new_version(\n        v2,\n        topics=dict(v1[\"topics\"]),\n        threshold=v1[\"threshold\"],\n        origin=\"rollback\",\n        rationale=\"back\",\n        restored_version=1,\n    )\n    history = [\n        {\"version\": 2, \"policy\": v2, \"replaced_policy_hash\": policy_mod.policy_hash(v1)},\n        {\n            \"version\": 3,\n            \"origin\": \"rollback\",\n            \"policy\": v3,\n            \"replaced_policy_hash\": policy_mod.policy_hash(v2),\n        },\n    ]\n    entries = [\n        {\n            \"round\": 1,\n            \"findings\": [],\n            \"occurred_at\": \"2026-09-14T15:00:00Z\",\n            \"policy_version\": 3,\n            \"policy_hash\": policy_mod.policy_hash(v3),\n        }\n    ]\n    measurement = measure.measure(entries, v3, None)\n    decision = revise.decide(entries, v3, history, measurement, NOW)\n    assert decision[\"action\"] == \"none\"\n    assert \"waiting for\" in decision[\"reason\"]\n\n\ndef test_a_rolled_back_configuration_is_never_a_rollback_target():\n    # v2 was rolled back; the restored configuration (v3) must not be judged\n    # \"worse than v2\" on coverage two rounds later and rolled back INTO v2,\n    # or the loop would ping-pong forever.\n    v1 = policy_mod.builtin_policy()\n    v2 = policy_mod.new_version(\n        v1,\n        topics={**v1[\"topics\"], \"queue-overflow\": {\"keywords\": [\"queue overflow\"], \"weight\": 1.0}},\n        threshold=v1[\"threshold\"],\n        origin=\"revision\",\n        rationale=\"mined\",\n    )\n    v3 = policy_mod.new_version(\n        v2,\n        topics=dict(v1[\"topics\"]),\n        threshold=v1[\"threshold\"],\n        origin=\"rollback\",\n        rationale=\"worse\",\n        restored_version=1,\n    )\n    history = [\n        {\"version\": 1, \"policy\": v1},\n        {\"version\": 2, \"policy\": v2, \"replaced_policy_hash\": policy_mod.policy_hash(v1)},\n        {\n            \"version\": 3,\n            \"origin\": \"rollback\",\n            \"policy\": v3,\n            \"replaced_policy_hash\": policy_mod.policy_hash(v2),\n        },\n    ]\n    entries = [\n        {**e, \"policy_version\": 3, \"policy_hash\": policy_mod.policy_hash(v3)} for e in _archive()\n    ]\n    decision = revise.decide(entries, v3, history, measure.measure(entries, v3, None), NOW)\n    assert decision[\"action\"] != \"rollback\"\n    assert revise.rolled_back_hashes(history) == {policy_mod.policy_hash(v2)}\n\n\ndef test_coverage_lost_at_a_grandparent_still_triggers_rollback():\n    # v1 knew \"quartz\"; v2 dropped it; v3 added something unrelated. On v3's\n    # own rounds v3 and v2 tie at 0 coverage, so a parent-only comparison\n    # never rolls back, but v1 covers everything (Codex full-branch review,\n    # finding 5).\n    v1 = policy_mod.builtin_policy()\n    v1[\"topics\"][\"quartz-crashes\"] = {\"keywords\": [\"quartz\"], \"weight\": 1.0}\n    v2 = policy_mod.new_version(\n        v1,\n        topics={k: v for k, v in v1[\"topics\"].items() if k != \"quartz-crashes\"},\n        threshold=v1[\"threshold\"],\n        origin=\"revision\",\n        rationale=\"dropped\",\n    )\n    v3 = policy_mod.new_version(\n        v2,\n        topics={**v2[\"topics\"], \"unrelated\": {\"keywords\": [\"zzunrelated\"], \"weight\": 1.0}},\n        threshold=v2[\"threshold\"],\n        origin=\"revision\",\n        rationale=\"added\",\n    )\n    history = [\n        {\"version\": 1, \"policy\": v1},\n        {\"version\": 2, \"policy\": v2, \"replaced_policy_hash\": policy_mod.policy_hash(v1)},\n        {\"version\": 3, \"policy\": v3, \"replaced_policy_hash\": policy_mod.policy_hash(v2)},\n    ]\n    entries = [\n        {\n            \"round\": n,\n            \"occurred_at\": f\"2026-09-14T1{n}:00:00Z\",\n            \"findings\": [f\"**[P2]** quartz renderer crashed again ({n}).\"],\n            \"policy_version\": 3,\n            \"policy_hash\": policy_mod.policy_hash(v3),\n        }\n        for n in (1, 2)\n    ]\n    decision = revise.decide(entries, v3, history, measure.measure(entries, v3, None), NOW)\n    assert decision[\"action\"] == \"rollback\"\n    assert decision[\"policy\"][\"restored_version\"] == 1\n\n\ndef test_main_refuses_a_policy_whose_lineage_metadata_was_edited(tmp_path, monkeypatch, capsys):\n    # Changing only `origin` used to switch the wait gate off: the hash does\n    # not cover lineage metadata (Codex full-branch review, finding 1).\n    v1 = policy_mod.builtin_policy()\n    v2 = policy_mod.new_version(\n        v1, topics=dict(v1[\"topics\"]), threshold=v1[\"threshold\"], origin=\"revision\", rationale=\"x\"\n    )\n    archive = tmp_path / \"archive.jsonl\"\n    archive.write_text(\"\\n\".join(json.dumps(e) for e in _archive()) + \"\\n\")\n    history = tmp_path / \"history.jsonl\"\n    history.write_text(json.dumps({\"version\": 2, \"policy\": v2}) + \"\\n\")\n    edited = {**v2, \"origin\": \"init\", \"parent\": None}\n    policy_path = tmp_path / \"policy.json\"\n    policy_path.write_text(json.dumps(edited))\n    m_path = tmp_path / \"m.json\"\n    m_path.write_text(json.dumps(measure.measure(_archive(), edited, None)))\n    code = revise.main(\n        [\n            \"r\",\n            str(archive),\n            \"--measurement\",\n            str(m_path),\n            \"--policy\",\n            str(policy_path),\n            \"--history\",\n            str(history),\n            \"--dry-run\",\n            \"--now\",\n            NOW,\n        ]\n    )\n    assert code == 1\n    assert \"lineage metadata\" in capsys.readouterr().err\n    with pytest.raises(ValueError, match=\"history records no versions\"):\n        policy_mod.assert_policy_matches_history(v2, [])\n    policy_mod.assert_policy_matches_history(v2, [{\"version\": 2, \"policy\": v2}])\n\n\ndef test_evidence_window_filters_by_round_identity_not_number():\n    # Two reviews share round 5; one predates the snapshot and one does not.\n    entries = [\n        {\n            \"round\": 5,\n            \"source_sha\": \"old\",\n            \"findings\": [\"**[P1]** a.\"],\n            \"occurred_at\": \"2026-09-14T15:00:00Z\",\n        },\n        {\n            \"round\": 5,\n            \"source_sha\": \"new\",\n            \"findings\": [\"**[P1]** b.\"],\n            \"occurred_at\": \"2026-09-14T17:00:00Z\",\n        },\n    ]\n    measurement = {\"anchor\": {\"collected_at\": \"2026-09-14T16:00:00Z\"}}\n    covered = revise.entries_covered_by_evidence(entries, measurement)\n    assert [e[\"source_sha\"] for e in covered] == [\"old\"]\n    policy[\"topics\"][\"empty\"] = {\"keywords\": [], \"weight\": 1.0}\n    with pytest.raises(ValueError, match=\"non-empty\"):\n        improvement_policy.validate_policy(policy)\n\n\ndef test_topic_names_may_not_contain_at_signs_and_origin_is_checked():\n    policy = improvement_policy.builtin_policy()\n    policy[\"topics\"][\"shell-semantics@custom\"] = {\"keywords\": [\"x\"], \"weight\": 1.0}\n    with pytest.raises(ValueError, match=\"'@'\"):\n        improvement_policy.validate_policy(policy)\n    policy = improvement_policy.builtin_policy()\n    policy[\"origin\"] = \"whatever\"\n    with pytest.raises(ValueError, match=\"origin\"):\n        improvement_policy.validate_policy(policy)\n\n\ndef test_hard_linked_outputs_are_refused(tmp_path, monkeypatch):\n    import os\n\n    archive = tmp_path / \"archive.jsonl\"\n    archive.write_text(\"{}\\n\")\n    alias = tmp_path / \"report.json\"\n    os.link(archive, alias)\n    with pytest.raises(PermissionError, match=\"input of this run\"):\n        improvement_policy.assert_safe_output(alias, inputs=[archive])\n    protected = tmp_path / \"docs\" / \"self-improvement-archive.jsonl\"\n    protected.parent.mkdir()\n    protected.write_text(\"{}\\n\")\n    monkeypatch.setattr(improvement_policy, \"REPO_ROOT\", tmp_path)\n    alias2 = tmp_path / \"out.json\"\n    os.link(protected, alias2)\n    with pytest.raises(PermissionError, match=\"same file as protected\"):\n        improvement_policy.assert_safe_output(alias2)\ndef test_negative_validity_is_drawn_below_zero_not_clamped_onto_it():\n    before = {\"epochs\": [{\"round\": 1, \"coverage\": 0.5}, {\"round\": 2, \"coverage\": 0.5}]}\n    after = {\n        \"policy_version\": 2,\n        \"epochs\": [\n            {\"round\": 1, \"coverage\": 0.5, \"validity\": -1.0},\n            {\"round\": 2, \"coverage\": 0.5, \"validity\": 1.0},\n        ],\n    }\n    svg = render.trigger_chart(before, after, [], 0.8)\n    rects = [line for line in svg.split(\"<rect\") if 'fill=\"#' in line][:2]\n    ys = [float(r.split('y=\"')[1].split('\"')[0]) for r in rects]\n    assert ys[0] > ys[1], \"a -1 correlation must sit lower on the page than +1\"\n    assert ys[0] > 140 + 3, \"a -1 correlation sits below the zero line, not on it\"\n    assert \"right axis, -1..+1\" in svg\n29:   history, automatic rollback.\n42:- 11 of 28 archived findings (39%) matched no topic at all, including every finding from rounds 8 to\n67:  Every version is appended to `docs/improvement-policy-history.jsonl` with a full snapshot.\n69:  the archive, the policy history and the evidence files: autonomy matrix, the loop with live\n87:   undated traces in historical epochs are _unknown_, never zero.\n102:   stay two rounds. The policy in force must equal the snapshot its history recorded for that\n117:   history, validates both destinations before writing either, refuses identical paths, and every\n120:10. **Rendered output is escaped.** Every string from the archive, history or evidence is\n141:recorded here as history — the committed `docs/rsi/trace-evidence.json` keeps per-topic session\n143:are not reproducible from the repository alone): 96 distinct failures across 3 sessions, validity\n144:0.55 against the review signal, 69 blind spots dominated by \"permission denied by the auto-mode\n145:classifier\" (28), tool input errors, and missing tools.\n153:With the verifier's own review sessions counted as the anchor, validity read 0.95 in the first local\n156:keeps only failures, and the verifier's sessions contain none), so that figure is historical and the\n96:          echo \"No Codex credentials secret is configured. Codex review did not run for this PR.\" > $RUNNER_TEMP/codex-review-status.txt\n166:          echo \"No trusted copy of scripts/redact-secrets.py exists on the base branch, so this PR cannot be safely reviewed by this job yet (failing closed rather than trusting the PR's own copy of the redactor).\" > $RUNNER_TEMP/codex-review-status.txt\n180:      # replace the reviewer's output or the redaction inputs.\n321:              body = fs.existsSync(process.env.RUNNER_TEMP + '/codex-review-status.txt')\n322:                ? fs.readFileSync(process.env.RUNNER_TEMP + '/codex-review-status.txt', 'utf8')\n326:              body = fs.existsSync(process.env.RUNNER_TEMP + '/codex-review-status.txt')\n327:                ? fs.readFileSync(process.env.RUNNER_TEMP + '/codex-review-status.txt', 'utf8')\n348:            body = body.replace(/codex-review-(status|sha)/g, 'codex-review-$1');\n357:              body: `### Codex independent review\\n\\n${body}\\n\\n---\\n*Generated by an independent model reviewing this diff with no stake in prior conclusions on it — see [docs/production-hardening-backlog.md](../blob/main/docs/production-hardening-backlog.md) item #4.*\\n<!-- codex-review-status: ${status} -->\\n<!-- codex-review-sha: ${context.payload.pull_request.head.sha} -->`,\n"}
{"chunk_id":"4bf257","wall_time_seconds":0.000150833,"exit_code":0,"original_token_count":63,"output":"............................................................... [ 45%]\n........................................................................ [ 90%]\n................                                                         [100%]\n160 passed in 3.25s\n"}

>sed -n '376,430p' scripts/measure_policy_validity_test.py; sed -n '383,490p' scripts/mine_trace_failures_test.py; sed -n '1158,1333p' scripts/revise_improvement_policy_test.py; sed -n '130,165p' scripts/improvement_policy_test.py; sed -n '272,300p' scripts/render_rsi_dashboard_test.py; rg -n '0.95|96|69|28|reproduc|illustr|histor' docs/plans/recursive-meta-improvement.md; rg -n 'codex-review-sha|replace|codex-review-status' .github/workflows/codex-review.yml

{"chunk_id":"23beae","wall_time_seconds":0.000006667,"exit_code":0,"original_token_count":5952,"output":"            # its classifications (Codex review of PR #10, round 14).\n            suffix += 1\n            name = f\"{base}-{suffix}\"\n        mined.append(\n            {\n                \"name\": name,\n                \"keywords\": topic_keywords,\n                \"evidence\": [\n                    {\"round\": item[\"round\"], \"finding\": item[\"finding\"][:160]} for item in covering\n                ],\n            }\n        )\n        taken.update(topic_keywords)\n        covered_ids = {id(item) for item in covering}\n        remaining = [item for item in remaining if id(item) not in covered_ids]\n        for item in remaining:\n            item[\"tokens\"] -= set(topic_keywords)\n    return mined\n\n\ndef validity_under(policy: dict, entries: list[dict], anchor: dict | None) -> float | None:\n    \"\"\"Validity a policy would score on the same findings and the same\n    anchor counts the measurement carried. Topics without a known anchor\n    count are excluded, exactly as the measurement excludes them.\"\"\"\n    if anchor is None:\n        return None\n    current = measure_mod.measure(entries, policy, None)[\"current\"]\n    known = [t for t in policy[\"topics\"] if anchor.get(t) is not None]\n    return measure_mod.spearman(\n        [float(current[\"dev_weighted\"][t]) for t in known], [float(anchor[t]) for t in known]\n    )\n\n\ndef entries_covered_by_evidence(entries: list[dict], measurement: dict) -> list[dict]:\n    \"\"\"Archive entries from rounds no later than the evidence snapshot's\n    collection time. Without a collection time, all entries.\"\"\"\n    collected_ms = measure_mod.parse_timestamp_ms(\n        (measurement.get(\"anchor\") or {}).get(\"collected_at\")\n    )\n    if collected_ms is None:\n        return entries\n    # By round identity, not number: two reviews sharing a number can\n    # straddle the snapshot time (Codex review of PR #10, round 37).\n    covered = {\n        r[\"key\"]\n        for r in measure_mod.rounds_in_order(entries)\n        if r[\"timestamp_ms\"] is not None and r[\"timestamp_ms\"] <= collected_ms\n    }\n    return [e for e in entries if policy_mod.round_key(e) in covered]\n\n\ndef validity_regressed(before: float | None, after: float | None) -> bool:\n    \"\"\"A candidate may not lower a defined validity, and may not turn a\n    defined validity into an undefined one (a constant weighted signal has\n    no rank agreement to measure, which would silence every later trigger —\n    Codex review of PR #10, round 3, finding 1).\"\"\"\ndef candidate_anchor(current: dict, policy: dict) -> dict | None:\n    \"\"\"Anchor counts for judging a specific policy: every topic the evidence\n    searched, with each of this policy's topics resolved to the evidence\n    searched under exactly its own keyword definition (plain name or a\n    `name@tag` variant kept for an older definition). A topic whose\n    definition was never searched is unknown, never borrowed from another\n    definition of the same name (Codex review of PR #10, rounds 9, 12, 18,\n    32 and 33).\"\"\"\n    anchor = current.get(\"anchor\")\n    if anchor is None:\n        return None\n    evidence = current.get(\"anchor_evidence\")\n    definitions = current.get(\"anchor_definitions\") or {}\n    keywords = policy_mod.topic_keywords(policy)\n    if evidence is None:\n        # A measurement without evidence-wide counts (older format): only\n        # the measured topics are known, and only where definitions match.\n        return {\n            topic: (\n                count\n                if topic not in keywords\n                or list(definitions.get(topic, [])) == list(keywords[topic])\n                else None\n            )\n            for topic, count in anchor.items()\n        }\n    result: dict[str, int | None] = {}\n    for topic, count in evidence.items():\n        if \"@\" in topic or topic in keywords:\n            continue\n        result[topic] = count  # a topic this policy does not define keeps its evidence\n    searched_names = {key.split(\"@\")[0] for key in evidence}\n    for topic, words in keywords.items():\n        if topic not in searched_names:\n            continue  # never searched under any definition: not part of the anchor\n        key = measure_mod.resolve_evidence_key(definitions, topic, list(words))\n        result[topic] = evidence.get(key) if key is not None else None\n    return result\n\n\ndef rolled_back_hashes(history: list[dict]) -> set[str]:\n    \"\"\"Configurations a rollback has already rejected. They are never\n    rollback targets themselves: after v2 is rolled back for validity, the\n    restored v1 would otherwise be judged worse than v2 on coverage two\n    rounds later and 'rolled back' to v2, and so on forever.\"\"\"\n    return {\n        e[\"replaced_policy_hash\"]\n        for e in history\n        if e.get(\"origin\") == \"rollback\" and isinstance(e.get(\"replaced_policy_hash\"), str)\n    }\n\n\ndef rejected_configuration(candidate: dict, history: list[dict], measurement: dict) -> dict | None:\n    \"\"\"A configuration rolled back on the same archive and evidence is not\n    retried: the rollback's history entry records the rejected hash and\n    what it was judged on.\"\"\"\n    wanted = policy_mod.policy_hash(candidate)\n    collected = (measurement.get(\"anchor\") or {}).get(\"collected_at\")\n    for entry in history:\n        if entry.get(\"origin\") != \"rollback\" or entry.get(\"replaced_policy_hash\") != wanted:\n            continue\n        if entry.get(\"archive_digest\") == measurement.get(\"archive_digest\") and (\n            entry.get(\"evidence_collected_at\") == collected\n        ):\n            return entry\n    return None\n\n\ndef unjudged_ancestors(policy: dict, history: list[dict]) -> list[dict]:\n    \"\"\"Snapshots this policy descends from, nearest first, following a\n    rollback through to the ancestry of the configuration it restored, up to\n    and including the first non-revision ancestor. Later evidence must be\n    able to expose a harmful ancestor that a newer revision or a rollback\n    was layered on before validity could be measured (Codex review of\n    PR #10, rounds 14 and 17).\"\"\"\n    chain: list[dict] = []\n    seen: set[int] = set()\n    version = judged_from(policy, history)\n    while isinstance(version, int) and version not in seen:\n        seen.add(version)\n        snapshot = snapshot_for_version(version, history)\n        if snapshot is None:\n            break\n        chain.append(snapshot)\n        if snapshot.get(\"origin\") == \"rollback\":\n            version = judged_from(snapshot, history)\n            continue\n        if snapshot.get(\"origin\") != \"revision\":\n            break\n        version = snapshot.get(\"parent\")\n    return chain\n\n\ndef judged_from(policy: dict, history: list[dict]) -> int | None:\n    \"\"\"Version whose ancestry a policy continues: the parent for a revision;\n    for a rollback, the parent of the restored version.\"\"\"\n    if policy.get(\"origin\") == \"rollback\":\n        restored = policy.get(\"restored_version\")\n        snapshot = snapshot_for_version(restored, history) if isinstance(restored, int) else None\n        if snapshot is None:\n            return None\n        return snapshot.get(\"parent\")\n    return policy.get(\"parent\")\n\n\ndef entries_under(entries: list[dict], policy: dict) -> list[dict]:\n    \"\"\"Archive entries from rounds stamped with this policy's version and\n    hash: the only rounds on which the policy can be judged (Codex review\n    of PR #10, round 25: older rounds decided by an ancestor must not enter\n    the comparison, or they can drown out the revision's own signal).\"\"\"\n    wanted = policy_mod.policy_hash(policy)\n    version = policy[\"version\"]\n    return [\n        e for e in entries if e.get(\"policy_hash\") == wanted and e.get(\"policy_version\") == version\n    ]\n\n\ndef rounds_under(entries: list[dict], policy: dict) -> int:\n    \"\"\"Rounds decided under this exact policy: archive-round.py stamps each\n    round with the policy hash in force when it was archived. Rounds from\n    before a revision was merged never count toward judging it, however\n    long its pull request sat open (Codex review of PR #10, round 4).\"\"\"\n    wanted = policy_mod.policy_hash(policy)\n    version = policy[\"version\"]\n    # Version AND hash: a later revision that recreates an earlier\n    # configuration must not inherit that configuration's old rounds\n    # (Codex review of PR #10, round 7).\n    return len(\n        {\n            measure_mod.round_key(e)\n            for e in entries\n            if e.get(\"policy_hash\") == wanted\n            and e.get(\"policy_version\") == version\n            and isinstance(e.get(\"round\"), int)\n        }\n    )\n\n\ndef decide(\n    entries: list[dict],\n    policy: dict,\n    history: list[dict],\n    measurement: dict,\n    now: str,\n    field_failures: dict | None = None,\n) -> dict:\n    \"\"\"Pure decision: returns {\"action\": \"none\"|\"revise\"|\"rollback\", ...}\n    without touching disk, so it can be tested and dry-run.\"\"\"\n    current = measurement[\"current\"]\n    coverage = current.get(\"coverage\")\n    validity = current.get(\"validity\")\n    keywords = policy_mod.topic_keywords(policy)\n\n    # 1. Safe inheritance: a revision that made things worse gets rolled back\n    #    before any new revision is layered on top of it. Both policies are\n    #    re-measured on the SAME findings: comparing today's coverage with\n    #    the parent's historical number would punish a revision merely for\n    #    being alive when unfamiliar findings arrived (Codex review of\n    #    PR #10, finding 1).\n    base_version = judged_from(policy, history)\n    if policy.get(\"origin\") in {\"revision\", \"rollback\"} and base_version is not None:\n        adopted = adoption_entry(policy[\"version\"], history)\n        if adopted is not None and rounds_under(entries, policy) >= MIN_ROUNDS_TO_JUDGE:\n            parent = snapshot_for_version(base_version, history)\n            if parent is not None and coverage is not None:\n                # Validity is judged only on rounds the evidence snapshot could\n                # have seen; rounds archived after collection would make an\n                # unchanged field look like a regression (Codex review of\n                # PR #10, round 7).\n                covered = entries_covered_by_evidence(entries_under(entries, policy), measurement)\n                # Coverage on the revision's own rounds, kept apart from the\n                # full-archive `coverage` that the trigger and candidate\n                # acceptance use (Codex review of PR #10, round 26).\n                own_rounds = entries_under(entries, policy)\n                parent_now = measure_mod.measure(own_rounds, parent, None)[\"current\"][\"coverage\"]\n                own_coverage = measure_mod.measure(own_rounds, policy, None)[\"current\"][\"coverage\"]\n                child_validity = validity_under(policy, covered, candidate_anchor(current, policy))\n                rejected = rolled_back_hashes(history)\n                worse_coverage = (\n                    parent_now is not None\n                    and own_coverage is not None\n                    and own_coverage < parent_now\n                    and policy_mod.policy_hash(parent) not in rejected\n                )\n                # Compare against every ancestor in the unjudged chain, not\n                # only the parent: the best-scoring ancestor is the rollback\n                # target when the current policy is worse than any of them.\n                best: dict | None = None\n                best_validity: float | None = None\n                # Coverage is compared with every ancestor too: a topic a\n                # grandparent recognised and the parent dropped is invisible\n                # in a parent-only comparison (Codex full-branch review,\n                # finding 5).\n                best_cov: dict | None = None\n                best_cov_value: float | None = None\n                for ancestor in unjudged_ancestors(policy, history):\n                    if policy_mod.policy_hash(ancestor) in rejected:\n                        continue\n                    v = validity_under(ancestor, covered, candidate_anchor(current, ancestor))\n                    if v is not None and (best_validity is None or v > best_validity):\n                        best, best_validity = ancestor, v\n                    c = measure_mod.measure(own_rounds, ancestor, None)[\"current\"][\"coverage\"]\n                    if c is not None and (best_cov_value is None or c > best_cov_value):\n                        best_cov, best_cov_value = ancestor, c\n                if (\n                    best_cov_value is not None\n                    and own_coverage is not None\n                    and best_cov_value > own_coverage\n                ):\n                    worse_coverage = True\n                    parent, parent_now = best_cov, best_cov_value\n                worse_validity = best_validity is not None and (\n                    child_validity is None or child_validity < best_validity\n                )\n                if worse_coverage or worse_validity:\n                    target = parent if worse_coverage else best\n                    assert target is not None\n                    # Record what the restored policy actually scores, not the\n                    # current one's number (Codex review of PR #10, round 15).\n                    target_coverage = measure_mod.measure(entries, target, None)[\"current\"][\n                        \"coverage\"\n                    ]\n                    what = (\n                        f\"coverage {own_coverage} vs {parent_now}\"\n                        if worse_coverage\n                        else f\"validity {child_validity} vs {best_validity}\"\n                    )\n                    return {\n                        \"action\": \"rollback\",\n                        \"reason\": (\n                            f\"on the same {current['findings_total']} findings and anchor, \"\n                            f\"v{policy['version']} scores {what} against v{target['version']}\"\n                        ),\n                        \"policy\": policy_mod.new_version(\n                            policy,\n                            topics=target[\"topics\"],\n                            threshold=target[\"threshold\"],\n                            origin=\"rollback\",\n                            rationale=f\"Rollback to v{target['version']}: v{policy['version']} scored worse ({what}).\",\n                            created_at=now,\n                            restored_version=target[\"version\"],\n                        ),\n                        \"coverage_before\": own_coverage,\n                        \"coverage_after\": target_coverage,\n                        \"validity_before\": child_validity,\n                        \"validity_after\": best_validity if not worse_coverage else None,\n                        \"changes\": [\n                            f\"restored taxonomy, weights and threshold of v{target['version']}\"\n                        ],\n                    }\n\n    # A revision that has not yet been judged must not be built on: a\n    # successor would only ever be compared with it, so a regression it\n    # introduced against ITS parent could never be rolled back (Codex review\n    # of PR #10, round 6). Wait until MIN_ROUNDS_TO_JUDGE rounds have run\n    # under it; the rollback check above already covered the judged case.\n    # The wait does not depend on having an ancestor to compare with: a\n    # rollback to the root version has none, and skipping the wait there let\n    # a rejected configuration be re-proposed after one round (round 36).\n    if policy.get(\"origin\") in {\"revision\", \"rollback\"}:\n        under = rounds_under(entries, policy)\n        if under < MIN_ROUNDS_TO_JUDGE:\n            return {\n                \"action\": \"none\",\n                \"reason\": (\n                    f\"v{policy['version']} has run under {under} round(s); waiting for \"\n                    f\"{MIN_ROUNDS_TO_JUDGE} before judging it or layering another revision\"\n                ),\n            }\n\n    triggers = []\n    if coverage is not None and coverage < MIN_COVERAGE:\n        triggers.append(f\"coverage {coverage} < {MIN_COVERAGE}\")\n    if validity is not None and validity < MIN_VALIDITY:\n        triggers.append(f\"validity {validity} < {MIN_VALIDITY}\")\n    blind = field_blind_spots(field_failures, keywords)\n    coverage_trigger = coverage is not None and coverage < MIN_COVERAGE\n    field_trigger = len(blind) >= MIN_FIELD_BLIND_SPOTS\n    if field_trigger:\n        triggers.append(f\"field: {len(blind)} failures match no topic\")\n\n    changes: list[str] = []\n    new_topics = dict(policy[\"topics\"])\n\n    # 2. Coverage repair: mine the blind spots.\n    # Explicit booleans, not string prefixes, decide what gets mined\n    # (Codex review of PR #10, round 28).\n    mining_input = list(current.get(\"unclassified_findings\", []))\n    if field_trigger or coverage_trigger:\n        mining_input.extend(blind)\n    mined = mine_topics(mining_input, keywords) if (coverage_trigger or field_trigger) else []\n    for topic in mined:\n        new_topics[topic[\"name\"]] = {\n            \"keywords\": topic[\"keywords\"],\n            \"weight\": 1.0,\n            \"mined_from\": topic[\"evidence\"],\n        }\n        changes.append(\n            f\"added topic {topic['name']} (keywords {topic['keywords']}) covering {len(topic['evidence'])} unclassified finding(s)\"\n        )\n\n    # 3. Weight repair: restorations are always evaluated; discounts only\n    #    when validity actually failed.\n    # Every validity comparison in this decision uses the rounds the evidence\n    # snapshot covers, exactly as the rollback check does, so a candidate\n    # cannot pass on later findings and then be rolled back on the snapshot\n    # (Codex review of PR #10, round 8).\n    covered = entries_covered_by_evidence(entries, measurement)\n    weighted_topics, weight_changes = weight_repair(\n        {**policy, \"topics\": new_topics},\n        current,\n        covered,\n        discount=any(t.startswith(\"validity\") for t in triggers),\n    )\n    new_topics = weighted_topics\n    changes.extend(weight_changes)\n\n    if not triggers and not any(c.startswith(\"restored\") for c in changes):\n        return {\n            \"action\": \"none\",\n            \"reason\": \"policy signal still predicts the field within thresholds\",\n        }\n    if not triggers:\n        triggers.append(\"field evidence corroborates a discounted topic\")\n\n    if not changes or all(\n        c.startswith(\"kept weights unchanged\") or c.startswith(\"skipped discounts\") for c in changes\n    ):\n        return {\n            \"action\": \"none\",\n            \"reason\": \"triggered (\"\n            + \"; \".join(triggers)\n            + \") but no bounded, evidence-backed change was available\",\n            \"triggers\": triggers,\n        }\n\n    revised = policy_mod.new_version(\n        policy,\n        topics=new_topics,\n        threshold=policy[\"threshold\"],\n        origin=\"revision\",\n        rationale=\"Revised because \" + \"; \".join(triggers) + \". \" + \" \".join(changes),\n        created_at=now,\n    )\n    # The whole candidate, not just its weight changes, must not regress\n    # validity against the policy it replaces (Codex review of PR #10, round 4).\n    v_before = validity_under(policy, covered, candidate_anchor(current, policy))\n    v_after = validity_under(revised, covered, can…706 tokens truncated…[\n                args.archive_path,\n                args.measurement,\n                args.policy,\n                args.history,\n                args.field_failures,\n            ],\n        )\n    entries = measure_mod.load_archive(args.archive_path)\n    policy = policy_mod.load_policy(args.policy)\n    history = policy_mod.load_history(args.history)\n    try:\n        policy_mod.assert_policy_matches_history(policy, history)\n    except ValueError as exc:\n        print(f\"::error::{exc}\", file=sys.stderr)\n        return 1\n    with open(args.measurement) as f:\n        measurement = json.load(f)\n    if measurement.get(\"policy_hash\") != policy_mod.policy_hash(policy):\n        print(\n            f\"::error::measurement was taken under policy hash {measurement.get('policy_hash')}, \"\n            f\"but {args.policy} hashes to {policy_mod.policy_hash(policy)}; re-measure first\",\n            file=sys.stderr,\n        )\n        return 1\n    digest = measure_mod.archive_digest(entries)\n    if measurement.get(\"archive_digest\") != digest:\n        print(\n            f\"::error::measurement was taken against archive digest {measurement.get('archive_digest')}, \"\n            f\"but {args.archive_path} now digests to {digest}; re-measure first\",\n            file=sys.stderr,\n        )\n        return 1\n    now = args.now or policy_mod.utc_now_iso()\n\n    field_failures = None\n    if args.field_failures:\n        with open(args.field_failures) as f:\n            field_failures = json.load(f)\n    decision = decide(entries, policy, history, measurement, now, field_failures)\n    if decision[\"action\"] == \"none\":\n        print(f\"no revision: {decision['reason']}\")\n    else:\n        verb = \"ROLLBACK\" if decision[\"action\"] == \"rollback\" else \"REVISION\"\n        print(\n            f\"{verb} -> policy v{decision['policy']['version']} (parent v{decision['policy']['parent']}): {decision['reason']}\"\n        )\n        for change in decision[\"changes\"]:\n            print(f\"  - {change}\")\n"}
{"chunk_id":"a69644","wall_time_seconds":0.000003291,"exit_code":0,"original_token_count":4116,"output":"def test_a_scan_that_matched_nothing_is_still_an_observed_field():\n    evidence = _evidence()\n    for topic in evidence[\"topics\"]:\n        evidence[\"topics\"][topic] = []\n    evidence[\"sessions\"] = [\"s1\", \"s2\", \"s3\"]\n    result = measure.measure(_archive(), policy_mod.builtin_policy(), evidence)\n    assert result[\"anchor\"][\"traces_considered\"] == 3\n    assert all(v == 0 for v in result[\"current\"][\"anchor\"].values())\n    # Without a session list, no matches really is no evidence.\n    del evidence[\"sessions\"]\n    current = measure.measure(_archive(), policy_mod.builtin_policy(), evidence)[\"current\"]\n    assert current[\"anchor\"] is None\n\n\ndef test_two_reviews_that_collided_on_a_round_number_stay_two_rounds():\n    # Two archive proposals opened before either merged both got round 11;\n    # their source SHAs differ, so they are two rounds (Codex full-branch\n    # review, workflows finding 2). Legacy placeholder+result pairs without a\n    # SHA still merge by number.\n    entries = [\n        {\n            \"round\": 11,\n            \"source_sha\": \"aaa\",\n            \"findings\": [\"**[P1]** one.\"],\n            \"occurred_at\": \"2026-09-14T15:00:00Z\",\n        },\n        {\n            \"round\": 11,\n            \"source_sha\": \"bbb\",\n            \"findings\": [\"**[P1]** two.\"],\n            \"occurred_at\": \"2026-09-14T15:05:00Z\",\n        },\n        {\"round\": 12, \"findings\": [], \"occurred_at\": \"2026-09-14T15:10:00Z\"},\n        {\"round\": 12, \"findings\": [\"**[P2]** result.\"], \"occurred_at\": \"2026-09-14T15:11:00Z\"},\n    ]\n    rounds = measure.rounds_in_order(entries)\n    assert [(r[\"round\"], r[\"key\"]) for r in rounds] == [\n        (11, \"sha:aaa\"),\n        (11, \"sha:bbb\"),\n        (12, \"round:12\"),\n    ]\n    assert measure.round_key({\"round\": 3}) == \"round:3\"\ndef test_evidence_records_every_scanned_session_even_without_matches(monkeypatch):\n    # A complete scan whose failures are all unclassified is an observed field\n    # with confirmed zero counts, not an empty anchor (Codex, round 36).\n    monkeypatch.setattr(mine, \"run_traces_json\", _fake_runner({\"t1\": _events(), \"t2\": []}))\n    keywords = {\"nothing-matches\": [\"zzzz-never\"]}\n    traces, complete = mine.list_traces(\"traces\", \"/repo\", [\"claude-code\"], 50)\n    failures = [f for t in traces for f in mine.mine_trace(\"traces\", t)]\n    evidence = mine.build_evidence(\n        failures, keywords, \"/repo\", [\"claude-code\"], complete, scanned=[t[\"id\"] for t in traces]\n    )\n    assert evidence[\"topics\"] == {\"nothing-matches\": []}\n    assert evidence[\"sessions\"] == [\"t1\", \"t2\"]\n    assert sys.modules[\"measure_policy_validity\"].evidence_trace_ids(evidence) == {\"t1\", \"t2\"}\n    result = sys.modules[\"measure_policy_validity\"].measure(\n        [\n            {\n                \"round\": 1,\n                \"findings\": [\"**[P1]** zzzz-never happened.\"],\n                \"occurred_at\": \"2026-09-14T15:00:00Z\",\n            }\n        ],\n        {\n            \"version\": 1,\n            \"parent\": None,\n            \"origin\": \"init\",\n            \"threshold\": 3,\n            \"topics\": {\"nothing-matches\": {\"keywords\": [\"zzzz-never\"], \"weight\": 1.0}},\n        },\n        evidence,\n    )\n    assert result[\"current\"][\"anchor\"] == {\"nothing-matches\": 0}\n    assert result[\"anchor\"][\"traces_considered\"] == 2\n\n\ndef test_namespace_mode_syncs_each_shared_session_before_reading_it(tmp_path, monkeypatch, capsys):\n    calls: list[list[str]] = []\n\n    def run(_bin, args):\n        calls.append(list(args))\n        if args[0] == \"list\":\n            assert args[1] == \"@gagan114\" and \"--all\" in args\n            return {\n                \"traces\": [\n                    {\"id\": \"remote-1\", \"agentId\": \"claude-code\", \"timestamp\": 5},\n                    {\"id\": \"remote-2\", \"agentId\": \"codex\", \"timestamp\": 6},\n                ]\n            }\n        if args[0] == \"sync\":\n            return {\"traceId\": args[1]}\n        if args[0] == \"show\":\n            return {\"events\": _events() if args[1] == \"remote-1\" else []}\n        raise AssertionError(args)\n\n    monkeypatch.setattr(mine, \"run_traces_json\", run)\n    monkeypatch.setattr(mine, \"EXTRA_CLI_ARGS\", [])\n    policy = tmp_path / \"policy.json\"\n    policy.write_text(json.dumps(policy_mod.builtin_policy()))\n    evidence = tmp_path / \"evidence.json\"\n    code = mine.main(\n        [\n            \"m\",\n            \"--namespace\",\n            \"gagan114\",\n            \"--agents\",\n            \"claude-code\",\n            \"--traces-key\",\n            \"tr_secret\",\n            \"--policy\",\n            str(policy),\n            \"--history\",\n            str(tmp_path / \"h.jsonl\"),\n            \"--save-evidence\",\n            str(evidence),\n        ]\n    )\n    assert code == 0\n    assert mine.EXTRA_CLI_ARGS == [\"--key\", \"tr_secret\"]\n    assert [\"sync\", \"remote-1\"] in calls\n    assert not any(c[:2] == [\"sync\", \"remote-2\"] for c in calls), \"codex sessions were filtered out\"\n    saved = json.loads(evidence.read_text())\n    assert saved[\"namespace\"] == \"gagan114\"\n    assert saved[\"sessions\"] == [\"remote-1\"]\n    assert saved[\"listing_complete\"] is True\n    assert \"tr_secret\" not in capsys.readouterr().out\ndef test_rollback_to_the_root_version_still_waits_before_being_judged():\n    # judged_from() is None for a rollback to v1 (no parent), which used to\n    # skip the waiting period and let the rejected configuration be\n    # re-proposed after a single clean round (Codex, round 36).\n    v1 = policy_mod.builtin_policy()\n    v2 = policy_mod.new_version(\n        v1, topics=dict(v1[\"topics\"]), threshold=v1[\"threshold\"], origin=\"revision\", rationale=\"x\"\n    )\n    v3 = policy_mod.new_version(\n        v2,\n        topics=dict(v1[\"topics\"]),\n        threshold=v1[\"threshold\"],\n        origin=\"rollback\",\n        rationale=\"back\",\n        restored_version=1,\n    )\n    history = [\n        {\"version\": 2, \"policy\": v2, \"replaced_policy_hash\": policy_mod.policy_hash(v1)},\n        {\n            \"version\": 3,\n            \"origin\": \"rollback\",\n            \"policy\": v3,\n            \"replaced_policy_hash\": policy_mod.policy_hash(v2),\n        },\n    ]\n    entries = [\n        {\n            \"round\": 1,\n            \"findings\": [],\n            \"occurred_at\": \"2026-09-14T15:00:00Z\",\n            \"policy_version\": 3,\n            \"policy_hash\": policy_mod.policy_hash(v3),\n        }\n    ]\n    measurement = measure.measure(entries, v3, None)\n    decision = revise.decide(entries, v3, history, measurement, NOW)\n    assert decision[\"action\"] == \"none\"\n    assert \"waiting for\" in decision[\"reason\"]\n\n\ndef test_a_rolled_back_configuration_is_never_a_rollback_target():\n    # v2 was rolled back; the restored configuration (v3) must not be judged\n    # \"worse than v2\" on coverage two rounds later and rolled back INTO v2,\n    # or the loop would ping-pong forever.\n    v1 = policy_mod.builtin_policy()\n    v2 = policy_mod.new_version(\n        v1,\n        topics={**v1[\"topics\"], \"queue-overflow\": {\"keywords\": [\"queue overflow\"], \"weight\": 1.0}},\n        threshold=v1[\"threshold\"],\n        origin=\"revision\",\n        rationale=\"mined\",\n    )\n    v3 = policy_mod.new_version(\n        v2,\n        topics=dict(v1[\"topics\"]),\n        threshold=v1[\"threshold\"],\n        origin=\"rollback\",\n        rationale=\"worse\",\n        restored_version=1,\n    )\n    history = [\n        {\"version\": 1, \"policy\": v1},\n        {\"version\": 2, \"policy\": v2, \"replaced_policy_hash\": policy_mod.policy_hash(v1)},\n        {\n            \"version\": 3,\n            \"origin\": \"rollback\",\n            \"policy\": v3,\n            \"replaced_policy_hash\": policy_mod.policy_hash(v2),\n        },\n    ]\n    entries = [\n        {**e, \"policy_version\": 3, \"policy_hash\": policy_mod.policy_hash(v3)} for e in _archive()\n    ]\n    decision = revise.decide(entries, v3, history, measure.measure(entries, v3, None), NOW)\n    assert decision[\"action\"] != \"rollback\"\n    assert revise.rolled_back_hashes(history) == {policy_mod.policy_hash(v2)}\n\n\ndef test_coverage_lost_at_a_grandparent_still_triggers_rollback():\n    # v1 knew \"quartz\"; v2 dropped it; v3 added something unrelated. On v3's\n    # own rounds v3 and v2 tie at 0 coverage, so a parent-only comparison\n    # never rolls back, but v1 covers everything (Codex full-branch review,\n    # finding 5).\n    v1 = policy_mod.builtin_policy()\n    v1[\"topics\"][\"quartz-crashes\"] = {\"keywords\": [\"quartz\"], \"weight\": 1.0}\n    v2 = policy_mod.new_version(\n        v1,\n        topics={k: v for k, v in v1[\"topics\"].items() if k != \"quartz-crashes\"},\n        threshold=v1[\"threshold\"],\n        origin=\"revision\",\n        rationale=\"dropped\",\n    )\n    v3 = policy_mod.new_version(\n        v2,\n        topics={**v2[\"topics\"], \"unrelated\": {\"keywords\": [\"zzunrelated\"], \"weight\": 1.0}},\n        threshold=v2[\"threshold\"],\n        origin=\"revision\",\n        rationale=\"added\",\n    )\n    history = [\n        {\"version\": 1, \"policy\": v1},\n        {\"version\": 2, \"policy\": v2, \"replaced_policy_hash\": policy_mod.policy_hash(v1)},\n        {\"version\": 3, \"policy\": v3, \"replaced_policy_hash\": policy_mod.policy_hash(v2)},\n    ]\n    entries = [\n        {\n            \"round\": n,\n            \"occurred_at\": f\"2026-09-14T1{n}:00:00Z\",\n            \"findings\": [f\"**[P2]** quartz renderer crashed again ({n}).\"],\n            \"policy_version\": 3,\n            \"policy_hash\": policy_mod.policy_hash(v3),\n        }\n        for n in (1, 2)\n    ]\n    decision = revise.decide(entries, v3, history, measure.measure(entries, v3, None), NOW)\n    assert decision[\"action\"] == \"rollback\"\n    assert decision[\"policy\"][\"restored_version\"] == 1\n\n\ndef test_main_refuses_a_policy_whose_lineage_metadata_was_edited(tmp_path, monkeypatch, capsys):\n    # Changing only `origin` used to switch the wait gate off: the hash does\n    # not cover lineage metadata (Codex full-branch review, finding 1).\n    v1 = policy_mod.builtin_policy()\n    v2 = policy_mod.new_version(\n        v1, topics=dict(v1[\"topics\"]), threshold=v1[\"threshold\"], origin=\"revision\", rationale=\"x\"\n    )\n    archive = tmp_path / \"archive.jsonl\"\n    archive.write_text(\"\\n\".join(json.dumps(e) for e in _archive()) + \"\\n\")\n    history = tmp_path / \"history.jsonl\"\n    history.write_text(json.dumps({\"version\": 2, \"policy\": v2}) + \"\\n\")\n    edited = {**v2, \"origin\": \"init\", \"parent\": None}\n    policy_path = tmp_path / \"policy.json\"\n    policy_path.write_text(json.dumps(edited))\n    m_path = tmp_path / \"m.json\"\n    m_path.write_text(json.dumps(measure.measure(_archive(), edited, None)))\n    code = revise.main(\n        [\n            \"r\",\n            str(archive),\n            \"--measurement\",\n            str(m_path),\n            \"--policy\",\n            str(policy_path),\n            \"--history\",\n            str(history),\n            \"--dry-run\",\n            \"--now\",\n            NOW,\n        ]\n    )\n    assert code == 1\n    assert \"lineage metadata\" in capsys.readouterr().err\n    with pytest.raises(ValueError, match=\"history records no versions\"):\n        policy_mod.assert_policy_matches_history(v2, [])\n    policy_mod.assert_policy_matches_history(v2, [{\"version\": 2, \"policy\": v2}])\n\n\ndef test_evidence_window_filters_by_round_identity_not_number():\n    # Two reviews share round 5; one predates the snapshot and one does not.\n    entries = [\n        {\n            \"round\": 5,\n            \"source_sha\": \"old\",\n            \"findings\": [\"**[P1]** a.\"],\n            \"occurred_at\": \"2026-09-14T15:00:00Z\",\n        },\n        {\n            \"round\": 5,\n            \"source_sha\": \"new\",\n            \"findings\": [\"**[P1]** b.\"],\n            \"occurred_at\": \"2026-09-14T17:00:00Z\",\n        },\n    ]\n    measurement = {\"anchor\": {\"collected_at\": \"2026-09-14T16:00:00Z\"}}\n    covered = revise.entries_covered_by_evidence(entries, measurement)\n    assert [e[\"source_sha\"] for e in covered] == [\"old\"]\n    policy[\"topics\"][\"empty\"] = {\"keywords\": [], \"weight\": 1.0}\n    with pytest.raises(ValueError, match=\"non-empty\"):\n        improvement_policy.validate_policy(policy)\n\n\ndef test_topic_names_may_not_contain_at_signs_and_origin_is_checked():\n    policy = improvement_policy.builtin_policy()\n    policy[\"topics\"][\"shell-semantics@custom\"] = {\"keywords\": [\"x\"], \"weight\": 1.0}\n    with pytest.raises(ValueError, match=\"'@'\"):\n        improvement_policy.validate_policy(policy)\n    policy = improvement_policy.builtin_policy()\n    policy[\"origin\"] = \"whatever\"\n    with pytest.raises(ValueError, match=\"origin\"):\n        improvement_policy.validate_policy(policy)\n\n\ndef test_hard_linked_outputs_are_refused(tmp_path, monkeypatch):\n    import os\n\n    archive = tmp_path / \"archive.jsonl\"\n    archive.write_text(\"{}\\n\")\n    alias = tmp_path / \"report.json\"\n    os.link(archive, alias)\n    with pytest.raises(PermissionError, match=\"input of this run\"):\n        improvement_policy.assert_safe_output(alias, inputs=[archive])\n    protected = tmp_path / \"docs\" / \"self-improvement-archive.jsonl\"\n    protected.parent.mkdir()\n    protected.write_text(\"{}\\n\")\n    monkeypatch.setattr(improvement_policy, \"REPO_ROOT\", tmp_path)\n    alias2 = tmp_path / \"out.json\"\n    os.link(protected, alias2)\n    with pytest.raises(PermissionError, match=\"same file as protected\"):\n        improvement_policy.assert_safe_output(alias2)\ndef test_negative_validity_is_drawn_below_zero_not_clamped_onto_it():\n    before = {\"epochs\": [{\"round\": 1, \"coverage\": 0.5}, {\"round\": 2, \"coverage\": 0.5}]}\n    after = {\n        \"policy_version\": 2,\n        \"epochs\": [\n            {\"round\": 1, \"coverage\": 0.5, \"validity\": -1.0},\n            {\"round\": 2, \"coverage\": 0.5, \"validity\": 1.0},\n        ],\n    }\n    svg = render.trigger_chart(before, after, [], 0.8)\n    rects = [line for line in svg.split(\"<rect\") if 'fill=\"#' in line][:2]\n    ys = [float(r.split('y=\"')[1].split('\"')[0]) for r in rects]\n    assert ys[0] > ys[1], \"a -1 correlation must sit lower on the page than +1\"\n    assert ys[0] > 140 + 3, \"a -1 correlation sits below the zero line, not on it\"\n    assert \"right axis, -1..+1\" in svg\n29:   history, automatic rollback.\n42:- 11 of 28 archived findings (39%) matched no topic at all, including every finding from rounds 8 to\n67:  Every version is appended to `docs/improvement-policy-history.jsonl` with a full snapshot.\n69:  the archive, the policy history and the evidence files: autonomy matrix, the loop with live\n87:   undated traces in historical epochs are _unknown_, never zero.\n102:   stay two rounds. The policy in force must equal the snapshot its history recorded for that\n117:   history, validates both destinations before writing either, refuses identical paths, and every\n120:10. **Rendered output is escaped.** Every string from the archive, history or evidence is\n141:recorded here as history — the committed `docs/rsi/trace-evidence.json` keeps per-topic session\n143:are not reproducible from the repository alone): 96 distinct failures across 3 sessions, validity\n144:0.55 against the review signal, 69 blind spots dominated by \"permission denied by the auto-mode\n145:classifier\" (28), tool input errors, and missing tools.\n153:With the verifier's own review sessions counted as the anchor, validity read 0.95 in the first local\n156:keeps only failures, and the verifier's sessions contain none), so that figure is historical and the\n96:          echo \"No Codex credentials secret is configured. Codex review did not run for this PR.\" > $RUNNER_TEMP/codex-review-status.txt\n166:          echo \"No trusted copy of scripts/redact-secrets.py exists on the base branch, so this PR cannot be safely reviewed by this job yet (failing closed rather than trusting the PR's own copy of the redactor).\" > $RUNNER_TEMP/codex-review-status.txt\n180:      # replace the reviewer's output or the redaction inputs.\n321:              body = fs.existsSync(process.env.RUNNER_TEMP + '/codex-review-status.txt')\n322:                ? fs.readFileSync(process.env.RUNNER_TEMP + '/codex-review-status.txt', 'utf8')\n326:              body = fs.existsSync(process.env.RUNNER_TEMP + '/codex-review-status.txt')\n327:                ? fs.readFileSync(process.env.RUNNER_TEMP + '/codex-review-status.txt', 'utf8')\n348:            body = body.replace(/codex-review-(status|sha)/g, 'codex-review-$1');\n357:              body: `### Codex independent review\\n\\n${body}\\n\\n---\\n*Generated by an independent model reviewing this diff with no stake in prior conclusions on it — see [docs/production-hardening-backlog.md](../blob/main/docs/production-hardening-backlog.md) item #4.*\\n<!-- codex-review-status: ${status} -->\\n<!-- codex-review-sha: ${context.payload.pull_request.head.sha} -->`,\n"}
{"chunk_id":"4bf257","wall_time_seconds":0.000150833,"exit_code":0,"original_token_count":63,"output":"............................................................... [ 45%]\n........................................................................ [ 90%]\n................                                                         [100%]\n160 passed in 3.25s\n"}

{"chunk_id":"23beae","wall_time_seconds":0.000006667,"exit_code":0,"original_token_count":5952,"output":"            # its classifications (Codex review of PR #10, round 14).\n            suffix += 1\n            name = f\"{base}-{suffix}\"\n        mined.append(\n            {\n                \"name\": name,\n                \"keywords\": topic_keywords,\n                \"evidence\": [\n                    {\"round\": item[\"round\"], \"finding\": item[\"finding\"][:160]} for item in covering\n                ],\n            }\n        )\n        taken.update(topic_keywords)\n        covered_ids = {id(item) for item in covering}\n        remaining = [item for item in remaining if id(item) not in covered_ids]\n        for item in remaining:\n            item[\"tokens\"] -= set(topic_keywords)\n    return mined\n\n\ndef validity_under(policy: dict, entries: list[dict], anchor: dict | None) -> float | None:\n    \"\"\"Validity a policy would score on the same findings and the same\n    anchor counts the measurement carried. Topics without a known anchor\n    count are excluded, exactly as the measurement excludes them.\"\"\"\n    if anchor is None:\n        return None\n    current = measure_mod.measure(entries, policy, None)[\"current\"]\n    known = [t for t in policy[\"topics\"] if anchor.get(t) is not None]\n    return measure_mod.spearman(\n        [float(current[\"dev_weighted\"][t]) for t in known], [float(anchor[t]) for t in known]\n    )\n\n\ndef entries_covered_by_evidence(entries: list[dict], measurement: dict) -> list[dict]:\n    \"\"\"Archive entries from rounds no later than the evidence snapshot's\n    collection time. Without a collection time, all entries.\"\"\"\n    collected_ms = measure_mod.parse_timestamp_ms(\n        (measurement.get(\"anchor\") or {}).get(\"collected_at\")\n    )\n    if collected_ms is None:\n        return entries\n    # By round identity, not number: two reviews sharing a number can\n    # straddle the snapshot time (Codex review of PR #10, round 37).\n    covered = {\n        r[\"key\"]\n        for r in measure_mod.rounds_in_order(entries)\n        if r[\"timestamp_ms\"] is not None and r[\"timestamp_ms\"] <= collected_ms\n    }\n    return [e for e in entries if policy_mod.round_key(e) in covered]\n\n\ndef validity_regressed(before: float | None, after: float | None) -> bool:\n    \"\"\"A candidate may not lower a defined validity, and may not turn a\n    defined validity into an undefined one (a constant weighted signal has\n    no rank agreement to measure, which would silence every later trigger —\n    Codex review of PR #10, round 3, finding 1).\"\"\"\ndef candidate_anchor(current: dict, policy: dict) -> dict | None:\n    \"\"\"Anchor counts for judging a specific policy: every topic the evidence\n    searched, with each of this policy's topics resolved to the evidence\n    searched under exactly its own keyword definition (plain name or a\n    `name@tag` variant kept for an older definition). A topic whose\n    definition was never searched is unknown, never borrowed from another\n    definition of the same name (Codex review of PR #10, rounds 9, 12, 18,\n    32 and 33).\"\"\"\n    anchor = current.get(\"anchor\")\n    if anchor is None:\n        return None\n    evidence = current.get(\"anchor_evidence\")\n    definitions = current.get(\"anchor_definitions\") or {}\n    keywords = policy_mod.topic_keywords(policy)\n    if evidence is None:\n        # A measurement without evidence-wide counts (older format): only\n        # the measured topics are known, and only where definitions match.\n        return {\n            topic: (\n                count\n                if topic not in keywords\n                or list(definitions.get(topic, [])) == list(keywords[topic])\n                else None\n            )\n            for topic, count in anchor.items()\n        }\n    result: dict[str, int | None] = {}\n    for topic, count in evidence.items():\n        if \"@\" in topic or topic in keywords:\n            continue\n        result[topic] = count  # a topic this policy does not define keeps its evidence\n    searched_names = {key.split(\"@\")[0] for key in evidence}\n    for topic, words in keywords.items():\n        if topic not in searched_names:\n            continue  # never searched under any definition: not part of the anchor\n        key = measure_mod.resolve_evidence_key(definitions, topic, list(words))\n        result[topic] = evidence.get(key) if key is not None else None\n    return result\n\n\ndef rolled_back_hashes(history: list[dict]) -> set[str]:\n    \"\"\"Configurations a rollback has already rejected. They are never\n    rollback targets themselves: after v2 is rolled back for validity, the\n    restored v1 would otherwise be judged worse than v2 on coverage two\n    rounds later and 'rolled back' to v2, and so on forever.\"\"\"\n    return {\n        e[\"replaced_policy_hash\"]\n        for e in history\n        if e.get(\"origin\") == \"rollback\" and isinstance(e.get(\"replaced_policy_hash\"), str)\n    }\n\n\ndef rejected_configuration(candidate: dict, history: list[dict], measurement: dict) -> dict | None:\n    \"\"\"A configuration rolled back on the same archive and evidence is not\n    retried: the rollback's history entry records the rejected hash and\n    what it was judged on.\"\"\"\n    wanted = policy_mod.policy_hash(candidate)\n    collected = (measurement.get(\"anchor\") or {}).get(\"collected_at\")\n    for entry in history:\n        if entry.get(\"origin\") != \"rollback\" or entry.get(\"replaced_policy_hash\") != wanted:\n            continue\n        if entry.get(\"archive_digest\") == measurement.get(\"archive_digest\") and (\n            entry.get(\"evidence_collected_at\") == collected\n        ):\n            return entry\n    return None\n\n\ndef unjudged_ancestors(policy: dict, history: list[dict]) -> list[dict]:\n    \"\"\"Snapshots this policy descends from, nearest first, following a\n    rollback through to the ancestry of the configuration it restored, up to\n    and including the first non-revision ancestor. Later evidence must be\n    able to expose a harmful ancestor that a newer revision or a rollback\n    was layered on before validity could be measured (Codex review of\n    PR #10, rounds 14 and 17).\"\"\"\n    chain: list[dict] = []\n    seen: set[int] = set()\n    version = judged_from(policy, history)\n    while isinstance(version, int) and version not in seen:\n        seen.add(version)\n        snapshot = snapshot_for_version(version, history)\n        if snapshot is None:\n            break\n        chain.append(snapshot)\n        if snapshot.get(\"origin\") == \"rollback\":\n            version = judged_from(snapshot, history)\n            continue\n        if snapshot.get(\"origin\") != \"revision\":\n            break\n        version = snapshot.get(\"parent\")\n    return chain\n\n\ndef judged_from(policy: dict, history: list[dict]) -> int | None:\n    \"\"\"Version whose ancestry a policy continues: the parent for a revision;\n    for a rollback, the parent of the restored version.\"\"\"\n    if policy.get(\"origin\") == \"rollback\":\n        restored = policy.get(\"restored_version\")\n        snapshot = snapshot_for_version(restored, history) if isinstance(restored, int) else None\n        if snapshot is None:\n            return None\n        return snapshot.get(\"parent\")\n    return policy.get(\"parent\")\n\n\ndef entries_under(entries: list[dict], policy: dict) -> list[dict]:\n    \"\"\"Archive entries from rounds stamped with this policy's version and\n    hash: the only rounds on which the policy can be judged (Codex review\n    of PR #10, round 25: older rounds decided by an ancestor must not enter\n    the comparison, or they can drown out the revision's own signal).\"\"\"\n    wanted = policy_mod.policy_hash(policy)\n    version = policy[\"version\"]\n    return [\n        e for e in entries if e.get(\"policy_hash\") == wanted and e.get(\"policy_version\") == version\n    ]\n\n\ndef rounds_under(entries: list[dict], policy: dict) -> int:\n    \"\"\"Rounds decided under this exact policy: archive-round.py stamps each\n    round with the policy hash in force when it was archived. Rounds from\n    before a revision was merged never count toward judging it, however\n    long its pull request sat open (Codex review of PR #10, round 4).\"\"\"\n    wanted = policy_mod.policy_hash(policy)\n    version = policy[\"version\"]\n    # Version AND hash: a later revision that recreates an earlier\n    # configuration must not inherit that configuration's old rounds\n    # (Codex review of PR #10, round 7).\n    return len(\n        {\n            measure_mod.round_key(e)\n            for e in entries\n            if e.get(\"policy_hash\") == wanted\n            and e.get(\"policy_version\") == version\n            and isinstance(e.get(\"round\"), int)\n        }\n    )\n\n\ndef decide(\n    entries: list[dict],\n    policy: dict,\n    history: list[dict],\n    measurement: dict,\n    now: str,\n    field_failures: dict | None = None,\n) -> dict:\n    \"\"\"Pure decision: returns {\"action\": \"none\"|\"revise\"|\"rollback\", ...}\n    without touching disk, so it can be tested and dry-run.\"\"\"\n    current = measurement[\"current\"]\n    coverage = current.get(\"coverage\")\n    validity = current.get(\"validity\")\n    keywords = policy_mod.topic_keywords(policy)\n\n    # 1. Safe inheritance: a revision that made things worse gets rolled back\n    #    before any new revision is layered on top of it. Both policies are\n    #    re-measured on the SAME findings: comparing today's coverage with\n    #    the parent's historical number would punish a revision merely for\n    #    being alive when unfamiliar findings arrived (Codex review of\n    #    PR #10, finding 1).\n    base_version = judged_from(policy, history)\n    if policy.get(\"origin\") in {\"revision\", \"rollback\"} and base_version is not None:\n        adopted = adoption_entry(policy[\"version\"], history)\n        if adopted is not None and rounds_under(entries, policy) >= MIN_ROUNDS_TO_JUDGE:\n            parent = snapshot_for_version(base_version, history)\n            if parent is not None and coverage is not None:\n                # Validity is judged only on rounds the evidence snapshot could\n                # have seen; rounds archived after collection would make an\n                # unchanged field look like a regression (Codex review of\n                # PR #10, round 7).\n                covered = entries_covered_by_evidence(entries_under(entries, policy), measurement)\n                # Coverage on the revision's own rounds, kept apart from the\n                # full-archive `coverage` that the trigger and candidate\n                # acceptance use (Codex review of PR #10, round 26).\n                own_rounds = entries_under(entries, policy)\n                parent_now = measure_mod.measure(own_rounds, parent, None)[\"current\"][\"coverage\"]\n                own_coverage = measure_mod.measure(own_rounds, policy, None)[\"current\"][\"coverage\"]\n                child_validity = validity_under(policy, covered, candidate_anchor(current, policy))\n                rejected = rolled_back_hashes(history)\n                worse_coverage = (\n                    parent_now is not None\n                    and own_coverage is not None\n                    and own_coverage < parent_now\n                    and policy_mod.policy_hash(parent) not in rejected\n                )\n                # Compare against every ancestor in the unjudged chain, not\n                # only the parent: the best-scoring ancestor is the rollback\n                # target when the current policy is worse than any of them.\n                best: dict | None = None\n                best_validity: float | None = None\n                # Coverage is compared with every ancestor too: a topic a\n                # grandparent recognised and the parent dropped is invisible\n                # in a parent-only comparison (Codex full-branch review,\n                # finding 5).\n                best_cov: dict | None = None\n                best_cov_value: float | None = None\n                for ancestor in unjudged_ancestors(policy, history):\n                    if policy_mod.policy_hash(ancestor) in rejected:\n                        continue\n                    v = validity_under(ancestor, covered, candidate_anchor(current, ancestor))\n                    if v is not None and (best_validity is None or v > best_validity):\n                        best, best_validity = ancestor, v\n                    c = measure_mod.measure(own_rounds, ancestor, None)[\"current\"][\"coverage\"]\n                    if c is not None and (best_cov_value is None or c > best_cov_value):\n                        best_cov, best_cov_value = ancestor, c\n                if (\n                    best_cov_value is not None\n                    and own_coverage is not None\n                    and best_cov_value > own_coverage\n                ):\n                    worse_coverage = True\n                    parent, parent_now = best_cov, best_cov_value\n                worse_validity = best_validity is not None and (\n                    child_validity is None or child_validity < best_validity\n                )\n                if worse_coverage or worse_validity:\n                    target = parent if worse_coverage else best\n                    assert target is not None\n                    # Record what the restored policy actually scores, not the\n                    # current one's number (Codex review of PR #10, round 15).\n                    target_coverage = measure_mod.measure(entries, target, None)[\"current\"][\n                        \"coverage\"\n                    ]\n                    what = (\n                        f\"coverage {own_coverage} vs {parent_now}\"\n                        if worse_coverage\n                        else f\"validity {child_validity} vs {best_validity}\"\n                    )\n                    return {\n                        \"action\": \"rollback\",\n                        \"reason\": (\n                            f\"on the same {current['findings_total']} findings and anchor, \"\n                            f\"v{policy['version']} scores {what} against v{target['version']}\"\n                        ),\n                        \"policy\": policy_mod.new_version(\n                            policy,\n                            topics=target[\"topics\"],\n                            threshold=target[\"threshold\"],\n                            origin=\"rollback\",\n                            rationale=f\"Rollback to v{target['version']}: v{policy['version']} scored worse ({what}).\",\n                            created_at=now,\n                            restored_version=target[\"version\"],\n                        ),\n                        \"coverage_before\": own_coverage,\n                        \"coverage_after\": target_coverage,\n                        \"validity_before\": child_validity,\n                        \"validity_after\": best_validity if not worse_coverage else None,\n                        \"changes\": [\n                            f\"restored taxonomy, weights and threshold of v{target['version']}\"\n                        ],\n                    }\n\n    # A revision that has not yet been judged must not be built on: a\n    # successor would only ever be compared with it, so a regression it\n    # introduced against ITS parent could never be rolled back (Codex review\n    # of PR #10, round 6). Wait until MIN_ROUNDS_TO_JUDGE rounds have run\n    # under it; the rollback check above already covered the judged case.\n    # The wait does not depend on having an ancestor to compare with: a\n    # rollback to the root version has none, and skipping the wait there let\n    # a rejected configuration be re-proposed after one round (round 36).\n    if policy.get(\"origin\") in {\"revision\", \"rollback\"}:\n        under = rounds_under(entries, policy)\n        if under < MIN_ROUNDS_TO_JUDGE:\n            return {\n                \"action\": \"none\",\n                \"reason\": (\n                    f\"v{policy['version']} has run under {under} round(s); waiting for \"\n                    f\"{MIN_ROUNDS_TO_JUDGE} before judging it or layering another revision\"\n                ),\n            }\n\n    triggers = []\n    if coverage is not None and coverage < MIN_COVERAGE:\n        triggers.append(f\"coverage {coverage} < {MIN_COVERAGE}\")\n    if validity is not None and validity < MIN_VALIDITY:\n        triggers.append(f\"validity {validity} < {MIN_VALIDITY}\")\n    blind = field_blind_spots(field_failures, keywords)\n    coverage_trigger = coverage is not None and coverage < MIN_COVERAGE\n    field_trigger = len(blind) >= MIN_FIELD_BLIND_SPOTS\n    if field_trigger:\n        triggers.append(f\"field: {len(blind)} failures match no topic\")\n\n    changes: list[str] = []\n    new_topics = dict(policy[\"topics\"])\n\n    # 2. Coverage repair: mine the blind spots.\n    # Explicit booleans, not string prefixes, decide what gets mined\n    # (Codex review of PR #10, round 28).\n    mining_input = list(current.get(\"unclassified_findings\", []))\n    if field_trigger or coverage_trigger:\n        mining_input.extend(blind)\n    mined = mine_topics(mining_input, keywords) if (coverage_trigger or field_trigger) else []\n    for topic in mined:\n        new_topics[topic[\"name\"]] = {\n            \"keywords\": topic[\"keywords\"],\n            \"weight\": 1.0,\n            \"mined_from\": topic[\"evidence\"],\n        }\n        changes.append(\n            f\"added topic {topic['name']} (keywords {topic['keywords']}) covering {len(topic['evidence'])} unclassified finding(s)\"\n        )\n\n    # 3. Weight repair: restorations are always evaluated; discounts only\n    #    when validity actually failed.\n    # Every validity comparison in this decision uses the rounds the evidence\n    # snapshot covers, exactly as the rollback check does, so a candidate\n    # cannot pass on later findings and then be rolled back on the snapshot\n    # (Codex review of PR #10, round 8).\n    covered = entries_covered_by_evidence(entries, measurement)\n    weighted_topics, weight_changes = weight_repair(\n        {**policy, \"topics\": new_topics},\n        current,\n        covered,\n        discount=any(t.startswith(\"validity\") for t in triggers),\n    )\n    new_topics = weighted_topics\n    changes.extend(weight_changes)\n\n    if not triggers and not any(c.startswith(\"restored\") for c in changes):\n        return {\n            \"action\": \"none\",\n            \"reason\": \"policy signal still predicts the field within thresholds\",\n        }\n    if not triggers:\n        triggers.append(\"field evidence corroborates a discounted topic\")\n\n    if not changes or all(\n        c.startswith(\"kept weights unchanged\") or c.startswith(\"skipped discounts\") for c in changes\n    ):\n        return {\n            \"action\": \"none\",\n            \"reason\": \"triggered (\"\n            + \"; \".join(triggers)\n            + \") but no bounded, evidence-backed change was available\",\n            \"triggers\": triggers,\n        }\n\n    revised = policy_mod.new_version(\n        policy,\n        topics=new_topics,\n        threshold=policy[\"threshold\"],\n        origin=\"revision\",\n        rationale=\"Revised because \" + \"; \".join(triggers) + \". \" + \" \".join(changes),\n        created_at=now,\n    )\n    # The whole candidate, not just its weight changes, must not regress\n    # validity against the policy it replaces (Codex review of PR #10, round 4).\n    v_before = validity_under(policy, covered, candidate_anchor(current, policy))\n    v_after = validity_under(revised, covered, can…706 tokens truncated…[\n                args.archive_path,\n                args.measurement,\n                args.policy,\n                args.history,\n                args.field_failures,\n            ],\n        )\n    entries = measure_mod.load_archive(args.archive_path)\n    policy = policy_mod.load_policy(args.policy)\n    history = policy_mod.load_history(args.history)\n    try:\n        policy_mod.assert_policy_matches_history(policy, history)\n    except ValueError as exc:\n        print(f\"::error::{exc}\", file=sys.stderr)\n        return 1\n    with open(args.measurement) as f:\n        measurement = json.load(f)\n    if measurement.get(\"policy_hash\") != policy_mod.policy_hash(policy):\n        print(\n            f\"::error::measurement was taken under policy hash {measurement.get('policy_hash')}, \"\n            f\"but {args.policy} hashes to {policy_mod.policy_hash(policy)}; re-measure first\",\n            file=sys.stderr,\n        )\n        return 1\n    digest = measure_mod.archive_digest(entries)\n    if measurement.get(\"archive_digest\") != digest:\n        print(\n            f\"::error::measurement was taken against archive digest {measurement.get('archive_digest')}, \"\n            f\"but {args.archive_path} now digests to {digest}; re-measure first\",\n            file=sys.stderr,\n        )\n        return 1\n    now = args.now or policy_mod.utc_now_iso()\n\n    field_failures = None\n    if args.field_failures:\n        with open(args.field_failures) as f:\n            field_failures = json.load(f)\n    decision = decide(entries, policy, history, measurement, now, field_failures)\n    if decision[\"action\"] == \"none\":\n        print(f\"no revision: {decision['reason']}\")\n    else:\n        verb = \"ROLLBACK\" if decision[\"action\"] == \"rollback\" else \"REVISION\"\n        print(\n            f\"{verb} -> policy v{decision['policy']['version']} (parent v{decision['policy']['parent']}): {decision['reason']}\"\n        )\n        for change in decision[\"changes\"]:\n            print(f\"  - {change}\")\n"}
{"chunk_id":"a69644","wall_time_seconds":0.000003291,"exit_code":0,"original_token_count":4116,"output":"def test_a_scan_that_matched_nothing_is_still_an_observed_field():\n    evidence = _evidence()\n    for topic in evidence[\"topics\"]:\n        evidence[\"topics\"][topic] = []\n    evidence[\"sessions\"] = [\"s1\", \"s2\", \"s3\"]\n    result = measure.measure(_archive(), policy_mod.builtin_policy(), evidence)\n    assert result[\"anchor\"][\"traces_considered\"] == 3\n    assert all(v == 0 for v in result[\"current\"][\"anchor\"].values())\n    # Without a session list, no matches really is no evidence.\n    del evidence[\"sessions\"]\n    current = measure.measure(_archive(), policy_mod.builtin_policy(), evidence)[\"current\"]\n    assert current[\"anchor\"] is None\n\n\ndef test_two_reviews_that_collided_on_a_round_number_stay_two_rounds():\n    # Two archive proposals opened before either merged both got round 11;\n    # their source SHAs differ, so they are two rounds (Codex full-branch\n    # review, workflows finding 2). Legacy placeholder+result pairs without a\n    # SHA still merge by number.\n    entries = [\n        {\n            \"round\": 11,\n            \"source_sha\": \"aaa\",\n            \"findings\": [\"**[P1]** one.\"],\n            \"occurred_at\": \"2026-09-14T15:00:00Z\",\n        },\n        {\n            \"round\": 11,\n            \"source_sha\": \"bbb\",\n            \"findings\": [\"**[P1]** two.\"],\n            \"occurred_at\": \"2026-09-14T15:05:00Z\",\n        },\n        {\"round\": 12, \"findings\": [], \"occurred_at\": \"2026-09-14T15:10:00Z\"},\n        {\"round\": 12, \"findings\": [\"**[P2]** result.\"], \"occurred_at\": \"2026-09-14T15:11:00Z\"},\n    ]\n    rounds = measure.rounds_in_order(entries)\n    assert [(r[\"round\"], r[\"key\"]) for r in rounds] == [\n        (11, \"sha:aaa\"),\n        (11, \"sha:bbb\"),\n        (12, \"round:12\"),\n    ]\n    assert measure.round_key({\"round\": 3}) == \"round:3\"\ndef test_evidence_records_every_scanned_session_even_without_matches(monkeypatch):\n    # A complete scan whose failures are all unclassified is an observed field\n    # with confirmed zero counts, not an empty anchor (Codex, round 36).\n    monkeypatch.setattr(mine, \"run_traces_json\", _fake_runner({\"t1\": _events(), \"t2\": []}))\n    keywords = {\"nothing-matches\": [\"zzzz-never\"]}\n    traces, complete = mine.list_traces(\"traces\", \"/repo\", [\"claude-code\"], 50)\n    failures = [f for t in traces for f in mine.mine_trace(\"traces\", t)]\n    evidence = mine.build_evidence(\n        failures, keywords, \"/repo\", [\"claude-code\"], complete, scanned=[t[\"id\"] for t in traces]\n    )\n    assert evidence[\"topics\"] == {\"nothing-matches\": []}\n    assert evidence[\"sessions\"] == [\"t1\", \"t2\"]\n    assert sys.modules[\"measure_policy_validity\"].evidence_trace_ids(evidence) == {\"t1\", \"t2\"}\n    result = sys.modules[\"measure_policy_validity\"].measure(\n        [\n            {\n                \"round\": 1,\n                \"findings\": [\"**[P1]** zzzz-never happened.\"],\n                \"occurred_at\": \"2026-09-14T15:00:00Z\",\n            }\n        ],\n        {\n            \"version\": 1,\n            \"parent\": None,\n            \"origin\": \"init\",\n            \"threshold\": 3,\n            \"topics\": {\"nothing-matches\": {\"keywords\": [\"zzzz-never\"], \"weight\": 1.0}},\n        },\n        evidence,\n    )\n    assert result[\"current\"][\"anchor\"] == {\"nothing-matches\": 0}\n    assert result[\"anchor\"][\"traces_considered\"] == 2\n\n\ndef test_namespace_mode_syncs_each_shared_session_before_reading_it(tmp_path, monkeypatch, capsys):\n    calls: list[list[str]] = []\n\n    def run(_bin, args):\n        calls.append(list(args))\n        if args[0] == \"list\":\n            assert args[1] == \"@gagan114\" and \"--all\" in args\n            return {\n                \"traces\": [\n                    {\"id\": \"remote-1\", \"agentId\": \"claude-code\", \"timestamp\": 5},\n                    {\"id\": \"remote-2\", \"agentId\": \"codex\", \"timestamp\": 6},\n                ]\n            }\n        if args[0] == \"sync\":\n            return {\"traceId\": args[1]}\n        if args[0] == \"show\":\n            return {\"events\": _events() if args[1] == \"remote-1\" else []}\n        raise AssertionError(args)\n\n    monkeypatch.setattr(mine, \"run_traces_json\", run)\n    monkeypatch.setattr(mine, \"EXTRA_CLI_ARGS\", [])\n    policy = tmp_path / \"policy.json\"\n    policy.write_text(json.dumps(policy_mod.builtin_policy()))\n    evidence = tmp_path / \"evidence.json\"\n    code = mine.main(\n        [\n            \"m\",\n            \"--namespace\",\n            \"gagan114\",\n            \"--agents\",\n            \"claude-code\",\n            \"--traces-key\",\n            \"tr_secret\",\n            \"--policy\",\n            str(policy),\n            \"--history\",\n            str(tmp_path / \"h.jsonl\"),\n            \"--save-evidence\",\n            str(evidence),\n        ]\n    )\n    assert code == 0\n    assert mine.EXTRA_CLI_ARGS == [\"--key\", \"tr_secret\"]\n    assert [\"sync\", \"remote-1\"] in calls\n    assert not any(c[:2] == [\"sync\", \"remote-2\"] for c in calls), \"codex sessions were filtered out\"\n    saved = json.loads(evidence.read_text())\n    assert saved[\"namespace\"] == \"gagan114\"\n    assert saved[\"sessions\"] == [\"remote-1\"]\n    assert saved[\"listing_complete\"] is True\n    assert \"tr_secret\" not in capsys.readouterr().out\ndef test_rollback_to_the_root_version_still_waits_before_being_judged():\n    # judged_from() is None for a rollback to v1 (no parent), which used to\n    # skip the waiting period and let the rejected configuration be\n    # re-proposed after a single clean round (Codex, round 36).\n    v1 = policy_mod.builtin_policy()\n    v2 = policy_mod.new_version(\n        v1, topics=dict(v1[\"topics\"]), threshold=v1[\"threshold\"], origin=\"revision\", rationale=\"x\"\n    )\n    v3 = policy_mod.new_version(\n        v2,\n        topics=dict(v1[\"topics\"]),\n        threshold=v1[\"threshold\"],\n        origin=\"rollback\",\n        rationale=\"back\",\n        restored_version=1,\n    )\n    history = [\n        {\"version\": 2, \"policy\": v2, \"replaced_policy_hash\": policy_mod.policy_hash(v1)},\n        {\n            \"version\": 3,\n            \"origin\": \"rollback\",\n            \"policy\": v3,\n            \"replaced_policy_hash\": policy_mod.policy_hash(v2),\n        },\n    ]\n    entries = [\n        {\n            \"round\": 1,\n            \"findings\": [],\n            \"occurred_at\": \"2026-09-14T15:00:00Z\",\n            \"policy_version\": 3,\n            \"policy_hash\": policy_mod.policy_hash(v3),\n        }\n    ]\n    measurement = measure.measure(entries, v3, None)\n    decision = revise.decide(entries, v3, history, measurement, NOW)\n    assert decision[\"action\"] == \"none\"\n    assert \"waiting for\" in decision[\"reason\"]\n\n\ndef test_a_rolled_back_configuration_is_never_a_rollback_target():\n    # v2 was rolled back; the restored configuration (v3) must not be judged\n    # \"worse than v2\" on coverage two rounds later and rolled back INTO v2,\n    # or the loop would ping-pong forever.\n    v1 = policy_mod.builtin_policy()\n    v2 = policy_mod.new_version(\n        v1,\n        topics={**v1[\"topics\"], \"queue-overflow\": {\"keywords\": [\"queue overflow\"], \"weight\": 1.0}},\n        threshold=v1[\"threshold\"],\n        origin=\"revision\",\n        rationale=\"mined\",\n    )\n    v3 = policy_mod.new_version(\n        v2,\n        topics=dict(v1[\"topics\"]),\n        threshold=v1[\"threshold\"],\n        origin=\"rollback\",\n        rationale=\"worse\",\n        restored_version=1,\n    )\n    history = [\n        {\"version\": 1, \"policy\": v1},\n        {\"version\": 2, \"policy\": v2, \"replaced_policy_hash\": policy_mod.policy_hash(v1)},\n        {\n            \"version\": 3,\n            \"origin\": \"rollback\",\n            \"policy\": v3,\n            \"replaced_policy_hash\": policy_mod.policy_hash(v2),\n        },\n    ]\n    entries = [\n        {**e, \"policy_version\": 3, \"policy_hash\": policy_mod.policy_hash(v3)} for e in _archive()\n    ]\n    decision = revise.decide(entries, v3, history, measure.measure(entries, v3, None), NOW)\n    assert decision[\"action\"] != \"rollback\"\n    assert revise.rolled_back_hashes(history) == {policy_mod.policy_hash(v2)}\n\n\ndef test_coverage_lost_at_a_grandparent_still_triggers_rollback():\n    # v1 knew \"quartz\"; v2 dropped it; v3 added something unrelated. On v3's\n    # own rounds v3 and v2 tie at 0 coverage, so a parent-only comparison\n    # never rolls back, but v1 covers everything (Codex full-branch review,\n    # finding 5).\n    v1 = policy_mod.builtin_policy()\n    v1[\"topics\"][\"quartz-crashes\"] = {\"keywords\": [\"quartz\"], \"weight\": 1.0}\n    v2 = policy_mod.new_version(\n        v1,\n        topics={k: v for k, v in v1[\"topics\"].items() if k != \"quartz-crashes\"},\n        threshold=v1[\"threshold\"],\n        origin=\"revision\",\n        rationale=\"dropped\",\n    )\n    v3 = policy_mod.new_version(\n        v2,\n        topics={**v2[\"topics\"], \"unrelated\": {\"keywords\": [\"zzunrelated\"], \"weight\": 1.0}},\n        threshold=v2[\"threshold\"],\n        origin=\"revision\",\n        rationale=\"added\",\n    )\n    history = [\n        {\"version\": 1, \"policy\": v1},\n        {\"version\": 2, \"policy\": v2, \"replaced_policy_hash\": policy_mod.policy_hash(v1)},\n        {\"version\": 3, \"policy\": v3, \"replaced_policy_hash\": policy_mod.policy_hash(v2)},\n    ]\n    entries = [\n        {\n            \"round\": n,\n            \"occurred_at\": f\"2026-09-14T1{n}:00:00Z\",\n            \"findings\": [f\"**[P2]** quartz renderer crashed again ({n}).\"],\n            \"policy_version\": 3,\n            \"policy_hash\": policy_mod.policy_hash(v3),\n        }\n        for n in (1, 2)\n    ]\n    decision = revise.decide(entries, v3, history, measure.measure(entries, v3, None), NOW)\n    assert decision[\"action\"] == \"rollback\"\n    assert decision[\"policy\"][\"restored_version\"] == 1\n\n\ndef test_main_refuses_a_policy_whose_lineage_metadata_was_edited(tmp_path, monkeypatch, capsys):\n    # Changing only `origin` used to switch the wait gate off: the hash does\n    # not cover lineage metadata (Codex full-branch review, finding 1).\n    v1 = policy_mod.builtin_policy()\n    v2 = policy_mod.new_version(\n        v1, topics=dict(v1[\"topics\"]), threshold=v1[\"threshold\"], origin=\"revision\", rationale=\"x\"\n    )\n    archive = tmp_path / \"archive.jsonl\"\n    archive.write_text(\"\\n\".join(json.dumps(e) for e in _archive()) + \"\\n\")\n    history = tmp_path / \"history.jsonl\"\n    history.write_text(json.dumps({\"version\": 2, \"policy\": v2}) + \"\\n\")\n    edited = {**v2, \"origin\": \"init\", \"parent\": None}\n    policy_path = tmp_path / \"policy.json\"\n    policy_path.write_text(json.dumps(edited))\n    m_path = tmp_path / \"m.json\"\n    m_path.write_text(json.dumps(measure.measure(_archive(), edited, None)))\n    code = revise.main(\n        [\n            \"r\",\n            str(archive),\n            \"--measurement\",\n            str(m_path),\n            \"--policy\",\n            str(policy_path),\n            \"--history\",\n            str(history),\n            \"--dry-run\",\n            \"--now\",\n            NOW,\n        ]\n    )\n    assert code == 1\n    assert \"lineage metadata\" in capsys.readouterr().err\n    with pytest.raises(ValueError, match=\"history records no versions\"):\n        policy_mod.assert_policy_matches_history(v2, [])\n    policy_mod.assert_policy_matches_history(v2, [{\"version\": 2, \"policy\": v2}])\n\n\ndef test_evidence_window_filters_by_round_identity_not_number():\n    # Two reviews share round 5; one predates the snapshot and one does not.\n    entries = [\n        {\n            \"round\": 5,\n            \"source_sha\": \"old\",\n            \"findings\": [\"**[P1]** a.\"],\n            \"occurred_at\": \"2026-09-14T15:00:00Z\",\n        },\n        {\n            \"round\": 5,\n            \"source_sha\": \"new\",\n            \"findings\": [\"**[P1]** b.\"],\n            \"occurred_at\": \"2026-09-14T17:00:00Z\",\n        },\n    ]\n    measurement = {\"anchor\": {\"collected_at\": \"2026-09-14T16:00:00Z\"}}\n    covered = revise.entries_covered_by_evidence(entries, measurement)\n    assert [e[\"source_sha\"] for e in covered] == [\"old\"]\n    policy[\"topics\"][\"empty\"] = {\"keywords\": [], \"weight\": 1.0}\n    with pytest.raises(ValueError, match=\"non-empty\"):\n        improvement_policy.validate_policy(policy)\n\n\ndef test_topic_names_may_not_contain_at_signs_and_origin_is_checked():\n    policy = improvement_policy.builtin_policy()\n    policy[\"topics\"][\"shell-semantics@custom\"] = {\"keywords\": [\"x\"], \"weight\": 1.0}\n    with pytest.raises(ValueError, match=\"'@'\"):\n        improvement_policy.validate_policy(policy)\n    policy = improvement_policy.builtin_policy()\n    policy[\"origin\"] = \"whatever\"\n    with pytest.raises(ValueError, match=\"origin\"):\n        improvement_policy.validate_policy(policy)\n\n\ndef test_hard_linked_outputs_are_refused(tmp_path, monkeypatch):\n    import os\n\n    archive = tmp_path / \"archive.jsonl\"\n    archive.write_text(\"{}\\n\")\n    alias = tmp_path / \"report.json\"\n    os.link(archive, alias)\n    with pytest.raises(PermissionError, match=\"input of this run\"):\n        improvement_policy.assert_safe_output(alias, inputs=[archive])\n    protected = tmp_path / \"docs\" / \"self-improvement-archive.jsonl\"\n    protected.parent.mkdir()\n    protected.write_text(\"{}\\n\")\n    monkeypatch.setattr(improvement_policy, \"REPO_ROOT\", tmp_path)\n    alias2 = tmp_path / \"out.json\"\n    os.link(protected, alias2)\n    with pytest.raises(PermissionError, match=\"same file as protected\"):\n        improvement_policy.assert_safe_output(alias2)\ndef test_negative_validity_is_drawn_below_zero_not_clamped_onto_it():\n    before = {\"epochs\": [{\"round\": 1, \"coverage\": 0.5}, {\"round\": 2, \"coverage\": 0.5}]}\n    after = {\n        \"policy_version\": 2,\n        \"epochs\": [\n            {\"round\": 1, \"coverage\": 0.5, \"validity\": -1.0},\n            {\"round\": 2, \"coverage\": 0.5, \"validity\": 1.0},\n        ],\n    }\n    svg = render.trigger_chart(before, after, [], 0.8)\n    rects = [line for line in svg.split(\"<rect\") if 'fill=\"#' in line][:2]\n    ys = [float(r.split('y=\"')[1].split('\"')[0]) for r in rects]\n    assert ys[0] > ys[1], \"a -1 correlation must sit lower on the page than +1\"\n    assert ys[0] > 140 + 3, \"a -1 correlation sits below the zero line, not on it\"\n    assert \"right axis, -1..+1\" in svg\n29:   history, automatic rollback.\n42:- 11 of 28 archived findings (39%) matched no topic at all, including every finding from rounds 8 to\n67:  Every version is appended to `docs/improvement-policy-history.jsonl` with a full snapshot.\n69:  the archive, the policy history and the evidence files: autonomy matrix, the loop with live\n87:   undated traces in historical epochs are _unknown_, never zero.\n102:   stay two rounds. The policy in force must equal the snapshot its history recorded for that\n117:   history, validates both destinations before writing either, refuses identical paths, and every\n120:10. **Rendered output is escaped.** Every string from the archive, history or evidence is\n141:recorded here as history — the committed `docs/rsi/trace-evidence.json` keeps per-topic session\n143:are not reproducible from the repository alone): 96 distinct failures across 3 sessions, validity\n144:0.55 against the review signal, 69 blind spots dominated by \"permission denied by the auto-mode\n145:classifier\" (28), tool input errors, and missing tools.\n153:With the verifier's own review sessions counted as the anchor, validity read 0.95 in the first local\n156:keeps only failures, and the verifier's sessions contain none), so that figure is historical and the\n96:          echo \"No Codex credentials secret is configured. Codex review did not run for this PR.\" > $RUNNER_TEMP/codex-review-status.txt\n166:          echo \"No trusted copy of scripts/redact-secrets.py exists on the base branch, so this PR cannot be safely reviewed by this job yet (failing closed rather than trusting the PR's own copy of the redactor).\" > $RUNNER_TEMP/codex-review-status.txt\n180:      # replace the reviewer's output or the redaction inputs.\n321:              body = fs.existsSync(process.env.RUNNER_TEMP + '/codex-review-status.txt')\n322:                ? fs.readFileSync(process.env.RUNNER_TEMP + '/codex-review-status.txt', 'utf8')\n326:              body = fs.existsSync(process.env.RUNNER_TEMP + '/codex-review-status.txt')\n327:                ? fs.readFileSync(process.env.RUNNER_TEMP + '/codex-review-status.txt', 'utf8')\n348:            body = body.replace(/codex-review-(status|sha)/g, 'codex-review-$1');\n357:              body: `### Codex independent review\\n\\n${body}\\n\\n---\\n*Generated by an independent model reviewing this diff with no stake in prior conclusions on it — see [docs/production-hardening-backlog.md](../blob/main/docs/production-hardening-backlog.md) item #4.*\\n<!-- codex-review-status: ${status} -->\\n<!-- codex-review-sha: ${context.payload.pull_request.head.sha} -->`,\n"}
{"chunk_id":"4bf257","wall_time_seconds":0.000150833,"exit_code":0,"original_token_count":63,"output":"............................................................... [ 45%]\n........................................................................ [ 90%]\n................                                                         [100%]\n160 passed in 3.25s\n"}

>sed -n '450,492p' scripts/mine-trace-failures.py; sed -n '135,166p' docs/plans/recursive-meta-improvement.md; sed -n '1,70p' scripts/revise_improvement_policy_test.py; sed -n '270,340p' scripts/measure-policy-validity.py; sed -n '330,365p' .github/workflows/codex-review.yml; rg -n 'full|excerpt|long' scripts/mine_trace_failures_test.py; sed -n '810,840p' scripts/revise-improvement-policy.py

{"chunk_id":"d09530","wall_time_seconds":0.000002417,"exit_code":0,"original_token_count":3278,"output":"                pass  # \"all\" deliberately includes the verifier's own sessions\n            if args.namespace:\n                sync_trace(args.traces_bin, trace[\"id\"])\n            failures.extend(mine_trace(args.traces_bin, trace))\n    except TracesCliError as exc:\n        print(f\"::error::{exc}\", file=sys.stderr)\n        return 1\n\n    lines, summary = report(failures, current_keywords)\n    summary[\"traces_scanned\"] = len(traces)\n    summary[\"listing_complete\"] = complete\n    summary[\"repo_dir\"] = args.repo_dir\n    summary[\"namespace\"] = args.namespace\n    for line in lines:\n        print(line)\n    if args.save_evidence:\n        Path(args.save_evidence).write_text(\n            json.dumps(\n                build_evidence(\n                    failures,\n                    search_keywords,\n                    args.repo_dir or \"\",\n                    agents,\n                    complete,\n                    scanned=[t[\"id\"] for t in traces],\n                    namespace=args.namespace,\n                ),\n                indent=2,\n            )\n            + \"\\n\"\n        )\n        print(f\"evidence written to {args.save_evidence}\")\n    if args.out_json:\n        Path(args.out_json).write_text(json.dumps(summary, indent=2) + \"\\n\")\n    print(\"---\")\n    print(json.dumps(summary, indent=2))\n    return 0\n\n\nif __name__ == \"__main__\":\n    sys.exit(main(sys.argv))\nmatched independently against every topic's keywords, and written as evidence with the keyword\ndefinitions it was searched under. Failures no topic claims are the field's blind spots; when at\nleast `MIN_FIELD_BLIND_SPOTS` of them exist, `revise-improvement-policy.py --field-failures` mines\ntopics from their output the same way it mines unclassified review findings.\n\nFirst strict run over the working sessions in this folder (a local observation on 2026-09-14,\nrecorded here as history — the committed `docs/rsi/trace-evidence.json` keeps per-topic session\nreferences and the aggregate failure count, not the per-failure breakdown, so these three figures\nare not reproducible from the repository alone): 96 distinct failures across 3 sessions, validity\n0.55 against the review signal, 69 blind spots dominated by \"permission denied by the auto-mode\nclassifier\" (28), tool input errors, and missing tools.\n\n## First real run\n\nMeasured against the archive as of round 10 with policy v1: coverage 0.61, anchor empty (no working\nsessions for this repository are indexed in Traces yet). The rule fired on coverage and proposed v2:\none mined topic covering 8 of the 11 blind-spot findings, coverage 0.61 → 0.89 (the remaining three\nare single-occurrence findings no bounded rule may claim). A second pass under v2 proposes nothing.\nWith the verifier's own review sessions counted as the anchor, validity read 0.95 in the first local\nrun: the number agrees with the review signal because it _is_ the review signal, which is why the\ndefault excludes them. The committed `docs/rsi/trace-evidence-verifier.json` is empty (the miner\nkeeps only failures, and the verifier's sessions contain none), so that figure is historical and the\ndashboard's echo check now reports no verifier traces rather than a number.\n\nReproduce:\n\n```bash\npython3 scripts/mine-trace-failures.py --repo-dir . --save-evidence docs/rsi/trace-evidence.json\nsed -n '/^---/,$p' <(python3 scripts/measure-policy-validity.py docs/self-improvement-archive.jsonl --trace-evidence docs/rsi/trace-evidence.json) | tail -n +2 > docs/rsi/measurement.json\npython3 scripts/revise-improvement-policy.py docs/self-improvement-archive.jsonl --measurement docs/rsi/measurement.json --dry-run\npython3 scripts/render-rsi-dashboard.py docs/self-improvement-archive.jsonl --trace-evidence docs/rsi/trace-evidence.json --out docs/rsi/dashboard.html\n```\n\"\"\"Tests for revise-improvement-policy.py.\n\nRun with: python3 -m pytest scripts/revise_improvement_policy_test.py -q\n\"\"\"\n\nimport importlib.util\nimport json\nimport sys\nfrom pathlib import Path\n\nimport pytest\n\n_MODULE_PATH = Path(__file__).parent / \"revise-improvement-policy.py\"\n_spec = importlib.util.spec_from_file_location(\"revise_improvement_policy\", _MODULE_PATH)\nassert _spec is not None and _spec.loader is not None\nrevise = importlib.util.module_from_spec(_spec)\nsys.modules[\"revise_improvement_policy\"] = revise\n_spec.loader.exec_module(revise)\npolicy_mod = sys.modules[\"improvement_policy\"]\nmeasure = sys.modules[\"measure_policy_validity\"]\n\nNOW = \"2026-09-14T19:00:00Z\"\n\n\ndef _archive():\n    return [\n        {\n            \"round\": 1,\n            \"occurred_at\": \"2026-09-14T15:00:00Z\",\n            \"findings\": [\"[P1] Secret leaked into logs.\"],\n        },\n        {\n            \"round\": 2,\n            \"occurred_at\": \"2026-09-14T16:00:00Z\",\n            \"findings\": [\n                \"[P2] Archive concurrency drops queued rounds.\",\n                \"[P2] Token exposed in a comment.\",\n            ],\n        },\n        {\n            \"round\": 3,\n            \"occurred_at\": \"2026-09-14T17:00:00Z\",\n            \"findings\": [\n                \"[P2] Archive threshold crossings are permanently missed under concurrency.\",\n                \"[P2] Archive PR creation cannot recover after a partial failure.\",\n            ],\n        },\n    ]\n\n\ndef _measurement(policy, evidence=None):\n    return measure.measure(_archive(), policy, evidence)\n\n\ndef test_no_revision_when_coverage_and_validity_hold():\n    policy = policy_mod.builtin_policy()\n    policy[\"topics\"][\"archive-ops\"] = {\"keywords\": [\"archive\"], \"weight\": 1.0}\n    decision = revise.decide(_archive(), policy, [], _measurement(policy), NOW)\n    assert decision[\"action\"] == \"none\"\n\n\ndef test_low_coverage_triggers_a_bounded_mined_revision():\n    policy = policy_mod.builtin_policy()\n    measurement = _measurement(policy)\n    assert measurement[\"current\"][\"coverage\"] < revise.MIN_COVERAGE\n    decision = revise.decide(_archive(), policy, [], measurement, NOW)\n    assert decision[\"action\"] == \"revise\"\n    revised = decision[\"policy\"]\n    assert revised[\"version\"] == 2 and revised[\"parent\"] == 1 and revised[\"origin\"] == \"revision\"\n    new_names = [t for t in revised[\"topics\"] if t not in policy[\"topics\"]]\n\n\ndef measure_epoch(\n    rounds: list[dict],\n    keywords: dict[str, list[str]],\n    weights: dict[str, float],\n    evidence: dict | None,\n    until_ms: int | None,\n    *,\n    historical: bool = False,\n) -> dict:\n    topics = list(keywords)\n    dev_rounds: dict[str, set[int]] = {t: set() for t in topics}\n    total = 0\n    classified = 0\n    unclassified: list[dict] = []\n    for rnd in rounds:\n        for finding in rnd[\"findings\"]:\n            total += 1\n            topic = policy_mod.classify_finding(finding, keywords)\n            if topic is None:\n                unclassified.append({\"round\": rnd[\"round\"], \"finding\": finding})\n                continue\n            classified += 1\n            dev_rounds[topic].add(rnd.get(\"key\", rnd[\"round\"]))\n    dev = {t: len(dev_rounds[t]) for t in topics}\n    # The detector decides on weighted recurrence, so validity must be\n    # measured on the same signal, or discounting a topic could never\n    # change what is measured (Codex review of PR #10, finding 4).\n    dev_weighted = {t: round(dev[t] * weights.get(t, 1.0), 4) for t in topics}\n    if historical and until_ms is None and evidence is not None:\n        # A historical epoch with no usable timestamp has no defensible\n        # evidence window: unknown, not \"everything\" (Codex review of\n        # PR #10, round 18).\n        anchor: dict[str, int | None] | None = dict.fromkeys(topics)\n    else:\n        anchor = anchor_counts_at(evidence, topics, until_ms, keywords)\n    validity = None\n    known = [t for t in topics if anchor is not None and anchor[t] is not None]\n    if anchor is not None:\n        validity = spearman(\n            [float(dev_weighted[t]) for t in known], [float(anchor[t]) for t in known]\n        )\n    coverage = round(classified / total, 4) if total else None\n    return {\n        \"round\": rounds[-1][\"round\"] if rounds else None,\n        \"timestamp_ms\": until_ms,\n        \"findings_total\": total,\n        \"findings_classified\": classified,\n        \"coverage\": coverage,\n        \"dev\": dev,\n        \"dev_weighted\": dev_weighted,\n        \"anchor\": anchor,\n        \"anchor_unknown_topics\": sorted(\n            t for t in topics if anchor is not None and anchor[t] is None\n        ),\n        \"validity\": validity,\n        \"unclassified_findings\": unclassified,\n        \"dev_only_topics\": sorted(\n            t for t in known if dev[t] >= 2 and anchor is not None and anchor[t] == 0\n        ),\n        \"anchor_only_topics\": sorted(\n            t for t in known if dev[t] == 0 and anchor is not None and (anchor[t] or 0) > 0\n        ),\n    }\n\n\ndef evidence_trace_ids(evidence: dict | None) -> set[str]:\n    \"\"\"Every session the evidence observed: the scanned-session list when\n    the snapshot records one, plus any session a topic matched. A complete\n    scan whose failures were all unclassified is an observed field with\n              status = 'failed';\n              body = '**Review did not complete successfully** (command failed, crashed, or timed out — see job logs). This is not a passing review; no findings below should be read as \"nothing found.\"';\n            } else if (fs.existsSync(process.env.RUNNER_TEMP + '/codex-review-output.txt')) {\n              status = 'completed';\n              body = fs.readFileSync(process.env.RUNNER_TEMP + '/codex-review-output.txt', 'utf8');\n            } else {\n              // Should not happen given the states above, but never claim a\n              // review happened without an output file to back it up.\n              status = 'unknown';\n              body = 'Codex review status is unknown — no output file was produced and no failure was recorded. Treat as unreviewed.';\n            }\n\n            if (body.length > 60000) {\n              body = body.slice(0, 60000) + '\\n\\n...(truncated)';\n            }\n            // The review text must not be able to forge either footer marker:\n            // a review of commit B that quotes A's SHA marker would otherwise be\n            // archived as A's review (Codex full-branch review, workflows finding 1).\n            body = body.replace(/codex-review-(status|sha)/g, 'codex-review-$1');\n            await github.rest.issues.createComment({\n              owner: context.repo.owner,\n              repo: context.repo.repo,\n              issue_number: context.issue.number,\n              // The SHA marker lets .github/workflows/archive-and-recommend.yml\n              // bind a workflow_run event to the exact review comment it\n              // produced, rather than trusting \"the latest comment that\n              // looks like a review\" — which any PR commenter could forge.\n              body: `### Codex independent review\\n\\n${body}\\n\\n---\\n*Generated by an independent model reviewing this diff with no stake in prior conclusions on it — see [docs/production-hardening-backlog.md](../blob/main/docs/production-hardening-backlog.md) item #4.*\\n<!-- codex-review-status: ${status} -->\\n<!-- codex-review-sha: ${context.payload.pull_request.head.sha} -->`,\n            });\n\n      - name: Fail on a [P1] finding or a failed review\n        if: |\n          steps.review.outputs.found_p1 == 'true' ||\n          steps.review.outputs.review_failed == 'true' ||\n          (steps.has-key.outputs.present == 'true' && steps.extract.outputs.available == 'false')\n        run: |\n117:    kinds = {f[\"excerpt\"]: f[\"kind\"] for f in failures}\n261:        \"excerpt\": \"secret token expired\",\n380:    assert any(\"failed\" in b[\"excerpt\"] for b in report[\"blind_spots\"])\n        threshold=policy[\"threshold\"],\n        origin=\"revision\",\n        rationale=\"Revised because \" + \"; \".join(triggers) + \". \" + \" \".join(changes),\n        created_at=now,\n    )\n    # The whole candidate, not just its weight changes, must not regress\n    # validity against the policy it replaces (Codex review of PR #10, round 4).\n    v_before = validity_under(policy, covered, candidate_anchor(current, policy))\n    v_after = validity_under(revised, covered, candidate_anchor(current, revised))\n    rejected = rejected_configuration(revised, history, measurement)\n    if rejected is not None:\n        return {\n            \"action\": \"none\",\n            \"reason\": (\n                f\"candidate reproduces configuration {policy_mod.policy_hash(revised)}, rolled back as \"\n                f\"v{rejected.get('replaced_version', '?')} on the same archive and evidence; needs new evidence\"\n            ),\n            \"triggers\": triggers,\n            \"rejected_changes\": changes,\n        }\n    if validity_regressed(v_before, v_after):\n        return {\n            \"action\": \"none\",\n            \"reason\": f\"candidate revision would move validity {v_before} -> {v_after}; refused\",\n            \"triggers\": triggers,\n            \"rejected_changes\": changes,\n        }\n    after = measure_mod.measure(entries, revised, None)[\"current\"]\n    if coverage is not None and after[\"coverage\"] is not None and after[\"coverage\"] < coverage:\n        return {\n            \"action\": \"none\",\n"}

>python3 - <<'PY'
import sys, importlib.util, tempfile, pathlib, json, os, subprocess
sys.path.insert(0,'scripts')
import revise_improvement_policy_test as rt
import render_rsi_dashboard_test as dt
p=rt.policy_mod; r=rt.revise; m=rt.measure
s=importlib.util.spec_from_file_location('miner','scripts/mine-trace-failures.py'); mine=importlib.util.module_from_spec(s);s.loader.exec_module(mine)
v1=p.builtin_policy();v2=p.new_version(v1,topics=v1['topics'],threshold=3,origin='revision',rationale='test')
with tempfile.TemporaryDirectory() as td:
 d=pathlib.Path(td); entries=rt._archive(); edited={**v2,'origin':'init'}
 for name,data in [('a', '\n'.join(map(json.dumps,entries))),('p',json.dumps(edited)),('h',json.dumps({'version':2,'policy':v2})),('m',json.dumps(m.measure(entries,edited,None)))]: (d/name).write_text(data)
 code=r.main(['r',str(d/'a'),'--policy',str(d/'p'),'--history',str(d/'h'),'--measurement',str(d/'m'),'--dry-run']);assert code==1;print('A1: origin-only edit refused, exit',code)
 os.link(d/'a',d/'report.json')
 try: m.main(['m',str(d/'a'),'--out-json',str(d/'report.json')])
 except PermissionError as e: print('A6:',e)
 else: raise AssertionError('archive overwritten')
 assert (d/'a').read_text()=='\n'.join(map(json.dumps,entries))
bad=p.builtin_policy();bad['topics']['bad@tag']={'keywords':['x']}
try:p.validate_policy(bad)
except ValueError as e:print('A2:',e)
else:raise AssertionError()
rt.test_rollback_to_the_root_version_still_waits_before_being_judged();print('A4/R36b: one-round root rollback waits')
rt.test_coverage_lost_at_a_grandparent_still_triggers_rollback();print('A5: quartz grandparent rollback to v1')
entries=[{'round':1,'source_sha':sha,'policy_version':1,'policy_hash':p.policy_hash(v1),'findings':['secret'], 'occurred_at':'2026-09-14T15:00:00Z'} for sha in ['aaa','bbb']]
assert len(m.rounds_in_order(entries))==r.rounds_under(entries,v1)==2;print('B2: measurement rounds=2; rounds_under=2')
dt.test_negative_validity_is_drawn_below_zero_not_clamped_onto_it();print('C1: negative chart coordinate assertions passed')
doc=pathlib.Path('docs/plans/recursive-meta-improvement.md').read_text();assert 'not reproducible from the repository alone' in doc and 'figure is historical' in doc;print('C2: historical/unreproducible qualifiers present')
base='Traceback (most recent call last):\n'+'x'*300+'\n'
def ev(n,tail):return {'type':'tool_result','status':'error','toolName':'shell','eventNumber':n,'output':base+tail}
trace={'id':'t1','agentId':'claude-code'};kw={'credential':['secret']}
mine.iter_events=lambda *a:iter([ev(1,'ValueError: secret exposed')]);f=mine.mine_trace('traces',trace);e=mine.build_evidence(f,kw,'',None,scanned=['t1']);assert len(e['topics']['credential'])==1;print('A3 original: late keyword counted; truncated=',e['truncated'])
mine.iter_events=lambda *a:iter([ev(1,'ValueError: quartz'),ev(2,'ValueError: secret exposed')]);f=mine.mine_trace('traces',trace);e=mine.build_evidence(f,kw,'',None,scanned=['t1']);print('A3 collision: failures=',len(f),'count=',f[0]['count'],'topics=',e['topics'],'truncated=',e['truncated']);assert e['topics']['credential']==[]
res=m.measure(entries,{'version':1,'origin':'init','parent':None,'threshold':3,'topics':{'credential':{'keywords':['secret'],'weight':1}}},e);assert res['anchor']['traces_considered']==1 and res['current']['anchor']=={'credential':0};print('R36a: unclassified scan retained:',res['current']['anchor'])
mine.EXTRA_CLI_ARGS[:]=['--key','SENTINEL_KEY']
def timeout(*args,**kwargs):raise subprocess.TimeoutExpired(args[0],300)
mine.subprocess.run=timeout
try:mine.run_traces_json('traces',['list','@slug','--all'])
except subprocess.TimeoutExpired as exc:
 assert 'SENTINEL_KEY' in str(exc);print('NEW key leak: uncaught timeout=',str(exc))
PY

{"chunk_id":"380c8b","wall_time_seconds":0.147339375,"exit_code":0,"original_token_count":230,"output":"::error::policy.origin='init' differs from the recorded v2 snapshot ('revision'); lineage metadata may not be edited in place\nA1: origin-only edit refused, exit 1\nA6: /private/var/folders/79/x0hsq2wj3qz0d3k35271zm5w0000gn/T/tmpsaw9qc0n/report.json is an input of this run; choose another output path\nA2: topic name 'bad@tag' is invalid (non-empty, no '@', no surrounding whitespace)\nA4/R36b: one-round root rollback waits\nA5: quartz grandparent rollback to v1\nB2: measurement rounds=2; rounds_under=2\nC1: negative chart coordinate assertions passed\nC2: historical/unreproducible qualifiers present\nA3 original: late keyword counted; truncated= []\nA3 collision: failures= 1 count= 2 topics= {'credential': []} truncated= []\nR36a: unclassified scan retained: {'credential': 0}\nNEW key leak: uncaught timeout= Command '['traces', 'list', '@slug', '--all', '--key', 'SENTINEL_KEY', '--json']' timed out after 300 seconds\n"}

All 160 tests pass, and Ruff is clean. The original A3 case passes, but two failures with the same excerpt still lose the later keyword and report a confirmed zero count. I also reproduced an uncaught timeout whose error text contains the Traces key; GitHub’s secret masking may hide it in Actions logs, but the script exposes it in raw stderr.

>sed -n '488,543p' scripts/revise_improvement_policy_test.py; sed -n '1,95p' scripts/measure-policy-validity.py; sed -n '240,315p' scripts/revise-improvement-policy.py; nl -ba scripts/mine-trace-failures.py | sed -n '108,145p;246,265p'; nl -ba scripts/revise-improvement-policy.py | sed -n '475,490p'; nl -ba .github/workflows/revise-improvement-policy.yml | sed -n '170,242p'

{"chunk_id":"ab9c3d","wall_time_seconds":0.000002917,"exit_code":0,"original_token_count":4706,"output":"Warning: truncated output (original token count: 4706)\nTotal output lines: 373\n\ndef test_rolled_back_configuration_is_not_retried_on_the_same_evidence():\n    parent = policy_mod.builtin_policy()\n    measurement = measure.measure(_archive(), parent, None)\n    first = revise.decide(_archive(), parent, [], measurement, NOW)\n    assert first[\"action\"] == \"revise\"\n    rejected_hash = policy_mod.policy_hash(first[\"policy\"])\n    history = [\n        {\"version\": 1, \"policy\": parent},\n        {\n            \"version\": 2,\n            \"parent\": 1,\n            \"origin\": \"revision\",\n            \"coverage_before\": 0.4,\n            \"policy\": first[\"policy\"],\n        },\n        {\n            \"version\": 3,\n            \"parent\": 2,\n            \"origin\": \"rollback\",\n            \"replaced_policy_hash\": rejected_hash,\n            \"replaced_version\": 2,\n            \"archive_digest\": measurement[\"archive_digest\"],\n            \"evidence_collected_at\": None,\n            \"policy\": {**parent, \"version\": 3, \"parent\": 2, \"origin\": \"rollback\"},\n        },\n    ]\n    current = {**parent, \"version\": 3, \"parent\": 2, \"origin\": \"rollback\", \"restored_version\": 1}\n    # The rollback has to serve its own waiting period first (Codex, round 36).\n    waiting = revise.decide(\n        _archive(), current, history, measure.measure(_archive(), current, None), NOW\n    )\n    assert waiting[\"action\"] == \"none\" and \"waiting for\" in waiting[\"reason\"]\n    stamped = [\n        {**e, \"policy_version\": 3, \"policy_hash\": policy_mod.policy_hash(current)}\n        for e in _archive()\n    ]\n    measurement_r = measure.measure(stamped, current, None)\n    rollback_entry = next(e for e in history if e.get(\"origin\") == \"rollback\")\n    rollback_entry[\"archive_digest\"] = measurement_r[\"archive_digest\"]\n    again = revise.decide(stamped, current, history, measurement_r, NOW)\n    assert again[\"action\"] == \"none\"\n    assert \"rolled back\" in again[\"reason\"]\n    # New archive content lifts the block.\n    grown = [\n        *stamped,\n        {\n            \"round\": 4,\n            \"occurred_at\": \"2026-09-14T18:00:00Z\",\n            \"findings\": [\"[P2] Archive queue overflow again.\"],\n        },\n    ]\n    retry = revise.decide(grown, current, history, measure.measure(grown, current, None), NOW)\n    assert retry[\"action\"] == \"revise\"\n\n\ndef test_evidence_against_a_removed_topic_survives_for_candidates():\n#!/usr/bin/env python3\n\"\"\"Measure whether the improvement policy's own signal predicts what the\nfield shows -- the L5 trigger from docs/plans/recursive-meta-improvement.md.\n\nThe loop in this repo decides \"target fix vs mechanism fix\" from Codex\nreview findings bucketed by docs/improvement-policy.json's taxonomy. That\nbucketed count is the loop's development score: it is what the mechanism\nsees. It can be wrong in two ways the mechanism itself cannot notice:\n\n  1. Coverage: findings the taxonomy does not classify are simply dropped,\n     so a class of problem the loop keeps hitting never accumulates toward\n     the threshold. Measured as classified / total findings.\n  2. Predictive validity: a topic the taxonomy credits heavily may never\n     show up in actual working sessions, while one it barely credits does.\n     Measured as the Spearman rank correlation, across topics, between the\n     review-derived recurrence (rounds with a finding) and an independent\n     anchor: Traces evidence from working sessions in this repository.\n\nThe anchor deliberately excludes the verifier's own transcripts (Codex\nreview sessions) by default: those contain the findings themselves, so\ncounting them would make the anchor echo the development score instead of\nchecking it (paper failure mode 3, \"reliable verification\").\n\nBoth measures are replayed per archive round, using only the rounds and\ntraces that existed at that round's timestamp, so the dashboard can show\nwhen a revision would have fired, not just where things stand now.\n\nUsage:\n    python3 measure-policy-validity.py <archive.jsonl>\n        [--policy PATH] [--trace-evidence EVIDENCE.json] [--out-json PATH]\n\nEvidence comes from `mine-trace-failures.py --repo-dir DIR --save-evidence\nEVIDENCE.json`. Without --trace-evidence the anchor is absent: coverage is\nstill measured, validity is reported as null, and the JSON says so plainly.\nPrints human-readable lines, then a `---` separator, then a JSON object.\n\"\"\"\n\nfrom __future__ import annotations\n\nimport argparse\nimport hashlib\nimport importlib.util\nimport json\nimport sys\nfrom datetime import UTC, datetime\nfrom pathlib import Path\n\n\ndef _load_sibling_module(name: str, filename: str):\n    if name in sys.modules:\n        return sys.modules[name]\n    path = Path(__file__).parent / filename\n    spec = importlib.util.spec_from_file_location(name, path)\n    assert spec is not None and spec.loader is not None\n    module = importlib.util.module_from_spec(spec)\n    sys.modules[name] = module\n    spec.loader.exec_module(module)\n    return module\n\n\npolicy_mod = _load_sibling_module(\"improvement_policy\", \"improvement_policy.py\")\n\nDEFAULT_ANCHOR_AGENTS = [\"claude-code\", \"antigravity\", \"cursor\", \"droid\", \"openclaw\", \"pi\"]\nMIN_TOPICS_FOR_VALIDITY = 3\n\n\n# --- archive replay ---------------------------------------------------------\n\n\ndef load_archive(path: str) -> list[dict]:\n    entries = []\n    with open(path) as f:\n        for line in f:\n            line = line.strip()\n            if line:\n                entries.append(json.loads(line))\n    return entries\n\n\ndef archive_digest(entries: list[dict]) -> str:\n    \"\"\"Content digest of the archive a measurement was taken against, so a\n    decision can refuse a measurement from a different archive (Codex\n    review of PR #10, round 2, finding 2).\"\"\"\n    canonical = json.dumps(entries, sort_keys=[REDACTED] separators=(\",\", \":\"))\n    return hashlib.sha256(canonical.encode()).hexdigest()[:12]\n\n\ndef parse_timestamp_ms(value: object) -> int | None:\n    if not isinstance(value, str):\n        return None\n    try:\n        parsed = datetime.fromisoformat(value.replace(\"Z\", \"+00:00\"))\n    except ValueError:\n        return None\n    if parsed.tzinfo is None:\n    ]\n)\n\n_TOKEN_RE = re.compile(r\"[a-z][a-z_-]{2,}\")\n\n\ndef tokenize(text: str) -> set[str]:\n    return {\n        tok.strip(\"-_\")\n        for tok in _TOKEN_RE.findall(text.lower())\n        if len(tok) >= MIN_TOKEN_LENGTH\n        and tok not in STOPWORDS\n        and not tok.startswith(\"http\")\n        and not tok.endswith(\"ly\")  # adverbs name manner, never a class of problem\n    }\n\n\ndef existing_keyword_tokens(keywords: dict[str, list[str]]) -> set[str]:\n    return {k.lower() for words in keywords.values() for k in words}\n\n\ndef mine_topics(\n    unclassified: list[dict],\n    keywords: dict[str, list[str]],\n) -> list[dict]:\n    \"\"\"Greedy, auditable topic mining over findings the policy could not\n    classify: the most frequent significant token names a topic; its\n    keywords are that token plus the tokens that co-occur with it most;\n    findings the new topic covers are removed and the process repeats.\"\"\"\n    taken = existing_keyword_tokens(keywords)\n    remaining = [\n        {\n            \"round\": item[\"round\"],\n            \"finding\": item[\"finding\"],\n            \"tokens\": tokenize(item[\"finding\"]) - taken,\n        }\n        for item in unclassified\n    ]\n    mined: list[dict] = []\n    while len(mined) < MAX_NEW_TOPICS and remaining:\n        df = Counter(tok for item in remaining for tok in item[\"tokens\"])\n        candidates = [t for t, n in df.items() if n >= MIN_FINDINGS_PER_TOPIC]\n        if not candidates:\n            break\n        head = sorted(candidates, key=lambda t: (-df[t], t))[0]\n        covering_head = [item for item in remaining if head in item[\"tokens\"]]\n        co = Counter(tok for item in covering_head for tok in item[\"tokens\"] if tok != head)\n        companions = [\n            t\n            for t, n in sorted(co.items(), key=lambda kv: (-kv[1], kv[0]))\n            if n >= MIN_FINDINGS_PER_TOPIC\n        ][: MAX_KEYWORDS_PER_TOPIC - 1]\n        topic_keywords = [head, *companions]\n        # A topic claims exactly the findings it would classify: any keyword,\n        # by the same substring rule the detector uses. Those findings are\n        # then unavailable to later topics, so no finding supports two\n        # topics (Codex review of PR #10, round 2, finding 3).\n        covering = [\n            item\n            for item in remaining\n            if policy_mod.classify_finding(item[\"finding\"], {\"_\": topic_keywords}) is not None\n        ]\n        if len(covering) < MIN_FINDINGS_PER_TOPIC:\n            break\n        base = \"-\".join(topic_keywords[:2]) if companions else head\n        taken_names = set(keywords) | {m[\"name\"] for m in mined}\n        name = base\n        suffix = 1\n        while name in taken_names:\n            # Never reuse a name: an overwritten topic would silently drop\n            # its classifications (Codex review of PR #10, round 14).\n            suffix += 1\n            name = f\"{base}-{suffix}\"\n        mined.append(\n            {\n                \"name\": name,\n   108\tdef run_traces_json(traces_bin: str, args: list[st…206 tokens truncated….read()\n   125\t        except OSError as exc:\n   126\t            raise TracesCliError(f\"Could not run `{traces_bin}`: {exc}\") from exc\n   127\t        if result.returncode != 0:\n   128\t            raise TracesCliError(f\"`{traces_bin} {' '.join(args)}` failed: {result.stderr.strip()}\")\n   129\t        payload = parse_cli_json(stdout)\n   130\t        if payload is not None:\n   131\t            if not payload.get(\"ok\"):\n   132\t                raise TracesCliError(f\"`{traces_bin} {' '.join(args)}` reported failure: {payload}\")\n   133\t            return payload[\"data\"]\n   134\t        last_error = stdout[:120].replace(\"\\n\", \" \")\n   135\t        if attempt < retries:\n   136\t            continue\n   137\t    raise TracesCliError(f\"Non-JSON output from `{traces_bin} {' '.join(args)}`: {last_error}\")\n   138\t\n   139\t\n   140\tdef list_traces(\n   141\t    traces_bin: str, repo_dir: str, agents: list[str] | None, limit: int\n   142\t) -> tuple[list[dict], bool]:\n   143\t    \"\"\"Sessions recorded in the folder, and whether the listing was complete.\n   144\t    A listing that fills its limit may have missed sessions, and evidence\n   145\t    built from it must say so rather than report confirmed zero counts\n   246\t        output = str(event.get(\"output\") or event.get(\"text\") or \"\")\n   247\t        tool = str(event.get(\"toolName\") or \"\")\n   248\t        call = calls.get(str(event.get(\"callId\")), {})\n   249\t        args = call.get(\"args\") or {}\n   250\t        command = str(args.get(\"command\") or args.get(\"file_path\") or args.get(\"pattern\") or \"\")\n   251\t        excerpt = excerpt_for(kind, output)\n   252\t        # The command is part of identity: two commands with the same output\n   253\t        # are two failures, and classification reads the command\n   254\t        # (Codex review of PR #10, round 28).\n   255\t        key = (tool, \" \".join(command.split())[:EXCERPT_CHARS], excerpt)\n   256\t        if key in failures:\n   257\t            failures[key][\"count\"] += 1\n   258\t            continue\n   259\t        failures[key] = {\n   260\t            \"_text\": f\"{command} {output}\".lower(),\n   261\t            \"trace_id\": trace[\"id\"],\n   262\t            \"agent\": trace.get(\"agentId\"),\n   263\t            \"event_number\": event.get(\"eventNumber\"),\n   264\t            \"timestamp\": event.get(\"timestamp\"),\n   265\t            \"tool\": tool,\n   475\tdef rejected_configuration(candidate: dict, history: list[dict], measurement: dict) -> dict | None:\n   476\t    \"\"\"A configuration rolled back on the same archive and evidence is not\n   477\t    retried: the rollback's history entry records the rejected hash and\n   478\t    what it was judged on.\"\"\"\n   479\t    wanted = policy_mod.policy_hash(candidate)\n   480\t    collected = (measurement.get(\"anchor\") or {}).get(\"collected_at\")\n   481\t    for entry in history:\n   482\t        if entry.get(\"origin\") != \"rollback\" or entry.get(\"replaced_policy_hash\") != wanted:\n   483\t            continue\n   484\t        if entry.get(\"archive_digest\") == measurement.get(\"archive_digest\") and (\n   485\t            entry.get(\"evidence_collected_at\") == collected\n   486\t        ):\n   487\t            return entry\n   488\t    return None\n   489\t\n   490\t\n   170\t        env:\n   171\t          GH_TOKEN: ${{ github.token }}\n   172\t          ACTION: ${{ steps.revise.outputs.action }}\n   173\t          SOURCE_SHA: ${{ steps.source.outputs.sha }}\n   174\t        run: |\n   175\t          set -euo pipefail\n   176\t          if git diff --quiet -- docs/improvement-policy.json docs/improvement-policy-history.jsonl docs/rsi/; then\n   177\t            echo \"Nothing changed (action: $ACTION).\"\n   178\t            exit 0\n   179\t          fi\n   180\t          version=$(python3 -c 'import json; print(json.load(open(\"docs/improvement-policy.json\"))[\"version\"])')\n   181\t          default_branch=$(gh repo view --json defaultBranchRef --jq .defaultBranchRef.name)\n   182\t          git config user.name \"github-actions[bot]\"\n   183\t          git config user.email \"41898282+github-actions[bot]@users.noreply.github.com\"\n   184\t          if git diff --quiet -- docs/improvement-policy.json docs/improvement-policy-history.jsonl; then\n   185\t            # Only the evidence, measurement and dashboard moved (the hourly\n   186\t            # refresh with no decision). One standing evidence branch, updated\n   187\t            # in place, so the hourly run never opens a new PR per hour; a\n   188\t            # human merges it whenever, and merging it re-triggers this\n   189\t            # workflow through the trace-evidence.json path filter.\n   190\t            # A standing policy proposal that the current decision no longer\n   191\t            # supports is withdrawn, not left open to merge (Codex review of\n   192\t            # PR #10, round 37): new rounds can legitimately turn a\n   193\t            # `revise` into `none`.\n   194\t            stale_pr=$(gh pr list --head \"improvement-policy-proposal\" --base \"$default_branch\" --state open \\\n   195\t              --json number,isCrossRepository --jq '[.[] | select(.isCrossRepository == false)][0].number // empty')\n   196\t            if [ -n \"$stale_pr\" ]; then\n   197\t              gh pr close \"$stale_pr\" --delete-branch \\\n   198\t                --comment \"Withdrawn: re-evaluated at \\`${SOURCE_SHA}\\` the fixed acceptance rule proposes no policy change (action: **${ACTION}**), so this proposal no longer satisfies it. A new proposal will be opened if a later round supports one.\"\n   199\t              echo \"Closed the stale policy proposal PR #$stale_pr.\"\n   200\t            fi\n   201\t            branch=\"improvement-evidence-refresh\"\n   202\t            title=\"chore(rsi): refresh the Traces field anchor\"\n   203\t            body=\"Automatically refreshed by [\\`revise-improvement-policy.yml\\`](../blob/main/.github/workflows/revise-improvement-policy.yml) from the shared Traces namespace at \\`${SOURCE_SHA}\\`. The fixed acceptance rule proposed no policy change on this evidence (action: **${ACTION}**). This changes only the evidence snapshot, the measurement and the dashboard; the policy is untouched.\"\n   204\t            git checkout -b \"$branch\"\n   205\t            git add docs/rsi/\n   206\t            git commit -m \"$title\"\n   207\t            git push --force origin \"$branch\"\n   208\t            existing_pr=$(gh pr list --head \"$branch\" --base \"$default_branch\" --state open \\\n   209\t              --json number,isCrossRepository --jq '[.[] | select(.isCrossRepository == false)][0].number // empty')\n   210\t            if [ -n \"$existing_pr\" ]; then\n   211\t              echo \"Updated the open evidence PR #$existing_pr in place.\"\n   212\t              exit 0\n   213\t            fi\n   214\t            gh pr create --title \"$title\" --body \"$body\" --label \"self-improvement-archive\" \\\n   215\t              --base \"$default_branch\" --head \"$branch\"\n   216\t            exit 0\n   217\t          fi\n   218\t          # One standing proposal branch: a newer archive round supersedes an\n   219\t          # open proposal instead of opening a competing one with the same\n   220\t          # version and parent (Codex review of PR #10, round 5).\n   221\t          branch=\"improvement-policy-proposal\"\n   222\t          git checkout -b \"$branch\"\n   223\t          git add docs/improvement-policy.json docs/improvement-policy-history.jsonl docs/rsi/\n   224\t          git commit -m \"chore(rsi): ${ACTION} of the improvement policy (v${version})\"\n   225\t          git push --force origin \"$branch\"\n   226\t          # Same-repository PRs only: `--head` matches by branch name alone, so\n   227\t          # a fork PR using this branch name must not be mistaken for the\n   228\t          # bot's own proposal (Codex review of PR #10, round 19).\n   229\t          existing_pr=$(gh pr list --head \"$branch\" --base \"$default_branch\" --state open \\\n   230\t            --json number,isCrossRepository --jq '[.[] | select(.isCrossRepository == false)][0].number // empty')\n   231\t          if [ -n \"$existing_pr\" ]; then\n   232\t            gh pr comment \"$existing_pr\" --body \"Superseded by a newer archive round at \\`${SOURCE_SHA}\\`: this proposal was regenerated from the current archive (action: **${ACTION}**, policy v${version}).\"\n   233\t            echo \"Updated the open proposal PR #$existing_pr in place.\"\n   234\t            exit 0\n   235\t          fi\n   236\t          gh pr create \\\n   237\t            --title \"chore(rsi): ${ACTION} of the improvement policy (v${version})\" \\\n   238\t            --body \"Automatically proposed by [\\`revise-improvement-policy.yml\\`](../blob/main/.github/workflows/revise-improvement-policy.yml) after the archive changed at \\`${SOURCE_SHA}\\`. Action: **${ACTION}**. See \\`docs/improvement-policy-history.jsonl\\` for the evidence and \\`docs/rsi/dashboard.html\\` for the rendered state. This changes only the AI-owned policy files; a human merges it, same as every other change in this repo.\" \\\n   239\t            --label \"self-improvement-archive\" \\\n   240\t            --base \"$default_branch\" \\\n   241\t            --head \"$branch\"\n"}
{"chunk_id":"5fe904","wall_time_seconds":0.00882325,"exit_code":0,"original_token_count":17,"output":"B1: both embedded markers neutralized; wrong-SHA footer rejected\n"}

>python3 - <<'PY'
from pathlib import Path
import subprocess
post=Path('.github/workflows/codex-review.yml').read_text()
line=next(x.strip() for x in post.splitlines() if 'body = body.replace(' in x)
archive=Path('.github/workflows/archive-and-recommend.yml').read_text()
assert '(c.body || "").trimEnd().endsWith(marker)' in archive
js="let body='quoted <!-- codex-review-sha: A --> and <!-- codex-review-status: completed -->';\n"+line+"\nif(body.includes('codex-review-sha: A') || body.includes('codex-review-status: completed')) throw Error('spoof'); const marker='<!-- codex-review-sha: A -->'; const comment='### Codex independent review\\n'+body+'\\n<!-- codex-review-sha: B -->'; if(comment.trimEnd().endsWith(marker)) throw Error('wrong SHA accepted'); console.log('B1: both embedded markers neutralized; wrong-SHA footer rejected');"
subprocess.run(['node','-e',js],check=True)
PY

{"chunk_id":"ab9c3d","wall_time_seconds":0.000002917,"exit_code":0,"original_token_count":4706,"output":"Warning: truncated output (original token count: 4706)\nTotal output lines: 373\n\ndef test_rolled_back_configuration_is_not_retried_on_the_same_evidence():\n    parent = policy_mod.builtin_policy()\n    measurement = measure.measure(_archive(), parent, None)\n    first = revise.decide(_archive(), parent, [], measurement, NOW)\n    assert first[\"action\"] == \"revise\"\n    rejected_hash = policy_mod.policy_hash(first[\"policy\"])\n    history = [\n        {\"version\": 1, \"policy\": parent},\n        {\n            \"version\": 2,\n            \"parent\": 1,\n            \"origin\": \"revision\",\n            \"coverage_before\": 0.4,\n            \"policy\": first[\"policy\"],\n        },\n        {\n            \"version\": 3,\n            \"parent\": 2,\n            \"origin\": \"rollback\",\n            \"replaced_policy_hash\": rejected_hash,\n            \"replaced_version\": 2,\n            \"archive_digest\": measurement[\"archive_digest\"],\n            \"evidence_collected_at\": None,\n            \"policy\": {**parent, \"version\": 3, \"parent\": 2, \"origin\": \"rollback\"},\n        },\n    ]\n    current = {**parent, \"version\": 3, \"parent\": 2, \"origin\": \"rollback\", \"restored_version\": 1}\n    # The rollback has to serve its own waiting period first (Codex, round 36).\n    waiting = revise.decide(\n        _archive(), current, history, measure.measure(_archive(), current, None), NOW\n    )\n    assert waiting[\"action\"] == \"none\" and \"waiting for\" in waiting[\"reason\"]\n    stamped = [\n        {**e, \"policy_version\": 3, \"policy_hash\": policy_mod.policy_hash(current)}\n        for e in _archive()\n    ]\n    measurement_r = measure.measure(stamped, current, None)\n    rollback_entry = next(e for e in history if e.get(\"origin\") == \"rollback\")\n    rollback_entry[\"archive_digest\"] = measurement_r[\"archive_digest\"]\n    again = revise.decide(stamped, current, history, measurement_r, NOW)\n    assert again[\"action\"] == \"none\"\n    assert \"rolled back\" in again[\"reason\"]\n    # New archive content lifts the block.\n    grown = [\n        *stamped,\n        {\n            \"round\": 4,\n            \"occurred_at\": \"2026-09-14T18:00:00Z\",\n            \"findings\": [\"[P2] Archive queue overflow again.\"],\n        },\n    ]\n    retry = revise.decide(grown, current, history, measure.measure(grown, current, None), NOW)\n    assert retry[\"action\"] == \"revise\"\n\n\ndef test_evidence_against_a_removed_topic_survives_for_candidates():\n#!/usr/bin/env python3\n\"\"\"Measure whether the improvement policy's own signal predicts what the\nfield shows -- the L5 trigger from docs/plans/recursive-meta-improvement.md.\n\nThe loop in this repo decides \"target fix vs mechanism fix\" from Codex\nreview findings bucketed by docs/improvement-policy.json's taxonomy. That\nbucketed count is the loop's development score: it is what the mechanism\nsees. It can be wrong in two ways the mechanism itself cannot notice:\n\n  1. Coverage: findings the taxonomy does not classify are simply dropped,\n     so a class of problem the loop keeps hitting never accumulates toward\n     the threshold. Measured as classified / total findings.\n  2. Predictive validity: a topic the taxonomy credits heavily may never\n     show up in actual working sessions, while one it barely credits does.\n     Measured as the Spearman rank correlation, across topics, between the\n     review-derived recurrence (rounds with a finding) and an independent\n     anchor: Traces evidence from working sessions in this repository.\n\nThe anchor deliberately excludes the verifier's own transcripts (Codex\nreview sessions) by default: those contain the findings themselves, so\ncounting them would make the anchor echo the development score instead of\nchecking it (paper failure mode 3, \"reliable verification\").\n\nBoth measures are replayed per archive round, using only the rounds and\ntraces that existed at that round's timestamp, so the dashboard can show\nwhen a revision would have fired, not just where things stand now.\n\nUsage:\n    python3 measure-policy-validity.py <archive.jsonl>\n        [--policy PATH] [--trace-evidence EVIDENCE.json] [--out-json PATH]\n\nEvidence comes from `mine-trace-failures.py --repo-dir DIR --save-evidence\nEVIDENCE.json`. Without --trace-evidence the anchor is absent: coverage is\nstill measured, validity is reported as null, and the JSON says so plainly.\nPrints human-readable lines, then a `---` separator, then a JSON object.\n\"\"\"\n\nfrom __future__ import annotations\n\nimport argparse\nimport hashlib\nimport importlib.util\nimport json\nimport sys\nfrom datetime import UTC, datetime\nfrom pathlib import Path\n\n\ndef _load_sibling_module(name: str, filename: str):\n    if name in sys.modules:\n        return sys.modules[name]\n    path = Path(__file__).parent / filename\n    spec = importlib.util.spec_from_file_location(name, path)\n    assert spec is not None and spec.loader is not None\n    module = importlib.util.module_from_spec(spec)\n    sys.modules[name] = module\n    spec.loader.exec_module(module)\n    return module\n\n\npolicy_mod = _load_sibling_module(\"improvement_policy\", \"improvement_policy.py\")\n\nDEFAULT_ANCHOR_AGENTS = [\"claude-code\", \"antigravity\", \"cursor\", \"droid\", \"openclaw\", \"pi\"]\nMIN_TOPICS_FOR_VALIDITY = 3\n\n\n# --- archive replay ---------------------------------------------------------\n\n\ndef load_archive(path: str) -> list[dict]:\n    entries = []\n    with open(path) as f:\n        for line in f:\n            line = line.strip()\n            if line:\n                entries.append(json.loads(line))\n    return entries\n\n\ndef archive_digest(entries: list[dict]) -> str:\n    \"\"\"Content digest of the archive a measurement was taken against, so a\n    decision can refuse a measurement from a different archive (Codex\n    review of PR #10, round 2, finding 2).\"\"\"\n    canonical = json.dumps(entries, sort_keys=[REDACTED] separators=(\",\", \":\"))\n    return hashlib.sha256(canonical.encode()).hexdigest()[:12]\n\n\ndef parse_timestamp_ms(value: object) -> int | None:\n    if not isinstance(value, str):\n        return None\n    try:\n        parsed = datetime.fromisoformat(value.replace(\"Z\", \"+00:00\"))\n    except ValueError:\n        return None\n    if parsed.tzinfo is None:\n    ]\n)\n\n_TOKEN_RE = re.compile(r\"[a-z][a-z_-]{2,}\")\n\n\ndef tokenize(text: str) -> set[str]:\n    return {\n        tok.strip(\"-_\")\n        for tok in _TOKEN_RE.findall(text.lower())\n        if len(tok) >= MIN_TOKEN_LENGTH\n        and tok not in STOPWORDS\n        and not tok.startswith(\"http\")\n        and not tok.endswith(\"ly\")  # adverbs name manner, never a class of problem\n    }\n\n\ndef existing_keyword_tokens(keywords: dict[str, list[str]]) -> set[str]:\n    return {k.lower() for words in keywords.values() for k in words}\n\n\ndef mine_topics(\n    unclassified: list[dict],\n    keywords: dict[str, list[str]],\n) -> list[dict]:\n    \"\"\"Greedy, auditable topic mining over findings the policy could not\n    classify: the most frequent significant token names a topic; its\n    keywords are that token plus the tokens that co-occur with it most;\n    findings the new topic covers are removed and the process repeats.\"\"\"\n    taken = existing_keyword_tokens(keywords)\n    remaining = [\n        {\n            \"round\": item[\"round\"],\n            \"finding\": item[\"finding\"],\n            \"tokens\": tokenize(item[\"finding\"]) - taken,\n        }\n        for item in unclassified\n    ]\n    mined: list[dict] = []\n    while len(mined) < MAX_NEW_TOPICS and remaining:\n        df = Counter(tok for item in remaining for tok in item[\"tokens\"])\n        candidates = [t for t, n in df.items() if n >= MIN_FINDINGS_PER_TOPIC]\n        if not candidates:\n            break\n        head = sorted(candidates, key=lambda t: (-df[t], t))[0]\n        covering_head = [item for item in remaining if head in item[\"tokens\"]]\n        co = Counter(tok for item in covering_head for tok in item[\"tokens\"] if tok != head)\n        companions = [\n            t\n            for t, n in sorted(co.items(), key=lambda kv: (-kv[1], kv[0]))\n            if n >= MIN_FINDINGS_PER_TOPIC\n        ][: MAX_KEYWORDS_PER_TOPIC - 1]\n        topic_keywords = [head, *companions]\n        # A topic claims exactly the findings it would classify: any keyword,\n        # by the same substring rule the detector uses. Those findings are\n        # then unavailable to later topics, so no finding supports two\n        # topics (Codex review of PR #10, round 2, finding 3).\n        covering = [\n            item\n            for item in remaining\n            if policy_mod.classify_finding(item[\"finding\"], {\"_\": topic_keywords}) is not None\n        ]\n        if len(covering) < MIN_FINDINGS_PER_TOPIC:\n            break\n        base = \"-\".join(topic_keywords[:2]) if companions else head\n        taken_names = set(keywords) | {m[\"name\"] for m in mined}\n        name = base\n        suffix = 1\n        while name in taken_names:\n            # Never reuse a name: an overwritten topic would silently drop\n            # its classifications (Codex review of PR #10, round 14).\n            suffix += 1\n            name = f\"{base}-{suffix}\"\n        mined.append(\n            {\n                \"name\": name,\n   108\tdef run_traces_json(traces_bin: str, args: list[st…206 tokens truncated….read()\n   125\t        except OSError as exc:\n   126\t            raise TracesCliError(f\"Could not run `{traces_bin}`: {exc}\") from exc\n   127\t        if result.returncode != 0:\n   128\t            raise TracesCliError(f\"`{traces_bin} {' '.join(args)}` failed: {result.stderr.strip()}\")\n   129\t        payload = parse_cli_json(stdout)\n   130\t        if payload is not None:\n   131\t            if not payload.get(\"ok\"):\n   132\t                raise TracesCliError(f\"`{traces_bin} {' '.join(args)}` reported failure: {payload}\")\n   133\t            return payload[\"data\"]\n   134\t        last_error = stdout[:120].replace(\"\\n\", \" \")\n   135\t        if attempt < retries:\n   136\t            continue\n   137\t    raise TracesCliError(f\"Non-JSON output from `{traces_bin} {' '.join(args)}`: {last_error}\")\n   138\t\n   139\t\n   140\tdef list_traces(\n   141\t    traces_bin: str, repo_dir: str, agents: list[str] | None, limit: int\n   142\t) -> tuple[list[dict], bool]:\n   143\t    \"\"\"Sessions recorded in the folder, and whether the listing was complete.\n   144\t    A listing that fills its limit may have missed sessions, and evidence\n   145\t    built from it must say so rather than report confirmed zero counts\n   246\t        output = str(event.get(\"output\") or event.get(\"text\") or \"\")\n   247\t        tool = str(event.get(\"toolName\") or \"\")\n   248\t        call = calls.get(str(event.get(\"callId\")), {})\n   249\t        args = call.get(\"args\") or {}\n   250\t        command = str(args.get(\"command\") or args.get(\"file_path\") or args.get(\"pattern\") or \"\")\n   251\t        excerpt = excerpt_for(kind, output)\n   252\t        # The command is part of identity: two commands with the same output\n   253\t        # are two failures, and classification reads the command\n   254\t        # (Codex review of PR #10, round 28).\n   255\t        key = (tool, \" \".join(command.split())[:EXCERPT_CHARS], excerpt)\n   256\t        if key in failures:\n   257\t            failures[key][\"count\"] += 1\n   258\t            continue\n   259\t        failures[key] = {\n   260\t            \"_text\": f\"{command} {output}\".lower(),\n   261\t            \"trace_id\": trace[\"id\"],\n   262\t            \"agent\": trace.get(\"agentId\"),\n   263\t            \"event_number\": event.get(\"eventNumber\"),\n   264\t            \"timestamp\": event.get(\"timestamp\"),\n   265\t            \"tool\": tool,\n   475\tdef rejected_configuration(candidate: dict, history: list[dict], measurement: dict) -> dict | None:\n   476\t    \"\"\"A configuration rolled back on the same archive and evidence is not\n   477\t    retried: the rollback's history entry records the rejected hash and\n   478\t    what it was judged on.\"\"\"\n   479\t    wanted = policy_mod.policy_hash(candidate)\n   480\t    collected = (measurement.get(\"anchor\") or {}).get(\"collected_at\")\n   481\t    for entry in history:\n   482\t        if entry.get(\"origin\") != \"rollback\" or entry.get(\"replaced_policy_hash\") != wanted:\n   483\t            continue\n   484\t        if entry.get(\"archive_digest\") == measurement.get(\"archive_digest\") and (\n   485\t            entry.get(\"evidence_collected_at\") == collected\n   486\t        ):\n   487\t            return entry\n   488\t    return None\n   489\t\n   490\t\n   170\t        env:\n   171\t          GH_TOKEN: ${{ github.token }}\n   172\t          ACTION: ${{ steps.revise.outputs.action }}\n   173\t          SOURCE_SHA: ${{ steps.source.outputs.sha }}\n   174\t        run: |\n   175\t          set -euo pipefail\n   176\t          if git diff --quiet -- docs/improvement-policy.json docs/improvement-policy-history.jsonl docs/rsi/; then\n   177\t            echo \"Nothing changed (action: $ACTION).\"\n   178\t            exit 0\n   179\t          fi\n   180\t          version=$(python3 -c 'import json; print(json.load(open(\"docs/improvement-policy.json\"))[\"version\"])')\n   181\t          default_branch=$(gh repo view --json defaultBranchRef --jq .defaultBranchRef.name)\n   182\t          git config user.name \"github-actions[bot]\"\n   183\t          git config user.email \"41898282+github-actions[bot]@users.noreply.github.com\"\n   184\t          if git diff --quiet -- docs/improvement-policy.json docs/improvement-policy-history.jsonl; then\n   185\t            # Only the evidence, measurement and dashboard moved (the hourly\n   186\t            # refresh with no decision). One standing evidence branch, updated\n   187\t            # in place, so the hourly run never opens a new PR per hour; a\n   188\t            # human merges it whenever, and merging it re-triggers this\n   189\t            # workflow through the trace-evidence.json path filter.\n   190\t            # A standing policy proposal that the current decision no longer\n   191\t            # supports is withdrawn, not left open to merge (Codex review of\n   192\t            # PR #10, round 37): new rounds can legitimately turn a\n   193\t            # `revise` into `none`.\n   194\t            stale_pr=$(gh pr list --head \"improvement-policy-proposal\" --base \"$default_branch\" --state open \\\n   195\t              --json number,isCrossRepository --jq '[.[] | select(.isCrossRepository == false)][0].number // empty')\n   196\t            if [ -n \"$stale_pr\" ]; then\n   197\t              gh pr close \"$stale_pr\" --delete-branch \\\n   198\t                --comment \"Withdrawn: re-evaluated at \\`${SOURCE_SHA}\\` the fixed acceptance rule proposes no policy change (action: **${ACTION}**), so this proposal no longer satisfies it. A new proposal will be opened if a later round supports one.\"\n   199\t              echo \"Closed the stale policy proposal PR #$stale_pr.\"\n   200\t            fi\n   201\t            branch=\"improvement-evidence-refresh\"\n   202\t            title=\"chore(rsi): refresh the Traces field anchor\"\n   203\t            body=\"Automatically refreshed by [\\`revise-improvement-policy.yml\\`](../blob/main/.github/workflows/revise-improvement-policy.yml) from the shared Traces namespace at \\`${SOURCE_SHA}\\`. The fixed acceptance rule proposed no policy change on this evidence (action: **${ACTION}**). This changes only the evidence snapshot, the measurement and the dashboard; the policy is untouched.\"\n   204\t            git checkout -b \"$branch\"\n   205\t            git add docs/rsi/\n   206\t            git commit -m \"$title\"\n   207\t            git push --force origin \"$branch\"\n   208\t            existing_pr=$(gh pr list --head \"$branch\" --base \"$default_branch\" --state open \\\n   209\t              --json number,isCrossRepository --jq '[.[] | select(.isCrossRepository == false)][0].number // empty')\n   210\t            if [ -n \"$existing_pr\" ]; then\n   211\t              echo \"Updated the open evidence PR #$existing_pr in place.\"\n   212\t              exit 0\n   213\t            fi\n   214\t            gh pr create --title \"$title\" --body \"$body\" --label \"self-improvement-archive\" \\\n   215\t              --base \"$default_branch\" --head \"$branch\"\n   216\t            exit 0\n   217\t          fi\n   218\t          # One standing proposal branch: a newer archive round supersedes an\n   219\t          # open proposal instead of opening a competing one with the same\n   220\t          # version and parent (Codex review of PR #10, round 5).\n   221\t          branch=\"improvement-policy-proposal\"\n   222\t          git checkout -b \"$branch\"\n   223\t          git add docs/improvement-policy.json docs/improvement-policy-history.jsonl docs/rsi/\n   224\t          git commit -m \"chore(rsi): ${ACTION} of the improvement policy (v${version})\"\n   225\t          git push --force origin \"$branch\"\n   226\t          # Same-repository PRs only: `--head` matches by branch name alone, so\n   227\t          # a fork PR using this branch name must not be mistaken for the\n   228\t          # bot's own proposal (Codex review of PR #10, round 19).\n   229\t          existing_pr=$(gh pr list --head \"$branch\" --base \"$default_branch\" --state open \\\n   230\t            --json number,isCrossRepository --jq '[.[] | select(.isCrossRepository == false)][0].number // empty')\n   231\t          if [ -n \"$existing_pr\" ]; then\n   232\t            gh pr comment \"$existing_pr\" --body \"Superseded by a newer archive round at \\`${SOURCE_SHA}\\`: this proposal was regenerated from the current archive (action: **${ACTION}**, policy v${version}).\"\n   233\t            echo \"Updated the open proposal PR #$existing_pr in place.\"\n   234\t            exit 0\n   235\t          fi\n   236\t          gh pr create \\\n   237\t            --title \"chore(rsi): ${ACTION} of the improvement policy (v${version})\" \\\n   238\t            --body \"Automatically proposed by [\\`revise-improvement-policy.yml\\`](../blob/main/.github/workflows/revise-improvement-policy.yml) after the archive changed at \\`${SOURCE_SHA}\\`. Action: **${ACTION}**. See \\`docs/improvement-policy-history.jsonl\\` for the evidence and \\`docs/rsi/dashboard.html\\` for the rendered state. This changes only the AI-owned policy files; a human merges it, same as every other change in this repo.\" \\\n   239\t            --label \"self-improvement-archive\" \\\n   240\t            --base \"$default_branch\" \\\n   241\t            --head \"$branch\"\n"}
{"chunk_id":"5fe904","wall_time_seconds":0.00882325,"exit_code":0,"original_token_count":17,"output":"B1: both embedded markers neutralized; wrong-SHA footer rejected\n"}

>python3 - <<'PY'
import sys, copy
sys.path.insert(0,'scripts')
import revise_improvement_policy_test as t
p=t.policy_mod;r=t.revise;m=t.measure
v1=p.builtin_policy(); first=r.decide(t._archive(),v1,[],m.measure(t._archive(),v1,None),t.NOW)
v3={**v1,'version':3,'parent':2,'origin':'rollback','restored_version':1}
a=[{**e,'policy_version':3,'policy_hash':p.policy_hash(v3)} for e in t._archive()]
e={'collected_at':'2026-09-14T19:00:00Z','sessions':['s1'],'definitions':p.topic_keywords(v1),'topics':{k:[] for k in v1['topics']},'truncated':[]}
measurement=m.measure(a,v3,e)
h=[{'version':1,'policy':v1},{'version':2,'policy':first['policy']},{'version':3,'origin':'rollback','policy':v3,'replaced_policy_hash':p.policy_hash(first['policy']),'archive_digest':measurement['archive_digest'],'evidence_collected_at':e['collected_at']}]
before=r.decide(a,v3,h,measurement,t.NOW)
e2={**e,'collected_at':'2026-09-14T20:00:00Z'}
after=r.decide(a,v3,h,m.measure(a,v3,e2),t.NOW)
print('Same observations:',before['action'],before['reason']);print('Only collection time advanced:',after['action'],after['reason'])
assert before['action']=='none' and after['action']=='revise'
assert p.policy_hash(after['policy'])==p.policy_hash(first['policy']);print('Identical rejected configuration re-proposed: yes')
PY

{"chunk_id":"4dcb85","wall_time_seconds":0.129650834,"exit_code":0,"original_token_count":63,"output":"Same observations: none candidate reproduces configuration e81ee3a6e789, rolled back as v? on the same archive and evidence; needs new evidence\nOnly collection time advanced: revise coverage 0.4 < 0.8\nIdentical rejected configuration re-proposed: yes\n"}

>python3 - <<'PY'
from pathlib import Path
import os, tempfile, subprocess, json
workflow=Path('.github/workflows/revise-improvement-policy.yml').read_text()
block=workflow.split('      - name: Propose the result as a pull request',1)[1].split('        run: |\n',1)[1]
script='\n'.join(line[10:] for line in block.splitlines())
with tempfile.TemporaryDirectory() as td:
 d=Path(td);(d/'bin').mkdir();(d/'docs').mkdir();(d/'docs/improvement-policy.json').write_text('{"version":2}')
 stub='''#!/usr/bin/env python3
import os,sys,json
from pathlib import Path
args=sys.argv[1:]; name=Path(sys.argv[0]).name
with open(os.environ['LOG'],'a') as f:f.write(json.dumps([name,*args])+'\\n')
if name=='git' and args[:2]==['diff','--quiet']:
 sys.exit(1 if os.environ['MODE']=='policy' or 'docs/rsi/' in args else 0)
if name=='gh':
 if args[:2]==['repo','view']: print('main')
 if args[:2]==['pr','list']:
  branch=args[args.index('--head')+1]
  if branch=='improvement-evidence-refresh':print('42')
  elif os.environ['MODE']=='policy':print('43')
'''
 for tool in ['git','gh']:
  f=d/'bin'/tool;f.write_text(stub);f.chmod(0o755)
 env={**os.environ,'PATH':str(d/'bin')+':'+os.environ['PATH'],'LOG':str(d/'calls'),'SOURCE_SHA':'abc','ACTION':'none','MODE':'evidence'}
 for _ in range(2): subprocess.run(['bash','-c',script],cwd=d,env=env,check=True,capture_output=True)
 calls=[json.loads(x) for x in (d/'calls').read_text().splitlines()]
 print('Two evidence-only runs: PR creates=',sum(c[:3]==['gh','pr','create'] for c in calls),'pushes=',sum(c[:2]==['git','push'] for c in calls))
 assert not any(c[:3]==['gh','pr','create'] for c in calls)
 adds=[c for c in calls if c[:2]==['git','add']];assert all(c==['git','add','docs/rsi/'] for c in adds); print('Evidence-only staged paths:',adds)
 (d/'calls').write_text('');env.update(MODE='policy',ACTION='revise')
 for _ in range(2):subprocess.run(['bash','-c',script],cwd=d,env=env,check=True,capture_output=True)
 calls=[json.loads(x) for x in (d/'calls').read_text().splitlines()]; print('Two unchanged-source policy runs: comments=',sum(c[:3]==['gh','pr','comment'] for c in calls),'PR creates=',sum(c[:3]==['gh','pr','create'] for c in calls))
PY

{"chunk_id":"5002b1","wall_time_seconds":1.742588792,"exit_code":0,"original_token_count":49,"output":"Two evidence-only runs: PR creates= 0 pushes= 2\nEvidence-only staged paths: [['git', 'add', 'docs/rsi/'], ['git', 'add', 'docs/rsi/']]\nTwo unchanged-source policy runs: comments= 2 PR creates= 0\n"}
{"chunk_id":"2cebcd","wall_time_seconds":0.000004,"exit_code":0,"original_token_count":9,"output":"?? .claude/settings.json\n5bd4d6e9\n"}

>git status --short; git diff --exit-code; git rev-parse --short HEAD

{"chunk_id":"5002b1","wall_time_seconds":1.742588792,"exit_code":0,"original_token_count":49,"output":"Two evidence-only runs: PR creates= 0 pushes= 2\nEvidence-only staged paths: [['git', 'add', 'docs/rsi/'], ['git', 'add', 'docs/rsi/']]\nTwo unchanged-source policy runs: comments= 2 PR creates= 0\n"}
{"chunk_id":"2cebcd","wall_time_seconds":0.000004,"exit_code":0,"original_token_count":9,"output":"?? .claude/settings.json\n5bd4d6e9\n"}

The hourly review found two more issues: advancing only collected_at re-enables a previously rejected policy, and repeated runs on the same source post repeated comments to an open policy PR. In isolated tests, evidence-only runs reused the existing PR and staged only docs/rsi/. No GitHub writes were made.

A1: CLOSED — Changing only origin to "init" exits with code 1 because it disagrees with history; a regression test covers lineage tampering.
A2: CLOSED — A topic containing @ raises ValueError before acceptance; a regression test pins validation.
A3: STILL OPEN — The single-failure case passes, but two errors sharing a shortened excerpt discard the later secret exposed match, producing zero evidence with truncated=[].
A4: CLOSED — A rollback restoring v1 waits after one round even though judged_from() returns None; a regression test pins this.
A5: CLOSED — The quartz v1→v2→v3 reproduction rolls back to v1; a regression test covers grandparent coverage.
A6: CLOSED — Passing a hard-linked archive as --out-json raises PermissionError and preserves the archive; hard-link protection has regression tests.
B1: CLOSED — Executing the posting sanitizer neutralized both embedded markers, and the footer check rejected the wrong SHA; verified with targeted assertions, not a committed workflow test.
B2: CLOSED — Two SHAs sharing one round number produce two measured rounds and rounds_under()==2; committed and targeted tests cover these cases.
C1: CLOSED — Negative validity plots below zero; the chart-coordinate regression test passes.
C2: CLOSED — Both numerical claims now explicitly describe historical observations and reproduction limits; targeted document assertions pass.
R36a: CLOSED — A complete unclassified scan retains its session and yields a zero-valued anchor instead of no anchor; regression tests pass.
R36b: CLOSED — The root-rollback waiting reproduction passes, as for A4.

New findings

[P2] Collection time alone re-enables a rejected policy. scripts/revise-improvement-policy.py:480–485
Reproduction: after the rollback waiting period, record a rejected candidate with the current archive digest and evidence timestamp. The decision returns none (“needs new evidence”). Change only evidence collected_at from 19:00 to 20:00, preserving every observation and archive entry. The decision becomes revise, proposing the identical rejected configuration hash, e81ee3a6e789. Hourly refreshes therefore bypass this guard without new observations.
[P2] Traces timeout exceptions expose the API key. scripts/mine-trace-failures.py:115–126
Reproduction: set EXTRA_CLI_ARGS=['--key','SENTINEL_KEY'] and inject subprocess.TimeoutExpired at the subprocess boundary. It escapes uncaught and formats:
text
```
Command '['traces', 'list', '@slug', '--all', '--key', 'SENTINEL_KEY', '--json']' timed out after 300 seconds
```
```
Command '['traces', 'list', '@slug', '--all', '--key', 'SENTINEL_KEY', '--json']' timed out after 300 seconds
```
Raw stderr exposes the key. GitHub secret masking may conceal it in Actions; I did not test live Actions masking.
[P3] Hourly policy proposals generate repetitive comments. .github/workflows/revise-improvement-policy.yml:231–232
Reproduction: execute the actual posting shell twice with stubbed git/gh, identical source SHA, ACTION=revise, and an existing policy PR. Observed two comments, despite no new source round. The standing branch prevents duplicate open PRs but does not prevent notification spam.

The A3 residual failure occurs at scripts/mine-trace-failures.py:255–258: deduplication increments the count and skips the later full text.

Verification

git diff origin/main...HEAD — executed against HEAD 5bd4d6e9.
python3 -m pytest scripts/ -q -p no:cacheprovider — 160 passed in 3.25s.
ruff check scripts/ — All checks passed.
Targeted python3 - <<'PY' ... PY reproductions — exercised every listed finding, excerpt collisions, timestamp-only retry, timeout disclosure, and workflow branching.
Posting sanitizer executed through Node; workflow shell executed with isolated command stubs.
Two evidence-only runs with an existing PR: zero PR creations, two branch pushes, staging restricted to docs/rsi/.
git diff --exit-code — clean; no tracked files modified. Pre-existing untracked .claude/settings.json remains.

Live Traces and GitHub operations were not exercised.

Verdict

The “all fixed” claim does not hold. A3 still reproduces, and hourly refreshes introduce material retry and credential-handling gaps despite the passing suite.

No content