Skip to content

type: Concept title: "Cron Tool Provenance & Error Taxonomy — additive v1 record fields for regime_snapshot.py" description: Proposal adapting the TradingAgents point-in-time-provenance and vendor-error-taxonomy patterns: a provenance block and a structured missing_errors mirror on every regime journal record, gating-neutral by shadow proof, applied only on owner approval. tags: [proposal, regime-snapshot, provenance, point-in-time, error-taxonomy, journal-v1, tradingagents] generated: { by: agent/hermes, at: 2026-09-27 } status: stable verified: { by: human:yasu, at: '2026-10-06' } owner_approval: approved 2026-09-27 applied: 2026-09-27

Summary

An additive v1 proposal for the regime snapshot's journal records, adapting two engineering patterns from the TradingAgents framework review:

  1. Provenance block (their v0.5.0 point-in-time pattern): every record carries where each dated input came from and when it was fetched.
  2. Error-class taxonomy (their Sep-24 vendor-error pattern): every fail-closed reason is classified — vendor_unreachable / no_data_in_window / staleness / internal — instead of only a prose string.

Both are record-shape additions — no gate input, no label computation, no fail-closed boundary changes — but the change is not purely additive in the strict sense: forecast_model_version changes value at the deploy boundary (declared below per review r1 R1-A). The design follows the two defects this loop actually ate: the 2026-09-17 mixed-instant term ratio (corrected under 2026-09-17:regime) and the 2026-09-25 FRED-freeze fail-closed, whose record showed that data was unusable but not how or from where.

Declared Deviation — forecast_model_version at the deploy boundary (R1-A)

CORRECTED 2026-09-29 (2026-09-29 corrects 2026-09-27:cron-tool-provenance-taxonomy-apply). The framing below is wrong and is kept only so the correction has something to point at. forecast_model_version is a per-run identifier: the hash mixes the file bytes with window_end and n_obs, so it changes every trading day regardless of code. Six consecutive rounds on one unchanged file produced six different values (09-21 531416d0… through 09-28 92b61f04…). There is therefore no visible deploy boundary in this field — a code change is indistinguishable from an ordinary day. The real evidence that the model is unchanged is that forecast_rv30 (0.1453) and forecast_window_end match across the apply for a fixed asof, which is stated below and remains true.

forecast_model_version is sha256 of har_forecast.py's own source bytes (plus ticker/window/window_end/n_obs), so the proposed pair file necessarily changes its value. Deployed and proposed values:

  • deployed: 531416d07752cc2b
  • proposed (r2 pair file): 2c8b70851012bdcb

The field says "same model" to later readers, so the boundary must be findable in the journal, not reconstructed: the apply record (on owner approval) must restate both hashes, the deploy date, and this evidence that the model is unchanged — forecast_rv30 (0.1453) and forecast_window_end are identical between deployed and proposed code on 2026-09-19 (proof, ops/tool-proposals/proof-2026-09-27/proof-output.txt); only error emission changed. This page carries the same declaration so the concept never asserts "existing fields never change meaning" without it.

The Two Additive Fields

provenance (emitted on every run, including fail-closed runs — an outage record itself carries provenance):

"provenance": {
  "schema": "v1-prov1",
  "fetched_at": "2026-09-27T18:13:45Z",
  "term_source": "FRED fredgraph.csv VIXCLS,VXVCLS (one request, same-date closes)",
  "har_source": "tools/har_forecast.py --ticker SPY",
  "asof_arg": "2026-09-27",
  "vix_date": "2026-09-22",
  "har_window_end": "2026-09-25",
  "max_input_age_business_days": 2
}

missing_errors (structured mirror of missing, same order, messages verbatim):

"missing_errors": [
  {"message": "stale vix/vix3m close: 2026-09-22 is 4 business days before 2026-09-27 (max 2)",
   "error_class": "staleness"}
]

Classes: vendor_unreachable (request itself failed), no_data_in_window (source answered, no usable value — e.g. a publication freeze), staleness (value exists but exceeds max_input_age_business_days), internal (our tooling/config failed). Per review r1 R1-B, the HAR classes are assigned by HTTP status inside har_forecast.py itself: 401/403 (our credential — expired/revoked key) and 400 (our programmatically-shaped request) → internal; 404 → no_data_in_window; connection refused / timeout / DNS / 5xx → vendor_unreachable. missing strings are unchanged for every failure mode deployed code already structures (staleness, thin history, HAR fit invalid, key-not-found); for vendor/auth fetch failures the pair file replaces deployed's raw traceback with a status-specific line, so those strings differ — the fail-closed exit code, prompt rules, and report structure are untouched. The taxonomy makes "volatility of data supply" evidence (the 9/25 team-insights observation) machine-loggable.

Gating-Neutrality Proof — revision r2 (2026-09-27, per owner review r1)

The first proof (stdout-record comparison, --no-append) was rejected by owner review as methodologically insufficient (B2: it never exercised the write path). The revised proof runs in production mode — append ENABLED — in a sandbox SEEDED with a copy of the real regime.jsonl, and compares the appended journal records, not stdout. Owner review r1 independently re-ran it from commit 64081e8: 10/10 PASS, same forecast_rv30 0.1453, same staleness message — then requested the r2 changes below (fault-name fidelity, the missing third class, the model-version declaration, dead harness code). The archived harness additionally proves it can catch the B1 defect class by fault injection. Full output + harness: ops/tool-proposals/proof-2026-09-27/ (proof-output.txt, run_proof.py); proposal file sha256s are in the decisions records. Exact commands (each in a sandbox with tools/ populated from the file under test and 96-journal/regime.jsonl seeded from production):

  • python3 regime_snapshot.py --asof 2026-09-19 — healthy path, deployed vs proposed (both files of the pair proposed), exit 0 both, record appended both. (Both proof dates are Saturdays, guaranteed absent from the journal, so the idempotency branch cannot suppress the writes under comparison.)
  • python3 regime_snapshot.py --asof 2026-09-19 (second run, same sandbox) — idempotency: proposed code suppresses its own identical re-run (N1 normalization below).
  • python3 regime_snapshot.py --asof 2026-09-26 — fail-closed path, exit 2 both, records appended both.
  • python3 har_forecast.py with no Tiingo key (config → internal, stderr prose byte-identical to deployed); with a dummy key against real Tiingo (HTTP 401/403 → internal, message names the credential, not the vendor); via a closed-port proxy (connection refused → vendor_unreachable, the injected fault matching the test's name per R1-B); with fetch_closes truncated to 12 closes (thin history → no_data_in_window, per R1-C).

Results (ALL PASS, 12/12): healthy and fail-closed appended records from the proposed code equal the deployed code's records apart from (a) the two additive fields and (b) forecast_model_version — declared above per R1-A, with forecast_rv30 (0.1453) and forecast_window_end asserted equal. The 2026-09-26 freeze reason classifies staleness.

Fault-injection control: the B1 defect (read-mode journal open) reintroduced into deployed code yields exit 1, io.UnsupportedOperation, journal unchanged — the harness demonstrably fails such a candidate, closing the "a proof that could not have failed" gap.

Idempotency interaction (review note N1, addressed): the same-day suppression check normalizes away provenance.fetched_at, so new-code re-runs still suppress; the first re-run over a date journaled by current deployed code still appends once (old records never carry the additive fields) — documented here rather than discovered in the journal.

Honest scope note: for failure modes where the pair file wraps the stderr prose (vendor/auth fetch failures), the missing string differs from what deployed code would have produced for that failure — previously-structured failures (thin closes, HARInvalid, key-not-found, staleness) are byte-identical.

What This Does NOT Do

  • Does not change any regime label, band, threshold, or the hysteresis state machine; regime_config.json and mapping_version are untouched (an operational record-shape change, not a mapping amendment — same class as the staleness guard). The single declared exception to "additive" is the forecast_model_version boundary above.
  • Does not alter the fail-closed boundary: a vendor_unreachable day still gets no label, exactly as a freeze day does today. Classification is for the record, not for gating.
  • Does not wire anything into cron or prompts. Consumers (round prompt, report-builder) read missing strings unchanged; a future prompt amendment to surface the provenance/error-class in round prose is a separate governance event.
  • Does not implement fallback data sources for FRED staleness — that 9/25 AMENDMENT-PROPOSAL idea remains owner-review-only and unaffected.
  • Does not self-apply: approval pending; the pair sits in ops/tool-proposals/, off every deploy path (deploy.sh contains no reference to ops/ — verified in review r0) — application is the owner-approved act of replacing the two tools/ files with the proposed content.

Procedure

  • Decision-log records: 2026-09-27:cron-tool-provenance-taxonomy (proposed) → owner review …-revise (REVISE) → revision r1 …-revise-r1 (proposed) → owner review …-revise-r2 (REVISE, close) → this revision r3 …-revise-r3 (approval pending). Review responses: r0, r1.
  • Proposal is a TWO-FILE PAIR (review B3): ops/tool-proposals/regime_snapshot.py.proposed + ops/tool-proposals/har_forecast.py.proposed, both based on the deployed files at repo commit 735222a (byte-identical wiki copies verified 2026-09-27), off every deploy path.
  • On owner approval: copy both pair files over tools/regime_snapshot.py and tools/har_forecast.py in the repository, re-run the archived harness in place (ops/tool-proposals/proof-2026-09-27/run_proof.py), commit, deploy, and record the apply — restating both forecast_model_version hashes, the deploy date, and the model-unchanged evidence per R1-A. Rejection retires both proposal files.
  • Invalidation: if either deployed tool moves past commit 735222a before approval, rebase the pair and re-prove.