Skip to content

ADR-0080 — FTF post-run extraction, dossier, and Story-update mechanics (operator-triggered)

Status: active Date: 2026-05-01 Introduced by: CVN-N001-EE-S06 (Track 11 closure) — operator request 2026-05-01 after Track 11 dossier had to invent the PDF-extraction path on the fly (kubectl exec into Console pod + psycopg2 against PG finetune_runs.pdf_report). Track 9 used the local-results/ filesystem path — Track 11 had to switch sources because the PDF wasn't dropped locally. This ADR codifies the canonical extraction + dossier + Story-update mechanics so future closures are mechanical. Supersedes: none — complementary to ADR-0079 : ADR-0079 defines the what (8-step workflow + verdict decision tree + dossier template + invariants). This ADR defines the how (executable mechanics — trigger semantics, extraction path, single-PR closure pattern).

Context

ADR-0079 codified the FTF-sweep → Story-closure 8-step workflow. Three subsequent closures (Tracks 5 / 6 / 9) followed it cleanly. Track 11 closure exposed three operational gaps that ADR-0079 did not specify :

  1. PDF extraction path is ambiguous. ADR-0079 step 3 says "Console 'Download PDF' button → documentation/missions/<mission>/reports/" — but :
  2. The Console UI runs Streamlit ; its st.download_button blob has no static URL — Claude / automation cannot click it.
  3. The local results/ filesystem is operator-machine-specific and gitignored — relying on it means the workflow is non-reproducible (Track 9 worked because the operator had recently downloaded ; Track 11 broke because the operator hadn't).

  4. The actual source-of-truth is finetune_runs.pdf_report (PostgreSQL bytea column) — but this isn't documented anywhere as the canonical extraction target.

  5. Trigger semantics are implicit. The Track 11 closure was triggered by the operator message "il est fini - tu dois récupérer le report, analyser, documenter…". This is the canonical pattern but it isn't named anywhere — risk of drift if a future operator says "auto-close on completion" thinking that's how it works (it isn't and shouldn't be — the verdict decision is operator authority per ADR-0079 invariant 5).

  6. Story-update path is fragmented. Today the operator must :

  7. Update F1 plan §10 row (or equivalent mission tracker)
  8. Update mkdocs nav for the dossier
  9. Either manually transition OP wp status OR wait for the auto-syncer (PR #796, still in CR review at the time of writing)
  10. Possibly post an OP comment with run_id + dossier link + verdict

This is 4 fragmented surfaces with no single closure command. The auto-syncer (PR #796) only handles the OP transition leg ; the others remain manual.

Failure mode if this ADR is not written : the next FTF closure (likely Track 1 wp#43 once block A merges + sweep runs + verdict decided) will re-invent the extraction path. By then, Track 11's kubectl exec console pod workaround will live only in this dossier's prose — not as a make target, not as a runbook step, not as anything callable. ADR-0079's promise of "mechanical Story closure ~30 min instead of ~2h" is not achievable without this ADR's mechanics.

Decision

Every FTF-sweep-driven Story closure is operator-triggered, follows a single canonical extraction → dossier → Story-update sequence, and ships as a single docs-only PR. No auto-on-completion ; no fragmented partial closures.

1. Trigger : operator-driven, never auto-on-completion

The post-FTF ritual is initiated only by an explicit operator request after the sweep status reaches completed OR failed-with-useful-data in finetune_runs. The canonical phrasing pattern :

operator: "<run X> is done — extract, analyse, document, close the story"

Or the equivalent French ("il est fini — tu dois récupérer le report, analyser, documenter et fermer la story"). The trigger is never :

  • Automatic on Airflow DAG completion (the verdict requires operator judgement per ADR-0079 invariant 5)
  • Triggered by a cron / webhook (same reason)
  • Inferred from PG row state changes (same reason)

Rationale : the verdict (LOCK / KEEP_AVAILABLE / ABANDON) and the dossier's hypothesis-pick (§10 of the dossier template) are operator decisions that compose with cross-Track context only the operator has (e.g., "we already abandoned 3 tiers — this is the 4th, write the cross-Track lesson differently"). Auto-execution would either drop the cross-Track context OR force an LLM to invent it, neither acceptable.

2. Canonical extraction path : PG finetune_runs.pdf_report

The source-of-truth for the FTF report PDF is the PostgreSQL finetune_runs.pdf_report bytea column. Not the local results/ filesystem (operator-machine-specific, gitignored), not the Console UI download button (no static URL, not scriptable), not MLflow artefacts (the FTF report engine writes to PG).

The canonical extraction surface is :

make ftf-extract RUN_ID=<run_id>

This Make target :

  • Reads RUN_ID (mandatory ; format-validated against ^ftf_\d{8}_\d{6}_[0-9a-f]+_[A-Z0-9._]+$)
  • Selects the prod cluster context (Scaleway Kapsule per kubectl config current-context = cvntrade-prod-*)
  • Runs kubectl exec -n cvntrade <console pod> -- python3 -c '<extraction script>' against the in-cluster Postgres (host 172.16.16.4 per ConfigMap cvntrade-env-config)
  • Pipes the bytea content to documentation/missions/<mission>/reports/ftf_report_<run_id>.pdf (path inferred from the operator's working dossier OR explicit MISSION= override — defaults to ml-boost)
  • Verifies the PDF integrity (file ... → PDF document, version 1.7)
  • Prints the run metadata (factor, cryptos, status, error count, duration, git SHA) to stderr for the dossier metadata block

Fallback when make ftf-extract is unavailable (e.g., new operator machine without kubectl configured) : the operator opens console.cvntrade.eu/runs, clicks "Download PDF" on the run, drops the file in documentation/missions/<mission>/reports/ftf_report_<run_id>.pdf. Both paths produce the bit-identical PDF (the Console reads the same finetune_runs.pdf_report column).

3. Dossier scaffold + analysis sections

The dossier follows ADR-0079 invariant 2 template (§1-§12 + sign-off checklist). The §1-§9 sections (numerical tables) are populated from the PDF per ADR-0079 invariant 4 — never from independent PG queries. The §10-§12 sections (hypotheses, decisions, OP closure) are operator + AI judgement.

No automated dossier generation today — the LLM reads the PDF (Read tool supports PDFs up to 20 pages) and writes the dossier in conversation. A future automation (e.g., make ftf-dossier-scaffold RUN_ID=... that emits a partially-filled markdown stub) is out of scope for this ADR and would require its own design (pinning report-engine version, etc.). Until then, the LLM-in-the-loop is the canonical scaffold.

4. Single-PR closure pattern

The closure ships as a single docs-only PR containing :

  • New dossier documentation/missions/<mission>/<YYYY-MM-DD>-track<N>-<slug>-results.md
  • New PDF documentation/missions/<mission>/reports/ftf_report_<run_id>.pdf
  • Update to mission tracker (F1 plan §10 row OR equivalent for non-F1 missions) — status In progress → Closed ABANDONED / Closed KEEP_AVAILABLE / Closed LOCKED
  • Update to mission cross-track lesson (e.g., F1 plan §6 outcomes block) when applicable
  • Update to mkdocs.yml nav (new dossier entry under the mission)
  • Optional : update to ADR / runbook if the closure surfaces a new operational lesson (rare ; gate via committee pr_review if it touches code path)

The PR is labelled documentation + guardrails-waiver (G3 doesn't apply to docs-only closure of an already-plan-reviewed Story per ADR-68 scoping). The PR body MUST contain Closes #<dossier issue> for G2 + a ## Guardrails waiver section per the established pattern.

5. Story update — auto-syncer first, manual fallback

OP wp transition In progress → Closed happens via the path :

  • Primary : the auto-syncer (scripts/op_story_sync.py, deployed via .github/workflows/op-story-sync.yml — push-to-main + hourly cron) reads the mission tracker (F1 plan §10 OR equivalent), parses the Closed ABANDONED / Closed KEEP_AVAILABLE / Closed LOCKED bold prefix, and PATCHes the OP wp status. SLA : 5 min post-merge OR ≤ 1 h cron tick.
  • Fallback : if the auto-syncer is unavailable (PR #796 not yet merged at the time of this ADR ; OP API key unavailable ; OP outage), the operator transitions the wp manually via OP UI within 1 h of PR merge. The OP comment template in dossier §12.1 is appended at the same time.

Discrepancy detection : per ADR-76, OP is the SSoT. If the mission tracker says Closed but OP says In progress after 1 h, that's a sync incident — file under the auto-syncer runbook.

Invariants

  • Invariant 1 (operator-triggered) : The post-FTF ritual is initiated by an explicit operator request after the sweep reaches a useful-data state. Auto-execution on completion is forbidden (the verdict and cross-Track lesson require operator judgement).
  • Invariant 2 (PG is the PDF source-of-truth) : The canonical PDF source is finetune_runs.pdf_report (PostgreSQL bytea). The local results/ filesystem is never authoritative — it's a download cache. Dossier PDFs are committed to documentation/missions/<mission>/reports/ and verified to match the PG bytea bit-identically.
  • Invariant 3 (extraction surface) : Extraction uses make ftf-extract RUN_ID=... (preferred) OR Console UI "Download PDF" (fallback). Ad-hoc kubectl exec shell commands ARE permitted in emergencies but MUST be folded back into make ftf-extract within the same PR or a follow-up PR — the Track 11 case (which used ad-hoc kubectl exec) is the last acceptable instance ; from this ADR forward, the Make target is the surface.
  • Invariant 4 (single PR) : Every closure ships as one docs-only PR containing dossier + PDF + mission tracker update + mkdocs nav. Splitting these across PRs is forbidden (atomicity preserves the auto-syncer's read consistency — partial state would leave OP and tracker out-of-sync).
  • Invariant 5 (no PDF copies in results/) : Once the PDF is committed to documentation/missions/<mission>/reports/, the local results/ copy is ephemeral and may be cleaned. The dossier links to the docs path, not the local path.
  • Invariant 6 (Story closure post-merge) : OP wp transition happens after the dossier PR merges (auto-syncer reads the merged main branch, OR operator transitions manually within 1 h). Pre-merge OP transitions are forbidden (would leave OP claiming Closed while the dossier is still under review).
  • Invariant 7 (extraction idempotency) : make ftf-extract RUN_ID=<x> is idempotent — re-running against an already-extracted run produces a bit-identical PDF (PG bytea is immutable post-sweep-completion). The target may overwrite the local file silently ; it MUST NOT corrupt or partial-write.
  • Invariant 8 (run_id ↔ git SHA cross-check) : The dossier metadata block records both the sweep run_id AND the git SHA from the PDF metadata. The git SHA must match the squash commit of the contract-surface PR (or its merge ancestry). Mismatch indicates a stale PDF or a botched extraction — fail loud.

Alternatives rejected

  • Auto-execute the ritual on Airflow DAG completion : rejected. The verdict + cross-Track lesson are operator-judgement (per ADR-0079 invariant 5). Auto-execution would either silence those decisions or force an LLM to invent them. Either way, the dossier becomes ceremonial paperwork instead of a real review artefact.
  • Use Console UI as the sole extraction surface : rejected. The Streamlit st.download_button blob has no static URL ; Claude / automation cannot click it. Operator-only extraction creates a single point of failure (operator unavailable = ritual blocked).
  • Use MLflow as the PDF artefact store : rejected. The FTF Report Engine writes to PG finetune_runs.pdf_report, not MLflow. Re-architecting the report sink is out of scope (and arguably worse — MLflow artefact retention is also per-run, with cleanup policies that conflict with the "PDF must be retrievable forever" need ; PG bytea + git-committed copy is more durable).
  • Skip the single-PR rule, allow split closures : rejected. The auto-syncer reads main ; partial state (e.g., dossier merged but mkdocs nav update pending) would create a transient broken doc site. Atomic single-PR is the simplest correctness story.
  • Extend ADR-0079 with new invariants instead of new ADR : rejected. ADR-0079 is the what (workflow + invariants + verdict tree, abstract). This ADR is the how (executable mechanics, concrete CLI, prescribed paths). Bundling them in one ADR would dilute both — they have different audiences (ADR-0079 = future-Claude / operators reasoning about closures ; ADR-0080 = whoever automates the ritual next).

Consequences

Positive

  • Reproducible extraction : the next operator (or future-Claude) finds the canonical path documented + scripted, not invented per-closure.
  • Single trigger, single PR, single closure : reduces the cognitive surface of "what am I supposed to do now" to a 1-paragraph runbook.
  • PG-as-source-of-truth alignment : closes the loop with ADR-77 (mkdocs SSoT) + ADR-76 (OP SSoT) — every artefact has exactly one canonical source.
  • Cross-validation cheap : the run_id ↔ git SHA invariant (I8) catches stale PDFs, wrong-branch extractions, and "I think I'm looking at the right run" mistakes for free.

Negative

  • make ftf-extract not yet implemented at the time of this ADR : Track 11 closure used ad-hoc kubectl exec per Invariant 3's grace clause. The Make target is forward debt — must ship in a follow-up PR. Without it, future closures still need ad-hoc shell, which is the failure mode this ADR was written to prevent. Tracked as a follow-up Story (operator-triggered, ~half-day work).
  • Tight coupling to Console pod presence : make ftf-extract shells through the in-cluster Console pod. If the Console deployment is down, extraction breaks until the fallback (Console UI download) is exercised — but the UI requires the Console pod to be UP too. Single point of failure that should be addressed by a sidecar / standalone extractor pod in a future iteration.
  • Operator-trigger discipline : Invariant 1 forbids auto-execution. If the operator forgets to trigger after a sweep completes, the dossier never gets written and the Story stays In progress forever. Mitigation : the auto-syncer can warn (e.g., "wp#X has been In progress > 7 days with a completed sweep behind it") — operational follow-up, not blocking this ADR.

Neutral

  • Backwards compatibility : the 4 already-closed dossiers (Tracks 5 / 6 / 9 / 11) all retroactively comply with this ADR (PDFs in documentation/missions/ml-boost/reports/, dossiers in missions/ml-boost/, single-PR pattern, operator-triggered). No retrofit needed.

Rollback

If the canonical extraction path proves unworkable (e.g., the Console pod is decommissioned, PG schema changes, the FTF report engine moves to MLflow), this ADR is amended in place — bumped to superseded by ADR-XXXX and the Make target retired. The dossier convention + single-PR pattern + operator-trigger semantics survive any extraction-path migration ; only Invariants 2-3 + 7-8 are coupled to the current PG-bytea path.

References

  • Parent need : CVN-N014 (continuous improvement)
  • Parent ADR : ADR-0079 — what (workflow + verdict tree)
  • Cross-cutting ADRs : ADR-0058 (FTF factor + guardrail) ; ADR-0068 (committee scoping for pr_review exemption on docs-only) ; ADR-0069 (OP wp transitions) ; ADR-0076 (OP SSoT) ; ADR-0077 (mkdocs SSoT)
  • Implementation evidence : Track 11 results dossier (first closure to retrieve PDF from PG via kubectl exec — the workaround this ADR codifies as make ftf-extract) ; Track 9 results dossier (last closure to use the local results/ shortcut — Invariant 2 retroactively forbids that path)
  • PRs : #793 (Track 11 contract surface, sweep ran on this) ; #796 (auto-syncer — the Story-update mechanism cited in §5) ; #800 (Track 11 closure dossier — the PR this ADR ships in)

Open questions / forward debt

  1. make ftf-extract implementation : forward debt. Half-day Make target + a thin Python script that does the PG bytea extraction. Should land in a follow-up PR with a unit test that mocks the PG cursor.
  2. Console pod sidecar / standalone extractor : the SPOF on Console-pod uptime is real. A standalone cvntrade-ftf-extractor Job (read-only PG access, no UI) would decouple. Out of scope for this ADR ; revisit if Console availability becomes a closure-blocking issue.
  3. Auto-trigger warning : Invariant 1's "operator-trigger discipline" needs a backstop — a daily Grafana alert / Slack message for In progress Stories with completed sweeps older than 7 days. Forward debt ; tracked under the auto-syncer runbook.
  4. Multi-mission tracker convention : F1 plan §10 is the F1-mission tracker. Future missions (filter-tuning, economic-value) will need their own equivalent — the auto-syncer's parser today only knows about F1 plan §10. ADR-XXXX (forward debt) will codify the mission-tracker convention so non-F1 missions can plug in.
  5. PDF retention policy : ADR-0079 invariant 3 commits PDFs to git. At ~200 KB × 1 sweep/week × N years, the docs base grows linearly. A future cleanup ADR may move PDFs > 2 years old to S3 with stub markdown links — out of scope today.