Skip to content

FTF engine improvements — incremental polish for the ablation framework

epic_id: CVN-N014-EA need_id: CVN-N014 (parent — see Continuous improvement) OP: wp#89 Status: active Created: 2026-04-29 Target release: rolling Owner: dococeven

1. Objective

Polish the FTF (Fine-Tuning Framework) engine — the system that runs ablation sweeps, persists finetune_results, computes statistics, and produces PDF reports. The engine has reached production maturity (used for Tracks 5+, label_smoothing, cleanlab variants) but accumulates small inconsistencies as new factors land.

This Epic is the home for incremental Stories that improve clarity, consistency, ergonomics, and robustness of the FTF engine without changing what it computes.

2. Architecture notes

The FTF engine spans :

ADRs introduced/touched :

  • ADR-56 — every pipeline change must be FTF-testable / A/B-testable by design
  • ADR-58 — every FTF factor must have a guardrail + integration test
  • ADR-59 — params in PostgreSQL Console-only
  • ADR-65 — Airflow DAG params are run-level only

Files expected to change (high level) : scripts/ftf_config_ui.py, src/commun/finetune/dag_config.py, src/commun/finetune/guardrails.py, possibly the Console PARAM_OPTIONS dict

3. Stories in this epic

Story Title Status OP GH
CVN-N014-EA-S01 Deduplicate PTE configuration in Console (slug vs SL/TP/H components) active wp#90 #772
CVN-N014-EA-S02 power_mode=deep preset self-supplies history_months (fix #807) done wp#100 #807 (PR #810)
~~CVN-N014-EA-S03~~ ~~FTF runner skip-with-recorded-reason on empty training window~~ rejected 2026-05-15 — superseded by ADR-25 fail-fast (PR #613 CR r3) ; underlying data-quality issue belongs in Epic CVN-N011-EA if it resurfaces wp#104 #813

4. Acceptance criteria (epic level)

  • No duplicate sources of truth for any FTF parameter (single canonical key per concept, derived values explicitly computed)
  • Console UI rejects inconsistent / divergent inputs at submit time (fail-loud per ADR-25)
  • Every Story under this Epic has a regression test that locks the new invariant
  • Operator-facing documentation reflects the canonical configuration model (no stale duplicate references)

5. Dependencies

  • Depends on: per-Story plan dossier + committee plan_review (per ADR-68) before any implementation PR ; no upstream Epic dependency
  • Blocks: nothing — Stories under this Epic must not block strategic missions

6. Risks + mitigations

  • Risk: Stories drift toward refactoring scope. Mitigation: each Story acceptance bar must be a single observable user-facing change, not "code is cleaner".
  • Risk: Conflicts with active sweeps if Console schema changes. Mitigation: Console changes ship behind feature flag or are paired with migration guidance.

7. Closure

Filled when status moves to done (rolling Epic — likely to stay active for a long time as long as new improvement Stories land here).