Validation

The Validation dashboard compares model-selected fields and rankings against historical CFP outcomes and committee-published lists.

What validation measures

Validation surfaces metrics such as:

• Field overlap: how often the model's projected 4-team (pre-2024) or 12-team field matches actual participants.
• Ranking alignment: correlation and rank error versus committee top-25 or top-12 lists where available.
• Team-level flags: model-added, model-dropped, and first-out cases when the payload includes them.

These metrics describe historical fit under documented assumptions, not real-time prediction accuracy.

What validation does not prove

Validation does not prove that Selection Room replicates committee deliberation, insider criteria, or future outcomes. Pre-2024 field checks use actual participants; committee top-12 overlap is a separate metric with its own limits. Sample and Live CFBD runs are not directly comparable.

How to refresh validation artifacts

Validation data is a single repo-level artifact (data/output/api/validation.json), not a per-run export. To refresh:

1. Run the validation harness: make validate (or ./bin/sroom validate --years 2014:2024).
2. Confirm data/output/api/validation.json regenerated for the target seasons.
3. Reload the Validation page. Offline development falls back to the seeded fixture (pnpm seed-fixtures copies it to web/lib/fixtures/validation.json).

If the artifact is missing, the dashboard shows an empty or partial state rather than inferred numbers.

Open the app

Use Validation when validation artifacts are present in the active run catalog.