Skip to content

Production-Readiness Audit — football-trackers

Date: 2026-08-03 · Scope: firmware/, server/, client/, vision/, infra & delivery, cross-cutting quality Scale assumed: ~20 devices, ~100 msg/s, one host, one operator. This system tracks children — privacy outranks everything.

1. Executive summary

The engineering quality here is genuinely high. Auth hardening, DoS bounds, privacy-by-design in the data model, the WS frame validator, the accessibility mirror, and the 23-ADR decision log are all better than most production systems. The test suites are not just present, they all pass (§2).

The gap is not code quality — it is that the safety net is not connected to anything. The repo is not under version control, so no CI has ever run; the well-built client-ci.yml is inert. And the single deployable artifact that exists — the dev Docker stack — publishes children's live positions and names to the entire LAN.

Three themes are severe enough to state plainly:

  1. Children's names and live positions are readable by any host on the LAN, unauthenticated. Proven live against the running stack (§4.1). A bare curl with no headers at all returns a child's displayName; forging one Origin header streams live coordinates.
  2. The firmware's outage backlog loses ~99% of what it exists to save. The blocking reconnect starves the 10 Hz GPS loop, so a 4-minute dropout preserves ~16 of ~2,400 fixes (§4.2).
  3. Right-to-erasure is broken five separate ways (§4.5) — the most carefully-built subsystem in the repo is the least functional. Every defect was reproduced by execution, not inferred. In its worst form the CLI prints a success receipt while the child's data remains fully recoverable.

Nothing here requires new infrastructure. Every fix is small, local, and in keeping with the zero-dependency ethos — the largest single change proposed is ~30 lines.

Severity distribution

Count Meaning
P0 9 Child-privacy exposure, silent data loss, or security bypass
P1 20 Match-day failure an operator would actually hit
P2 32 Compounding operational debt
P3 2 Polish

2. Baseline — what passes today

Everything that can be executed was executed. No pre-existing test failures.

Gate Command Result
Server typecheck bunx tsc -p tsconfig.json --noEmit PASS (clean)
Server suites all 20 in server/test/ 20/20 PASS
Client typecheck bun run typecheck PASS
Client lint bun run lint PASS
Client unit bun test 32/32 PASS
Vision docker compose run --rm test 101/101 PASS
Firmware pio run SUCCESS — RAM 14.9%, Flash 71.9% (943 KB)
Client e2e bunx playwright test COULD NOT RUN — see below

Two honest caveats:

  • Playwright could not run. playwright.config.ts hardcodes SERVER_PORT = 3000 with no env override, and port 3000 was held by an unrelated local service. The acceptance gate cannot run on any machine where anything else uses 3000. (New finding — P3, §6.)
  • The six "unreachable" server suites all pass. auth-cli, auth-dos, auth-loader, events, events-e2e, scan-load have no package.json script, so nothing but a human who remembers them will ever run them. They are healthy; they are simply unwired.

Flash headroom note: at 943 KB the image fits comfortably in a 1.25 MB OTA slot, so the OTA recommendation (§5, F-4) is feasible with ~28% headroom.

3. Method

Findings were produced by one pass and independently verified by a different pass (maker ≠ checker), with verifiers instructed to refute by default and to correct line numbers against the code as it exists today. 55 claims were adjudicated: 43 confirmed, 12 partially confirmed.

Verifiers also rejected several plausible-sounding claims, which is why the list below is shorter than the raw survey:

  • "No graceful shutdown means data loss" — measured false. 33 rows persisted, kill -TERM, reopened read-only: still 33 rows. WAL survives process death. The real cost is a stall and abrupt socket drops, not lost telemetry. Ranked P2, not P0.
  • "Per-player metric series are never removed" — true but a documented, deliberate residual (purge-player.ts, observability.md). Not a bug.
  • "The captive-portal password is fixed" — true but an accepted ADR-0022 decision. Not re-litigated.
  • "A naive git add . would commit .venv, model weights, footage" — false. The existing per-directory .gitignores handle all of it correctly.

Two further passes ignored the claim list entirely and hunted independently — a child-privacy lens (tracing every path by which a child's identity, location or image can leave the intended boundary) and a completeness critic (deliberately looking where the survey had not: the root-level CLIs, cross-boundary failure semantics, retention/erasure interaction with a live match, resource exhaustion). Between them they found the entire §4.5 erasure cluster, which the structured survey missed completely — the strongest argument for not letting the finder be the checker.

Where a claim could be tested against running software, it was. §4 marks these [PROVEN LIVE]; every §4.5 defect was reproduced with the real CLI in a scratch directory. The user's own telemetry.db was never modified — the nine synthetic rows injected during live probing were deleted, verified 0 remaining.

4. P0 findings

4.1 Children's names and live positions are LAN-readable, unauthenticated [PROVEN LIVE]

Where: docker-compose.yml:54-56, server/src/server.ts:76-89, server/src/auth.ts:411-412

The dev stack binds 0.0.0.0:3007 with ALLOW_ANONYMOUS_LIVE=true. ANON_MODE short-circuits currentPrincipal before any cookie check, so the only gate is an Origin comparison — and Origin is a header the browser enforces, not the server. Two independent bypasses, both demonstrated against the running stack:

curl (no Origin at all)          -> 200  {"displayName":"CANARY-CHILD-NAME"}
curl -H 'Origin: http://evil...' -> 403  {"error":"forbidden_origin"}
WebSocket, forged Origin         -> ACCEPTED, 4 live frames with real lat/lon
WebSocket, no Origin             -> 1008 forbidden origin

The gate is inverted in effect: originOkLenient (server.ts:87-89) is !origin || allowed.includes(origin), so absent is treated as trusted — and absent is exactly what every non-browser client sends by default. It blocks the hard attack (a browser) and waves through the easy one (a script). /sessions, /roster, /config, /history and /auth/me all answer a bare curl; /auth/me reports "authenticated":true.

This is not the ADR-0007 plaintext-on-isolated-LAN trade: the bench runbook (local-bench-runbook.md:122) instructs the operator to disable Wi-Fi client isolation so the wearable can reach the broker — which is ADR-0013's escalation trigger #2 verbatim. And restart: unless-stopped means this survives reboots.

Failure: during an outdoor bench run, any device on the home Wi-Fi — a guest phone, a neighbour with the PSK, a compromised IoT bulb — scans the subnet and retrieves a named child's full location history and a live 10 Hz feed. Every request logs as an ordinary read, indistinguishable from the coach's tablet.

Fix (~12 lines): bind 127.0.0.1:3007:3000 (the Vite proxy is the only consumer), and make the invariant structural in server.ts — const PUBLIC_HOST = process.env.PUBLIC_HOST ?? (ANON_MODE ? '127.0.0.1' : '0.0.0.0'). Origin checks are CSWSH defence; they must never carry authorization weight.

Verify: test -z "$(lsof -nP -iTCP:3007 -sTCP:LISTEN | grep -v 127.0.0.1)"

4.2 The firmware backlog loses ~99% of an outage — the opposite of its purpose

Where: firmware/src/main.cpp:121,126-129,161,568,573

wifiEnsure() busy-waits delay(150) + up to 15,000 ms (while … delay(250)), and is re-entered from mqttEnsure() on every loop() pass while down. gnss.getPVT() is only reached after that block. Two compounding causes: the 256-byte UART ring saturates in 0.256 s at 10 Hz (1 KB/s), and the SparkFun driver retains only the newest NAV-PVT — so even with an infinite buffer the loop can stash at most one fix per pass.

Measured: ~15.15 s per attempt, during which the GPS produces ~151 fixes and the code stores 1. That is 99.3% loss. A .local broker adds a further 2 s block; a blackholed broker adds up to 15 s more.

This directly contradicts main.cpp:6-8 ("a dropout on the field never loses the session") and ADR-0003.

Three defects interlock, and fixing one alone makes things worse:

  1. The blocking reconnect (above) means the backlog rarely fills.
  2. The 256 KB cap holds only ~197 s and drops the newest fix, not the oldest.
  3. The server's own rate limiter would reject a working replay: INGEST_RATE_CAP=15/s vs an unpaced flush of ~1,971 lines in ~1 s → ~97% dropped as reason="rate". Currently masked because the backlog is always tiny.

Fix (~25 lines firmware + ~4 server): convert wifiEnsure/mqttEnsure to a millis() state machine (no blocking waits); drain with while (gnss.getPVT()); Serial2.setRxBufferSize(1024); pace the flush to ~40 packets/pass; drop-oldest rotation; raise INGEST_RATE_CAP to 60 with burst 120 (still 6× nominal, still bounded).

Verify: cut the AP for 60 s, then assert the device's stash counter advanced ≥550 (today: ~4).

4.3 The vision web UI accepts children's footage, contradicting ADR-0023's hard gate

Where: vision/webui/runner.py:23,37-40, vision/webui/index.html:74-81

ADR-0023 §2 is unambiguous — "No youth footage in any phase" — with the real-youth gate deferred to a future ADR requiring a DPIA, verified parental consent, and a documented lawful basis. Yet ATTEST_KINDS = {"public_adult", "consented_youth"} and the UI offers "Dečji — imam saglasnost roditelja".

The mechanism is worse than "a checkbox unlocks it": attest_kind is a pure ledger string with zero downstream effect. It is recorded and never read again — download, decode, detect, annotate, write and serve behave identically. No consent evidence, controller, lawful basis or retention date is captured, so it cannot even discharge GDPR Art. 7(1) demonstrability. consented_youth appears in exactly two places in the repo and is authorised by no ADR; the web UI itself is absent from ADR-0023 entirely.

Combined with §4.4 (no retention) and the unauthenticated 0.0.0.0:8077 bind, this is a live egress path for face-bearing derivatives that ADR-0023 §14 explicitly places under no-egress rules.

Fix (~20 lines): ATTEST_KINDS = {"public_adult"}; delete the UI selector; reword the refusal; add a test asserting the youth path is refused. If youth footage is ever genuinely wanted, it returns through the §14 ADR — not through runner.py.

Verify: ! grep -rn 'consented_youth' vision/webui/ vision/README.md

4.4 Committing the repo would write children's names into git history permanently

Where: no root .gitignore; server/.gitignore covers only node_modules/ and *.db*

Four sensitive runtime files are unignored by anything in the repo:

File Contents
server/roster.json plaintext playerId → child's full name (roster.ts:5-9: "THIS IS THE ONLY PLACE PLAYER NAMES LIVE AT REST")
server/auth-accounts.json coaches' argon2id hashes + session assignments
server/mosquitto/ft.passwd broker password hashes
server/session-config.json per-session config

None exist yet — the exposure materialises the moment the operator provisions for the first real match, which is precisely when git init is planned. git rm does not remove data from history. .claude/settings.local.json is protected only by the user's machine-global ignore file, not the repo.

Separately — a subtle bug that silently breaks model integrity [PROVEN LIVE]: vision/.gitignore excludes models/ and then tries !models/MANIFEST.json. Git cannot re-include a file whose parent directory is excluded, so the SHA-256 weight-integrity manifest — the anchor for resolve_weight's verification — would not be committed at all. Verified empirically, including the fix:

models/  + !models/MANIFEST.json  ->  MANIFEST.json IGNORED   (integrity pin lost)
models/* + !models/MANIFEST.json  ->  MANIFEST.json STAGED, *.pt still ignored

Fix (~10 lines, must land BEFORE git init): root .gitignore with roster.json, auth-accounts.json, session-config.json, *.passwd, .env*, .DS_Store, .claude/settings.local.json; change models/ → models/*; add a ~6-line server/test/gitignore-guard.ts that shells git check-ignore -q for each sensitive path.

4.5 Right-to-erasure is broken five separate ways [ALL REPRODUCED BY EXECUTION]

This is the most important cluster in the audit. purge-player.ts is thoughtfully built — receipts, exit codes, secure_delete, a documented retry contract — and it is the subsystem I would trust least. Five independent defects, each verified by running the real CLI against real modules:

# Defect Reproduced result
a Erased data stays byte-recoverable in the WAL receipt {"erased":300} → erased playerId still appears 9,214× in t.db-wal
b Fail-closed loader = fail-OPEN erasure duplicate playerId → exit 0, success receipt, name still on disk
c Collateral destruction of other children erasing one Saturday player deleted all 3 Sunday children's names
d Wrong store in Docker deletes the name on the host, cannot reach positions in the named volume
e Missing DB = success non-existent DB_PATH is silently created; exit 0, nothing erased

(a) The WAL retains everything. db.ts:16-26 sets journal_mode=WAL + secure_delete=ON and comments that erasure therefore "actually destroy[s] the bytes". But secure_delete only zeroes freed pages in the page images the deleting connection writes — which land in the -wal sidecar; the pre-delete images survive in both files. Nothing in shipped code ever checkpoints: grep -rn 'wal_checkpoint' server/src server/*.ts returns nothing — the only hits are in test/retention.ts, where the test manually checkpoints, compensating for what production never does.

main db:  4,096 bytes   → 0 occurrences of the erased child
WAL:  3,827,512 bytes   → 9,214 occurrences of the erased child

Because the WAL is never truncated, essentially all data lives there — so the erasure zeroed almost nothing. Any file-level copy, backup, or recovered disk yields the "erased" child's full trace.

Fix (~6 lines): PRAGMA journal_size_limit = 0 in db.ts, and PRAGMA wal_checkpoint(TRUNCATE) after the purge completes.

(b) The fail-closed loader makes erasure fail open. purgeRosterPlayer (roster.ts:186) reads via loadRoster() — the serving loader, which correctly returns empty on a malformed file and drops an entire session on a duplicate playerId. Then roster.ts:192 guards the rewrite with if (removed > 0), so "found nothing" means "write nothing", while purge-player.ts exits 0. Fail-closed for reads is fail-open for erasure. Reproduced verbatim:

{"erased":0,"rosterEntriesErased":0,"playerId":"07","scope":"all sessions"}   exit 0
→ roster.json still contains ALEX_M / ALEX_MORGAN

(c) Erasing one child deletes others. roster.ts:195-200 rebuilds the whole file from the filtered map, so every entry the loader silently dropped is erased from disk — triggered by removed > 0 on a completely unrelated session:

before: u12-sat[DEPARTING_CHILD, KEEP_ME_SAT]   u12-sun[SUNDAY_A, SUNDAY_B, SUNDAY_C]
after:  u12-sat[KEEP_ME_SAT]                    u12-sun GONE ENTIRELY

Nobody notices until the next Sunday match renders every dot as a bare pseudonymous id — and the receipt said success.

Fix for (b)+(c) (~10 lines): give purgeRosterPlayer a permissive read-modify-write (the raw JSON.parse round-trip roster-user.ts:46-59 already uses), preserving every byte it does not target, and make an unreadable file throw so the existing non-zero exit path fires.

(d) Wrong store under Docker. The split is exact and unfortunate:

  • Names → roster.json resolves against /app, which is bind-mounted from the host. The documented host-side command finds and deletes it.
  • Positions → DB_PATH=/data/telemetry.db in the named volume server_data, addressable by no host path. The command cannot touch them.

So the documented command destroys the child's name, leaves 30 days of 10 Hz location data intact, reports {erased:0, retry:true} — and observability.md correctly tells the operator that means "not erased, retry". They retry forever. Meanwhile the data is now pseudonymous and unattributable, because the identifier that linked it to the request has been deleted. A stale server/telemetry.db on the host adds genuine "which DB is authoritative?" ambiguity.

Fix (~10 lines): bind-mount ./server/data:/data so the store is a visible host file; add the docker compose exec -T server bun run purge-player.ts … invocation to the runbook as the only correct form while the stack is up; delete the stale host DB.

(e) A missing database counts as success. DB_PATH pointing at a non-existent file causes bun:sqlite to create it, so the CLI erases 0 rows from an empty new database and exits 0 — the exact outcome of (d), reported as compliance. Fix (~4 lines): existsSync(dbPath) check with a distinct exit code, so "wrong file" is never presented as "transient failure".

Two further erasure-adjacent defects (P1/P2):

  • The purge freezes the server. db.ts:87's DELETE FROM telemetry WHERE player_id = $player has no LIMIT and there is no index on player_id — EXPLAIN QUERY PLAN yields SCAN telemetry, while the retention delete beside it uses a covering index and is carefully batched. With secure_delete zeroing every freed page, running the documented lost-device wipe during a match holds the write lock for tens of seconds: dots stop moving while children are actually running. Fix: add idx_telemetry_player, batch with the rowid-subquery pattern already in db.ts:83-85.
  • Retention never touches roster.json. The sweep bounds the telemetry table only, so the name↔playerId re-identification map outlives every fix it identifies, with no time bound — and ft_oldest_raw_fix_age_seconds, described as "the data-minimisation SLI proving the retention window holds", is blind to it. After a season the SLI reads healthy while the file still names every child who ever wore a device. Fix (~15 lines): after each sweep, drop roster sessions with no remaining telemetry.

4.6 The dev broker is anonymous and LAN-published [PROVEN LIVE]

Where: deploy/mosquitto/mosquitto.conf:6-7, docker-compose.yml:29-30

allow_anonymous true on 0.0.0.0:1883. Confirmed live: an anonymous publish from the host was accepted. Any LAN host can mosquitto_sub -t 'football-trackers/#' for every child's 10 Hz feed, or publish forged telemetry — and since ingest.ts only checks that body pl/id agree with the topic, a consistent forgery is accepted, server-stamped, persisted as authoritative and fanned out to the coach.

The enforcing config already exists (server/mosquitto/mosquitto.conf + ft.acl with the correct %u pattern); compose simply mounts the wrong one. This is not the ADR-0007 trade — that decision was per-device credentials plus ACLs with plaintext transport; here the load-bearing inner controls are absent entirely.

Fix (~8 lines): point the dev broker at the same ft.passwd/ft.acl with absolute container paths. Bonus: every bench run then exercises the real auth path, eliminating the prod-only-auth-never-tested class of bug.

5. P1 findings (condensed)

# Finding Where Fix
S-1 Wire-field types unvalidated → metrics injection. A string fix passes raw.fix < 2 (NaN compare) and is interpolated raw into the exposition. [PROVEN LIVE] — injected ft_injected_metric 999. A rogue device can forge ft_anon_mode_active 0, silencing the alarm for §4.1, and duplicate metric names break the entire scrape (up=0, all alerts dead). ingest.ts:137-159, metrics.ts:82-98 ~20 LOC: coerce at boundary; Number.isFinite guard on Gauge.set
S-2 Status frame needs no attacker. Only s.up is typechecked; a firmware skew missing batt writes literal undefined into /metrics, killing the whole scrape, and freezes that device's health card silently. ingest.ts:224-248 same coercion
S-3 Fail-open env parsing. Math.max(1, Number('6h')) is NaN and every downstream compare is false. Measured: typo'd HISTORY_MAX_SPAN_MS → a 10-year export of children's raw location accepted, rate limiter 10000/10000. Also kills session TTL (cookie valid forever), argon2id inflight cap, roster rate limit, the shared scan cap. Three knobs fail hard (1 ms hot loops). history.ts, ingest.ts, auth.ts, server.ts, scanLoad.ts, retention.ts promote events.ts's envCount to shared env.ts, log resolved config at boot
S-4 /health lies. [PROVEN LIVE] Broker down → {"ok":true,"mqtt":true} while ft_mqtt_connected 0. Write-once latch, never reset. No DB probe. server.ts:181,526 ~10 LOC: onDisconnected callback
S-5 Unbounded metric cardinality. ACL leaves the session segment as bare +; metrics.received.inc() runs before parse, validation and rate-limit. Measured: 200 garbage publishes → 201 series. ingest.ts:128, ft.acl:10-11 ~14 LOC session cap + bucket sweep
S-6 No server CI gate. The only workflow leaves auth, authz, origin, rate-limit, retention and erasure logic completely ungated — inherited as-is at git init. .github/workflows/ server-ci.yml cloned from client-ci
C-1 Clock skew breaks freshness. No NTP on an isolated LAN. Tablet ahead >10 s → healthy feed renders an empty pitch; tablet behind → a dead tracker shows as a live dot forever, defeating ADR-0018's honesty rule. 5 Date.now() sites ~25 LOC serverClock.ts (running min of Date.now()-serverTs)
C-2 Terminal reconnect give-up. 8 retries ≈ 37 s expected / 75 s worst case, then dead for the match. No retry button, no online listener. Recovery exists only by accident (Review→Live remount) and is undiscoverable. useLiveTelemetry.ts:32-36,274-287 ~20 LOC: reconnectNow() + online listener + button
F-1 Backlog replay is not crash-safe. Deleted only after full success → reboot mid-flush re-sends everything; no sequence number and no unique index → duplicate rows inflate a child's distance/sprint stats. main.cpp:234-250, db.ts:28-49 NVS offset cursor + sq field + unique index
F-2 Replayed fixes are temporally wrong. Device ts is never read by anything; replayed rows get serverTs = now, so a 90 s outage collapses into ~1 s — the child appears to teleport, fabricating a burst of "high-intensity efforts". GPS UTC is already parsed and unused. main.cpp:582, ingest.ts:186 publish gts; trust it for replay
F-3 Provisioning accepts topic-unsafe ids. set player 07/b → broker accepts, server's single-level + never matches: a silent black hole with no drop counter. Over-long ids overflow buf[256] → every fix diverted to backlog. main.cpp:384,452 ~14 LOC charset check matching TOPIC_ID_RE
F-4 No watchdog, no reset reason, no version. A wedged device is dead on a child until power-cycled; brownout loops are invisible. main.cpp esp_task_wdt + reset reason/boot count/version in status
F-5 Plaintext child location persists on flash. 256 KB ≈ 1,971 fixes at ~1 cm precision, retained indefinitely if the session ends offline. A fourth copy ADR-0010 does not enumerate — purge-player provably cannot reach it. main.cpp:71-72,249 age-based purge at boot + wipe command
F-6 SESSION_ID compiled in as "test". Every match accumulates under one session id — and session id is the unit of access control. A U10 coach can read U14 children's history. main.cpp:36 NVS key, mirroring ADR-0022's mqtt_host
V-1 Web UI unauthenticated on 0.0.0.0:8077, unbounded thread per POST, and _serve_out hands back the raw source clip the UI never links. webui/server.py:105-144 loopback bind + Semaphore(1) + artifact allow-list
V-2 run_v1 retains every decoded frame. 90 min @720p/5fps ≈ 69.5 GiB — the stated goal is physically impossible. Frames are dead weight; the writer re-decodes anyway. pipeline.py:37,41,139 drop frame from the tuple → O(tracks)
I-1 No production artifact. No Caddyfile, no prod compose, no unit. The authenticated broker's password_file ./ft.passwd is cwd-relative, and the conf's own header contradicts its README — verified: mosquitto exits rather than starting. deploy/, server/mosquitto/ absolute paths + deploy/production/compose.yml
Q-1 Empty provenance ledger. samples.manifest.jsonl is 0 bytes while samples/ holds 3 clips — and the test that guards it passes vacuously on an empty file, so the control cannot fail in the one state where it matters. vision/ backfill + coverage assertion

6. P2 / P3 (grouped)

Delivery & quality: no aggregate test script and no typecheck script in server/package.json (7 of 22 test files unscripted) · no vision/firmware CI · vision/README.md claims CI that does not exist · lint/format only for client · playwright.config.ts hardcodes port 3000 so the gate cannot run alongside other local work (new) · firmware has zero tests; the wire contract is three hand-maintained copies reconciled by a comment.

Server: mode: 0o600 on the child-name and credential files is a no-op whenever the file already exists (POSIX applies mode only at creation — verified: a 644 file stayed 644 after both write paths), so the documented at-rest posture silently fails for any roster restored from backup, scped, or created by an editor — fix with an explicit chmod or a temp-file + rename · WS fan-out drops are counted as successful sends: server.publish()'s return is discarded (server.ts:517), and Bun documents 0 = dropped and -1 = backpressure, so a stalled coach tablet loses frames while ft_ws_sent_total climbs at full rate · no migrations (PRAGMA user_version) or backup tooling · no uncaughtException/unhandledRejection handlers (Bun's default is exit(1), and publish() sits outside the ingest try/catch) · in-memory sessions log out every coach on restart · no graceful shutdown (measured: no data loss; cost is a 10 s stall and abrupt socket drops) · sh as PID 1 (container exits 137/SIGKILL, measured in 1.2 s — zero drain window, not the assumed 10 s).

Client: ErrorBoundary only wraps the live canvas — Review white-screens the whole root, leaving no way back except reload · PITCH_CORNERS compile-time (and currently pointing at a Belgrade bench spot, so every real pitch maps to the wrong box) · no fetch deadlines, and the "try again" copy is misleading because re-pressing Apply is a no-op · no client observability · touch targets ~24-25 px (WCAG 2.5.5 wants 44) · off-pitch players clipped invisibly while still counted · inbound frames not checked against the subscribed sessionId (defence-in-depth).

Vision: run_v2/run_v3 are ... stubs returning success · no retention/TTL (out/ = 51 MB after 3 jobs) · attestation ledger lives inside the prunable directory · no subprocess timeouts · ffmpeg exit status unchecked · lockfile is placeholder text nothing consumes · model fetch is trust-on-first-use (the fetcher overwrites the manifest with what it just downloaded).

Infra: no healthchecks despite a purpose-built /health · no log rotation · no resource limits · containers run as root · deps installed from network at container start without --frozen-lockfile.

Docs: README still says "NestJS" · CLAUDE.md documents the removed VITE_WS_URL · vision/ is invisible from every entry point while README presents Track B as un-started.

7. Verified strengths — do not re-recommend

Confirmed present and correct; audited only for regressions:

  • Auth: argon2id + constant-work dummy hash (no enumeration oracle), per-IP buckets, concurrent-hash inflight cap, per-username soft-lock, per-user session caps, __Host- cookies, constant-time CSRF.
  • AuthZ ordering is consistent on every session-scoped surface and deliberately avoids a 400/401 oracle.
  • Privacy by design: names only in roster.json (0600), never in DB/logs/metrics; explicit-field copying (no spread); secure_delete; retention sweep; erasure CLI with receipts; loopback-only /metrics.
  • DoS bounds: per-player buckets, shared off-loop scan slot, span caps, keyset paging that yields the loop.
  • Client: strict WS frame validation with structural stripping; bounded maps/trails; honest reconnect state machine; shared freshness classifier; no client-side storage; strict CSP default-src 'none'; rAF/ref render discipline with a 50-player frame-budget gate; first-class A11yMirror.
  • Firmware: NVS secret provisioning (one image, refuses to run un-enrolled); per-device MQTT identity with broker ACL %u; defensive GPS bring-up.
  • Process: 23 cross-linked ADRs; the deterministic simulator as shared e2e substrate; vision's offline guards, SHA-verified weights and privacy-firewall gitignore.

8. Roadmap

Each phase closes with machine-checkable acceptance criteria. A separate checker verifies before a phase closes (maker ≠ checker). Out-of-scope discoveries go to a triage list, not into the phase.

Phase 1 — Foundation: make the safety net real (everything depends on this)

  1. Root .gitignore (roster.json, auth-accounts.json, session-config.json, *.passwd, .env*, .DS_Store, .claude/settings.local.json); fix vision/.gitignore models/ → models/*.
  2. server/test/gitignore-guard.ts.
  3. git init + initial commit only after 1-2 verify clean.
  4. server/package.json: add typecheck + aggregate test (covering all 20 suites).
  5. server-ci.yml, vision-ci.yml, firmware pio run job, cloned from client-ci.yml's shape.
  6. Make playwright.config.ts port env-overridable.

Accept: git check-ignore -q passes for every sensitive path · bun run test exits 0 and runs 20 suites · pio run exits 0 · all workflows present.

✅ DONE 2026-08-03. All four acceptance criteria met: the guard's 286 checks pass · bun run test exits 0 running 21 suites in 19 s (20 as scoped, plus the guard itself) · pio run exits 0 · five workflows present, actionlint clean. Beyond the letter of the scope: the Playwright gate ran for the first time (20 passed — it was un-runnable during the audit because :3000 was held), and client/e2e/ is now typechecked, having been outside every static gate.

A five-lens independent checker pass found seven real defects in this phase's own work, all fixed and re-verified by execution before the commit. The three worth remembering, because each is a gate that reported green while not gating: - The guard was path-filtered out of CI for exactly the commits that leak — adding docs/roster.json or a root-level file matched no filter, so nothing ran. It now has its own unfiltered repo-guard.yml. - run-all.ts's per-suite timeout could not stop a hung suite: the e2e suites' mosquitto/server grandchildren inherit the stdout pipe, so awaiting the drain alongside proc.exited blocked forever. - The guard's sweep was blind to non-ASCII and uppercase filenames — git C-quotes "samples/André.mp4", and cameras write .MP4. The child with an accented name was the one who would not have been caught.

Deliberately left for later, not silently dropped: platformio.ini declares platform = espressif32 unversioned, so firmware CI proves "still compiles", not "compiles identically to the bench image".

Phase 2 — Close the P0 exposure (no new deps; ~40 lines total)

  1. Bind 127.0.0.1:3007; make ANON_MODE ⇒ loopback structural in server.ts.
  2. Dev broker auth parity (mount ft.passwd/ft.acl, absolute paths).
  3. Remove consented_youth; add the refusal test.
  4. Consider gating /roster and /history behind real auth even in anon mode — anon exists so the live pitch view needs no login, which does not require handing out names and bulk history.

Accept: lsof shows no non-loopback listener on 3007 · anonymous mosquitto_sub refused · ! grep -r consented_youth.

✅ DONE 2026-08-03. All four items, including #4 (the "consider" one — the user chose to gate them). Acceptance, all by execution against the running stack: curl http://<LAN-IP>:3007/... → connection refused, lsof shows 127.0.0.1:3007 only · anonymous mosquitto_sub -t 'football-trackers/#' → Connection Refused: not authorised, from both loopback and the LAN address · the consented_youth grep is clean · anon /roster, /history, /events → 403 login_required while /live still opens. Gates: 23 server suites (2 new: anon-scope.ts, deploy-posture.ts), 21 Playwright specs, 104 vision tests, client typecheck/lint/units.

Four defects in this phase's own work, each found by running it rather than reading it: - The gate made names unreachable for everyone. currentPrincipal returned the anon principal before parsing the cookie (this document says so at §4.1), so a coach who logged in was silently downgraded to the shared anon identity — and could never reach the endpoints the login was for. Cookie first, anon as fallback; audit lines now name the coach instead of username: null. - Signing out left the names on screen. useRoster keyed its effect on sessionId alone, and on the anon stack sign-out does not unmount the shell — so the previous coach's roster stayed painted over a live feed until a page reload. Client cache quietly undoing the server's authz. Caught in the browser; the e2e that pins it fails with 12 stale names against the old code. - No way to log in at all on the anon stack: /auth/me answers 200-anonymous, so the app never showed a login form. Added a "Sign in for names & review" affordance, and simulate.ts now provisions the coach account in both postures. - docker compose restart does not re-read .env, so rotating broker credentials left the server authenticating with the old password — which presents as a broker fault. Documented in three places.

Also closed, beyond the letter of the scope: deploy/mosquitto/mosquitto.conf (the anonymous config that caused §4.6) is deleted rather than left one copy-paste from being remounted, and deploy-posture.ts — in the unfiltered repo-guard workflow, because a revert edits root-level files no path filter covers — statically pins the loopback publish, the authenticated mount, the required credentials and the absolute broker paths. All five of its checks were mutation-tested: each caught, guard green again after restore.

Known consequence, stated plainly: the bench coach view is now this-machine-only. A second tablet on the Wi-Fi cannot reach it — that is the point, and a pitch-side tablet is the Caddy + real-auth deployment.

Phase 2b — Repair erasure (GDPR-load-bearing; do not defer)

  1. journal_size_limit = 0 + wal_checkpoint(TRUNCATE) after purge.
  2. Permissive read-modify-write in purgeRosterPlayer; throw on unreadable file.
  3. existsSync(DB_PATH) guard with a distinct exit code; bind-mount ./server/data:/data; runbook docker compose exec line; delete the stale host DB.
  4. idx_telemetry_player + batched delete.
  5. Extend retention to prune orphaned roster sessions.

Accept: after purge, ! strings -a $DB $DB-wal | grep -q <playerId> · duplicate-playerId roster → non-zero exit and name removed · erasing one session leaves all others byte-identical · missing DB_PATH → distinct non-zero exit · EXPLAIN QUERY PLAN shows SEARCH, not SCAN · erasure e2e passes against the containerized store.

✅ DONE 2026-08-23. All five items. Acceptance, by execution: the strings -a scan of telemetry.db + -wal finds 0 hits after a purge — in server/test/erasure-audit.ts with the erased rows interleaved with survivors (3:1 burst and a 10-player round-robin), and against the running Docker stack via docker compose exec -T server bun run purge-player.ts … with the scan run from the host on the bind-mounted ./server/data/ (4,500 rows, receipt walTruncated:true, vacuumed:true, WAL 0 bytes, server kept serving, 0 error lines) · erasing one session leaves every other session semantically identical — order, loader-rejected entries and unknown keys preserved (the file is re-serialised, so "byte-identical" was the wrong word) · missing, empty or non-SQLite DB_PATH → exit 5, retry:false, file untouched · both purge lookups SEARCH idx_telemetry_player · 24 server suites green (1 new: erasure-audit.ts).

One acceptance clause amended, deliberately: "duplicate-playerId roster → non-zero exit and name removed" is self-contradictory — if the name is removed, non-zero would tell the operator to re-run a completed erasure. The implementation erases every occurrence (rosterEntriesErased:2) and exits 0; what must never happen — a success receipt with the name still on disk — is what the test pins. An unreadable roster is the non-zero case (exit 3), and it is checked before any row is deleted, so "nothing was changed" is literally true.

The six-lens checker pass found six real defects in this phase's own first cut, each reproduced independently before it counted (26 candidates confirmed, about half downgraded on impact), all fixed and re-verified by execution: - secure_delete + TRUNCATE was not enough. Freed pages are zeroed, but a leaf page a surviving player still occupies is rebalanced in place and keeps the erased rows' bytes in its gap — ~0.2–0.5 % of an erased player's rows recoverable in the everyday round-robin layout, behind a green receipt. The first cut's test inserted each player's rows contiguously and could not see it. Now: VACUUM before the checkpoint; the test interleaves. - The TRUNCATE checkpoint froze the live server for 5 s per attempt when a reader pinned the WAL — it busy-waits holding the write lock, and the CLI inherited busy_timeout=5000. Now: PASSIVE copy first, 100 ms busy timeout for eight short TRUNCATE attempts — exit 4 in ~2.6 s instead of ~26 s, and the writer is never held for more than 100 ms. - A roster failure after the rows were gone printed erased:0 / "nothing was changed" and skipped the checkpoint. Now: the roster is read (and validated) before the delete, the failure receipt reports the true counts, and VACUUM + checkpoint run in a finally. - The hourly sweep's rewrite could race a CLI purge and write a just-erased name back behind a success receipt. Now: every writer of roster.json (sweep, purge CLI, roster-user.ts) takes a lock file; the purge re-reads to verify. A process.exit inside the lock in roster-user.ts would have left it behind — caught by its own test. - SELECT DISTINCT session_id for the pruner was a full covering-index scan on the event loop, linear in rows. Now: one indexed LIMIT 1 probe per roster session. - A 0-byte DB_PATH passed the existsSync guard and was initialised as a fresh schema — "erased 0, exit 0" by another route. Now: regular-file + size + SQLite-header check.

Smaller items folded in from the same pass: ids validated ([A-Za-z0-9._-]{1,64}, exit 2) so a typo cannot become an "erased 0" record; a read-only store is exit 5 (retry:false), not "retry forever" — the Linux root-owned bind mount; future-dated provisioning stamps clamped; a session literally named __proto__ stamped as an own key; atomic rewrite at the symlink target with the temp file removed on failure; the runbook's exit-5 remedy no longer exits 5 itself; the new counter is in the metrics table. Known, documented bound: names for a session that never receives a fix expire RETENTION_DAYS after the last roster-user.ts set (re-run set to renew) — a WARN names the session.

A second checker pass on the fixed code (lock, VACUUM, CLI) found no erasure-integrity defect but hardened the new machinery: the roster lock was held across the whole DB delete (minutes on a huge store, past its own 60 s stale rule) — now two millisecond windows, a dead holder broken atomically, a live one never broken; permanent conditions (bad roster path/permissions, malformed file, un-removable lock, full disk, a name nested where the rewrite cannot reach) are exit 5 retry:false, not "retry"; receipts carry storeBytes + per-stage timings; the CLI refuses when the disk cannot hold the ~2.5× rebuild; the strings | grep verification one-liner (false-pass on a missing file, false-fail on substrings) is replaced. Known, documented cost: on a ~1 GB store the rebuild is tens of seconds to ~2 minutes and a live server drops fixes meanwhile — erase between sessions.

Two test-harness lessons worth keeping: an un-finalised EXPLAIN statement on a bun:sqlite connection pins a WAL snapshot once any later statement runs (prepare + finalize it); and an unreferenced Database in a helper process is garbage-collected mid-await, silently closing the connection — the "pinned reader" that was not pinning.

Phase 3 — Boundary correctness

  1. Coerce every wire field in ingest.ts (telemetry and status); Number.isFinite guard on Gauge.set.
  2. Shared env.ts with loud fallback; log resolved config at boot.
  3. Truthful /health (+ DB probe); session label cap + bucket sweep.

Accept: injection test finds no non-numeric value in /metrics · typo'd env still enforces caps · /health flips to mqtt:false within 5 s of broker loss · 500 novel sessions ≤ 33 series.

✅ DONE 2026-08-23. All three items. Acceptance, by execution (server/test/e2e.ts §12–15 against a real broker + the real server, and the new server/test/boundary.ts): six malformed frames (the audit's fix: "3\nft_injected_metric 999", a numeric-looking string, a null, a NaN, a string lat, an object ts) → zero WS frames, zero rows, bad_payload ×6, no ft_injected_metric and every sample value a plain decimal · a status frame with everything but up missing exports ft_device_battery_percent -1 (the unmetered sentinel) and a finite-number health envelope; a non-numeric up is dropped · HISTORY_MAX_SPAN_MS=6h (and every other knob) falls back to its default with a WARN naming the rejected value, and the boot log lists the whole resolved config · /health → 503 {ok:false, mqtt:false} within 5 s of killing the broker (it used to latch true forever); it also carries a db probe · 500 publishes under novel session ids → ≤ 33 distinct session label values (32 + _other); the real session keeps its own series · 25 server suites + 21 Playwright specs green.

The five-lens checker pass then found three defects in this phase's own first cut (18 findings confirmed in all, the rest hardening/pre-existing), each reproduced independently and fixed before commit: - The new /health db probe could not fail — SELECT 1 runs entirely in SQLite's VM, so the table dropped or the disk full still answered db:true. Now it reads the telemetry table and folds in the last insert outcome; the e2e drops the table and watches db:false appear. - The label cap was first-come for ANY traffic — 32 junk publishes (even unparseable ones) could reserve every session slot and evict the real match into _other for the process lifetime, inside the exact S-5 threat model. Now configured sessions are seeded at boot, only validated frames admit, and unvalidated traffic reads _other without reserving (cost: the first valid packet of a new stream counts under _other). - The status rssi sentinel was 0 — the strongest possible signal, so a firmware omitting rssi rendered a GREEN card where the old stack showed a blank one. The sentinel is now -127 (classifies bad), and every status/telemetry field has a physical range (a wrapped pct: 250 reads unmetered, spd < 0 is rejected — two of those once overflowed a /history average to null). Also folded in from that pass: timer knobs are bounded below the 32-bit setInterval clamp (a "sweep monthly" value used to become a 1 ms hot loop the boot log vouched for); sane maxima on TTL/history-span/retention; envBool stays strict-lowercase (case-insensitivity would have loosened anon/proxy/cookie knobs); URL userinfo is redacted from the boot log; the invalid-config summary logs at ERROR so LOG_LEVEL=error still shows it; a 1 KiB server-side payload cap (too_large); MQTT keepalive 15 s (~22 s half-open detection, was ~90 s).

What changed, structurally: src/wire.ts is now the single boundary for both device frames (explicit fields, finite numbers, bounded ids — no string coercion: a device sending "3" is a bug to surface, not to paper over); src/env.ts replaces 40-odd ad-hoc Number(process.env.X ?? d) reads (the Math.max(1, NaN) pattern that admitted a 10-year export) and records every knob for the boot log; the registry refuses non-finite values and caps label cardinality (session 32, player 256, overflow _other); the ingest rate buckets are swept when idle; /health follows the MQTT client's close/offline events and returns 503 when not ok — Playwright's webServer wait (200–403 = available) and a future compose healthcheck then both mean what they say. One test (ws-origin.ts, deliberately brokerless) had been using HTTP .ok as "server up" and now waits for the truthful 503 body instead.

Phase 4 — Field resilience (firmware)

Non-blocking connect state machine + jittered backoff · GPS drain loop + larger RX buffer · paced flush + NVS offset cursor + sq dedupe + drop-oldest · GPS UTC timestamps · player-id validation · watchdog + reset reason/version · backlog age purge · runtime SESSION_ID. Server side: raise ingest rate cap in the same phase (they must land together).

Accept: 60 s AP outage preserves ≥92% of fixes · no duplicate (player_id, seq) rows · replayed rows span ~60 s, not ~1 s · invalid ids rejected at enrollment.

✅ DONE 2026-08-24 — code + server-side acceptance; the hardware halves await the bench drill (runbook §7). What is PROVEN today, by execution: no duplicate rows — 30 publishes with 10 duplicate (player, device, seq) triples persist exactly 20 rows, the re-sends counted as dropped{duplicate}, and a replacement tracker's fresh sequence is NOT swallowed (the dedupe key is per-device — server/test/e2e.ts P4a, plus a server-restart dedupe check in the checker pass) · replayed rows span the outage — a 20-fix backlog with gts spread over ~57 s persists with that span, not the arrival second, and /history aggregates computed over spanned vs collapsed timestamps differ exactly as F-2 predicted (e2e P4a + checker) · a sustained ~45/s replay+live load is fully accepted (cap raised 15→50; the test outlasts the burst window, so a cap regression fails it — e2e P4b) · invalid ids are rejected at enrollment — ft_id_valid (charset-exact with the server's TOPIC_ID_RE, host-tested incl. 07/b, 65 chars, +, #) gates save, the portal and configLoad, so a badly-provisioned device refuses to run rather than black-hole · the crash-safety logic (seq high-water, two-file rotation, cursor windows, expiry, backoff bounds) is host-tested C++ run locally and in firmware-ci; pio run compiles (flash 72.5%). PENDING THE BENCH (no device on USB): the literal ≥92%-of-60 s measurement and the mid-replay power-cut drill — the runbook's §7 is the script.

The five-lens checker pass (host-compiled C++ probes + live server probes) found real defects in this phase's own first cut, all fixed and re-verified: - The both-full rotation deleted BOTH backlog halves — main.cpp deleted 1 - plan.target while the plan's older half sits AT target, so a >22 min outage would have destroyed all 256 KB and kept one record, logging "dropped the OLDEST half". The plan flag is renamed (drop_oldest_at_target) with an I/O contract comment, and the host test now pins which slot the sacrifice sits in. - The "8 KB buffer rides out the stall" theory was wrong — the SparkFun driver parses the whole UART ring into ONE PVT struct, so fixes buffered during a blocking mqtt.connect collapse to the last one (~71–76% preserved in a broker-down outage). The stall itself is now short: a 400 ms TCP pre-connect (PubSubClient then skips its own blocking connect), a 1 s CONNACK bound, and a cached mDNS result — ~4–14 fixes lost per backoff attempt, ~5 attempts per 60 s ⇒ ≥92% with margin. - The player-scoped dedupe would have swallowed a replacement tracker (and a clear-re-enrolled one) for up to 30 days. The index is now (player_id, device_id, seq); clear no longer wipes seq_hw (keys are removed individually); a device with no stored high-water starts from a random base. - Smaller, each reproduced: torn-append tails are healed at boot (\n terminator) so they can't merge with — and destroy — the next record; only position fixes (2/3/4) are stashed (pre-match indoor no-fix churn at 10 Hz was evicting real fixes); u-blox signed gSpeed/headMot are clamped/normalised at the source AND tolerated by the server (a near-stationary fix is no longer dropped whole); fixType 5 (time-only, no position) is no_fix, not a dot in the Atlantic; a session change at re-enrollment wipes the pending backlog (those fixes belong to the OLD session's access-control scope); the freshness gauge uses arrival time (a backlog drain no longer false-fires the staleness SLO); a poison record can't wedge the flush; the reset-reason legend was off by one (sw=3, panic=4, task-wdt=6 — now correct in /metrics HELP and the docs); per-file checkpoint counters; the wipe message no longer overclaims flash erasure.

Phase 5 — Coach-view reliability

serverClock.ts skew correction · reconnectNow() + online listener + retry button · root and Review error boundaries · fetch deadlines · pitch corners via session config · off-pitch indicator · 44 px touch targets · minimal client beacon.

Accept: skew unit test within 100 ms · e2e: kill server → "gave up" → click retry → feed returns · induced Review throw shows the boundary, not a blank page.

✅ DONE 2026-08-26 — all three acceptance criteria met, by execution, plus ten defects the checker pass found in this phase's own first cut.

ACCEPTANCE. Skew within 100 ms — client/src/serverClock.test.ts drives a 45 s-ahead tablet through 15 samples of realistic transit jitter and lands within 3 ms; the negative-skew (dead-tracker-looks-live) direction is covered too. In the browser: reliability.spec.ts runs the whole app with Date.now() shifted +30 s and requires the pitch to populate AND the "Last fix (s)" cell to read fresh — with the correction disabled, that test fails at the first assertion (verified). Kill server → "gave up" → retry → feed returns — the same spec kills a REAL server process, waits for the give-up text, presses the button, restarts the server and requires the feed back. Induced Review throw — a DEV-only crash switch (dead-code-eliminated from production, and client-ci's new guard:bundle step fails the build if the token survives — proven by injecting it) makes Review throw for real; the boundary renders, the shell survives, and "Back to live" returns to a working canvas.

ALSO SHIPPED: the pitch's four corners moved out of the bundle into per-session config (ADR-0019 amendment), validated on both sides — a 426-quad differential run (both hemispheres, 78°N, the antimeridian, 400 random quads) found zero disagreements between the server and client validators; off-pitch players pinned to the canvas edge with a distinct marker and the word "off pitch" in the accessible mirror, instead of being clipped into invisibility while the HUD counted them; deadlines on every client read; retries that actually re-fetch; 44 px touch targets; and a four-value client beacon (ADR-0024) so a dark tablet is visible from /metrics.

The six-lens checker pass (browser probes, a stopped server, host-run repros) confirmed 10 defects in the first cut — one refuted — every one fixed and re-verified: - The clock estimator was fed by telemetry, whose serverTs since Phase 4 may be a replayed fix's GPS time. A page loading while a tracker drained a backlog would have inferred an offset of HOURS and then rendered stale positions as live dots — ADR-0018's honesty rule failing in its dangerous direction. Fixed with a new {event:'hello'} envelope (the server's clock, first frame on every socket) plus .../status as the only sources; telemetry is now excluded by design, and a test pins the consequence it prevents. - A silent stall left the view saying "live" forever — the commonest field failure of all (walk behind the clubhouse; the AP drops the flow) produces no close event, so the socket stayed OPEN, the phase stayed 'live', and BOTH new recovery paths were explicit no-ops there. A watchdog now treats 15 s of silence on a socket that had been carrying data as death. The first version of that watchdog called close() and waited for onclose — measured against a SIGSTOPped server, that event did not arrive for 40 s while the banner still read "connected". It now detaches the socket and drives the reconnect itself; a SIGSTOP e2e case pins it. - The fetch deadline bounded only the response HEADERS. fetch() resolves before the body streams, so every await res.json() was unbounded — the exact indefinite hang the module's own docstring claimed to fix, and the likelier shape for a multi-packet /history page. The body is now read inside the deadline. - 8 s was the wrong deadline for a scan, justified by a comment claiming a server-side scan-time cap that does not exist: a legitimate long review read was aborted and shown as a failure that every retry reproduced, while the abandoned scan kept its shared off-loop slot. Scans now get 30 s; small reads keep 8. - The simulator's pitch was 7.1 km from the client's fallback, so every hardware-free run rendered the whole fleet off-pitch — invisible before this phase, an edge full of markers after it. The simulator now publishes its own corners through the session config (so every run also exercises that path), and a new live-spec assertion fails if any tracked player renders off-pitch. - One transient /config failure stranded the coach for the whole match on U14 defaults and the placeholder pitch, with the footer positively asserting no pitch was measured. The hook now retries with backoff and reports an explicit error status the footer distinguishes from "none configured". (Its documented "last-good retention" was dead code — with one fetch per session it could never fire; removed.) - Smaller, each reproduced: the beacon's rate-limit map had no sweep and is IP-keyed for the anonymous principal (unlike /roster, which requires a login) — now swept like the ingest buckets; an online- triggered reconnect was counted as ws_manual_retry, which would have made a healthy Wi-Fi flap read as a UX failure in the one metric ADR-0024 defines as "a coach had to intervene"; the reconnect button was offered during the FIRST connect (shouldOfferReconnect, now one rule with its own unit gate); the accessible mirror rebuilt its homography on every 1 Hz render; the bundle guard advertised a child-name check it did not perform (now performed); and the timer-leak test could not fail — deleting clearTimeout left it green — now it watches the timer itself.

The one REFUTED claim: that Review's default window is computed from an inert estimator. The verifier showed the feeder does exist — and the hello frame makes the estimate available from the moment the socket opens, before Review can be reached.

The checker's completeness pass then drove a second round, because most of those fixes had shipped WITHOUT a test that fails without them: the clock-source rule is now a function (clockSampleFrom) with its own unit gate, so re-feeding telemetry fails a test rather than a code review; a browser test strips the pitch out of the config response to exercise the off-pitch render path (which the simulator fix had just stopped exercising) and another fails the config read outright to pin the honest footer; the beacon's bucket sweep is proven by a new ft_client_beacon_buckets gauge returning to 0; the bundle guard has a self-test that requires every forbidden token to be detected; and a mid-body caller abort is pinned as an abort, not a timeout. The session config also re-reads on online, so a session that exhausts its retry budget is not stranded for the rest of the match.

Deliberately deferred to Phase 6 (operability), not fixed here: an abandoned /history or /events request keeps scanning and keeps one of OFFLOOP_MAX_INFLIGHT slots until it finishes — the server never observes the client's disconnect (request.signal is unused), and there is no wall-clock scan budget at all. Phase 5 reduces how often that happens (scans get a 30 s deadline rather than the 8 s that was aborting legitimate reads), but the mechanism is server-side cancellation plumbing and belongs with the rest of the operability work.

Phase 6 — Operability

Graceful shutdown + exec-form PID 1 · compose healthcheck on /health · log rotation · migrations via user_version · VACUUM INTO backup with erasure-aware rotation · deploy/production/compose.yml with absolute broker paths · uncaughtException handlers.

Accept: docker stop exits 0 (not 137) in <2 s · healthcheck reports healthy/unhealthy correctly · backup restores to a byte-identical row count · purged player absent from every backup.

✅ DONE 2026-08-27 — all four acceptance criteria met by execution against the real stack, plus the item Phase 5 deferred here and the whole of §6's "Server" and "Infra" groups.

ACCEPTANCE, measured. (1) docker stop — the baseline was re-measured first and reproduced the audit exactly: exit 137 after 1.29 s, sh as pid 1, no teardown at all. After exec + init: true + src/shutdown.ts: exit 0 in 0.23 s on the dev stack and 0.10 s on the production image, with db closed {checkpointed:2} in the container's own log. test/shutdown-e2e.ts pins the in-process half at 7–9 ms and asserts the step ORDER, not merely that steps ran. (2) Healthcheck — docker stop ft-mosquitto flipped the server container to unhealthy at +50 s (interval 15 s × 3 retries, as configured), with the probe's own output recording {"ok":false,"mqtt":false,"db":true,...}; restarting the broker returned it to healthy within 10 s. The probe is bun run healthcheck.ts, because oven/bun:1.3 ships no curl, wget or nc (verified). (3) Backup row count — test/backup.ts compares source and copy in total AND per player per session, then re-opens the copy independently. (4) Purged player absent from every backup — purge-player.ts runs the same erasure statements against every telemetry-*.db in BACKUP_DIR and re-counts each file; a copy that cannot be erased is exit 4 with the file named, and a test proves that failure path fires (a read-only backup) rather than being skipped silently.

ALSO SHIPPED (ADR-0025): the user_version migration ladder, with a store NEWER than the build refusing the boot (proven end to end — the process exits non-zero and never opens a listener); VACUUM INTO backups whose rotation is bounded by both BACKUP_KEEP and RETENTION_DAYS, because a copy is a fix and ADR-0010 applies to it; the deferred scan-cancellation item — request.signal plus a 25 s wall-clock budget, checked inside the existing yield helper so a new paged loop cannot forget it, with a test that fills every shared slot with abandoned scans and requires a fresh read to succeed; a production stack (deploy/production/compose.yml + server/Dockerfile) that is non-root, installs from the lockfile, publishes nothing on 0.0.0.0, has no anonymous access, and carries no roster/accounts/store in any image layer — all of it guarded statically by test/deploy-posture.ts (37 checks) which runs unfiltered in repo-guard; log rotation on both services; and the rest of §6's "Server" group: uncaughtException/unhandledRejection handlers, mode: 0o600 that is no longer a no-op on an existing file, WS fan-out drops counted as drops, and coaches staying logged in across a restart.

TWO FINDINGS FROM THIS PHASE'S OWN WORK, both fixed: - The SLO tests were asserting on a lie. Making server.publish()'s return meaningful turned history-e2e and events-e2e red — because they used ft_ws_messages_sent_total as a proxy for "the live loop is being serviced" with no WebSocket client connected at all. The counter had been counting attempts. Both now attach a real /live subscriber, so the SLO measures delivery rather than intent, and a new e2e.ts case pins the accounting directly: publish into an empty room and sent must stay flat while dropped rises. - The store itself was 0644. Noticed while checking the production image: the name and credential files are 0600, but the file holding the positions was left at the process umask. Tightened on open, sidecars included, best-effort so a mount that cannot chmod warns instead of refusing to boot.

Each fix was then verified NON-VACUOUS by breaking it and watching its gate go red — ten of them: the downgrade guard, the yield-point cancellation check, the SIGTERM handler, the drop accounting, the backup retention bound, the backup erasure-failure report, the 0600 writer (all three CLIs), the session handover, and both compose posture guards.

THE SIX-LENS CHECKER PASS then found twenty-two more defects in this phase's own first cut, every one reproduced by execution before it was believed. The four that mattered most:

  • A docker stop during BOOT was worse than the bug being fixed. The signal handlers were installed as the last statement of server.ts, leaving ~150 ms with none — and because bun is pid 1, the kernel DISCARDS a signal pid 1 has no handler for, so a stop in that window waited out the entire grace period and SIGKILLed: exit 137 after 5.1 s, 3/3, against a 1.3 s baseline. Handlers are now the first line of the module. The same window also destroyed the session handover (loadSessions() consumes the file during boot, and the step that writes it back had not been registered yet), so auth.ts now registers that step itself, at the moment it consumes the file. A six-point sweep across the window is the gate.
  • A mistyped DB_PATH wrote this schema into somebody else's database — created telemetry, converted the file to WAL, and overwrote its user_version, the byte other migration tools key on, while this server served an empty pitch behind a green /health. assertOurs() now runs before any pragma that writes, and a foreign store is refused byte-for-byte.
  • The erasure receipt could say "erased" over backups it never opened. backups: [] was indistinguishable from a wrong BACKUP_DIR (host path vs container path), and a checker reproduced exit 0 with a clean receipt while every real copy still held 1,800 of the child's rows — the shape of §4.5(e), re-opened on the new surface. The receipt now carries backupDir and backupsFound, the same signal rosterFound provides, with an operator-visible note.
  • One authenticated principal could take the review surface offline for the whole club. The per-principal rate bucket is not a fairness control for a SHARED cap: a caller well inside its own budget held every slot continuously and denied another coach 39 of 40 reads over 40 s. The slots now have a per-principal share (2 of 4), so a coach's own Review page still works and no principal can reach zero for everyone else.

And the rest, each with a test that fails without it: the session handover applied neither the current TTL nor the per-user cap (so "shorten sessions and restart" — the response to a lost tablet — changed nothing), was not consumed when the unlink failed (a signed-out coach came back on every boot, silently), trusted a hand-written e of 1e308, and read a 400 MB file into a 512 MB container before checking its size; abortAllScans() marked budgets and returned, so reason="shutdown" was a permanently zero metric and a coach mid-review got a socket reset instead of the promised 503; an aborted scan recorded no volume and no principal in the bulk-export audit trail; the default ScanBudget leaked the very set its leak-detector reads; migrations used a deferred BEGIN whose SQLITE_BUSY_SNAPSHOT the busy handler never retries; test/migrate.ts did not test its own central claim (deleting a column from migration 1 left all six cases green — there is a frozen schema snapshot now, because migration 1 is append-only forever); five concurrent auth-user.ts adds silently lost two accounts that reported success while persisting two that reported failure (the roster's proven lock is now shared by all three CLIs); deploy-posture passed 37/37 on a Dockerfile ending USER root; rotation only ran as a side effect of a SUCCESSFUL backup, so a failing nightly cron expired nothing (--rotate-only and ft_backup_oldest_age_seconds close that); a future-dated copy could never age out; an unreadable BACKUP_DIR threw a stack trace outside the exit contract; an unwritable /data produced a raw SQLITE_CANTOPEN crash-loop with no diagnostic; and the production README left you with a healthy backend and no coach UI, because nothing told you to build the client.

The checkers also REFUTED plenty, which is the other half of the value: no byte-level residue of a purged player in any backup (8,000 interleaved rows, 0 hits, -wal included); no missed site in the session re-keying (11 sites, 14/14 assertions); no timing oracle; no drift between migration 1 and the pre-Phase-6 schema; no slot/budget desync from 400 malformed requests; no rate-limiter evasion via abort-and-retry; no personal data in any image layer (all 12 blobs, zero whiteouts, content-grepped against the real roster); and deploy.resources.limits DOES apply under Compose v5 — the "swarm only" folklore is out of date.

Deliberately NOT done, and named rather than faked: TLS termination for a field box. There is no Caddyfile because the real decision is an internal CA and getting it trusted on the coaches' tablets, which belongs with the person who owns those tablets. deploy/production/README.md says so, and says what to do until then.

Phase 7 — Vision & docs

Fail-fast stubs · job queue + timeouts + artifact allow-list · streaming pipeline · TTL prune · ledger out of out/ · checksum-pinned fetch · consumed lockfile · README/CLAUDE.md drift + vision/ visibility · docs-as-tests guard.

Accept: --ball exits non-zero · second concurrent POST returns 429 · _iter_world_states peak < 1/10 naive · docs guard passes.

✅ DONE 2026-08-28 — all four acceptance criteria met by execution, plus the whole of §6's "Vision" and "Docs" groups and P1 items V-1, V-2 and Q-1.

ACCEPTANCE, measured. (1) --ball exits non-zero — --ball, --radar and --stats now exit 3 (a status distinct from argparse's 2 and from a real failure's 1) with the reason on stderr, and create nothing: ls /tmp/o1 /tmp/o2 /tmp/o3 → no such file. --selftest still exits 0. (2) 429 — driven over a real socket in test/test_webui_server.py: the first POST gets 202, the second while it runs gets 429 with a message, and the slot is released when the job ends (verified by polling until a third POST is accepted). (3) Peak retention — measured at 720p rather than asserted: 200 frames naive = 553 MB, streaming peak = 1 frame = 2.8 MB, a ratio of 1/200 against the required 1/10. At the stated 90-min/720p/5 fps target that is the difference between 69.5 GiB and ~3 MB. (4) Docs guard — two of them: server/test/docs-guard.ts, 24 checks, wired into the unfiltered repo-guard workflow and the 31-suite server gate; and vision/test/test_docs_guard.py for the subproject's own claims.

ALSO SHIPPED: the v2/v3 ... stubs replaced by NotImplementedStage raised as the FIRST statement (so the refusal is identical with or without weights, and cannot be mistaken for a missing manifest); _iter_world_states rebuilt as a two-decode-pass stream keeping only detections plus one copied sample crop per track id; the web UI bound to loopback, capped at one job, given deadlines on both subprocess stages that kill the process group (yt-dlp shells out to ffmpeg; killing the child alone leaves the pipe open and the read blocked forever), and an artifact allow-list — clip.<ext>, the raw downloaded source, was reachable to anyone who guessed a 12-hex job id; a 24 h TTL prune of out/, with the attestation ledger moved OUT of it first; checksum-pinned weight fetch, where the manifest records the pin and never the digest of what arrived; both lockfiles made real and actually installed from (the CPU one is a full verified pip freeze, rebuilt and re-run); and ffmpeg's exit status finally read.

THREE FINDINGS FROM THIS PHASE'S OWN WORK, all beyond the brief: - The README's third-party attribution was false, not merely unfilled. It said Roboflow sports (MIT) "is vendored … at commit <RECORD SHA HERE>". Nothing was ever copied — the module's own header has always said these are "NOT a verbatim copy … independent minimal equivalents", ~90 lines of original code. Filling the blank would have meant inventing a SHA for code nobody took. The accurate claim is now in vendor/sports/PROVENANCE.json with a copied_code flag the guard keys off; if real source is ever brought in, the guard starts demanding the 40-char commit. - CI was testing a different major version of the decoder than the pipeline runs on. The CPU test image resolved opencv-python 5.0.0.93 while the inference images pin <5. A test image is only evidence about the run image to the extent the two agree; the bound is now in both. - samples/ held footage ADR-0023 §3 requires to be discarded. The empty manifest was not a paperwork gap: two clips of an amateur pitch whose competition and players' ages cannot be identified from the footage — exactly the ambiguous case §3's default-deny rule is written for. Reported to the owner, who deleted them. clip.mp4 was verified by eye to be synthetic (a green rectangle and white bars, no people) and is recorded as such.

NON-VACUITY: 21 mutations, each breaking one fix and requiring its gate to go red. Three tests failed that bar and were rewritten, which is the whole reason for running it: - the streaming test passed with _crop reverted to a numpy view, because the fake provider reused three track ids so every crop pinned the same single frame, and the measurement ran after the crops were released. Real tracking emits a stream of NEW ids — that is what track_id_space: "raw" is confessing — and under views each pins its own parent frame for all of pass 1. There is now an id-churn case measuring inside embed(), the one instant every crop is live at once; - the no-pin test asserted "pin" in message, which the mismatch path also satisfies ("expected the pinned None"). It now asserts the downloader is never called — refusing before a 137 MB download is the behaviour that matters; - the lockfile-consumed check searched the whole Dockerfile stage for requirements-test.lock and so matched the COPY line, staying green when the pip install was flipped back to the range file. It now reads the install line specifically.

Deliberately NOT done, and named rather than faked: v2 (ball + radar) and v3-over-video remain unbuilt. Wiring them needs real weights, a calibrated clip and a GPU — none of which CI has — so they refuse loudly instead of returning a hollow success. That is the fix; building them is not this phase.

Phases 1, 2 and 2b are the ones I would not defer: Phase 1 because every other gate is inert without it, Phase 2 because the exposure is live today on a stack with restart: unless-stopped, and Phase 2b because an erasure mechanism that reports success without erasing is worse than none — it produces a written audit trail asserting compliance that did not happen. Phases 3-4 carry the most engineering value per line. Phase 5 onward is comfort and durability.

A note on §4.5: nothing there reflects carelessness. Each defect is a reasonable decision applied one context too far — a fail-closed loader (right for serving) reused on an erasure path, secure_delete trusted without accounting for WAL semantics, a receipt contract that predates the Docker layout. That is exactly the class of bug that survives review and only surfaces under execution, which is why the acceptance criteria above are all byte-level assertions rather than "verify erasure works".

The outdoor-session milestone (README's own open item) is blocked specifically by Phase 4 — until the reconnect path is non-blocking, a real match will silently lose most of any out-of-range period, and the resulting data would be misleading rather than merely incomplete.