CTO dashboard plan

Product outcome

Give technical leadership a compact view of delivery predictability, risk and forecast confidence while preserving drill-down into the underlying work items and transitions.

Current state

State: repository-local release candidate implemented; post-launch calibration gate active.

The committed and online implementation has an organization-scoped endpoint with request/response validation, provider-neutral work-item contracts, a React route/page, bounded WIP drill-down, flow cards/charts and an explainable portfolio table. Inclusive UTC ranges, per-project workflows, deduplicated completion, explicit reopen/rework populations, complete-week rates, estimate distributions, WIP/cycle percentiles and forecast withholding are implemented with focused regression fixtures. A representative integration fixture evaluates 5,000 work items and 7,500 transitions while bounding the 2,500-item WIP population to 100 returned rows. Durable snapshots start at feature enablement; post-launch forecast calibration and usefulness remain observation, so positive health claims remain intentionally withheld.¹²

Delivery rules

1. Organization and project scope is mandatory in every query and test fixture.
2. Every metric identifies whether it represents current state, an event window or a durable historical snapshot.
3. A selected end date is the reference clock for windowed calculations; Date.now() is not used inside metric formulas.
4. Start and done classifications are resolved per project, never from a cross-project union without a per-project check.
5. Rates publish numerator, denominator, exclusions and sample size alongside the result.
6. A forecast is unavailable rather than falsely precise when coverage, history or sample size is insufficient.
7. Commit and line-count statistics remain contextual engineering activity signals and are never used as individual productivity or delivery-health inputs.
8. New aggregations are bounded at the database or pagination layer and receive representative-volume performance evidence.

Execution sequence

CD0 — Metric correctness and regression baseline

Priority: P0

Effort: M

State: implemented locally

1. Extract pure metric helpers where useful and add focused unit fixtures for two organizations, projects with different workflow mappings, reopen/reclose sequences, rework, unestimated work and empty history.
2. Add route integration coverage proving non-admin, stale membership, cross-tenant organization and foreign project IDs fail closed or return no foreign data.
3. Normalize inclusive date parsing and anchor rolling calculations to the selected to value. Label current-state and event-window sections separately in the response and UI.
4. Classify completion and reopen transitions with the owning project’s effective workflow; define and test the explicit completion-event deduplication rule.
5. Calculate rework from qualifying regressive transitions rather than only transitions to done. Count a reopen only for a done-to-non-done transition and publish distinct-issue and event counts where they answer different questions.
6. Return unknown/censored WIP age explicitly when transition history is absent; do not report it as zero.
7. Replace the unweighted organization coverage average with aggregate estimated issues divided by aggregate eligible issues.
8. Mark the current ETA and health label experimental. Suppress On Track claims until confidence and health components are implemented.
9. Remove repeated issue scans from per-project ETA calculation, bound the returned WIP rows and expose total count separately.

Exit criteria: dedicated metric and route tests pass; historical date selections are deterministic; mixed-workflow fixtures cannot cross-classify; the UI distinguishes current state from the selected event period; no simplified label claims a project is on track.

CD1 — Trustworthy base metrics

Priority: P1

Effort: M

State: implemented locally

1. Publish issue and story-point completion separately, plus remaining and unestimated populations.
2. Add estimate-size distribution and explicit exclusion/mapping metadata to the response.
3. Use complete-week buckets for throughput and velocity and expose partial-week handling.
4. Define the policy for estimates changed after completion. Until completion-time estimates are snapshotted, label historical story-point delivery as recalculated from current estimates.
5. Add response schemas at the HTTP boundary rather than relying only on TypeScript interfaces.

Exit criteria: metric definitions, API fields, fixtures and UI labels agree; all totals reconcile from project to organization; mutable-estimate limitations are visible.

CD2 — Flow and quality

Priority: P1

Effort: L

State: implemented locally

1. Add WIP age distribution and thresholds, not only a sorted issue list.
2. Implement cycle/flow time P50 and P85 with censored/open-item handling.
3. Persist or configure workflow phase mappings needed to distinguish lead, cycle and development time.
4. Add configurable excluded states/types and retain the applied policy with every result.
5. Separate backlog bug ratio, created-bug ratio and post-completion leakage; do not collapse them into one unlabeled percentage.

Exit criteria: percentile calculations pass deterministic fixtures; workflow and exclusion configuration changes invalidate or version affected aggregates; every quality rate exposes its exact population.

CD3 — Forecasting and predictability

Priority: P2

Effort: XL

State: implemented locally

1. Add the target models and a clean-baseline migration for weekly project snapshots, completion-time estimate facts and sprint-start planned work.
2. Start snapshots and completion facts when the feature lands. There is no MonStaff history to backfill; fields that need elapsed history remain unavailable until enough real periods have accumulated.
3. Replace the linear ETA with a baseline forecast that states minimum history, estimate-coverage and sample-size requirements.
4. Add uncertainty percentiles, forecast drift and calibration against completed periods.
5. Calculate sprint predictability from immutable sprint-start scope and delivered work, including added/removed scope as separate signals.

Exit criteria: snapshots are idempotent and tenant-scoped; repeated calculation from the same synthetic or newly accumulated snapshot is deterministic; forecasts show sample size and confidence and are withheld below the accepted data threshold. Forecast usefulness remains a post-launch observation gate, not a schema-implementation blocker.

CD4 — Decision-oriented portfolio experience

Priority: P2

Effort: L

State: implemented, committed and online; post-launch calibration retained as residual observation

1. Keep the page to at most ten decision-oriented widgets and move detail into project drill-downs.
2. Introduce health only after its component signals, missing-data behavior and thresholds are accepted. Always show the underlying signals and confidence.
3. Add accessible loading, empty, unavailable and partial-data states; retain applied date, timezone, workflow and exclusion context.
4. Add pagination and server-side aggregation for portfolio and WIP detail.
5. Establish query and render performance budgets on representative organization sizes.

Repository-local evidence: the dashboard reconciles 5,000 work items/7,500 transitions in the integration environment, returns a bounded 100-row WIP page and completes under its ten-second test ceiling. Post-launch calibration and decision usefulness still require real elapsed history.

Exit criteria: a decision maker can explain every health or forecast result from visible source signals; incomplete data cannot render as healthy; performance budgets pass at representative volume.

Dependencies and decisions

• The complete remediation plan has implemented tenant authorization locally; independent route review and production canary remain rollout gates.
• CD3 requires accepted policies for timezone, working calendar, exclusions, minimum forecast sample, snapshot retention and treatment of scope changes.
• No historical MonStaff dataset exists. Synthetic fixtures can prove formulas and failure behavior, while forecast calibration and usefulness can be accepted only after representative post-launch history accumulates.
• Commit and LOC context depends on accessible GitHub history and must degrade independently from delivery metrics.

Expected change map

Area	Expected work
apps/backend/src/modules/admin/delivery-dashboard.service.ts	Correct classifications and windows, separate pure calculations, bound results
apps/backend/src/modules/admin/admin-analytics.controller.ts	Validate query and response schemas
apps/backend/prisma/schema.prisma	CD3 target snapshot and sprint-start schema through a clean-baseline migration
packages/shared/src/types/admin.ts	Versioned metric metadata, confidence and unavailable states
apps/admin/src/pages/CtoDashboard.tsx	Temporal labels, warnings, confidence and drill-down behavior
apps/admin/src/components/cto-dashboard/	Accessible charts, bounded tables and explicit uncertainty
apps/backend/src/**/*.test.ts, apps/backend/test/	Formula fixtures, tenant route matrix, query-count and volume coverage

Verification matrix

• two organizations with colliding project/status names and a foreign projectId filter;
• distinct project-level start/done mappings in one organization;
• done-to-done, done-to-active, active-to-done and repeated reopen/reclose histories;
• multiple rework events for one issue and rework without completion;
• null, zero, fractional and changed story-point estimates;
• empty, partial-week, historical and timezone-boundary date ranges;
• WIP with full, missing and stale transition history;
• insufficient forecast history, low coverage and zero velocity;
• representative-volume plans proving bounded queries and response sizes.

Definition of done

A stage is complete only when code, schema, shared contracts, UI labels, tests and this wiki agree; local and CI quality gates pass; repository-derived claims are captured in an immutable snapshot; and remaining rollout or calibration limits are explicit.

Non-goals

• A health score without visible components.
• Precise delivery promises from sparse history.
• In-memory aggregation of unbounded organization history.
• Productivity judgment based solely on input activity.

Footnotes

1. Greenfield release-candidate local implementation evidence. ↩

2. Residual active-plan repository-local closure. ↩