Eight stages, two clocks

Define the task as a bounded unit: typed input, typed output, explicit success shape, explicit termination. An unbounded task cannot be evaluated, so it cannot be gated, so it cannot progress.

Technique	When
Typed result envelope (success / failure / partial as first-class states)	Default for any agent task
Bounded loops (max iterations, max cost, max wall time)	Any task with retry or multi-step reasoning
Task decomposition (tree of bounded subtasks)	Work too large for one boundary
Explicit non-goals in the task definition	Tasks prone to scope creep

Evidence produced

The task definition itself. The boundary is the first auditable artifact.

Common failure

Boundary defined by prompt language only (“do X, don’t do Y”) with no typed contract. The model can violate prose; it cannot violate a type.

Make execution observable before making it evaluable. Every run emits a trace: inputs, outputs, intermediate states, costs, timing. If you cannot observe it, you cannot measure it; if you cannot measure it, you cannot gate it.

Technique	When
Structured trace logging (per-step, machine-readable)	Default
State-as-spine (single canonical state object, transitions logged atomically)	Pipelines where partial state corruption is possible
Cost and latency capture per step	Anything with a budget or SLA
Replay capability (trace sufficient to reconstruct the run)	High-stakes domains, debugging evaluator disagreements

Evidence produced

The raw material of the Evidence Log. Traces are what promotion decisions cite.

Common failure

Logging designed for human debugging (print statements, prose) rather than for evaluation (structured, queryable). Retrofitting structure onto prose logs costs more than instrumenting correctly at the start.

Attach an evaluator to the bounded, instrumented task. The evaluator declares all four choices: target (what layer to inspect), technique (what produces the verdict), oracle (what the verdict is measured against), and position policy (where the evaluator runs per spectrum region). The oracle is an evaluator property, not a separate declaration at the task level.

Technique	Verdict basis	Cost	When
Exact match / assertion	Deterministic comparison	Near zero	Output has a single correct form
Property checks	Invariants hold (schema valid, sums balance, no PII present)	Near zero	Correctness is partially checkable even when the full answer is not
Reference-based scoring	Distance from known-good examples	Low	Golden datasets exist
Rubric LLM judge	Model applies written criteria	High per verdict	Fuzzy quality, cold start, no labels yet
Trained classifier	Learned from accumulated labels	Low per verdict, high setup	Enough labels accumulated (see stage 8)
Human review	Person renders verdict	Highest	Cold start, calibration, stakes exceed all automated confidence

Evidence produced

Verdicts, attached to traces.

Common failure

One evaluator asked to judge everything about the output. Verdicts become uninterpretable. Decompose: one evaluator per property, composed.

Position the evaluator's verdict in the action path. The gate decides what happens between verdict and effect. Position is a policy choice per task, per autonomy level.

Position	Behavior	When
Pre-execution (blocking)	Action does not fire without a passing verdict	Irreversible or costly actions
Post-execution (audit)	Action fires, verdict recorded, failures trigger review	Reversible actions, high volume
Sampled	Fraction of actions gated, rest pass	Matured tasks, cost control
HITL surface	Human is the gate; system produces a review artifact — a plan, a brief, a proposed action with evidence attached	Cold start, high stakes, calibration
Off	No gate	Fully matured tasks with demotion rules armed

Evidence produced

Gate decisions (passed, blocked, escalated), which are themselves evidence for stage 5.

Common failure

Gate position chosen once and never revisited. Position should move as evidence accumulates; a permanently pre-execution gate on a matured task is paying the cold-start tax forever.

Collect verdicts over a defined window into metrics a promotion rule can consume. A single passing verdict is an anecdote; a rolling window of verdicts is evidence.

Technique	When
Rolling window over N cases	Default; volume-based tasks
Rolling window over time period	Low-volume tasks where N takes too long
Stratified windows (per input category)	Task performance varies by input type; aggregate hides weak strata
Disagreement tracking (evaluator vs human on overlapping cases)	Any HITL stage; feeds stage 8

Evidence produced

Windowed metrics: pass rate, escalation precision/recall, cost per verdict, human-override rate.

Common failure

Window defined after the fact to make the numbers pass. The window and threshold must be declared before accumulation starts, or the promotion is a guess with paperwork.

The core state transition of PAA. A declared rule consumes windowed evidence and moves the autonomy level: gate repositions, human involvement decreases, and the change is logged with the evidence that justified it. Promotion without evidence is a guess, not a promotion.

Technique	When
Threshold rule (metric ≥ bar over window, promote one level)	Default
Staged promotion (HITL → post-execution audit → sampled → off)	Any task; skip levels only with explicit justification
Promotion with probation (elevated sampling for M cases after promotion)	First promotion of any task; cheap insurance
Human sign-off on the promotion itself	Regulated domains; the rule proposes, a person ratifies

Evidence produced

The promotion record: prior level, new level, rule fired, window, metrics, timestamp. This record is what makes autonomy auditable.

Common failure

Promotion happens informally. Someone gets comfortable and stops reviewing. The system's actual autonomy level and its declared level diverge, and no artifact records when or why.

The symmetric transition. A demotion rule watches the same metrics and moves autonomy down when they degrade. Fallbacks define what fires when a verdict fails or a boundary is hit. Promotion without an armed demotion rule is a one-way ratchet, and one-way ratchets are how autonomous systems fail publicly.

Technique	When
Metric-triggered demotion (mirror of the promotion rule)	Default; arm it at promotion time
Retry with modification	Transient failures
Fallback route (cheaper model, simpler method, cached answer)	Availability matters more than peak quality
Human escalation	Verdict fails and stakes are high
Rollback (undo the action)	Only where actions are actually reversible; verify, do not assume
Circuit breaker (task suspended entirely after K failures in window)	Failures are correlated, not independent

Evidence produced

Demotion records and fallback firings, same shape as promotion records.

Common failure

Non-atomic state transitions. A failure mid-transition leaves the system in a state no rule anticipated. Transitions between autonomy levels must be atomic — the same failure class that causes pipeline seam bugs applies to the autonomy ladder directly.

The evaluator has its own lifecycle on its own clock. Cold start runs expensive verdicts (human review, rubric judges); those verdicts are labels; labels train cheap classifiers; the cheap classifier takes over verdict production and the expensive path drops to a calibration sample. The task and its gate are both moving, each on its own clock, and the gate's economics are what make gating everything affordable.

Technique	When
Judge-as-labeler (rubric judge verdicts become training data)	From day one; labels are a byproduct, capture them
Human-review-as-labeler	Every HITL decision is a free label; capture or lose it
Distillation ladder (judge → embedding classifier → fine-tuned small model)	Label count crosses viability thresholds per rung
Eval-the-eval (calibration sample: classifier vs judge vs human on overlap)	Always, once a cheap evaluator is in production
Evaluator versioning (verdicts tagged with evaluator version)	Always; unversioned verdicts poison windows across evaluator changes

Evidence produced

Evaluator performance records: agreement rates, cost per verdict over time, label counts.

Common failure

Labels thrown away. Months of human review decisions with no capture path, then a classifier project starts from zero. The single cheapest mistake to avoid in the whole lifecycle.