Build PAA / Worked Example

One task, walked end to end

Start with a task whose boundary is explicit and whose work remains under human control. Observe the existing work, run a candidate beside it, and collect behavioral and economic evidence. First choose a configuration at unchanged authority. Later, decide whether continued evidence justifies changing oversight.

All numbers and operating decisions on this page are illustrative. This is a teaching example, not a report of production results or a real autonomy transition.

The executable quickstart uses a refund task, but follows the same governance sequence shown here. The refund approval quickstart runs propose, approve, and emergency demotion against a disposable store.

  1. Stage 1

    Declare the task

    State: none yet; the boundary is not yet declared.

    Before measuring performance, define the input, output, real-world action, and work that remains outside the boundary.

    Our running example is bounded_publication. It accepts a draft_post and may produce a published_post for either a community or professional channel. Publishing anything outside those channels is excluded.

    Artifact: declaration

    • Task bounded_publication, declaration version 1
    • Boundary: input draft_post, output published_post
    • initial_position: manual · deployment: disabled
    • scopes: publish:community , publish:professional

    deployment is disabled, required on every declaration, with no default, and set to disabled until Stage 2 instruments the existing manual effect.

    Top-level declaration fields →

    Rule: An unbounded workflow cannot enter the spectrum: no typed boundary, no evidence, no gate.

  2. Stage 2

    Instrument the existing work

    State: position manual, deployment active for scope publish:community

    Record the human-run workflow before adding authority to automation. Capture each draft, published result, and human decision so later comparisons use observed work rather than memory.

    A person still writes and publishes every post. The task is active at the manual position.

    Artifact: evidence_records

    • Scope publish:community, subject kind case
    • output_recorded: technique deterministic, evaluation basis kind invariant, epistemic status ground_truth, authority blocking, version v1

    A minimal structural evaluator on the human-produced output, enough to satisfy the evidence-record contract while the real quality evaluators are still absent.

    Evidence Records contract →

  3. Stage 3

    Run the worker in shadow

    State: position manual, deployment shadow for scope publish:community

    The candidate runs on real drafts, but its output never reaches a reader. The human continues to publish the real result.

    Record the candidate beside the human-controlled outcome so they can be compared. PAA calls this shadow operation.

    Compare worker configurations against the same behavioral bar and the operating cost per accepted outcome. The illustrative comparison below selects a worker while authority remains manual and deployment remains shadow.

    Illustrative configuration comparison

    Choose a worker; keep the same authority

    Both configurations run the same 200 frozen cases for bounded_publication, scope publish:community. Only the worker model changes. The prompt, evaluator set, rubric, human-review procedure, and fallback stay fixed. Position remains manual and deployment remains shadow.

    Before the comparison, the operator requires an acceptance rate of at least 95% and at most 0 hard safety failures. An accepted outcome meets the publication-quality rubric and all safety checks. Every candidate is reviewed; acceptance here does not authorize publication. The economic policy prefers lower operating cost per accepted outcome among qualified configurations.

    Illustrative results: same 200 cases per configuration; costs in USD
    Configuration Behavioral pass rate Hard failures Human-review rate Worker cost Evaluation cost Accepted outcomes Total operating cost Cost per accepted outcome
    Configuration A 96% · qualified 0 100% $12.00 $4.00 192 $60.00 $0.313
    Configuration B (selected) 98% · qualified 0 100% $6.00 $4.00 196 $52.00 $0.265

    Configuration A: $60.00 includes $12.00 worker inference, $4.00 evaluation, $40.00 human review, and $4.00 additional retry costs. Each charge is counted once, including attempts that did not produce an accepted outcome.

    Configuration B: $52.00 includes $6.00 worker inference, $4.00 evaluation, $40.00 human review, and $2.00 additional retry costs. Each charge is counted once, including attempts that did not produce an accepted outcome.

    Both clear the behavioral bar. The operator selects Configuration B for its lower effective cost. Humans continue to control publication; no autonomy event is produced by this choice. The implementation records the selected configuration and retains the incumbent as a fallback.

    These fabricated costs cover all components declared for this comparison. With real data, disclose missing costs and distinguish estimates from actual spend. Zero accepted outcomes makes the ratio undefined. Aggregate task or pipeline costs only when the underlying usage and pricing remain attributable.

    The later promotion window below is separate evidence for the selected configuration. A real experiment could instead justify retaining the incumbent, rejecting every candidate, or collecting more evidence.

    Artifact: evidence_records

    • Scope publish:community, subject kind case
    • publication_quality: technique llm_judge, evaluation basis kind rubric, epistemic status proxy, authority advisory, version v1
    • publication_quality: technique human, evaluation basis kind human_gold, epistemic status ground_truth, authority advisory, version v1

    Both evaluators share property publication_quality without sharing technique, evaluation basis, epistemic status, or authority. This is the multiple-evaluators-per-property case.

    Evidence Records contract →

  4. Stage 4

    Evaluate and establish evidence

    State: position manual, deployment shadow for scope publish:community

    A rubric-driven model and a human reviewer judge publication_quality for the same candidates. Each judgment records what was measured and which evaluator produced it.

    Freeze the exact cases used for the decision, including why any case was excluded. This decision artifact records the 200-case window and whether it cleared the promotion rule.

    This is a later window collected for the selected configuration, separate from the comparison population. The consumer retains configuration and cost attribution; the current runtime does not enforce worker-specific windows or read cost for eligibility.

    Artifact: decision_artifact

    • Scope publish:community, decision kind promotion, outcome eligible
    • Window: cases: 200
    • Included 194, excluded 6

    Every excluded reference carries a reason. An admissibility snapshot without a reason for each exclusion is not a valid decision artifact.

    Decision Artifacts contract →

    Rule: Recorded evidence is not automatically admissible evidence; admissibility is a declared, snapshotted decision.

  5. Stage 5

    Promotion eligibility

    State: position manual, deployment shadow for scope publish:community

    The evidence now clears the promotion rule. Nothing changes yet; the task is only eligible for a motion.

    Artifact: position_projection

    • deployment: shadow
    • Scope publish:community resolves to manual
    • Scope publish:professional resolves to manual

    Both declared scopes still resolve to manual.

    Autonomy Events contract →

  6. Stage 6

    Decide whether to change oversight

    State: position manual, deployment shadow for scope publish:community

    A motion proposes moving publish:community from manual to human-in-the-loop review and binds the proposal to its evidence.

    operator:jordan_lee reviews the evidence and approves or rejects the motion.

    Artifact: motion_events

    • Scope publish:community: manual to hitl
    • motion_proposed: actor system:promotion_rule: promotion window (200 cases) admissible and eligible per decision artifact
    • motion_approved: actor operator:jordan_lee: evidence reviewed; publication_quality agreement across both evaluators clears the promotion bar

    Both rows share one motion_id and bind to the same evidence_ref and evidence_sha256 as the Stage 4 decision artifact.

    Autonomy Events contract →

    Rule: A motion is proposed and resolved by a named actor or a system actor. It does not fire itself.

  7. Stage 7

    Apply the scoped change

    State: position hitl, deployment shadow for scope publish:community

    The approved motion moves publish:community from manual to human-in-the-loop review. The professional channel remains manual.

    The event stream records the scoped change and is used to reconstruct the current position.

    Artifact: motion_events

    • Scope publish:community: manual to hitl
    • position_changed: actor operator:jordan_lee: motion approved; committing publish:community from manual to hitl

    This row shares its motion_id with the motion_proposed and motion_approved rows from Stage 6: three rows, one motion, one scope.

    Autonomy Events contract →

  8. Stage 8

    Operate and keep evaluating

    State: position hitl, deployment active for scope publish:community

    The community worker now prepares each publication. Its blocking checks must pass, and a human still authorizes the final action. Low-confidence cases can route directly to the human without changing the position of the whole scope.

    Evaluation continues during active operation. Replacing an evaluator creates a new declaration version and evidence partition; replacing a worker also requires new evidence, although the current contract cannot enforce worker-version attribution.

    Artifact: position_projection

    • deployment: active
    • Scope publish:community resolves to hitl
    • Scope publish:professional resolves to manual

    The active declaration is a new version. Its two scopes resolve independently: publish:community reaches hitl through a new exact-scope motion, while publish:professional remains manual.

    Autonomy Events contract →

    Rule: Deployment and evaluator succession require new declaration versions; neither silently mutates a position or an evidence record already on file.

  9. Stage 9

    Demotion

    State: position manual, deployment active for scope publish:community

    One confirmed publishing failure is enough under this policy. The operator moves publish:community back to manual immediately; the professional channel is unaffected.

    Artifact: motion_events

    • Scope publish:community: hitl to manual
    • motion_proposed: actor operator:jordan_lee: confirmed publish_authorization failure on one case; demotion.window (1 case) satisfied
    • motion_approved: actor operator:jordan_lee: operator-initiated demotion, committed immediately
    • position_changed: actor operator:jordan_lee: committing publish:community from hitl back to manual

    All three rows share one motion_id and are committed in a single immediate operator action, for publish:community alone.

    Autonomy Events contract →

    Rule: A policy failure or an operator can demote immediately, on a one-case window, for the affected scope only.