Worker capability
The same judgments used to govern a task can also improve the system doing the work. Review decisions support changes in oversight and provide examples for improving the implementation. A candidate must still be compared with the current implementation at the same boundary and against the same evaluation bar.
Read Grading first for the definition of a verdict and the requirements for a training-ready log.
Moving up the spectrum does not mean the model got better. It means the evidence got stronger.
Autonomy position answers what the evidence justified. Capability answers how good the work is. They must be measured separately.
Moves on: Evidence
How much oversight does this task require?
Manual through autonomous, moved by admissible evidence over a declared window and a motion that resolves under the declared execution mode. This axis answers what the evidence justified. It says nothing about the quality of the work behind the boundary.
Moves on: Labels
How good is the work behind the boundary?
The quality of whatever produces the output, improved by consuming the verdicts governance already generates. A task can sit at human-in-the-loop with an excellent worker, or reach autonomous with a mediocre one that clears a modest bar. Position is not a capability claim.
Either dimension can change while the other holds still. They still support each other: review produces the labels used to improve the worker, and every improved worker must qualify against the same governance bar before entering production.
Deployment is another independent dimension. A task may be active, shadowed, or disabled at any autonomy position. Shadow deployment lets a candidate run and collect evidence without allowing its output to affect the governed result.
A task can sit at human-in-the-loop with an excellent worker, or reach autonomous with a mediocre one that clears a modest bar. Both are legitimate. Neither is readable from the position alone, which is why the two are declared and measured separately.
General practice orders these interventions by cost. This framework orders them by how hard they are to undo, because revocable is one of the three properties autonomy is required to have here. Cost happens to correlate; revocability is the dimension that matters. No rung is required or endorsed. Naming a required technique would bind the framework to that technique's lifespan.
| Rung | Intervention | Consumes | Revocability |
|---|---|---|---|
| Change what the worker sees | Revise instructions, or retrieve better context. Retrieval deserves the emphasis: selecting the few most relevant past judgments per item stays flat forever, where feeding every past judgment into context grows without bound. | No label budget; the stored judgments are read, not spent | Immediate, revert the configuration and the prior behavior returns |
| Teach by imitation | Adjust weights on examples of good outputs so the worker produces more like them. The right rung when the worker cannot reliably produce the required form at all. | Approved examples only, rejections are wasted at this rung | Requires holding the incumbent; reverting means swapping back to a retained version |
| Teach by preference | Adjust weights on pairs, a better and a worse output for the same input, shifting toward the better and away from the worse. Several named methods exist and the class is what matters, not the current favorite. | Both halves of the verdict stream, approvals and rejections alike | Requires holding the incumbent; reverting means swapping back to a retained version |
| Teach by trial | The worker attempts tasks, an automatic verifier scores the attempts, and weights move toward higher-scoring behavior. The right rung for long-horizon multi-step work; rarely what a single task needs first. | A trustworthy programmatic verifier and an environment to run attempts in | Lowest, behavior is least predictable from the training signal, so regressions are hardest to anticipate |
Anything may produce a candidate: training on exported labels, a revised prompt, a different base model, a distilled smaller model, or deterministic code replacing a model entirely. PAA does not prescribe the method. Its governance requirements begin when the candidate is considered for production.
Adjacent industry work, including training frameworks, reinforcement toolchains, verifier stacks, and environment stacks, owns the trainer. This framework owns the loop the trainer plugs into.
The boundary, instrumentation, and evaluation bar stay fixed while the candidate is compared with the current implementation.
Running a candidate against the same boundary and the same held-out split without
displacing the incumbent uses shadow deployment. A shadow deployment runs the
candidate and its evaluators and records evidence without allowing the candidate's result
to affect the governed outcome.
Two implementations measured against one bar is itself a native comparison, and it protects live output while the candidate arm earns its evidence. See Framework for the deployment axis in full.
This example starts with a modest but usable system so that each improvement can be measured. Starting with an already excellent system would show less of the process.
1
Cold start
A deliberately economical worker, one a cost-conscious team would plausibly deploy rather than a sandbagged one, enters at human-in-the-loop. Output quality is expected to be mediocre. Review volume is high, work volume is low. This is the affordable end of the maturity curve.
2
Accumulation
Every review is a structured verdict against the training-readiness commitments. The held-out split is frozen early. Labels are profiled once, before first use.
3
First candidate
Accumulated verdicts are exported, deduplicated, stripped of administrative fields, and attached to evaluator identity. They produce an improved candidate through whichever rung of the ladder fits the label shape.
4
Governed re-entry
Baseline on held-out, candidate on held-out, same bar, one variable. Entry only on clearance, and only at the position the task currently holds.
5
Earned promotion
The improved worker's live evidence accumulates under the standard rules until the task earns movement along the spectrum, on the same terms any task does.
6
Iterate
Each cycle produces richer labels because the worker is better and the volume is higher. Evaluator distillation proceeds independently, never in the same attribution window.
This limitation affects attribution across implementation changes.
The task declaration names the boundary, evaluators, deployment mode, gate policy, and allowed transitions. It does not identify the implementation producing work behind the boundary. The producer on an evidence record identifies the evaluator that produced the verdict, not the worker whose output was judged.
As a result, the contract cannot prove which worker version produced a window of evidence. It can hold the boundary, evaluator, and bar constant, but it cannot verify that the worker was the only changed variable or identify a previous worker version as the fallback.
A future contract revision can close this gap by declaring the worker implementation and carrying that identity on each evidence record. Until then, teams need an external operational record that ties evidence and fallback behavior to worker versions.