Canonical flow
PR review agent
Stacked semantic review for merge safety.
Abstract pattern
Stacked semantic review
The engineer-legible flagship showing how deterministic checks can sit underneath semantic judgment and keep review on the right side of the merge gate.
- Assisted Human-led with automation support.
- HITL Human approves each action.
- HOTL Human samples or monitors.
- Autonomous Automation acts within guardrails.
Task contract
PR review agent
The engineer-legible flagship showing how deterministic checks can sit underneath semantic judgment and keep review on the right side of the merge gate.
- Boundary
- Input is the PR diff plus context like the linked issue and test results. Output is a structured review verdict of approve, request-changes, or escalate with rationale.
- Evidence log
- Diff, test results, lint results, judge verdict, human decision when gated, and whether a later merge needed a revert or caused an incident.
- Evaluator
-
- Target
- Output
- Technique
- Deterministic tests and lint run first, then an LLM judge handles the semantic call the deterministic layer cannot make.
- Oracle
- Tests as reference for the deterministic layer and senior reviewer decisions as human-gold for judge validation.
- Position policy
-
- hitl
- blocking — agent verdict is held at the merge gate until a human reviewer clears it
- hotl
- sampling — only flagged or high-risk PRs re-enter blocking review; low-risk PRs proceed after sampled monitoring
- Promotion rule
- Judge agreement with senior reviewers stays above threshold over a window and deterministic escapes stay near zero, allowing the flow to shift from blocking review to sampled monitoring.
- Demotion
-
- Rule
- A merged auto-approved PR causes a revert or incident, or judge-vs-human agreement drifts down.
- Fallback
- Human review stays on the merge path for substantive changes.
Teaching point
What this flow proves
Cheap deterministic checks can stack under an expensive semantic judge, and the artifact boundary itself is the thing being reviewed.
Teaching context
Why this flow is governed this way
- Worker context
- LLM + tools
- Without PAA
- Merges rely on whoever happens to review the PR; semantic quality is informal and inconsistent; the merge gate has no systematic bar and no audit trail when something ships that should not have.
- Never full-auto
- Substantive semantic judgment — an LLM judge can own the first-pass review but not the final merge gate without ongoing human validation.
This page is linked from the canonical card set on the flows index.